Our Scripture reading today is 1 Peter 2:1-5.
Verse 3, "If so be ye have tasted that the Lord is gracious."
The reason we as Christians should have a desire for the Word of God is that we have experienced the grace of God over and over again.  This is what the word, "tasted", means in this verse.  God's grace has been poured out on us time and time again.  Titus 2:11, "For the grace of God that bringeth salvation hath appeared to all men,"  God's grace saves from our sin and sanctifies for His glory.  What a blessing to be under God's grace!  
As we consider all of the grace of God we have personally experienced, it ought to bring us to a place of desiring His Word and putting away all sin.  We do not earn favour with God.  We cannot earn our forgiveness.  We can, however, express our deep love for Him by walking in His Word and living according to His will.  Our Godly conduct is also made possible by His grace.  The taste of grace that we have all experienced is what keeps us hungry for more of God's Word.