The Book Pertaining to the Excellent Qualities of the Holy Prophet (may Peace be upon them) and His Companions (Kitab Al-Fada'il)
Muslim :: Book 30 : Hadith 5816
'A'isha reported that Allah's Messenger (may peace be upon him) granted permission for doing a thing, but some persons amongst the people avoided it. This was conveyed to Allah's Apostle (may peace be upon him), and he was so much annoyed that the sign of his anger appeared on his face. He then said: What has happened to the people that they avoid that for which permission has been granted to me? By Allah, I have the best knowledge of Allah amongst them, and fear Him most amongst them.