The Book of Pilgrimage (Kitab Al-Hajj)
Muslim :: Book 7 : Hadith 3078
'A'isha (Allah be pleased with her) reported: Allah's Messenger may peace be upon him) said to me: Had your people not been unbelievers in the recent past (had they not quite recently accepted Islam), I would have demolished the Ka'ba and would have rebuilt it on the foundation (laid) by Ibrahim; for when the Quraish had built the Ka'ba, they reduced its (area), and I would also have built (a door) in the rear.