According to the Qur'an, the Kaaba was built by Abraham and his son Ishmael. Islamic traditions assert that the Kaaba "reflects" a house in heaven called al-Baytu l-Ma?mur (Arabic: ????? ???????) and that it was first built by the first man, Adam. AbrahAm and Ishmael rebuilt the Kaaba on the old foundations.
The four corners of the Kaaba roughly face the four points of the compass. In the eastern corner of the Kaaba is the "Rukn-al-Aswad" (the Black Stone or al-?ajaru l-Aswad), generally thought to be a meteorite remnant. So, the Kabaa is the structure. Not the meteorite.
[Quran; 2:125] We have rendered the shrine (the Ka`aba) a focal point for the people, and a safe sanctuary. You may use Abraham's shrine as a prayer house. We commissioned Abraham and Ismail: "You shall purify My house for those who visit, those who live there, and those who bow and prostrate."