In terms of historical locations, St. Stephen's Basilica is quite impressive, while there are also a number of fortresses and castles you might want to visit, with Buda Castle probably being the most significant and extensive, it having been the seat of Hungarian kings, with parts of it date to the Middle Ages.