Seyyid Ali Baba, the Qadiri sheikh living in Lahore, was invited to Prizren in his dream in 1783 by Gözcü Mahmud Efendi, who was martyred during the conquest of the region. Sayyid Ali Baba accepted the invitation and reached Prizren after a long journey. First, he works as a tomb keeper at the grave of Gözcü Mehmed Efendi.

Then he builds a lodge here and guides the people. Even though it was damaged during wars and interregnum periods, dhikr and devrân are never missing in Seyyid Ali Baba Kâdirî-Rezzâkî Lodge today.