Questioning the heritage status of the city, the Congress has said thousands of people have been troubled by this status.