Campaigners want a replacement for the complex built on the same site after the local authority moved to demolish it.