Builder delaying delivery and not honouring the agreement

Hi We booked a flat. Possession date was April 2017, however till date only excavation has happened. We have paid 80% under subvention scheme and liability for interest payment will be falling on us starting Feb'19. As per Builder Buyer Agreement, there is a clause which states, if the builder fails to deliver booked unit ontime (and after 6 months of grace period) and the property owner does not want to take alternate property offered by builder, then money will be returned without any interest. We have written to builder and met them in-person, they have refused to honor this clause and are openly challenging us to take legal route. We are middle class family and this money was my father's retirement fund. How can we get resolution, our money back!!