Refund if no progress by builder till 3 years after taking booking

I booked a flat with a builder in Bhiwadi, 3 years back in 2013. But till date the builder has not started the construction. There pet answer is that they will start construction next month. I have already paid 20% of Basic cost of the Flat. They have made builder buyer agreement after One and a half year of booking in 2014 end. In the agreement he has mentioned project completion time as 4 years + 1 year as grace period. Also No bank is giving loan on this project till date. So I told the builder to return my money as I am no longer interested in the Flat anymore. But he is not giving my money back and saying he will make the Flat. So what to do in this case.