Issue with builder in buying a property

I gave advance of 4,00,000 to a local builder who was selling his newly constructed house in Durg Chattisgarh. Nect process was to get the remaining amount as bank loan. We applied for loan through ICICI bank and loan was approved for our credentials. Bank wanted to visit the property and verify the builder's document. The document which builder provided was not satisfactory for bank to give loan and hence the loan was rejected. We asked builder to return the advance amount but he denied and sent a legal notice saying that if the we don't book in a month the advance money will not be returned and the booking is deemed cancelled. Now, My Issue is banks will not give loan and I cannot complete the deal. Is this a consumer case and can I go to consumer forum against this or is this even not a case and I shall forget about my money now. The game which builder has played is accountable for all these mix ups. Any suggestions here will be really appreciated.