How can i proceed for cancellation of Booked Flat

I booked a flat which was under construction and paid 10 L as booking amount and upon reading the draft agreement was not comfortable for lot many point and since the builder was nor ready to change clauses hence proceeded with the cancellation. As per the booking form it was written that 1L will be in case cancellation is made after booking. No agreement has been initiated as of yet and its been 1 month since i made a request for cancellation and now the builder says he will refund the amount excluding 1L + tax chalans which has not been refunded by the government. Can i get the 1 L amount back which he has deducted for booking cancellation, also is the tax chalans amount refunded back to the buyer only after they are refunded by the government ?