Refund of token money for purchasing property

I was interested in buying a property from an individual seller, so I paid a token money of INR 25000 via IMPS. The seller committed to return the money within 3 days of payment if any issue arises. But due to poor build quality of the house, I didn't bought it. We didn't do any agreement. Now the seller is making excuses in refunding. He is also asking for property verification report which I got for INR 4000 and saying he will not refund unless he has the original report in hand. I cannot trust him as his tone and behaviour has changed drastically and things have become very stressed. He is a doctor and he is threatening me that he won't return my money until I hand him original property verification report at a place decided by him. Please suggest