Owner's association have penalized me for Party hall usage-BLR

So I asked the direct owner about this plan and he permitted me saying you will have to pay the nominal fee as this is not a party (for parties the hall is free for all) this is what he said and told me to take the keys from security guard. However the problem started when all the owners came to my home and asked why I was using the Part hall for sleeping purpose, I explained them about my discussion with the owner. Now the owner completely denies that he permitted and says that "he asked me to take the permission from the owners association" So the owners association allows me to continue for 2 days and also says that you need not pay anything and they all go. after a week's time when I have left the home the owners association puts a penalty of 5K on me, without me being in the meeting. so I ended up giving 5k to the owner as the owner as asked the 3rd party company (Pinclick) to hold 5k out of mt security deposit. Please help!!!