Amenities dispute between different category of society what can I do

Builder designed phase 1 with D1, D2 buildings having 56 flats and 56 rowhouses/bunglows. But he kept separate entry gate for buildings & rowhouse with big wall in between. He provided common amenities(club house, swimming pool, borewell) in rowhouse complex which supposed to be common for both. STP tank is also common and it is on side of D1, D2 building. But rowhouse people refusing us to use amenities on the basis that in their agreement builders mentioned amenties only for row houses. Because of that we people living in D1, D2 are not able use these amenities and builder is also not taki