Is it legal to live together without getting divorce from our spouse

Dear sir/madam, I would like some information on the following.Im a 42 year old Christian male married but due to a lot of problems with my wife I got into an extramarital affair with another married christian woman whose husband also is physically abusing her. Both of us going through problems with our spouses .so we ended up into a relationship. What I would like to know is that we both want to get out of our present marriage and we both want to get married.Also if you could tell me if we could stay with each other before getting the divorce from our respected spouses. Kindly advice me how to go about this as we want to start living together and also at the same time get our divorce from our spouses.Would really appreciate if you could enlighten me on legal aspects and is there are chances of our spouses filing any case against us if we living together without a divorce and what measures should we take legally without any of us going behind bars.Thanking you