Ex uses the court order to get to me. Noncomplianc

Advise please. Kids dad disappeared for nearly a year then submitted court papers, lying to make himself look good and me look bad. Court order issued, he never stuck to it constantly changing it. It was somehow my fault that he had to pay maintenance and he used that as an excuse for 8 months as to why he couldn't have the children. Constant false accusations via email. Kids hated going and do distressed. Now he has bitten back at me defending myself and said he is going back to the court order. He hasn't seen them for 8 weeks. They haven't slept over since October ( even that was intermittent). Since October the longest he has had them is 3 hours as hoc.
Where do I stand legally if I tell him it needs to be introduced slowly and he can't expect the boys to go from 0% to 100%???
I'm now a stupid anxious mess. It's all games and he just wants to get to me. He doesn't care about the kids and never has "

Feeling so sorry for you. Be strong. I am in same situation. Dont be kind to your ex in terms of contact. Go back to court and say he is not complying with the order.