Is it cheating if you are on a break and your girlfriend is pregnant...?

I have been in a really unhappy and miserable relationship for just over a year now. Long story but i fell pregnant and my partner wanted me to get rid of the baby. There are a lot of issues regarding his ex and he spends a lot of time with her and in constant daily contact. We argue and break up over it a lot, im very insecure over it all even more so when i was pregnant, and every time this happens he drives straight to her house and they will chat about it and do what ever...i dnt know what goes on really...im just left to my imagination. All through the pregnancy this happened...he even stayed nights there which he hasnt told me about but i found out. I have had the baby now and things are still not good...he has little time for the baby. in an argument he made a big thing out of saying that if were not together he can do what he wants with her and if he had had sex with her that wud be fine and i have no right to be upset......im confused... Is this an admission???? should i not be upset if anything happened? coz i think that treating a pregnant woman this way, is hurtful and disrespectful and i feel that if he cared enough for me he wouldnt even be thinking of sleeping with another woman.i couldnt imagine doing it to him so why wud he do it to me???? i know the obvious answer is that he doesnt love me....i guess i just need to hear it from someone else...