He distanced himself after things were going well

I desperately need help here….I’ve been talking to a guy for a while & in the beginning I will admit things did move fast. We both just hit it off. After two and a half months, We had a bit of a fall out over something really small, the issue was that I texted “hey, I’m still working” and he replied “oh okay Ms. 9-5 and then he said “30” so I thought he meant age but he meant something totally different and I kept asking him what it meant and I probably did come off a little aggressive in trying to figure it out because I just didn’t understand why I had to wait until we saw each other in person for him to explain a small thing. we didn’t talk for 3 weeks after that but he eventually reached out to me again. At first when he did he was so excited to see me and talk to me and after we saw each other we now talk on and off. He will text like once a week now. This pass week he text me good morning and I honestly forgot to respond to him but now I’m kind of happy I didn’t I’ve been feeling like I should pull away b/c I don’t want to be taken for granted or a fall back girl. Not to mention he did just get out of a five year relationship last year so I guess pulling back may be for the best right??? And to add the cherry on top his ex and him did reconnect recently. I can tell even from when we were together he was afraid to let him open up to the idea that he could be with someone else. Idk should I text him back even though it’s been days or should I give things time to rest and then text him later this month and take things slow?