Should I walk away or try?

This man and I grew up together and have reconnected after 20+ years. We started off as friends spending every weekend together. A couple of months into that we became intimate and now I have been stuck in this situationship for 7 months. I’m not great at stating what I want but finally, I told him we are not on the same page, he said that we are and we agreed that neither of us will see, sleep or emotionally connect with anyone else but now he’s pulled back a bit and during our convo he said I don’t like to label things, I like them to grow organically into what they’re going to be, to just become committed to each other.

I am not a chaser, I can count on one hand the times that I’ve reached out to him first so I am just letting it be but his phone calls and messaging have greatly declined.

I am horrible in relationships, I’m cold and distant, I’ve gone through two divorces. I let him in in the beginning because we were just friends at first and I had no feelings for him. I always pull away when I get feelings for someone. I’ve tried to end it with this guy 7 or 8 times now because I get nervous that I’m getting too close and he won’t let me go.

Do I walk away or do I actually try? I’ve been doing some work on my own issues and have purchased the “He’s Not That Complicated” so I know I’ve been handling my relationships all wrong. Or is he just not that into me?

Side notes: We spend time with his family and we still do message every day.