he says he is interested, i don't feel he is

I dated this guy 6 years ago, but things didn’t work.
For me it didn’t work that i thought he never listened to me and was self-centered, always all about his needs. For him, he thought i demand too much of him, that i was needy.
So we both agreed we needed different people and we called it off but remain friends, for 5 years. At first we didn’t have much contact, he moved away and we really didn’t have much to talk I guess. But for the last 3 years or so, because of his work, he was driving a lot and during those rides he used to call me and we talked a lot. We become really good friends, he actually listened to me on those calls and empathized a lot with whatever i would be going through.
Even dating, we gave each other advice and support and i really started considering him a really good friend.
Fast forward to a couple months ago, he breaks the news that he is coming back. He was convincing me that we should try again. I really didn’t want to at first because he is now a really good friend and what i am to lose is much more. But somehow, here i am…

He comes back and we met right away, just to chat and went on a walk. After that we met a couple more times with stay overs and that’s… two weeks ago.
He started back to work and he says he has been very busy. I know he is, he has all his family here and all his friends and they also probably missed him.
But it just doesn’t feel right to me that he hasn’t made any time to meet me in all this long.
He is also a bit special, he gets overwhelmed easily in relationships or around people, as i explained above that’s what didn’t work for him, so i have been patiently waiting for him to come to me. In that line, he is also very last minute, planning freaks him out, in case he needs his solo time.

He writes me every day asking me for my day and he has asked me to meet up a couple times for a run half way between us (we live 1h away) but always last minute and I haven’t been able. I joked about the fact that we seemed to not to be able to find time for each other and he called me very apologetic about it, but saying things will get worst because of this or that.

This weekend he is traveling for work, next weekend he is here. I told him i could stay around if he wanted to do something or i could join some friends on a trip, the following i am out for work. That just is enough for freaking him out, he said he didn’t want to make plans in case he didn’t want to do anything at all because he may be stressed out from work… so i joined my friend’s trip, it just didn’t feel right to make space for someone that may not feel like seeing you. Although my experience with him in the past, is that he always tells me to do something… but always last minute.
In any case, it means that for the next 3 weekends we won’t see each other… making it to a total of 5.

So i don’t know what to do or what to think. I have my things too, which may not be a good fit all together. I am impatient, although i have worked on it over the years, but sometimes i have difficulty recognizing if this is affecting my judgment. It’s fair that he has to spend time with his family and friends. I also understand his planning fear comes with being him and although that’s a hard thing, he cames with a lot of other awesome things that make that one tolerable… if we ever see each other.

And i also suffer from relationship anxiety, i tend to withdraw rather quick when i can’t understand things because i kinda think it would be less painful if i am the one leaving first. Instead of just doing that, i thought i would write here and see if i can have some more objective opinions that give me clarity.

thanks for reading!