I’m so disappointed at him

We have been together for 5 years. He bought a house shortly before we met and even though there is a goal to buy a house together and move in this new house down the road, right now we both love to live in his house, the location is gorgeous and we are very happy there.
He keeps telling me that this is my house as much as his and since we have been living together I contribute to all expenses including the mortgage. We are not married yet but we are considering it for later (it is not as important in my culture as it seems to be in the US right now) and we have other areas of our life where we want to invest money first.
I have been mad and disappointed at him for the past few days because his parents asked if they could install permanently their fifth-wheel on our property to come visit whenever they want and he said yes. He said yes knowing that I said no a while ago when they were planning to live with us permanently on our/his land using that same 5th wheel.
His family has not been the most welcoming to me and if his dad is nowadays pretty indifferent to me, his mom seems to dislike me. She has been rude a few times and has made a few passive-agressive comments towards me. She loves to criticize and judge everybody so I’m not feeling comfortable around her anyway wondering what she is saying of me behind my back. I started to keep myself busy and mostly stay away when they visit us over the weekend but having them living in the yard for up to a few weeks at a time, whenever they want, and for how long they want is another story.
My boyfriend keeps telling me he is going to be clear with them and explain about boundaries so they do not bother me especially when he is out of town for work (and that happens a lot). These people have a history of not respecting other people’s boundaries when it does not suit them and I doubt they are going to respect mine especially when my partner is not here. Since I am already at my wits end, I don’t see that going well on my side.
He keeps telling me that this my house as much as his but this is exactly where I can see that it is not and his family comes first.
He told me that I said yes to the project a long time ago (that was at the beginning of our relationship and he told his parents yes before asking me so what was I supposed to say being so new in our relationship…). He conveniently forgets that we talked about it again a year ago and that I clearly started that at my/our age I do not wish to live with my parents or his, be it long or short term. When I remind him that, he keeps saying that he thought that I was talking about long-term and not ”short” visits (for me a short visit is 5 days max, not a month). He also knows that we are not getting along great and that I don’t really enjoy myself when they visit.
I completely understand that on paper this is his house and he can do whatever he wants, but I can’t help but feel betrayed after all these years of ”let’s make my home your home too”.
I feel so disappointed at him, I don’t know how I am going to get over it. He literally set me up in a situation where I don’t want to be that is going to last years if not decades until his parents decides they want to move here permanently because they are too old to travel. I love him and he is a great partner but right now I can barely look at him.
Am I the idiot because I should have seen that coming since I’m not officially on the title ? What should I do now ?