He's a single, middle aged man who lives alone in a shitty rundown apartment with his cat. Supplemental material mentions that he regained his fear of women, and this even cost him his job as a security guard at a nightclub. He found work as a freelance bodyguard. We can give him the benefit of the doubt it's a suitable job for him because he is actually still very strong by human standards, but freelance isn't a consistent thing.
He does have friends, the main cast. However, he's probably not very close to them anymore. Most of them either got married, drifted away, or have nothing in common with him anymore. So he probably does spend most of his time alone either at home or working. The few times he does get invited, he has to watch the love of his life being happily married to the man who was indirectly responsible for his death. Not only that, but Bulma has TWO children with Vegeta. Everyone else either talks about the last bad guy who threatened Earth or family matters. It's never shown, but knowing all this, we can assume he often feels left out. What can he talk about? Relationships? Everyone's been married for at least a decade. Fighting? Everyone's arguably stronger than him. Life? Everyone's stable, except him. And because he has nothing to contribute, he always ends up saying something stupid that upsets everyone. He's not doing this to be an asshole. He's just been so disconnected with everyone, and wants to fit in, that he might has some social anxiety.
If you think about it, he probably related to Krillin and Tien the most. He probably felt comfortable with Krillin knowing both of them were hopelessly single and outclassed by the Saiyans. In fact, Yamcha on the exterior was more capable of finding love; he's good looking, has a proportionately well built body, and is still leagues above most Earthlings in strength. Krillin is a dwarf with no nose and is a textbook "virgin". But eventually even Krillin found love in his life, a hot blonde cyborg taller than him no less. He also got a very good job as a police officer, and lives at his own house with a family. Tien is a recluse, and it's been stated in supplemental material that he settled with Launch as well as running his own dojo.
Being invited to the baseball game was literally the best day of his life in decades. Even then, he was the butt of jokes, and won on a technicality. He had his 15 minutes of fame, but following that, he was already forgotten. Then that whole being forgotten for the ToP. He wasn't even invited to spectate. Yamcha wasn't even upset. He finally reached the point that this is how it is for the remainder of his life. Single. No stable job. Lives in a shithole apartment. Not even valued as a fighter. He's a perfect reflection on real people who lost their glory days and lives a mediocre life.
