Wes Hayden, The Bachelorette’s “baddest boy of them all,” just wrapped up a guys’ weekend in Austin with some other former contestants from the show.

“Tanner [Pope,] who lives in Dallas, drove up while Mike [Stagliano] and David [Good] and the rest of the guys flew in to have a good time and see me,” Hayden tells PEOPLE. “I loved hanging out with the guys. We had a great time.”

What does a group of bachelors do during a guys’ weekend? While some of the guys went to Lake Austin Spa Resort to get pampered, the whole group hit local hotspot Ranch 616, where they shared dinner, drinks and dish about Bachelorette winner Ed Swiderski — and those claims he was unfaithful to his new fiancée Jillian Harris.

“I had no clue whether Ed had a girlfriend … while we were on the show. I really didn’t know about any of it,” he says. “But if it’s true, I feel really bad for Jillian. I don’t know what to believe to be honest with you. I feel bad for Jillian and for Ed. But if in fact he was doing what everybody’s been saying, then man up and deal with it.”

The controversial country singer, who told PEOPLE not to believe everything you see on TV, expressed doubt about whether romance can blossom on the show.

“I have a hard time grasping that any of the show’s intentions are genuine,” he said. “If Jillian and Ed and their relationship is truly genuine, if she honestly came on the show looking for love and thought that she found it and then found out that it was all made up and Ed was cheating on her, then that’s awful.”

As for Jillian, Hayden says, “We connected on the show and we’re always going to be friends.”

But he’s moving on and focused on his music career, still pushing his first album Full Circle.

“I’m not dating anybody; I’m way too busy right now,” he says. “I see a lot of my sister right now though. She goes on the road with me wherever I go … But you know, if I meet the right woman, I’d like to settle down someday.”