From the gameplay I've had so far, I'm getting the impression of that noone ever uses the "join party" function on the adventure board. Rather, they create their own parties instead.
This is fully understandable, simply due to the fact that doing anything at all costs energy and using "join party" is essentially like gambling away 10-20 energy.
Here's what could happen: