When a group enters a graveyard, those players should not be allowed to leave the group until the group reaches the next floor. If the player dies, they could leave the group as well. It's tiring to have a good group, enter a graveyard, then have half the group leave. Then things get tough and that one other guy staying leaves.
Graveyards are supposed to be something to overcome as a group, not something to roll your eyes at and leave the group over.
Good gods, have you been to a T3 graveyard before? Even if you are properly equipped (shivermist, glacius, etc.), it is a long and tedious level, many players do not want to put in that much effort. I'll also wager that a good majority of players don't even know how to handle a full party T3 graveyard, I know I'm not so good at it myself. So don't be surprised when everyone ditches. If you're worried about party members ditching and leaving you with extra phantoms, make it clear before you step on the party button that you don't want anyone ditching, or go solo.