No, there is not a way to stop it. If you have slinglers in your group, Hunker down will prevent you being pulled. I pop it usually when a new wave starts so it doesn't interrupt what my job is. I dont know about other classes, but they should have an ability like hunker down.