A trigger for the problem is when we are grouped up before the match and when someone enters the match later on in the timer they seem to be left out of the group. It is like the system has a spot reserved for the late joiner (by late I mean not clicking join match right away) but another player enters the war zone taking the late persons spot on the raid frame. This consistently happens when we are in the war zone as the first group. We had a voidstar that I got in as soon as the timer popped up but my guild mate that was grouped with me joined the warzone when the timer was at 1:37 and the first group that we where supposed to join got filed and my guild mate was not on the window.
tl;dr When grouped, not joining the war zone right when it pops up is a trigger.