There should be a single-server queue for pve but a cross-server queue for pvp. Single-server queue works like WoW's system, but it only affects people from your server. This kills anonymity so people can't just be d*cks and get away with it, and the legacy system helps with this too as it spreads across all characters. The only avoiding way would be to avoid questing, which is annoying and difficult.
However, for pvp I believe there should be cross-server implementation, yes, I hate WoW's community of "GUISE *** DEFEND THE RAMP OMGH Y U SUK U NOT DEFENDING GR IMAD" and arguing ragers over various topics, but it is my personal belief that this is what the /ignore command is for.