The problem is the same reason you're bolstering as the reason you don't want it, fighting the same people over and over again. Sure it's fun at times, but fighting fresh faces adds a new element to the game as you won't know who they are and what spec they are (healer, tank, dps). It "should" also help with queue times for many servers with lower than average populations who are either forced to constantly play huttball during hours that the few republic players (on imp heavy servers) or imp players (on repub heavy servers) are on to create matches for voidstar and civil war. A lack of variety in terms of warzones to keep things interesting will hurt the pvp population and prevent it from growing.
Also, despite the forum trolls, WoW cross server pvp and dungeons were an amazing hit for the vast majority of the players that play the game. MMO Devs will always cater to the masses, not the elitist few who have the loudest voices. (I'm not accusing anyone here of anything, fyi) People don't like change, so when it happens they complain about it causing issues with this and that but in the end they will get used to it and either enjoy it or not.
In the end, you can't please everyone. So they should and will try to please as many as they can.