It's not a serious issue. The lack of population on some servers is a serious issue. The time it takes for updates and new content to appear is a serious issue. Endgame armour designs that look like a 12 year old came up with them are a serious issue.
These are what will drive people away, not taking slightly less time to form a group. You may notice that it would be faster to begin with, but in the end it will still take over an hour because people have left for other reasons.