I think most people don't care if gear matters a little. I personally like earning gear and playing around with it to min/max it. However, the operative word is "little". Once you get into large gaps in gear it becomes no fun for those that do not have gear, and also for those that do but like competitive matches.
Large gear gaps is also very bad for the general health of PvP as anyone new or people with alts will simply not bother with PvP which leads to less people, less pops etc.