Matchmaking - explaining the not so obvious

[Shoraan]

The devs have stated that their matchmaking algorithm is solid. Several players have stated from their own experience that it is not working well. With this post I want to point out a few obvious and not so obvious issues with implementing a good matchmaking system.

 

At first I was planning to write a small essay but I decided to rather put that into bullet points so it's easier to read.

 

1. A matchmaking algorithm for GSF should have four goals: creating Imp vs. Rep matches rather than wargames, creating two teams of equal strength, starting a match as soon as possible, and keeping the queue time for every player to a minimum.

 

2. The four goals in point 1 have to be put into an order of priority. Some people place more value on short queue times, others prefer teams of comparable strength, others hate wargames. Therefore the devs may prioritize them differently than some of us may.

 

3. We have a queue system, not a pool system (or whatever you want to call it). That means we don't have a fixed pool of 100 players that show up at the same time. Sorting players from a fixed pool into groups would be rather easy, but in a queue system we have to take into account the time someone has waited in the queue.

 

4. Calculating the strength of an individual player is rather complex. A few factors that have to be taken into account are experience, skill, and gear level of the individual pilot. These could be measured by the amount of legacy fleet comms, or time spent in GSF, or KDR, etc.

 

5. My guess is that they have an internal level system (1-6) to calculate a pilot's strength. That means, in theory two of the best (level 6) pilots would be measured against 12 of the worst (level 1). I don't think that they calculate the strength of players in relation to others in the queue but on an absolute scale, i.e. a pilot with 100 hours of experience will most likely be level 6, just like someone with 300 hours.

 

6. When putting together a match the algorithm has to take into account how long everyone has waited in the queue. If there are 2 spots left in one team and a 4-man-premade joins the queue, it will have to decide who to kick in order to put the 4-man-premade in. That may mean that it will throw out two level 1 instead of two level 6 players, because the level 6 players have waited much longer. That may result in an 8 vs 8 match with six level 6 players on one side and only two on the other side (at least for Imp vs. Rep matches) .

 

7. In wargames the behaviour described in point 6 should actually not be an issue. When the algorithm has finally finished the matchmaking it could try to sort the pilots into two new groups to make the teams more balanced.

However, we know that when we start a match we sometimes have to wait for more than a minute before the match starts. If one or more high level pilots don't accept the queue pop the algorithm may replace them with one or more low level pilots, because no one else is available or they waited long enough. Since the team assignments are set as soon as we enter the battle screen, rebalancing is out of the question at this point.

 

8. Some or all of my assumptions may be wrong.

Edited May 8, 2014 by Shoraan
Fixed wording.

[Verain]

T1. A matchmaking algorithm for GSF should have four goals: creating Imp vs. Rep matches rather than wargames, creating two teams of equal strength, starting a match as soon as possible, and keeping the queue time for every player to a minimum.

 

Agreed, though the weights to those would be hard to adjust.

 

2. The four goals in point 1 have to be put into an order of priority. Some people place more value on short queue times, others prefer teams of comparable strength, others hate wargames. Therefore the devs may prioritize them differently than some of us may.

 

In fact, this is the biggest reason why the devs can't even offer hints as to how it works. Even if the players understood the matchmaking, some would write massive rants upon some minor detail of it.

 

I'm pretty sure it prioritizes minimizing wait time, for instance- and I think it does this very aggressively. I'm also guessing that it doesn't look at existing games and predict their end times, which would also be helpful.

 

3. We have a queue system, not a pool system (or whatever you want to call it). That means we don't have a fixed pool of 100 players that show up at the same time. Sorting players from a fixed pool into groups would be rather easy, but in a queue system we have to take into account the time someone has waited in the queue.

 

Also note that it gives priority to grouped pilots versus solo. Presumably these are on some scale- ex, being in group queue probably gives you credit of some amount of "time served" in queue.

 

4. Calculating the strength of an individual player is rather complex.

 

Yes, it is. I suspect that it actually matches not at ALL based on some predicted skill, but just based on the gear you queued with- in other words, if you queue with a mastered ship and four unmastered ships, it probably says "ok, this is a fully mastered guy". If you queue without that mastered ship, it probably takes the second most mastered ship and goes based on that.

 

Aka: I'm almost positive it doesn't look at W/L, KDR, etc. And I don't really think it should, either. In a flat system, which this is, how much requisition you queued with is pretty much how much gear you are wearing (if they let us change layouts, then it would just be the ship you have that is the most mastered).

 

 

5. My guess is that they have an internal level system (1-6) to calculate a pilot's strength.

 

This doesn't seem warranted by anything I've observed. Why do you feel this way?

 

6. When putting together a match the algorithm has to take into account how long everyone has waited in the queue. If there are 2 spots left in one team and a 4-man-premade joins the queue, it will have to decide who to kick in order to put the 4-man-premade in.

 

It might instead decide that the 4 man has to wait, depending on how it weight it.

 

 

 

 

I think that whatever the matchmaking is for wargames, it's rather confused. I've definitely seen it take solo queue aces and put them alongside a premade, and then line up two ship food guys on the other side. It's not common to see that, by any means, but the fact that I do see it means that the matchmaking isn't very strict.

[mr_sim]

I agree with what you both are saying. I'm getting tired of all the QQ about match making, You've just demonstrated why the matchmaking itself is not actually broken. When in fact the unstable player que pool is the problem.

 

This is why I say we need 2 things.

 

- Ranked GSF

- Ranked Pvp que in server for both Ground and GSF

 

The nature of how a ranked server could work is when you que you get a copy of your character on the ranked server, possibly they make a lighter version of your character. Then when the que comes in your client connects to that server. This is a bit of a heavier process that would also move much slower then normal ques and require more infrastructure, So I feel for this reason and many others, that ranked should be the only cross server play.

 

The Ground PvPers are practically screaming for cross server ranked. As well many GSF pilots are looking for both better teams and tougher competition. It would also get some of the Aces and Elites off of the regular ques, and new pilots could have a more balanced environment to learn in.

 

Look BioWare everyone wins. So what do you say next project after strongholds hmmmmm.

[tommmsunb]

I'd say another thing to consider is the rotations going on, on the harbinger we were getting a lot of bad players on the other team, then we waited 10 minutes to get on another rotation and got significantly better players.

[SammyGStatus]

I'd say another thing to consider is the rotations going on, on the harbinger we were getting a lot of bad players on the other team, then we waited 10 minutes to get on another rotation and got significantly better players.

 

^ This. Unfortunately it's very hard to que with the premades that're in Europe playing on WC servers when we're running our nightly groups around 6pm-2am. We (Irrelevant Squadron / DO) have literally been doing EVERYTHING WE CAN to avoid playing the 2 shippers and get into the rotation with players who know what they're doing. Unfortunately, the player pool at the time we play on the servers doesn't seem to give a dang. When it was 5 of us (group of 2 and a group of 3), we were always matched together and faced pretty bad opponents. When it was just Tom and I towards the end of the night (maybe around 1am), we were the only players on our team who knew anything that was happening and lost to a what looked like a premade (entire team full of 5 shippers). It seems that our timings are off more than anything (the exact time and rotation that our two groups of 5 people total were queuing either weren't on the same rotation or the decent opposition hadn't logged in/ queued yet), because without everyone of similar statistics queuing together, all of our observations are going to be slightly skewed (Our Official Actuary can provide a more detailed analysis of why this is so). Till there is a ranked version of queuing (which most likely means x-server), we're going to still find ourselves, more times than not, in these predicaments. I won't hold my breath :/

Edited May 8, 2014 by SammyGStatus

[Sidenti]

I'm sorry. I think I'm going to need

to me.

[Shoraan]

This doesn't seem warranted by anything I've observed. Why do you feel this way?

 

I apologize. I should have elaborated a bit. The level range (1-6) was just an example and it could be 1-6 or 1-20 or 1-100. It's pure speculation on my side. With "internal level system" I actually meant that they probably use brackets for player strength, so that they can deal with extremes. Let's say they assign a weight of 1 to someone who is about to enter his first match. Let's also assume that having a mastered ship is the ceiling and assign a value of 100 for that. (It may make a difference whether you enter the queue with 1 mastered ship or with 5, but either way you end up with a relatively high number.)

 

If you have such large differences in weight values, matching small teams (8 vs 8) will lead to weird results in extreme situations, e.g. - assuming pure solo queue - having someone with five mastered ships together with seven newbies against 8 semi-competent pilots. The question really is "How many semi-competent pilots is this 5-mastered-ships-guy worth?" Putting them into brackets eliminates extremes and should prevent unfair situations where 5-mastered-ships-guys are punished by being constantly teamed with the least experienced and geared players.

 

It might instead decide that the 4 man has to wait, depending on how it weight it.

 

If the goal is to start a match asap rather than letting everyone wait indefinitely it will have to kick someone.

 

I think that whatever the matchmaking is for wargames, it's rather confused. I've definitely seen it take solo queue aces and put them alongside a premade, and then line up two ship food guys on the other side. It's not common to see that, by any means, but the fact that I do see it means that the matchmaking isn't very strict.

 

I agree. Wargames are really weird sometimes. I don't have an explanation for that.

[SithAceI]

Has anyone tried to base their matchmaking experiences to how many ships you Que with?

 

I noticed when I have que'd with only one ship I have gotten mostly noob 2 shippers as teammates, conversely when I have que'd with all five ships I have been matched with 5 shippers. I am starting to think that part of the matchmaking system weighs how many ships u que with. Of course this does not explain getting what u get for your enemies team in so much as good players or noobs.

 

If anyone can test this out and report back to see if at all I am on to one possible matrix in this messed up matchmaking system.

[Sidenti]

Has anyone tried to base their matchmaking experiences to how many ships you Que with?

 

I noticed when I have que'd with only one ship I have gotten mostly noob 2 shippers as teammates, conversely when I have que'd with all five ships I have been matched with 5 shippers. I am starting to think that part of the matchmaking system weighs how many ships u que with. Of course this does not explain getting what u get for your enemies team in so much as good players or noobs.

 

If anyone can test this out and report back to see if at all I am on to one possible matrix in this messed up matchmaking system.

 

I've done this in every possible configuration, from fielding just one stock Scout to a full loadout of buffed ships. The matchmaker algo appears to ignore it all.

 

One thing I HAVE noticed is that my main and alt get far different matchups. The former is consistently matched up with the better pilots, but queue time suffers. Conversely, the alt gets into a pile of matches against much weaker opposition and sports a 19-9 record.

 

Did I mention he was an Imperial alt? That seems like it'd be important when discussing wins and losses. -bp

[wishingwell]

On Jedi Covenant, I've been noticing things like this a lot:

 

http://i.imgur.com/lRl7pQv.jpg

 

Note that it's Pub on Pub, so everyone came from the same queue pool. We decimated these guys; final score was 7 to 1000. I hate matches like this, and it seems like they are happening more and more often. It's like whatever match-making is going on is actually lumping together similar ships but only on one side.

 

I would be inclined to believe that it largely comes down to the player pool varying so wildly if it weren't for things like this. It doesn't seem to matter what time of day I play any more -- the odds of getting a match like this, even when it's Pub vs. Pub or Imp vs. Imp, are just crazy.

 

Now, if you say the match filled up this way because 4 of the people on my team were in a premade and that's the way the cookie crumbles... I'm actually in favor of making the premade wait, and letting the little guys fight 6v6. I wasn't in a premade in this instance, nor am I ever (I am almost always in a duo) but honestly I think it's really bad for the future of GSF to let matches continue like this. Let the premade break up and end up with two on one side and two on the other if they don't want to wait. That stinks, but until we get our playerbase healthy again, I think more people are being turned off by this than they are being entertained.

 

EDIT: And if we assume my team has a 4-man premade, that *still* doesn't explain why some of those little guys didn't end up on my side, and some of us on theirs. If four of them were matched with the premade that would have at least made it slightly more even. Both sides would have had a chance at more than one gunship and at least one bomber. It doesn't make any sense.

Edited May 17, 2014 by wishingwell

[General_Brass]

On Jedi Covenant, I've been noticing things like this a lot:

 

http://i.imgur.com/lRl7pQv.jpg

 

.

 

If the one five ship guy left before the match started, chances are it was me. I see that all too often and yes it's only fun for people that are out to make others rage quit. Thankfully it's always nice to see that there are a few people that aren't out to do that.

[Kalphitis]

On Jedi Covenant, I've been noticing things like this a lot:

 

http://i.imgur.com/lRl7pQv.jpg

 

Note that it's Pub on Pub, so everyone came from the same queue pool. We decimated these guys; final score was 7 to 1000. I hate matches like this, and it seems like they are happening more and more often. It's like whatever match-making is going on is actually lumping together similar ships but only on one side.

 

I notice this all the time as well.

 

One important thing to consider, along with the game choosing whether or not to do a same faction or multi-faction match: how big will the game be?

 

If the system always started games as quickly as possible, then we would only ever see smaller 8v8 games. The fact that we have 12v12 games means that the system is programmed to wait for a set amount of time if it 'thinks' that more people will join to make the game larger. I believe that 12v12 is the priority over 8v8.

 

The obvious answer would be: if one faction has at least 12 players in the queue, and the other faction just had their 8th (or group of 8 to 11 max) join, the system MUST wait...but for how long? 5 seconds? 30 seconds? 2 mins?

 

Now that we've established that the selection process is already programmed to allow for wait times instead of playing games as quickly as possible (in order to create the largest game possible), why do groups get queue priority to 'fill' up games faster? Why not wait then also in order to allow for fairness?

 

OP: in your post, you pointed out that up to 3 people might get dropped if a group of 4 is needed to 'fill' up a game. At least that makes 'some' sense -- but I've also noticed that people get dropped in queue priority sometimes even if it doesn't matter. I obviously have no way to prove this, just observations.

[Kalphitis]

I If the system always started games as quickly as possible, then we would only ever see smaller 8v8 games. The fact that we have 12v12 games means that the system is programmed to wait for a set amount of time if it 'thinks' that more people will join to make the game larger. I believe that 12v12 is the priority over 8v8.

 

The obvious answer would be: if one faction has at least 12 players in the queue, and the other faction just had their 8th (or group of 8 to 11 max) join, the system MUST wait...but for how long? 5 seconds? 30 seconds? 2 mins?

 

I suppose the easy, and most plausible explanation, is that the queue is only checked by the system every so often....say 30 seconds.

 

So every 30 seconds, if there is at least 12 on each faction, a 12v12 game starts -- if not, an 8v8 or no game starts.

[Bolo_Yeung]

Well... if there is any matchmaker it is seriously broken.

We queued in 3 with good team 5/5/4/ships Our team was like 5/5/4/4/3/3/2/2. Second team - all 2-shippers with one player having 3.

Ofc it was wargame, so Rep vs Rep.

While I know that 2-shippers can be alts of experienced players, this wasnt the case. Even with us going in unupgraded ship it ended with like 50:5.

 

Thats all about 'working' matchmaker.

[General_Brass]

Well... if there is any matchmaker it is seriously broken.

We queued in 3 with good team 5/5/4/ships Our team was like 5/5/4/4/3/3/2/2. Second team - all 2-shippers with one player having 3.

Ofc it was wargame, so Rep vs Rep.

While I know that 2-shippers can be alts of experienced players, this wasnt the case. Even with us going in unupgraded ship it ended with like 50:5.

 

Thats all about 'working' matchmaker.

 

Anti Matchmaking is much more the rule than the exception in this game. The dynamics of what is allowed, and the prioritization pretty much insure it.

Edited May 19, 2014 by General_Brass

[Kalphitis]

So, 3 times in the last 2 days I have noticed that another person that is Solo q'ing has jumped me in priority and got in a game sooner than me.

 

Most recently, I logged in and saw Drakolich was in a game. I queue'd up, 5 mins later the game ended, and another game started. Drak got in, I did not. I asked Drak, and he said that he was solo queuing also.

 

What does this mean? Matchmaking does exist! How does it work? Who knows.

 

Drak was on his Drakolichsoloq character -- he doesn't have many ships on that toon. Also, his character is a low level, not 55.

 

Earlier today the same thing happened when I got jumped by MMORPG. MMO is famous for only playing with a gunship.

 

My conclusion: there is a matchmaker that matches players based on how many ships they either own on their current character, or how many ships are on their current load-out. It does not pay attention to legacy things, only current character.

 

This still doesn't make any sense to me....and frankly it does piss me off since I hate waiting...but yea.... I feel like I get jumped in priority all the time, be it groups or other solo'ers.

[Kalphitis]

Note that it did not say that I was AFK on either of these occasions.

[Sidenti]

So, 3 times in the last 2 days I have noticed that another person that is Solo q'ing has jumped me in priority and got in a game sooner than me.

 

Most recently, I logged in and saw Drakolich was in a game. I queue'd up, 5 mins later the game ended, and another game started. Drak got in, I did not. I asked Drak, and he said that he was solo queuing also.

 

What does this mean? Matchmaking does exist! How does it work? Who knows.

 

Drak was on his Drakolichsoloq character -- he doesn't have many ships on that toon. Also, his character is a low level, not 55.

 

Earlier today the same thing happened when I got jumped by MMORPG. MMO is famous for only playing with a gunship.

 

My conclusion: there is a matchmaker that matches players based on how many ships they either own on their current character, or how many ships are on their current load-out. It does not pay attention to legacy things, only current character.

 

This still doesn't make any sense to me....and frankly it does piss me off since I hate waiting...but yea.... I feel like I get jumped in priority all the time, be it groups or other solo'ers.

 

IF that's the case, it's probably paying attention to total ships owned by a character. I've tried moving a stock Blackbolt into my aces' loadouts just to see if it helped to get me a pop. No dice. But Nine popped regularly with a grand total of three unmastered ships. -bp

[Kalphitis]

IF that's the case, it's probably paying attention to total ships owned by a character. I've tried moving a stock Blackbolt into my aces' loadouts just to see if it helped to get me a pop. No dice. But Nine popped regularly with a grand total of three unmastered ships. -bp

 

I do feel like this is very probable, and that it doesn't pay attention to the better statistic (which would be lifetime total req or total games).

 

Not going to lie...I'm seriously considering making a new character that only ever purchases and plays with 2 or 3 ships just so I can get better q priority. See how I feel about it in a few days...

 

Would be cool to get some dev insight on this.

Edited May 20, 2014 by Kalphitis