Interview with BioWare’s Manager of Analytics – Stats and Game Design Decisions

[ImmortalLowlife]

I think some of you may be reading more into that than was intended...I don't believe that means only 10% ever get to the end and get played, I think he means 10% RUSH through the leveling (spacebar spacebar spacebar), most enjoy the story and take their time by watching cut-scenes.

 

No that's EXACTLY the conclusion I came to Tux. But listening to "certain members" of the community, they'd like us to believe that the majority of ppl using 12XP are rushing to endgame so they can ruin everyone's pug OPs groups by being under geared noobs & not knowing their classes." It's cool to see it said by a dev that the majority of players using the boost are using it to enjoy the story.

[TUXs]

No that's EXACTLY the conclusion I came to Tux. But listening to "certain members" of the community, they'd like us to believe that the majority of ppl using 12XP are rushing to endgame so they can ruin everyone's pug OPs groups by being under geared noobs & not knowing their classes." It's cool to see it said by a dev that the majority of players using the boost are using it to enjoy the story.

OK, sorry I agree...I like that people are using it to enjoy the story as well.

[NeuroniaSW]

"12x XP is a great recent example. Through analysis of the Rise of the Hutt Cartel expansion, we came to the realization that returning players found their old characters hard to engage with (too much stuff in your inventory, unfamiliar abilities, changed skill trees, etc.). We worked with designers to implement a solution that addressed this issue as well as gave us an opportunity to experiment with something that could well have been risky."

 

This 100X over. My favourite alts felt completely foreign to me under the new system so I just rolled new alts.

12xp was a boon though as I just rolled through Chapter 3 on my Sniper, Consular and now my Knight.

 

As for the Ranked PvP issue...I'm going to have to place some blame on poor information from the developers here. You get an infoguide about Warzones then it mentions ranked, but I'm still unclear about it. As far as I know you need to be level 60 and have 2018 gear...but nowhere in the game does it explain how to get 2018 gear, and I have had to rely on posters' signatures for that. Compare that to Galactic Starfighter, which had pages of stuff about its functionality.

 

Pretty sad to see 8 v 8 is gone though, that seems like a ton of fun.

[Jadescythe]

Less than 10% of our player base seems to be primarily motivated to “get to the end”; the rest choosing to experience the story that exists.

 

It's metrics like this that make you re-evaluate your perspective on the game. It helps you understand why there isn't so much more time put into operations and ranked PvP because the overall player base isn't focused on end game. The end game population is just the most vocal. I can't count the number of times I've seen someone join a guild and say they've never done an operation since launch and we're 3 years into the game.

[TUXs]

It's metrics like this that make you re-evaluate your perspective on the game. It helps you understand why there isn't so much more time put into operations and ranked PvP because the overall player base isn't focused on end game. The end game population is just the most vocal. I can't count the number of times I've seen someone join a guild and say they've never done an operation since launch and we're 3 years into the game.

That quote of his has absolutely nothing to do with what you seem to think it does...it didn't say "Less than 10% of the population does end game Ops".

[Jadescythe]

That quote of his has absolutely nothing to do with what you seem to think it does...it didn't say "Less than 10% of the population does end game Ops".

 

It was in reference to how players are using 12x XP. And it appears they are overwhelmingly using it to see story and take time playing the game rather than rushing to end game for operations or ranked PvP. Some of my post is merely my own experiences being relayed. Personally, I use it to power level alts, but I'm clearly in the minority.

[TUXs]

It was in reference to how players are using 12x XP. And it appears they are overwhelmingly using it to see story and take time playing the game rather than rushing to end game for operations or ranked PvP. Some of my post is merely my own experiences being relayed. Personally, I use it to power level alts, but I'm clearly in the minority.

OK lol, I agree with you! I'm like you too...but I don't believe we're the minority Bioware claims we are...less than 10% is low...I don't know how they can judge peoples motivations so well.

[Bobs_YourUncle]

OK lol, I agree with you! I'm like you too...but I don't believe we're the minority Bioware claims we are...less than 10% is low...I don't know how they can judge peoples motivations so well.

 

Depends on what they figure falls into the 10%. Would a player that has only one or two toons at 60, and 5 to 10 more that are sitting around between 1-50 be considered one that isn't rushing to get to the end using 12x? Seems they would have to include these in the "not so rushed to get to the end" using 12xp to be considered a % of their player base. I think the numbers they are basing this on is because most don't bother leveling alts through the generic stories of the expansions. The alternative is to only consider new players leveling their first toon with 12xp and 90% of them are taking their time and going no further than the regular story, which I would guess is a significantly lower number compared to the overall playerbase.

[hadoken]

OK lol, I agree with you! I'm like you too...but I don't believe we're the minority Bioware claims we are...less than 10% is low...I don't know how they can judge peoples motivations so well.

 

What I imagine they're looking at is how engaged they are at end game vs how many alts they're leveling. Or perhaps it's just simply time played at various levels vs time played at max level. The person rushing to cap will have less per level until the final level. The person just wanting to do stories will have the opposite.

 

At least that's how I'm interpreting it. It's kind of hard without actually seeing the data they're using.

[DarthDymond]

It's metrics like this that make you re-evaluate your perspective on the game. It helps you understand why there isn't so much more time put into operations and ranked PvP because the overall player base isn't focused on end game. The end game population is just the most vocal. I can't count the number of times I've seen someone join a guild and say they've never done an operation since launch and we're 3 years into the game.

This raises an interesting (to me at least) issue regarding the different player types.

 

It's no secret that some players treat this game as, for all intents and purposes, a single player game - that's true of any MMO and is probably even more prevalent here since we have a contingent of KOTOR fans who are just here for a KOTOR III fix.

 

Obviously those players have different priorities and preferences when it comes to gameplay, but something I hadn't really thought much about is how they're probably also much less likely to be engaged in player discussions about the game. I can't speak for anyone else, but I know I have pretty much never been involved in forums or online boards regarding any of the single player games I play - leaving a straightforward customer review is the most online engagement I think I've ever given one.

 

It stands to reason that the players who treat this game as a single player game are relatively less likely to be involved in the online discussions, as compared with the (for lack of a better term) more traditional MMO crowd. MMO players treat online engagement very differently - I've seen plenty of people on this board say that they would consider quitting over how much BW communicates here (and while I know most are empty threats, they still point to the players caring about the issue). With single player games, prioritizing a company's forum posting habits would usually be outright bizarre.

 

I've always said that it's important to remember that these boards represent a tiny minority of actual players and payers. But it is also a sample set that probably skews towards the type of player that has specific preferences - the 'traditional' MMO preferences. I am curious as to how the overall numbers actually lay out - as opposed to how they are represented in the forum community.

 

(That being said, even if fans of Ops, WZs, etc. are a minority, there is still the important dynamic of how 'sticky' a type of gameplay is - how effective it is at getting players to keep coming back or to stay around altogether. Even if 99% of the game's population prefer one-time content, if all this game provides is one-and-done experiences then it wouldn't be able to survive. Sure there are some people are still playing repeated runs of KOTOR, even 12 years later, but not enough of them to support a minimum MMO population.)

[hadoken]

Obviously those players have different priorities and preferences when it comes to gameplay, but something I hadn't really thought much about is how they're probably also much less likely to be engaged in player discussions about the game. I can't speak for anyone else, but I know I have pretty much never been involved in forums or online boards regarding any of the single player games I play - leaving a straightforward customer review is the most online engagement I think I've ever given one.

 

It stands to reason that the players who treat this game as a single player game are relatively less likely to be involved in the online discussions, as compared with the (for lack of a better term) more traditional MMO crowd. MMO players treat online engagement very differently - I've seen plenty of people on this board say that they would consider quitting over how much BW communicates here (and while I know most are empty threats, they still point to the players caring about the issue). With single player games, prioritizing a company's forum posting habits would usually be outright bizarre.

 

I wouldn't be so quick to assume they're always different groups of people. I'm an old MMO vet that has become a single player over time. I think forum browsing is a separate thing altogether from whether or not you're into the multiplayer or single player aspects of games. And I think a single player type can still enjoy the mechanics of the game to be interested in when classes are buffed/nerfed or what other enhancements the game is undergoing (thus desire to visit the forums). I know I am.

[Draqsko]

OK lol, I agree with you! I'm like you too...but I don't believe we're the minority Bioware claims we are...less than 10% is low...I don't know how they can judge peoples motivations so well.

 

Did someone slip a mickey in your coffee this morning? You are entirely too agreeable today and it's disturbing...

[DarthDymond]

I wouldn't be so quick to assume they're always different groups of people. I'm an old MMO vet that has become a single player over time. I think forum browsing is a separate thing altogether from whether or not you're into the multiplayer or single player aspects of games. And I think a single player type can still enjoy the mechanics of the game to be interested in when classes are buffed/nerfed or what other enhancements the game is undergoing (thus desire to visit the forums). I know I am.

Oh people absolutely have multiple facets to their gaming, particularly within an MMO where variety is an emphasized aspect of the game.

 

My point was, strictly speaking in terms of trends and tendencies, I think it is likely that the more someone treats this as a Single-Player Game, the less likely they are to engage in the online community; whereas to the extent that they are engaged with the MMO aspects of the game, the more likely they are to post.

 

I mean that it is a sliding scale, not at all a statement of absolutes - after all there have been more posts than I can count here on this board of people saying "I am only here for the Single-Player experience! I wish this game wasn't an MMO!" So again, just in terms of demographic trends, so to speak, I think there is a correlation where one end of the spectrum is generally associated with a game's forums and online community more than the other.

Edited May 29, 2015 by DarthDymond

[Ghisallo]

Some pretty interesting answers in there, and its great to have the new insight. I think its pretty reassuring to know there are guys that know there stuff in charge of this, whether you agree with everything that gets said / done or not.

 

What gets me is many of us have said (with less detail of course) that this is how things are done. Some people have said "BS!!!!!" other people have said "then show us the Data they get." We said to them "the amount of data ids freaking massive, there is simply no way they could show it to the general public and even if they did they wouldn't understand a lick of it. The later then said "BS!!!"

 

I am glad they were this open about it BUT at the same time they really shouldn't have to be because it is simple common sense (if you have even the smallest clue about how MMO devs operate.)

 

That all said, if someone doesn't like what they do now they will just say "they don't know what to do with all that data. they are still idiots."

[Ghisallo]

It was in reference to how players are using 12x XP. And it appears they are overwhelmingly using it to see story and take time playing the game rather than rushing to end game for operations or ranked PvP. Some of my post is merely my own experiences being relayed. Personally, I use it to power level alts, but I'm clearly in the minority.

 

Well here is the problem with that data and expanding it beyond what it says on it's face. I think...

 

What I imagine they're looking at is how engaged they are at end game vs how many alts they're leveling. Or perhaps it's just simply time played at various levels vs time played at max level. The person rushing to cap will have less per level until the final level. The person just wanting to do stories will have the opposite.

 

At least that's how I'm interpreting it. It's kind of hard without actually seeing the data they're using.

 

was pretty close.

 

You could easily make the argument that the data point there is a product of the game design itself. I am on an OPs team with my main. You can clear Ravagers in 2 hours (less once you get it down well) just as one example. What do you do with the rest of your week? You are likely either not playing (after you get your dailies done) or you are using the 12xp with alts SO the majority of my play time, even though I play end game, is not spent at endgame due NOT to my choice but the design of the game itself.

 

Lastly I think you hit a nail on the head, and read too much into a statement. They are experiencing (per the quote) the STORY. 12x exp actually is a disincentive to see the whole world and do the side quests. He also states

 

an opportunity to experience the substantial breadth of story the game provides without forcing them to repeat content that they’ve done on their primary.

 

So it seems like you are actually in the majority, not the minority. Since people don't have to repeat the "tedious" side quests they are more likely to level alts to the end of the story lines, which keeps them engaged in the game with the new story vs getting bored waiting for daily, weekly and instance lock out timers to reset.

 

It gets even more complicated when you ask "what is endgame" as there really is no one size answer there.

Edited May 29, 2015 by Ghisallo

[GavinRuneblade]

What gets me is many of us have said (with less detail of course) that this is how things are done. Some people have said "BS!!!!!" other people have said "then show us the Data they get." We said to them "the amount of data ids freaking massive, there is simply no way they could show it to the general public and even if they did they wouldn't understand a lick of it. The later then said "BS!!!"

 

I am glad they were this open about it BUT at the same time they really shouldn't have to be because it is simple common sense (if you have even the smallest clue about how MMO devs operate.)

 

That all said, if someone doesn't like what they do now they will just say "they don't know what to do with all that data. they are still idiots."

 

I no longer have a link and couldn't find it when I searched recently. But I want to say in 2011 or 2012 a zynga exec was talking about metrics and the design of farmville (at the time it was still the top Facebook game and they tracked EVERYTHING).

 

I specifically remember he was saying their metrics showed the players who logged in daily spent the most money. So they added mechanics to get people to log in daily, but while the new daily logging players spent more on average, they didn't spend as much as the natural daily login players and zynga didn't know why.

 

Meanwhile, any gamer could have told them that the natural players were the ones who enjoyed the game, the ones they coerced into logging in daily were invested or addicted, or just wanted the daily reward, but didn't necessarily enjoy it as much. And we're much more likely to spend on what we enjoy.

 

Ever since that video (and some of the discussions I've had with various game developers in many genres) I am always cautious about the disconnect between their interpretation of metrics and the experience for the player.

 

From this perspective of mine, Alex's comments about how much the bioware team play swtor was very comforting to hear.

[Mowermanx]

I no longer have a link and couldn't find it when I searched recently. But I want to say in 2011 or 2012 a zynga exec was talking about metrics and the design of farmville (at the time it was still the top Facebook game and they tracked EVERYTHING).

 

I specifically remember he was saying their metrics showed the players who logged in daily spent the most money. So they added mechanics to get people to log in daily, but while the new daily logging players spent more on average, they didn't spend as much as the natural daily login players and zynga didn't know why.

 

Meanwhile, any gamer could have told them that the natural players were the ones who enjoyed the game, the ones they coerced into logging in daily were invested or addicted, or just wanted the daily reward, but didn't necessarily enjoy it as much. And we're much more likely to spend on what we enjoy.

 

Ever since that video (and some of the discussions I've had with various game developers in many genres) I am always cautious about the disconnect between their interpretation of metrics and the experience for the player.

 

From this perspective of mine, Alex's comments about how much the bioware team play swtor was very comforting to hear.

 

Indeed, and Alex Tremblay sums it up perfectly with this and its the most important part of the interview.

Which brings me to an argument I had with a user on this forum about cartel packs. he used the "they have data" argument, to which I replied, no amount of data can tell you why someone is buying that pack as you cannot get into the mind of said customer, you cannot guess what item he/she is specifically after out of that pack, and thats if they are buying it for a specific item in the first place.

This is just an example, but it applies to many of the decisions, and as DarthDymond mentions in an earlier post, the player feedback and comments that Alex says is needed to draw conclusions to some of the questions posed to the analytic team, you will often get tainted results, as the sample size is too small and/or a different demographic to the overall player base.

 

Here is that statement:

The hard part, however, isn’t about having or not having the data. Unsurprisingly, the mission of my team is to analyse and interpret the data to understand people’s motivations and desires. And, to combine those data driven insights with player feedback and comments we receive to provide the qualitative data to join to the quantitative analysis. Since we don’t collect information on how you feel or think, it’s up to us to draw these conclusions by putting all the pieces together.

Edited May 30, 2015 by Mowermanx

[Danw]

This raises an interesting (to me at least) issue regarding the different player types.

 

It's no secret that some players treat this game as, for all intents and purposes, a single player game - that's true of any MMO and is probably even more prevalent here since we have a contingent of KOTOR fans who are just here for a KOTOR III fix.

 

 

This games is nothing like KOTOR (other then time frame...sort of) so I'm not sure where that fits in.

 

Anyway,

There metrics are all interpreted wrong and driven solely by CM not actually trying to get more subscribers.

They might have all the metrics in the world, but they still don't get it, for a classic mmo type game (wow theme park style) if you want subscribers you have to:

 

1. have constant new content, something new every month at a minimum, with better world design they could release a new mission every week on existing planets etc.

 

2. dynamic activities, conquest is a good example, but very limited and has nothing to do with actual conquest, imagine more of a planetary control via npc base capture system, something that engages the player that is not running old instanced content

 

3. more rewards for actually playing the game then are released in CM, example: there are more decos from CM then actually playing the game, way more outfits and visual weapons, etc, so many ways to distribute stuff for playing the game, but instead it's all done via the CM, no wonder no one wants to subscribe, why subscribe when you don't get stuff for playing (except the few things they think they have to add to make people happy...the exact opposite of reality)

 

4. non combat related content, i know they have their hidden reasons to why pazaak is not in the game, but that was one of their biggest mistakes, non-combat things to do that engage players is completely missed. a successful example of this is the Nightlife event, and that is no brain clicking pseudo gambling. imagine if they had pazaak (star wars blackjack), sabaak (star wars poker), and dejarik (star wars chess) in game, KOTOR style drag racing with best times daily, weekly, monthly with rewards, and a board for all time best runs for bragging rights.

 

5. end game (there should never be end game content, all content should be end game but that's a whole other story/game design) content should never be obsolete. but if you must do it that way you should be adding end game content and updating old endgame content (all of it), not adding new content and making the old crap useless. players do not stop at 50 or 55 (old caps) to run the old operations and FP's designed for those levels. updating your content for end game is a must, if you are not willing to do that, then your end game will always be low on content and low on subscribers.

[annabethchase]

5. end game (there should never be end game content, all content should be end game but that's a whole other story/game design) content should never be obsolete. but if you must do it that way you should be adding end game content and updating old endgame content (all of it), not adding new content and making the old crap useless. players do not stop at 50 or 55 (old caps) to run the old operations and FP's designed for those levels. updating your content for end game is a must, if you are not willing to do that, then your end game will always be low on content and low on subscribers.

 

This I 100% agree with. Horizontal progression at endgame would fit the client base so much better based on my observations on Shadowlands. The metrics to support that would be number of participants in older ops on GF day, non group finder day and by day of the week. Then advanced analysis would be to chart every ops participation count week by week and analyze trends based on changes in reward structure and aging of the ops.

[t-darko]

It's metrics like this that make you re-evaluate your perspective on the game. It helps you understand why there isn't so much more time put into operations and ranked PvP because the overall player base isn't focused on end game. The end game population is just the most vocal. I can't count the number of times I've seen someone join a guild and say they've never done an operation since launch and we're 3 years into the game.

 

Must admit that my strain of thought ended up in the same reflection; not long ago I recruited a player to my guild that turned out to be a founder(pre-order, beta player) and to my astonishment he had no clue as to the function and existents of augments, grant him, he was new to pvp but never the less I just was baffled.

 

There has been an increase of communication the last week from the devs, and considering recent events, I personally feel relieved, and the way its done is good thinking as well; through Fan sites and pod casts. But even though, I can't shake the feeling of randomness that I have when I come across this information.

 

I love the article, it inrichess, and that is indeed a rare feeling, good show Olibb.

[NeuroniaSW]

Must admit that my strain of thought ended up in the same reflection; not long ago I recruited a player to my guild that turned out to be a founder(pre-order, beta player) and to my astonishment he had no clue as to the function and existents of augments, grant him, he was new to pvp but never the less I just was baffled.

 

There has been an increase of communication the last week from the devs, and considering recent events, I personally feel relieved, and the way its done is good thinking as well; through Fan sites and pod casts. But even though, I can't shake the feeling of randomness that I have when I come across this information.

 

I love the article, it inrichess, and that is indeed a rare feeling, good show Olibb.

 

I'm not shocked at all about your founder friend. I'm not a founder but I had been playing since about two to three months after launch and didn't know about Augments, Dailies (Oricon/Section X/CZ/etc.) and a bunch of other stuff. You get small pop-ups from the game when you hit level or character milestones but documentation is really sparse outside of that. I generally try to look around for a "Guide section" but those aren't abundant either.

 

Thankfully I've got a really achiever-oriented playfriend that helps me max out.

 

P.S. Funny story about Augemnts, I was on Oricon with my Mara and my friend on his Inq and I was faceplanting a lot (yes, more than even a Mara should...) and he asked me if I had Augments and I was like: "What are Augments?" and he was "..." and then the game got a LOT easier.

[Transcendent]

They need to sticky these interviews so everyone gets wind.

 

^^Sums it up.

 

Nice interview, and good answers.