Suggestions for a more effective PTS

[Xavrath]

I have waited to post this comment because I felt it was important to get an accurate view of the situation before offering constructive criticism. I feel that enough time has passed for me to arrive at an accurat4e impression of where SWTOR is going and the problems that present themselves.

 

One thing that I am certain about, is that the SWTOR team has a lot on their plate. There are many things that need to be addressed in the game and limited resources to address them. So here are my suggestions to Bioware in order to improve their efficiency and response to the community:

 

I have seen strong indications that the public test server is not accomplishing the purpose for which it is intended. While Bioware has responded by saying that the feedback on the PTS is light, they need to ask a very important question...

 

Is the feedback light because people have no issues, or is that because the patches are not actually getting thoroughly tested?

 

I submit that the PTS is not affective as a test platform as it is currently implemented. We have seen patches go through multiple iterations in order to repair issues that should have been caught on the PTS initially. We have seen changes make their way to production servers that were poorly received because of the lack of feedback from the test servers.

 

Both of these facts are clear indications that the PTS does not have enough players doing what needs to be done. Please note that identifies two different problems.

 

In order to get more players on PTS, you have to give some incentive for doing so. SWTOR is new and shiny. Players will stay on the production servers unless you give them something for playing PTS. I suggest that you give play time credit as an incentive. It is a relatively low cost option for Bioware that would attract a significant number of people to play PTS.

 

I don't think it is effective to offer play credit just for being on PTS. If you do that then the lamers will just idle on PTS to get the game time credit. You have to give credit for helping resolve issues. So give the credit to people that make problem reports from PTS that lead to effective bug resolutions. I suggest maybe a day or two of credit for each effective bug report.

 

The second suggestion has been made before. You need to have the ability to create a character of any level that is appropriately geared. By creating such a mechanism, you allow players to quickly get to the content that you wish to test. It allows focused testing with a minimum of fuss.

 

The test environment is critical to the success of the game. Poor quality in patches, accompanied by repeated down time, leaves a bad impression. It leads to a very negative outlook if not dealt with properly.

 

The effectiveness of the PTS is absolutely critical to the process of customer service. Without an effective test environment there will be a continued slide toward dissatisfaction in the Customer Service that Bioware is providing. We already have the impression that there is a lot of "shooting from the hip" going on, and that there is not a whole lot of aiming beforehand. Without an effective PTS that impression could eventually become fatal.

[JRandim]

I think they can do a PTS without giving an incentive, alot of other MMOs do and they always have people on the PTS. However the rest of it you hit the nail on the head.

 

Nobody wants to level a character on a test server. And for people that truly care about working with the test server, this leads to a lack of population to actually test new content. So it seems like a new patch goes to the PTS, and nobody is there, so bioware doesn't hear or see any issues with the patch and pushes it to live. Then it finally hits live and it turns into a massive **** storm because of a glaring bug that would've been seen immediately on the PTS had the PTS actually been able to serve its purpose.

 

And as you said, this of course means we now need a second patch to fix what the first one broke. And lets not even get started on patches that are suppose to fix things, and in actuality fix nothing. I think that's everyone's favorite, when we spend 4 hours of down time to get a patch that broke something, or did nothing.

 

Nobody likes down time, and patching several times a week. We tolerate a scheduled weekly patch, and understand the need for it. But when you start patching multiple times a week and deploying semi surprise patches during like peak times on friday. It leaves a bad taste in your mouth, and gives off an impression of a Dev team that is somewhere between unprofessional and unprepared.

[DarthChamberlin]

new update go read

 

 

Hey everyone, thank you to all of you who gave constructive feedback here. We've closed the poll as we feel we have a good indication of your opinions. As I said previously, changes will be made to the global cooldown and we'll bring you more information when we have a solid timeline to share.

 

Possibly more than any part of the game, UI is a very subjective topic, so we understand the varied opinions expressed here. The long-term aim for The Old Republic is to give players flexibility in their UI choices and the changes coming for Game Update 1.2 will allow for more of that.

 

I wanted to also address concerns about the Public Test Server. We're working towards enabling character transfers so you can move your own character across and test new patches. Other ideas to allow the testing of high-level content have been explored, but ultimately character transfer is what we want to allow for everyone.

[Werefox]

Xav,

 

The game has been out for less than 2 months. I think you should give BW a break. There is little interest in the PTR because a lot of players are still leveling their toons. I have one level 50 and it took me a month to get there. Now I'm workng on my second one. No one's bored enough yet to want to check out the next big thing early. Over the next year I expect the PTR to blow up. Until then, give the CURRENT game a try before you mouth off about what's coming down the pipe or the method in which they bring it to us.

[Tenacity]

They need to stop locking out endgame operations from being tested when changes are being made to those operations.

 

This tuesday's patch made two changes to Soa in Eternity Vault. Both of those changes didnt go through properly (the same bugs are still occuring), and now there are two new bugs caused by that patch (phase 3 will randomly cause tanks to be unable to see party members or the boss, and will be unable to move - and dying during phase 3 and not being resurrected will prevent players from obtaining any loot from the boss).

 

I'm getting really tired of how bioware is handling endgame pve here, they continually make seemingly random changes without ever testing the effects those changes have, all because they dont want an insignificant number of people seeing new fight mechanics before they're released. It's bullcrap, and they know it.

 

 

 

We should not be paying a monthly fee to test bioware's game for them, these issues should have been fleshed out in beta.

Edited February 9, 2012 by Tenacity

[corebit]

Bioware trying to lay the blame on us for "not enough feedback" is simply irresponsible. They chose to ignore tons of feedback issued since beta, yet they have the gall to point the finger at us about not getting right the cooldown graphics.

 

Testing stuff on the PTS is us doing a service to them, not the other way around. We don't pay 15 bucks a month to test the game for them.

[DarthDemens]

No incentives required. Just make it to where its not a giant hassle to get to the point to where you can actually test. If its true that they're locking out Ops as well, then that's even more BS. (I haven't bothered to level up enough on the PTS to get into an Op)

 

1) Char Copy / Premades. Enough Said.

 

2) Surplus of all crafting mats (including mission skill rewards) on the GTN for 1 credit a stack. Remember, the point is to test, and nobody wants to grind 2 hour long crit required missions to test things like RE. Gear/Mods would be nice too, but are less vital to a testing atmosphere.

 

3) Since the PTS already has the patch, leave it online during the weekly/emergency maintenance. Have the next weeks release on it at that time so testing can begin.

 

4) More specific patch notes. Its nice to know that you "Implemented safeguards to address a Valor exploit", but we can't actually test that without knowing what the fix is supposed to do. Yes, I realize you may not want to broadcast an exploit over the forums before it can be fixed, but there needs to be some way to test it. Presumably, this is what caused this weeks Ilum break.

[Knewt]

Xav,

 

The game has been out for less than 2 months. I think you should give BW a break. There is little interest in the PTR because a lot of players are still leveling their toons. I have one level 50 and it took me a month to get there. Now I'm workng on my second one. No one's bored enough yet to want to check out the next big thing early. Over the next year I expect the PTR to blow up. Until then, give the CURRENT game a try before you mouth off about what's coming down the pipe or the method in which they bring it to us.

We should not give Bioware a break on testing. The PTS is only a tool. The heavy lifting needs to be done by their QA department. Issues like the ones in Ilum since 1.1 should have been caught by their QA team. The PTS is a great secondary testing mechanism, but it should never be treated as the primary way to test a patch. So far, BW has shown me that they don't really know how to test their game. The issues in Ilum this week and after the 1.1 patch were easily avoidable with a small team of testers going through a basic testing script.

 

Further, BW has shown that they don't understand how to gather appropriate player feedback. Everyone knew how lucrative slicing was in beta. It was reported to Bioware. They still released it and had to nerf it almost immediately. The cooldown issue is another example. They made a radical change to the UI without asking the community to perform some focused testing. A quick post in the PTS forum asking people to focus on the new cooldown animation would have generated some interest in the PTS and probably generated more feedback.

 

In 2012, it is not acceptable to release game breaking bugs to live servers. That might have worked 7-10 years ago when the MMO genre was in its infancy. Now that the genre is maturing, the demands and expectations are greater. The user base is no longer the same 200k people who played Ultima Online, Everquest and World of Warcraft. The MMO market is tens of millions of gamers who have come to expect a certain level of polish in the games they play. Bioware needs to grow their testing methodologies to meet the expectations of their customers.

Edited February 9, 2012 by Knewt

[xsmspiffs]

Bioware trying to lay the blame on us for "not enough feedback" is simply irresponsible. They chose to ignore tons of feedback issued since beta, yet they have the gall to point the finger at us about not getting right the cooldown graphics.

 

Testing stuff on the PTS is us doing a service to them, not the other way around. We don't pay 15 bucks a month to test the game for them.

 

They did not blame the people on the PTS for a lack of feedback. They said that the people who play on the PTS didn't provide feedback on the issue.

 

I play on the PTS and I didn't provide any feedback on the issue because I had nothing to say about it... I like the change (sorry for those of you who do not).

 

What would help is for people to come and test the changes BEFORE they go to live so that they can provide feedback BEFORE the changes go live.

 

The cool down UI issue doesn't require a 50, so that is no a reason for it not being addressed by the PTS players.

[littleming]

They did not blame the people on the PTS for a lack of feedback. They said that the people who play on the PTS didn't provide feedback on the issue.

 

I play on the PTS and I didn't provide any feedback on the issue because I had nothing to say about it... I like the change (sorry for those of you who do not).

 

What would help is for people to come and test the changes BEFORE they go to live so that they can provide feedback BEFORE the changes go live.

 

The cool down UI issue doesn't require a 50, so that is no a reason for it not being addressed by the PTS players.

 

Except that the initial patch notes didn't really say anything about a new GCD and thus no one was expecting it, or knew about it needing to be tested.

[Xavrath]

OK, I have one question about using character copy to test content....

 

If Bioware changes content in a level 35 quest in order to deal with issues concerning game balance (quest needs to be easier/harder), then how are you going to test that with a level 50 character that is copied over? Would a level 50 character give you any feel at all whether the changes accomplished the play balance that is desired?

 

How about when the changes that need to be tested are to a class or faction that you do not play? How do you test the content now?

 

What Bioware needs to realize is that character copy is not a real answer here. An existing character at any given class, faction, level combination is only able to appropriately test a small fraction of the content or changes.

 

I agree that as more players reach end-game, this problem becomes less pronounced. That does not change the fact that using character copy as a band-aid for character creation on the PTS is also implying that Bioware has no interest in properly testing changes that are appropriate to specific classes, faction, or levels.

 

I also seriously doubt that there is enough actual traffic on the test server to test such narrowly focused content appropriately. Every time I look the PTS is light on traffic.

[Xavrath]

Xav,

 

The game has been out for less than 2 months. I think you should give BW a break. There is little interest in the PTR because a lot of players are still leveling their toons. I have one level 50 and it took me a month to get there. Now I'm workng on my second one. No one's bored enough yet to want to check out the next big thing early. Over the next year I expect the PTR to blow up. Until then, give the CURRENT game a try before you mouth off about what's coming down the pipe or the method in which they bring it to us.

 

I thought that my commentary was quite well targeted and appropriate. We often learn much by listening to the opinions of others. I would suggest that there was vastly more information of value in my post than in yours. After all, my position is based on information gleaned from actual events. You can hardly claim that of your position.

 

The admonition that I should not "mouth off" implies that you are somehow taking offense to my post. There really is no other reason to use such an accusatory tone to address my position. If that is the case, I suggest that you go play the game yourself instead of trolling other people that are trying to make constructive criticism. There is simply no reason for you to be offended by my post since it is not directed to you. If such commentary offends you then you are obviously in the wrong place.

[NowakSs]

Xav,

 

The game has been out for less than 2 months. I think you should give BW a break. There is little interest in the PTR because a lot of players are still leveling their toons. I have one level 50 and it took me a month to get there. Now I'm workng on my second one. No one's bored enough yet to want to check out the next big thing early. Over the next year I expect the PTR to blow up. Until then, give the CURRENT game a try before you mouth off about what's coming down the pipe or the method in which they bring it to us.

 

Wow I have one word fan boy Not every one likes lvl char, most like doing End game pvp Or pve

[LifeOfMessiah]

Game Time incentive seems like a good idea, but it wouldn't be fair to only those on the PTS. I don't play PTS, but I have worked with the developers on NUMEROUS bugs that I have reported, and I am on a regular server. Why shouldn't that incentive, or any incentive revolving around effective bug reporting, go towards the regular players as well as the PTS players. Edited February 9, 2012 by LifeOfMessiah

[Aram_Darksun]

I thought that my commentary was quite well targeted and appropriate. We often learn much by listening to the opinions of others. I would suggest that there was vastly more information of value in my post than in yours. After all, my position is based on information gleaned from actual events. You can hardly claim that of your position.

 

The admonition that I should not "mouth off" implies that you are somehow taking offense to my post. There really is no other reason to use such an accusatory tone to address my position. If that is the case, I suggest that you go play the game yourself instead of trolling other people that are trying to make constructive criticism. There is simply no reason for you to be offended by my post since it is not directed to you. If such commentary offends you then you are obviously in the wrong place.

 

It was. Werefox is the only one I see mouthing off here. You raised a valid point though I can't say I agree on the incentive. If you factor in lost income it as anything but inexpensive.

 

However I can not agree more that we need to be able to 'beta buff' our characters. Forcing people to level test characters is very shortsighted and a lot to ask. That is a massive time investment they are asking for just so we can help them. I have no doubt that is extremely limiting testable content.

 

Copied characters would be a step in the right direction but I also agree it would not be enough. That still severely limits what any one tester can accomplish. Combine with a small test population and you will still see vast level ranges inadaquately represented. Pretty much everything short of endgame infact. The population will mature which just further compounds the issue when it is mostly the people who have reached endgame that will be looking at test.

 

Tbh, I naturally assumed there was either a copy or beta buff feature because something like that is so incredibly fundamental. We all know how important testing is but they are simply not giving us the tools we need to help them.

[Ellvaan]

Greetings everyone!

 

While this statement was made in another thread: Poll: What do you think about the new changes made to global cooldown UI in 1.1.2?; Senior Online Community Manager Stephen Reid did comment on the topic of character transfers to the Public Test Server specifically:

 

I wanted to also address concerns about the Public Test Server. We're working towards enabling character transfers so you can move your own character across and test new patches. Other ideas to allow the testing of high-level content have been explored, but ultimately character transfer is what we want to allow for everyone.

 

You can view more important posts in our Developer Tracker, which lists the latest posts from developers, as well as our Community Managers and Coordinators.

 

Thank you!

Edited February 9, 2012 by Ellvaan

[sdlaurin]

you guys are nuts if you think anyone but forum mods read any of this crap.

 

test forums are for QQ and not for feedback. no one reads your feedback.

[Knewt]

you guys are nuts if you think anyone but forum mods read any of this crap.

 

test forums are for QQ and not for feedback. no one reads your feedback.

That could be said of any forum for any game. It's a false perception thought. There is someone collecting feedback and forwarding interesting threads to the dev team. At this point, they know there is a growing portion of their player base that is unhappy with the constant changes and the issue of patches introducing new, game breaking bugs.

 

Please take your negativity back to the general forums. We were having a nice discussion here.

[Manigma]

That could be said of any forum for any game. It's a false perception thought. There is someone collecting feedback and forwarding interesting threads to the dev team. At this point, they know there is a growing portion of their player base that is unhappy with the constant changes and the issue of patches introducing new, game breaking bugs.

 

Please take your negativity back to the general forums. We were having a nice discussion here.

How do you know feedback is forwarded? You just make an assumption which is pure optimism, it doesn't help that they are so quiet about any issues they have read or indeed are looking at (if they even are).

 

They are quick at knee jerk reactions to exploits - i.e. Ilum spawn camping fiasco - but since there were so few 50s in the PTS this probably wasn't tested or looked, a glaring oversight and shows how out of touch they are with how players will react to certain changes.

[Mephane]

OP is way over board. Just being able to copy any of their live characters to the PTS would instantly solve all issues in regards to activity. So many people would be interested in seeing a patch before it goes live, but not on some fresh low level character, but on their main.

 

Look at all other MMOs that have a PTS - there was plenty of activity when people could try a patch out with their mains, without any other incentive even being required.

[Mephane]

you guys are nuts if you think anyone but forum mods read any of this crap.

 

test forums are for QQ and not for feedback. no one reads your feedback.

 

You've just proven yourself wrong by the very post above yours. Go spill your hate somewhere else.

[LaucianNailor]

I think the character copy has been registered loud and clear as an essential (and if all settings/binding are copied over as well, then a player can jump right in and not have to adjust to their 'live' play). This will help to throw up direct comparisons to show the impact of changes.

 

Second key thing for me is to please keep the PTS up when the live servers are taken down for a patch/maintenance. You'll get a big influx (especially from Europe) who can test content during this period.

[Drakkun-Scar]

Personally, I love the idea of being able to bring over my characters from live to pts, but one of the things I loved about testing in SWG and AOC was the leveling npc's for those that want to just dive in, roll several if not all of the classes and just give it a go.

 

'Course galaxies had tons of players/guilds dedicated to just bug hunting, not just for high end stuff either. Everything was available and TL was rarely taken down for any reason apart from content updates obviously. It worked for swg, nothing worked for aoc tested or not...thats another story though, and a lesson at that.

[havok_bloodcraft]

If we could copy existing characters to the pts it would solve alot of problems because we could go right from experiencing a problem on the live servers to trying it on the pts to see if it was fixed or not. it would also allow us to test balancing patches because we could go from fighting on live servers to fighting on pts and see how much we were affected by a patch with realistic transition. in this case being able to experience this exact situation with and without the patch with the same setup exactly.

[DarthDemens]

If Bioware changes content in a level 35 quest in order to deal with issues concerning game balance (quest needs to be easier/harder), then how are you going to test that with a level 50 character that is copied over? Would a level 50 character give you any feel at all whether the changes accomplished the play balance that is desired?

 

How about when the changes that need to be tested are to a class or faction that you do not play? How do you test the content now?

 

In either of those situations, its expected that people who are of the appropriate level/class will test the content.

 

To put your questions another way:

 

Am I going to level from 1-35 on the PTS just to test a level 35 quest change? No. I'm not willing to spend several hours just getting to the point where I can test one thing. More importantly, I'm not willing to do that knowing that the next change might involve a level 30 quest, and I get to re-do all that work again. With a CharCopy, I can bring over characters of varying levels from Live (or other players can, if I don't have the right level/class combo) to test it in a minimal amount of time. As it stands now, if they put out a patch on Thursday that changes a level 35 quest, that gives me 4 days to level to 35 on the PTS and test that content. That's most of my playtime for the weekend, and I'm not willing to give up 4 days worth of playtime to test one quest. So, it doesn't get tested. So bugs go through.

 

CharCopy is the best way to allow a maximum amount of testing in a minimal amount of time.

 

Alternatively, if they change a class quest (Say, they fixed one of the bugs affecting the final boss of the BH Act 1 quest) they could offer pre-made level 32 BH Mercs and Powertechs on the appropriate quest so that people could log in, test, and get their feedback in in a matter of minutes or hours, instead of taking days.