So about that CLOSED beta testing????

[Ghisallo]

with any software/application development, defects have severity level.

 

the entire revan fight alone either proves they were willing to launch with high severity defects, or it wasn't even tested period.

 

add in the fact that the game has unplayable ability lag and i question whether or not bw cares at all.

 

What it proves is that you don't understand is that things can work perfectly in the small closed beta environment and then have a problem when pushed live. It goes beyond possible corruption of code on the push to what processes may be shared with other things going on, stress due to much higher populations using said processes etc. In this case many people knowledgeable of the encounter in closed beta have said there was NO problem there. The lag is also an issue you will often miss until you hit a live population.

[Projawa]

I don't believe ROTHC, which did have an open beta, was this buggy.

 

They said the main reason for a closed beta was to keep the story under wraps.

I get the reasoning behind this, though the story better be amazing if this was the main reason and IMO the SOR story was good but not great.

[EllieAnne]

How about next time you know, maybe test something this big with people willing to help you out FOR FREE so that we aren't left with a bug ridden, barely playable system.

I thought that's what early release was all about

[Arkerus]

I think the biggest problem was that Sales and Marketing (usually abbreviated to the quite appropriate S&M) absolutely committed the company to a December 2nd launch with no wiggle room whatsoever. So even if all these bugs were being experienced on the test servers (or if some changes had come so late that sufficient testing wasn't able to be done), QA and the devs couldn't have said "wait! stop! you can't launch it's not ready!"

 

Remember, every department at the company except for QA has high incentives to launch. Sales wants the money from people buying the new stuff. Publicity wants all the great reviews to come out, high profile people to look at it. Developers in part want their baby out their and frankly may be sick of looking at the same code so want a break. Community people are tired of "when will it be out". Even we customers push towards early launch. Can you imagine what would've happened on the forums if December 2nd came around and they said "It's not ready yet, no launch this week, we'll let you know when it's ready".

 

Release dates on something as complex as MMO expansions need to be fluid. Decisions on when to flip the switch need to be in the hands of QA, not a calendar. Any company that leaves the decision in the hands of a calendar will have problems - maybe far, far, far worse than we got with this launch.

 

All that said - it won't be zero bugs. Some bugs QA will decide can be lived with (yes, that's part of their job, deciding which bugs are critical and highest priority, which are annoyances that can be dealt with). And you won't hit every bug. I have no trouble believing that the Jungle Bounty bug was a surprise to them, that none of the testers got killed by the beast. But the Revan fight, the only way it could've gone live was making the calendar the authority on when to launch.

 

I completely agree that marketing/sales and the company really as a whole, want the product out by date. They "fight" with development to get the product out the door even when development doesn't want it to go out. I am going to play devil's advocate for a second here...

 

They have to. They need to. The reality is that developers and QA will never find a time to release the product. Managers have to make really hard decisions regarding the "right" time to release. If development and QA isn't held to a release schedule, the product will never go out the door. QA will never be satisfied. Sometimes, SOMETIMES it is well worth it to release a buggy product and fix it live (or with patches) than to sit on the product and not release it.

 

It really comes down to cost/benefit and timelines. If the company has committed XYZ date then the product has to be released. Even if the date is flexible, if the game is playable its probably time to release it.

 

The longer games sit in QA the less money they make. The less money they make, the less investment the company puts back into. The less investment, the fewer numbers of coders, designers, artists get hired or stay on to work.

 

Am I advocating releasing a broken game that will actually COST them more money to fix? NO. I am not. Its about finding the right spot between 100% perfect and broken mess.

Edited December 8, 2014 by Arkerus

[Khevar]

I don't believe ROTHC, which did have an open beta, was this buggy.

2.0 testing had both "open" PTS servers, which didn't have the Makeb story, and "closed" PTS servers, which did.

 

I remember not being able to get into the closed server. This became frustrating when I "ran out of slots" for character copy, because I could only delete toons from the open server (and the PTS copy sent to both).

[GadgetDon]

If development and QA isn't held to a release schedule, the product will never go out the door. QA will never be satisfied. Sometimes, SOMETIMES it is well worth it to release a buggy product and fix it live (or with patches) than to sit on the product and not release it.

 

If QA really is never satisfied and would only approve launching with zero issues, you need to replace the QA management.

 

The main job of QA's management is to prioritize issues. Which are bad enough bugs that you can't ship with it - crashers, data loss, unusability. (My personal history is in productivity software so using terms mostly from that). Which are ones that can be dealt with but need fixing asap. And which ones, frankly, are minor enough to be dealt with until there are no more serious issues to look at which means they'll never be fixed.

 

There is no way that the expansion should've launched with the bugged Revan fight. It's the "core functionality" in a way, of the expansion, the thing everything led up to. I'd also have blocked release for the lack of rewards from the op bosses. Jungle Bounty, if it had been found, I wouldn't put in that #1 priority.

[Arkerus]

If QA really is never satisfied and would only approve launching with zero issues, you need to replace the QA management.

 

The main job of QA's management is to prioritize issues. Which are bad enough bugs that you can't ship with it - crashers, data loss, unusability. (My personal history is in productivity software so using terms mostly from that). Which are ones that can be dealt with but need fixing asap. And which ones, frankly, are minor enough to be dealt with until there are no more serious issues to look at which means they'll never be fixed.

 

There is no way that the expansion should've launched with the bugged Revan fight. It's the "core functionality" in a way, of the expansion, the thing everything led up to. I'd also have blocked release for the lack of rewards from the op bosses. Jungle Bounty, if it had been found, I wouldn't put in that #1 priority.

 

You think that. And QA probably knows about it.

 

And here is the shocker, are you ready for it:

They released it anyway. They made a concious decisions to (most likely) release a bugged product since the negative benefit of NOT releasing it was too immense to ignore. In other words, the benefit of releasing the product was too much to say no, even with the bugged Revan fight.

 

You can be all righteous and say "I WOULD NEVER DO THAT" and I can tell you right now, yes you would. When the numbers go up on the screen and the incredibly bad press from cancelling a release is in your face, you release the product unless there is something so broken it will bring the servers down.

 

I am fairly confident QA stood up and said "this is broken." They simply got overuled. I have no issues with Bioware's decision to release it into early access. While they get some bad press for releasing a bugged product, the bad press from cancelling the release would have been much worse.

 

I don't need a lesson on what QA does. I know what they do. They just got overruled. That's really it.

Edited December 8, 2014 by Arkerus

[Ghisallo]

Or since closed beta testers have said there was no problem with the Revan encounter or ability lag (issues that can be explained by lack of stress testing) they released because they said "the problems were have are so minor they aren't worth sweating over." Early release was likely done instead of an open beta for a number of reasons though. No need for more space devoted to the beta server for all the character copies that would have been done...save money...encourages people to buy the game a fair amount of time in advance...make money. If they would have said "purchase before 11/2 for beta access" people would have freaked.

 

Now some may call this shady but I take it as a matter of perspective. I am looking at you SOE. EQ Landmark is basically an EQNext long term beta you pay to play. Many of the features in it are features that they are/were planning on putting into EQNext. So why have a long term (and free) Alpha when you can have people give you money to test it?

Edited December 8, 2014 by Ghisallo

[Anzel]

You think that. And QA probably knows about it.

 

And here is the shocker, are you ready for it:

They released it anyway. They made a concious decisions to (most likely) release a bugged product since the negative benefit of NOT releasing it was too immense to ignore. In other words, the benefit of releasing the product was too much to say no, even with the bugged Revan fight.

 

You can be all righteous and say "I WOULD NEVER DO THAT" and I can tell you right now, yes you would. When the numbers go up on the screen and the incredibly bad press from cancelling a release is in your face, you release the product unless there is something so broken it will bring the servers down.

 

I am fairly confident QA stood up and said "this is broken." They simply got overuled. I have no issues with Bioware's decision to release it into early access. While they get some bad press for releasing a bugged product, the bad press from cancelling the release would have been much worse.

 

I don't need a lesson on what QA does. I know what they do. They just got overruled. That's really it.

 

So I've been in the room when this kind of stuff happens. QA doesn't always have the final say, especially with a product like this. However, they pushed back the Stronghold expansion for similar reasons and we still all bought oodles of Stronghold packs LOL. I dunno. It's been a few days for me and I still think this would have benefited from a public test of at least the new combat system.

 

More is broken than just the storyline and ability delay. GSF, is hosed (I know, I know like three people must be really mad) and a slew of other bugs. Just look at the Dev Tracker and other threads around the forums LOL.

[Psychopyro]

 

I am fairly confident QA stood up and said "this is broken." They simply got overuled. I have no issues with Bioware's decision to release it into early access. While they get some bad press for releasing a bugged product, the bad press from cancelling the release would have been much worse.

 

I don't need a lesson on what QA does. I know what they do. They just got overruled. That's really it.

 

This is honestly probably what happened. We had this problem in SWG a LOT with LucasArts saying you have to release this patch by this day or else... Considering that EA has a similar track record of releasing bugged stuff because of a hard date that they set in stone and refuse to bend on (Sims, SimCity, Civ, etc as recent releases that did)...

[LincolnAlphaZeta]

tl;dr - Take a breath. Don't put all the blame on Dev & QA. I think they were put on a deadline. Enjoy the content.

 

With each release there are expectations. With major releases even moreso. Considering the large scale of the system we're dealing with, it is not surprising that there have been a ton of issues. The problem is that there are far more issues than there were with 2.0 (RotHC). This shouldn't be surprising because the system has gotten larger (more FPs, GSF, Strongholds, more Ops, re-worked skill system, etc). With that all being said I have a hard time blaming the Engineering (Dev & QA) department for ALL of the issues. Given that there have been problems outside of the new contents area (GSF, exceptional lag) there is some blame to be laid at their feet.

 

The main issue around the general lack of Quality (IMHO) is that this appears to be a date-driven release in order to have revenue realized for the end of the quarter/year.

 

The fact that it was released at the start of December for early access, and then a week later for everyone else who bought 3.0 makes this all the clearer. The bottom line is that they had no control over releasing with this number of issues. It would have taken issues that made things completely unplayable in order to delay the launch. Folks might say that all the issues make the game unplayable, but that doesn't pass the sniff test. Folks are still playing. Release management might have taken into account the numbers of pre-orders and the numbers of regular orders. If pre-orders are only 25% of regular orders, then they could likely take the risk of having issues be there for a week if they know they'll be fixed when the other 75% of buyers get a chance to have at it.

 

I've been doing software QA for 20 years from 100 person startups to larger companies (15k+ employees). I've tested everything from printer drivers, to content delivery systems, to desktop and enterprise virtualization. It's the same in every industry where a product release directly generates revenue. For a company to gain revenue for something like this (the expansion), it must be released. The timing of that release determines when a company can count the money taken in as revenue for a time period (Quarter, Year). If 3.0 gets pushed out until 2015 Q1, then the numbers for 2014Q4 and FY2014 take a hit. Stock price drops, people are placed upon the Sacrificial Altar of Management, internal moral drops, etc.

 

I'd like to assume that 3.0 was released as best as was allowable with the time allotted. I have met very few folks QA folks that take a "Yeah, whatevs" mentality when a release goes out. I've been the victim of having to release stuff before I thought it was ready. I've also been part of some very high quality releases.

 

Gamers are among the most vocally critical customers that exist. On forums like this, the vitriol can be slathered on without much in the way of repercussion. Just another day on the interwebs. For the record, I was able to finish the 3.0 solo story on my 3rd attempt with my first character who tried (just so I could see what folks where up in arms about). I'll likely wait until the issues have been addressed util I try with other chars. I think my biggest beef is with the unreasonable lag spikes across the game (not just in 3.0 content), and general server stability (which has been on-going for a while)

 

I enjoyed the story a lot. I think they did a wonderful job in crafting Rishi and Yavin 4 (Rishi in particular). I paid my $20 and don't feel ripped off as I know I'll still play for a while.

 

But, Yeah, Whatevs

[GadgetDon]

You can be all righteous and say "I WOULD NEVER DO THAT" and I can tell you right now, yes you would. When the numbers go up on the screen and the incredibly bad press from cancelling a release is in your face, you release the product unless there is something so broken it will bring the servers down.

 

I am fairly confident QA stood up and said "this is broken." They simply got overuled. I have no issues with Bioware's decision to release it into early access. While they get some bad press for releasing a bugged product, the bad press from cancelling the release would have been much worse.

 

I don't need a lesson on what QA does. I know what they do. They just got overruled. That's really it.

 

Actually, I've been in a company where that happened, QA got overruled, and we got HAMMERED by it. Huge support costs, huge hit to our reputation. (Granted, this was productivity software that people used in businesses, not a game.) And from that point, it was hard and fast. QA can NOT be overruled. QA has an absolute veto.

 

You're right, Bioware would've been hammered for violating the date they guaranteed to everyone. And that's the other mistake. By tying themselves to a date, they removed all their flexibility - not just flexibility for bug problems, flexibility if competitors made last minute choices, flexibility if the flu came through and the people you'd need ready for the release got sick.

[Lotharofxev]

Pagy

 

so they changed something AFTER testing ended?

 

which is it? or are you suggesting the code started evolving and changing on its own?

 

Ok, that was good. SOR has become the SWTOR equivalent of Skynet.

 

Also LOL @ the suggestion the Devs were under a deadline. 20 months to release less content than Quesh? BAHAHAHHAHAHHAHAHAHAHAHHAHAHAHAHAHAHAHAHAHAHAHHAHAHAh

[Darkwords]

You have 2 kinds of beta testers. The average players that do turn in bug reports as they go but their trivial quest-bug reports (to Biofail) mostly go unheeded with Biofail figuring they can fix later. Then you have the Butt-buddy-always-get-in-beta-testing-coz-I'm-in-a-leet-guild players who fly through beta to get to the end game Ops and ONLY care about reporting bugs for those, (well after they try and kill bosses) they may as an afterthought do a bit of beta reporting. Seen it many times as a tester.

 

BTW if Bioware thinks their oh so secret beta testers don't spill the beans in guild, they are sadly mistaken lol. TS, Mumble,Vent are full of info pre-release

[Andryah]

You have 2 kinds of beta testers. The average players that do turn in bug reports as they go but their trivial quest-bug reports (to Biofail) mostly go unheeded with Biofail figuring they can fix later. Then you have the Butt-buddy-always-get-in-beta-testing-coz-I'm-in-a-leet-guild players who fly through beta to get to the end game Ops and ONLY care about reporting bugs for those, (well after they try and kill bosses) they may as an afterthought do a bit of beta reporting. Seen it many times as a tester.

 

BTW if Bioware thinks their oh so secret beta testers don't spill the beans in guild, they are sadly mistaken lol. TS, Mumble,Vent are full of info pre-release

 

I stopped reading at your second "Biofail". The pejorative offers no value to any point you are trying to make....it's just you taking a potato peeler to Bioware's internet back side, I guess to vent some frustration.

 

And regardless of how much you stomp your feet and name call..... no... player beta testers ARE NOT that good of a source for pre-launch testing. It's not without value... but players follow no organized test plan.... they just do whatever they want to do, and report whatever they choose/want to report. There is more chaff then wheat in player beta tester feedback. This by the way is not unique to MMOs.. it's true for virtually every software product, regardless of industry or product or service.

 

Also, your assumption that player bug reports are ignored is... mis-stated. I'm sure all bugs reported are captured and cataloged into their bug database and queued for 1) verification 2) determination of cause 3) fixed 4) tested to insure the fix. AND... not all bugs are treated with the same priority. There is an internal team of people somewhere that plows through the bug list and sorts and prioritizes what gets worked on first. Which is why some annoying but not serious bugs can go on for years.... because even though the player reporting it thought it was game breaking...at the macro level... Bioware put other more serious bugs ahead of it to be addressed. And, sometimes, they simply cannot verify, determine cause, and fix without a lot of effort. That is the nature of bug management in large complex software products (which are NEVER bug free.. EVER).

Edited December 9, 2014 by Andryah

[Deewe]

tl;dr - Take a breath. Don't put all the blame on Dev & QA. I think they were put on a deadline. Enjoy the content.

 

Actually, I've been in a company where that happened, QA got overruled, and we got HAMMERED by it.

Both great posts. Whole content. Although had to shrink it to continue the discussion.

 

I guess we all agree on the fact most bugs have been found before going live. Still it does not excludes majors ones being discovered after.

 

The issue here is players perception. BioWare in SWTOR has a long track of record of delivering non polished content. And then when they appear they manages them badly. Last example being the skills training and especially the lack of refund for players who pre-ordered.

 

I'd say BioWare is in dire need of publishing more polished content and having a better communication. It will go a long way to keep the players and make even more $$$ out of the game. Honestly it's sad to see how many player find actually normal when things aren't working as intended or poorly designed. Too many people have lowered their expectations not to be disappointed anymore. I feel bad for the devs, really.

[Andryah]

Both great posts. Whole content. Although had to shrink it to continue the discussion.

 

I guess we all agree on the fact most bugs have been found before going live. Still it does not excludes majors ones being discovered after.

 

The issue here is players perception. BioWare in SWTOR has a long track of record of delivering non polished content. And then when they appear they manages them badly. Last example being the skills training and especially the lack of refund for players who pre-ordered.

 

I'd say BioWare is in dire need of publishing more polished content and having a better communication. It will go a long way to keep the players and make even more $$$ out of the game. Honestly it's sad to see how many player find actually normal when things aren't working as intended or poorly designed. Too many people have lowered their expectations not to be disappointed anymore. I feel bad for the devs, really.

 

The funny part here is that the way you talk... they NEVER release polished content. I guess this mindset comes from a desire to only focus on what is wrong and never ever give a thumbs up to what is done well and polished well.

 

I don't think you actually believe the extreme you are presenting... which begs the question why prosecute only defects and not acknowledge things they get right.

 

They certainly do not release perfection. No MMO does really. But they have, and continue, to release a lot of fantastic and well polished content in this MMO.

Edited December 9, 2014 by Andryah

[Baraxis]

The funny part here is that the way you talk... they NEVER release polished content. I guess this mindset comes from a desire to only focus on what is wrong and never ever give a thumbs up to what is done well and polished well.

 

I don't think you actually believe the extreme you are presenting... which begs the question why prosecute only defects and not acknowledge things they get right.

 

They certainly do not release perfection. No MMO does really. But they have, and continue, to release a lot of fantastic and well polished content in this MMO.

 

Hmmm,

The funny part really is this the way YOU talk, as if this is perfectly normal for an MMO. I cant think of a single time your last line has applied to this game, since F2P. "Toss it out there and fix it when we can" has been the motto for too long here.

 

It isn't the new content that has my group scratching our heads over the expansion, it's the earlier game that got ignored when the new character builds were made. They really just speed balled this out there.....

 

Half-assed is the word for it......

[enigmacypher]

Both great posts. Whole content. Although had to shrink it to continue the discussion.

 

I guess we all agree on the fact most bugs have been found before going live. Still it does not excludes majors ones being discovered after.

 

The issue here is players perception. BioWare in SWTOR has a long track of record of delivering non polished content. And then when they appear they manages them badly. Last example being the skills training and especially the lack of refund for players who pre-ordered.

 

I'd say BioWare is in dire need of publishing more polished content and having a better communication. It will go a long way to keep the players and make even more $$$ out of the game. Honestly it's sad to see how many player find actually normal when things aren't working as intended or poorly designed. Too many people have lowered their expectations not to be disappointed anymore. I feel bad for the devs, really.

 

this is funny but it is in the whole of life..

100 years ago they built houses to last multiple generations NOW they start to fall apart the day you move in..

 

we live in quantity over quality every time..

 

some of these issue could be considered minor BUT they exist

 

Also customer server is not what it used to be .. I more than once let the customer be right then went and banged my head on the desk because they were OH SO WRONG..

 

Just the world we live in now and I am glad I am as old as I am since I won't have to watch it degrade too much more

[GrandLordMenace]

I love how people are complaining about a perceived "failing" of the closed beta.

 

Anyone who has knowledge of that could still be under NDA.

 

A few scenarios could've happened:

 

A: Story was broken the entire time during Testing, unlikely, due to the simple fact that many will run the story to see it before everyone else, inevitably.

 

B: It broke in a lateish patch and couldn't be fixed before launch, slightly more likely.

 

or C: A very late change in the patch, maybe even AFTER the testing server went down (because they were on a tight deadline) broke some things in the patch. This wouldn't be unlikely, look at the dev class blogs, some of them are outdated from what we have on live, they could've changed several things between then and now.

[Deewe]

Just the world we live in now and I am glad I am as old as I am since I won't have to watch it degrade too much more

I see where you are going here.

 

I'm a fan of history and it always amaze me hundreds if not thousand years old building are still up and ours last only a couple of years. Then the motto is to make thing that doesn't last so we have work for the future. Instead of using our energy into new or improved things.

[Anzel]

I love how people are complaining about a perceived "failing" of the closed beta.

 

Thank all that is HOLY they listened and made the correct decision to put 3.2 on PTS. Hopefully we won't have the disaster that was Yavin.

[DOHboy]

Thank all that is HOLY they listened and made the correct decision to put 3.2 on PTS. Hopefully we won't have the disaster that was Yavin.

 

what disaster?

[Jerba]

Thank all that is HOLY they listened and made the correct decision to put 3.2 on PTS. Hopefully we won't have the disaster that was Yavin.

No need to necro this thread. And I cannot believe you still blame the closed testing for the 3.0 bugs. Bugs do happen, no matter how much you test it, and just because a bug gets reported is not a guarantee it will get fixed. We already have multiple pages of bug reports on the PTS forums and I'm sure there'll still be bugs when 3.2 launches.

[undiess]

Thank all that is HOLY they listened and made the correct decision to put 3.2 on PTS. Hopefully we won't have the disaster that was Yavin.

 

3.2 is not an expansion. They will never put an expansion on the PTS that they will charge real money for.