Raise the NPC Limit -please-

[ScarletBlaze]

Hey folks!

 

I know the NPC limit is certainly something many of you would like to see raised so I wanted to explain why it is in place currently. As most of you suspect already, it is a performance issue. Without going into too much detail, if someone's Stronghold begins to load a large amount of NPCs it can create issues not only in your Stronghold, but in others as well.

 

As with all things we will continue to monitor this limit and raise it as we can. We want to give you the opportunity to place as many NPCs possible, but we also want to ensure you have the best experience in your Stronghold as well.

 

Hope that makes sense. Thanks!

 

 

-eric

 

 

 

Eric,

 

I have a question. I have been hearing there is a limit on Cartel furniture you can place. Is this correct? I need to know as I know some of my guild members wanted to purchase some for the guild house but if there is a limit I need to let them know about the limit.

 

 

Thank you.

[Kantaso]

Yes there is a limit, 50 small lights limit for example. And that is for all your strongholds, so if you have 2 strongholds only 25 lights per each, if you have 3 only 16 per each if you have 4 blah blah you know the deal

[Calsetes]

OR - how about if too many players enter at once, NPCs begin to disappear, starting with pets?

 

I'd be fine with this option. My current setup was to have my various companion NPCs set up in a room according to class, and have another room for vehicles and "robot" pets, then to have all the Imperial equivalents on the first floor in a similar manner. Hearing about the 25 NPC limit, it does kind of put a crimp in my plan as I'll have only 4 slots left over for NPCs just after covering a single faction's companions (I don't have HK or Treek unlocked yet.)

 

If it were possible to have just all the companions for both factions set up, I'd do that and disregard the pets entirely. Something as large as the Nar Shadda Sky Palace, it's just begging to have NPCs strewn about it for us social hermits who play MMOs alone!

 

 

.... Maybe section off some of the larger strongholds like the Sky Palace? Make the upstairs one map and the downstairs another map, with their own NPC limits or something?

[Malastare]

This is what I don't understand why. It's fine for animated NPCs, but how are Republic and Empire Guardsman different from any lamp or console or chair? They don't move, they don't make noises, they don't do anything.

 

My first guess would be that they are simply re-using the same object they used from the rest of the world instances. In that case, the difference between them being unliving objects and pseudo-NPCs might be more important. I know they're handled differently because I can see them loading at a different time. I think they might be subject to dynamic lighting, special texture behaviors (atlassing, texture level replacement) and some other minor effects. In the end, I don't see how any of those differences might affect strongholds. They seem to mostly be client-side changes.

 

Can't they just remove the animations from NPCs and treat them as static decorations?

 

Yup, that sounds like a decent way of doing it. The only possible drawback I can see is the detail of the model and texturing. I play at high detail with additional transparency AA that is based on the higher quality textures and texture aliasing. I'd never use the troopers if they were locked to be Medium-level models with medium-level textures surrounded by my fully anti-aliased furniture. Perhaps there is already a solution to that. I don't know.

 

Patience might be a good idea here. Knowing how developers think, they want to increase that limit, too, and they're probably trying whatever they can think of. Development schedules and priorities get in the way.

 

They can keep the NPC limit for vendors, companions and dancers for the time being, but it should really be removed from pets, mounts and guards.

 

Also, why do animal mounts count as NPCs when mechincal mounts count as objects?

 

I think this would be similar to the limitations I guessed at above. The game does seem to treat animal mounts differently in a few other places (most famously, in the preview screens). This hints that there is something fundamentally different about their models. Is it something that can't be solved? I doubt it. Was it ever going to be at the top of the improvement list? I doubt it.

[Jerba]

This is what I don't understand why. It's fine for animated NPCs, but how are Republic and Empire Guardsman different from any lamp or console or chair? They don't move, they don't make noises, they don't do anything. Can't they just remove the animations from NPCs and treat them as static decorations? Just make them unclickable objects like everything else in the Stronghold.

Like Malastare already wrote, there is a huge difference. For static objects like lamps, consoles and chairs, the game just has to store the position and rotation of the objects, and does not need to remember any interactibility.

NPCs on the other hand are much more complicated. When they are spawned, they also start of with a position and rotation. Then the game has to determine which level to spawn (for pets, it is a random level between 1 and 50), and store how many hitpoints it has (always 100%), and whether it can use abilities (in this case, no). All the NPCs are spawned when the stronghold is loaded on the server, and you can imagine that there will be a small lag spike for everyone on the server when the game needs to spawn 300+ NPCs at once every time someone enters a stronghold.

And everytime a player uses an ability next to those NPCs, the game needs to process that. Use an AoE taunt? Give every NPC nearby the taunt debuff, and remove it four seconds later. Use an AoE heal? Heal every NPCs in range.

 

Of course, they could have implemented pets just like chairs and lamps. They would look 100% identical (though you would no longer be able to target them), but would not cause too many performance issues.

The problem is that this is a lot more work. Pets already exist in-game as NPCs, so they could just reuse them for NPC decorations. Recreating all those pets as static decorations would take a long time.

I just hope that based on our feedback they are thinking whether it is worth the effort, but my hopes are not set too high.

Edited August 20, 2014 by Jerba

[mallize]

I posted this in another 'NPC limit' thread, but it probably belongs here.

 

The only thing I can think of that would cause the need for network traffic or DB hits (after the initial load) would be their ability to respond to quests. That could be a push event from the server and could be limited to one active at a time. All of the animations and phrases could be client side.

 

So, if you think about a mount or companion in game. There are all of these attributes associated with them. Their artificial intelligence, their pathing, their phrases, combat stat calculations, affection.. etc. I bet they didn't code them any differently for strongholds. They just let you 'call them'. So SWToR see's you having 20 active companions and has to use the RAM and CPU to deal with all of that.

 

The solution is to make 'dummy' versions of mounts and NPCs.

 

Edit: If it's really because of having a finite number of instances per stronghold type, that's bad design. Read about Amazon's auto scaling. You should be able to detect load and bring up new instances to accommodate the need. Scale up and scale down when not needed. That's the way the computing world is going. I would think MMO games would take sharding to the next level with this concept.

 

And I do write software for a living. Admitedly I know nothing of gaming developemnt though... https://www.linkedin.com/in/matthewlclement

[Grammarye]

I posted this in another 'NPC limit' thread, but it probably belongs here.

...<snipped for brevity>

As another person who writes software for a living (albeit somewhat cynical as a result), I think the reason for this is fairly straightforward. What gets a feature out with boxes ticked in as short a time as possible? Who hasn't worked with product managers and executives who don't want to hear anything other than 'we'll have it ready in a few weeks'? All signs point to 'get it out there; we'll make it faster later'. Not entirely an unreasonable strategy, though I think they've erred a bit far from the fast side of the spectrum.

 

TOR is, alas, saddled with design decisions that were poor before launch. Most of the devs that made those decisions are gone. The new devs are doubtless doing their best with outdated concepts, a bunch of hacks, and probably large areas of the code marked 'for gods' sake don't touch this, Bob wrote it back in 2009; nobody knows what the hell it does but all the NPCs start to rotate if you even move a semi-colon!'. We've all been there; Bob is responsible for much pain.

 

That we get weird effects of sound & image bleedthrough across instances points to a fundamentally buggy design; either the server data isn't well separated or the clients all get the full instance data and filter it, again badly. In fact, it's practically a visible race condition where something hasn't caught up with all the necessary data. None of this bodes well for scaling that problem out further, short of each player getting their own host (and that would be most wasteful, unless they moved to an actor/container framework like Docker).

 

Cloud scaling could help if all they needed was to throw more CPU at the problem (and I note that Firefall is built on AWS for that reason), but I suspect the problem is more a data-handling one. I work with AWS & big data on a daily basis, I should add, but not in gaming. As speculation, most of what the code will be doing is DB queries (hopefully cached, but how far away?), DB updates, client request/response, and if they have any sense, the edge servers handle some/all of the decision-making load for an instance. The NPC limitations don't feel like a pure 'moar hamsters' problem - they feel like 'oh god, the DBA is giving me that look again'; something where just getting the data from A to B is being bottlenecked. Pure speculation, mind you, but I'm arriving at that based on the notion that the limit isn't just for you; it's for all your noisy neighbours on the same host, which more typically is a network problem of getting all the data they want to them in a timely fashion.

 

Adding more hosts & CPU is relatively easy, and it's not like they're streaming textures in real-time like some engines. Fixing a fundamental data scaling problem that gets worse with more hosts added is harder. Changing a fundamental design where you can't split load across hosts; even harder. While it would terrify me no end to discover that one beefy machine is running every single Kaas Stronghold entry because they can't split an actual map for load, only put different maps on different physical machines, it also wouldn't surprise me. Eve Online has the same problem after all.

 

It does feel like something where I'd immediately want to throw a bunch of AutoScaled EC2 instances and Redis/memcache at it just to see if that solved the immediate scaling problem, but I'd bet if it were really that easy, they'd have done it by now. This brings me back to my original thought - all of this sort of re-engineering takes a huge amount of time, effort and money, and it's just not being spent. EA want to see a return on their investment, not more investment.

Edited August 20, 2014 by Grammarye
Typos

[Andryah]

Maybe stop armchair review of code that nobody has any actual visibility to. Seriously, none of us has enough information to validate performance concerns..... so..... we are left to take them at their word.

 

Personally, I too want the limit to be raised.

 

That said... let's explore a few things that are easily observable in game (right or wrong on Biowares part):

 

1) If you open the Map while in your Stronghold instance, you will see clearly that there are multiple player Stronghold instances inside each local planet instance. That means that they are nesting instances (which is normal for this game and is done on every planet.

 

2) They have bucketed (incorrectly, IMO) Companions, pets, animal mounts, and actual NPCs as -----> NPC count. This was probably a simplification choice by the devs. At first this seemed completely absurd to me..... but then I realized that each of these are not static, they have stationary animations in them. And such animations could very well represent a much higher game load then stationary decorations like inanimate chairs, mechanical mounts, etc. That said... not all mechanical mounts are completely without animations. The Walker for example.... sitting in my entry hanger.... wobbles and weaves on it's foot pads. But maybe mechanical mount animations are much simpler (due to limited graphical elements that animate, as opposed to an Orobird with all it's feathers).

 

3) They do appear to be running pretty close to the performance wall with the current limits they have set. Yesterday morning... before the prime time rush to play with strongholds... I noticed no performance issues. But at prime time last night on both serviers I play on (Harbinger and Ebon Hawk)..... there was notable delay inside the Strongold when placing/moving furniture, opening doors, opening cargo holds, and even the GTN Kiosk. This reminds me of the days after RoTHC released when the server would get crushed by population traffic, even though the servers were only showing HEAVY status at the time.

 

TL;DR: there does appear to be some substance to concern about performance when thousands of players are hitting Strongholds all at once (and frankly, last night it looked like more then half the server was playing with strongholds). That said..... NPC, pets, animal mounts, companions should all be separate object classes IMO, but I can see that such a separation would make things a lot more complicated when pushing up against performance concerns (not for a single user, but the collective performance load from a full server of players). I certainly do not want them to divide the 25 limit in fixed code, rather then let players decide if they want more pets or more of something else.

 

The 25 limit is a disappointment, but it's not a tragedy. I have adapted and moved on. If they increase the limits later on, great... we can always redecorate at any time.

Edited August 20, 2014 by Andryah

[Grammarye]

My giant wall of text above probably obscures this in trying to reply in detail, but I agree - the performance limit is there for a good reason, even if we have no idea what it is technically (though I'd still argue most big data systems hit the same class of problems). Hopefully as Strongholds settles down and load balances out, they can tweak it later.

 

It's unfortunate that TOR lacks more flexible scaling, but most of the stuff I was talking about really only became viable at scale in the last year or two; some are still in beta. It's unreasonable to suggest that TOR should already be using them - but we can hope for the future

 

You never know; all this hubbub might prompt someone to look into it.

[WEIRDLITTLEBRO]

...Without going into too much detail, if someone's Stronghold begins to load a large amount of NPCs it can create issues not only in your Stronghold, but in others as well.

 

As with all things we will continue to monitor this limit and raise it as we can.

 

I thought as much. Either way, the response is appreciated. Personally, I think the dev team has made a wiser choice by starting the limit low and potentially raising it later rather than allowing a bunch of people to run into performance issues and being forced to slap a limit down after the fact. Neither option is ideal, but the latter would've been much less welcome.

 

I do hope you are able to raise the limit sooner rather than later though.

Edited August 21, 2014 by WEIRDLITTLEBRO

[Bouma]

The NPC limit was understandable. Finding ways to reduce the load isn't always easy.

 

That said, have the DEV tried to NOT do holograms on Companions that are from other Characters? I personally think that the hologram effect is reducing the ambiance of what could be done with this. It is cool, if one, but to create a nice Companions BBQ gathering, in Holo, isn't really Socialable.

[saremun]

WOW, thats really sucks cause i had some planes for my stronghold and now they all are part of the past, good you told us Devs so we dont spend more money on decorating untill a solution is found!

 

Thanks for the respons and hope you fix it sooner than later...please...

Edited August 23, 2014 by saremun

[GadgetDon]

Limits suck. There should be no limits. I came from City of Heroes where there were almost no limits on the bases, people did AWESOME stuff with it all, it was so cool.

 

And the code base was so fragile that if a dev looked at it wrong, it broke And they couldn't make improvements or fix stuff because it was such an unholy mess.

 

So while it's good to provide feedback on which limits are most binding to find a way through - I'm glad they started with something they can build on, designed it for stability and future expansion. Haven't run into the NPC limit myself but there have been a few design decisions I've had to adapt my stronghold around. Nature of the beast.

[indytims]

Gotta throw my agreement out there for the negatives of housing so far.

 

Severe limits on NPCs/Pets

Hologram NPCs

A few seriously weird anchor point designs

Prestige shennanigans

 

I am cool with the huge prices on some of the items, one of the houses, the expansions, etc. I am totally fine with that.

 

I am also totally cool with making crafting more integral to making new items, even to the point of requiring resources from lower-level missions and such.

 

I'm good with just about eveything, but the NPC limits/holograms just really kill a lot of the enthusiasm I had for the system. And for folks who had hoped to take Prestige 'seriously' - you really can't. Not when a person only needs to spam as many 200 credit chairs throughout their stronghold to max out their % to get the highest score they can. A Voting system (as used in certain other games), though not perfet, would have still been far superior.

 

I like all the options users here have suggested:

 

A slider to lower the number of players allowed in an instance in exchange for raising the number of NPCs.

 

Raising the limit on NPCs, but have them start to drop the more players that enter the residence.

 

'Dummy' or 'Shell' models that do not move/animate/etc somehow purchasable, so that they do not take up any more resources than a chair or mailbox.

 

I personally prefer the 3rd option, as it seems like it would offer no 'burden' on the surprisingly antiquated engine this game runs on. The first two options I would wager still would add such a burden. So just give us 'dummy' models that don't move/animate that are little more than statues - I think most of your players would be satisfied with that.

[Cleet_Xia]

This idea that NPC/ pets/ animal mounts have to be limited because of perf issues due to animations.....

 

If animated decorations cause a server side perf hit ~ Why is it that the holocron decorations are not limited in number?

 

They are quite well animated.

[Zoom_VI]

This idea that NPC/ pets/ animal mounts have to be limited because of perf issues due to animations.....

 

If animated decorations cause a server side perf hit ~ Why is it that the holocron decorations are not limited in number?

 

They are quite well animated.

 

I'm pretty sure the holocrons are particle effect not a animation.

[GrimAce]

I'm pretty sure the holocrons are particle effect not a animation.

 

They do rotate, but it's a single animation, and probably a single animation bone for the model (five for the datacron racks), so it's vastly simplified compared to the NPC models for pets and whatnot. When you load an NPC, you're also loading all the information that's not used - as a member said previously, level data, stats, buffs/debuffs, inventory information etc etc are all being called for every NPC that gets spawned. A lot of that ifnormation still gets called for pets even if it's not used, like inventory.

 

Converting the NPC items to simplified static designs is probably the closest thing to a solution that allows the NPC limit to be absolved, but it'd also involve converting the models, rewriting data, creatign new entires etc. For the hundreds of pets, dozens of animal mounts and whatnot, that's a lot of work - and companions wouldn't be able to have customizable appearances based o ntheir gear either, so it's not exactly a foolproof idea.

Edited August 23, 2014 by GrimAce

[Cleet_Xia]

I'm pretty sure the holocrons are particle effect not a animation.

 

The color flame/smoke/sparks is certainly a particle effect, but the cubes themselves fluidly rotate on multiple axes.

[Transcendent]

Hey folks!

 

I know the NPC limit is certainly something many of you would like to see raised so I wanted to explain why it is in place currently. As most of you suspect already, it is a performance issue. Without going into too much detail, if someone's Stronghold begins to load a large amount of NPCs it can create issues not only in your Stronghold, but in others as well.

 

As with all things we will continue to monitor this limit and raise it as we can. We want to give you the opportunity to place as many NPCs possible, but we also want to ensure you have the best experience in your Stronghold as well.

 

Hope that makes sense. Thanks!

 

-eric

 

It makes perfect sense, and thanks for another post explaining the reasoning behind the design choices.

 

So with the thanks out of the way.

 

How about going back to those "in charge" and explaining that some of your subscribers are not happy with the limitations artificially imposed on this game by the poor choice of engine?

 

Strongholds - limited by engine.

PvP - limited by engine.

Group Finder - limited by engine.

Mission logs - limited by engine.

 

I'm sure sure the list is much longer than this, so when is it going to be recognised that the current engine for this game is inadequate for the task of an mmo? I'm not trying to detract from the hard work you guys do to bring us new content by the way, far from it. It's just becoming more frustrating playing a game that is hampered by the game engine, when the potential is there for a much better experience than being offered with what the developers have to work with.

[saremun]

It makes perfect sense, and thanks for another post explaining the reasoning behind the design choices.

 

So with the thanks out of the way.

 

How about going back to those "in charge" and explaining that some of your subscribers are not happy with the limitations artificially imposed on this game by the poor choice of engine?

 

Strongholds - limited by engine.

PvP - limited by engine.

Group Finder - limited by engine.

Mission logs - limited by engine.

 

I'm sure sure the list is much longer than this, so when is it going to be recognised that the current engine for this game is inadequate for the task of an mmo? I'm not trying to detract from the hard work you guys do to bring us new content by the way, far from it. It's just becoming more frustrating playing a game that is hampered by the game engine, when the potential is there for a much better experience than being offered with what the developers have to work with.

 

I fully agree with you, if the game to expand further then they really need to find a permnent solution for this messed up engine.

[MaximusRex]

This idea that NPC/ pets/ animal mounts have to be limited because of perf issues due to animations.....

 

If animated decorations cause a server side perf hit ~ Why is it that the holocron decorations are not limited in number?

 

They are quite well animated.

 

It has nothing to do with animations, as that is all done client side, it has everything to do with making data calls to the server to load them. It seems there is something different about their data structure that requires a lot more data to be transferred, and that is where the performance issues occur.

[iankalo]

Its so frustrating

 

http://prikachi.com/images/530/7561530A.jpg

http://prikachi.com/images/531/7561531t.jpg

 

There is a room for exactly 138 republic troopers on this yard. I wanted an army but ... I just hit 25 NPCs limit

This hurt more than an ice water bucket.

[KTap]

Can't even have the vendors + enough guards watching the perimeter on Tatooine

[Darth_Wicked]

Can't even have the vendors + enough guards watching the perimeter on Tatooine

 

I have only started decorating the Coruscant and DK Strongholds and I already have fourteen NPC's placed, seven on each Stronghold.

 

They just appear to be so lifeless with no one walking around... It is especially infuriating when you consider

and

. Not saying that was intentional misleading but it comes close.

 

I see several NPC's that we can't use yet, not to mention animations / emotes.

Edited August 25, 2014 by Darth_Wicked

[errant_knight]

If they'd placed the initial limit high and had walking paths with collision detection, adding innumerable calculations to every stronghold, and it had bogged the servers down, people would have been more pissed because it would have affected gameplay. Give them a chance to see how things function when each server has a full complement of npcs in each stronghold. Right now they'd only be guessing. The odds are good that they will add things as they find they can.