The words success and failure are tossed around often when talking about MMOs on blogs, especially here. And usually, someone will ask for a definition of success/failure, so here goes. Note that this ONLY applies to MMOs, not games in general.
To me there are three general categories of success for an MMO, which I’ll call ‘suits’, ‘devs’, and ‘players’.
Suit success is simple; did the investors or company behind the MMO make money? Was a profit turned? And was that profit a good return-on-investment? The tricky part of suit success is we generally can’t say if something was a success or failure unless it’s an extreme. WoW is a success, The Sims Online was a failure. But almost everything else is some shade of gray. For instance, SW:TOR likely hasn’t made back its original cost + ongoing expenses, and EA generally trying to distance themselves from the title on earnings calls is telling, but we can’t definitively prove SW:TOR is a failure, only make an educated guess based on what we know. Another odd example is Warhammer Online. The game is shut down, but (at least according to Mark Jacobs, who at this point I don’t think has anything to gain by lying) WAR was profitable overall. To a suit, WAR was a success.
Dev success is defined by whether or not the devs still have a job working on said MMO, and the rate of content being generated. This is a bit of a sliding scale metric. A game like LotRO has lost most of its devs, but it still has a skeleton crew updating the cash shop, so while not a ‘it’s shut down, everyone is fired’ failure, LotRO is heavily towards that end. WoW or EVE have kept their teams employed for over a decade, with steady and consistent content, so obvious success. This metric is important because unlike other genres, an MMO is only getting started at release, and ideally should be going strong for years, so keeping the core team around, interested, and paid is critical. Layoffs are a clear indicator of failure here, as are cutbacks in content delivery (no more expansions, patches being rolled out slower, etc).
Finally we have player success, which can roughly be identified by “are people playing?” and “are people playing the MMO they expected to play?”. The first one is easy, if your MMO is gaining players, that is success. If it’s losing players, that is failure. If an MMO has achieved a stable, supportable level of players, that is also success. Growth is always nice, but if you set out to build a niche MMO, and you hit and retain your niche such that the dev team is paid and providing updates, that is indeed success.
The second part of player success is more interesting IMO. An MMO changing drastically (UO Trammel, SWG NGE, sub->F2P switch) is almost never good for the players who bought into the original version, so for all of those players said MMO is a failure, even if a second group comes in and enjoys the newer offering. This can also apply to pre-release hype (GW2 manifesto) vs post-release reality (GW2 itself); while what was ultimately delivered may work for some, failing to meet the expectations you set is to some degree failure.
A fourth factor, or perhaps wildcard, is time. How long is it fair to judge an MMO? For instance, EQ1 was a huge success all around in the first few years of its existence, while isn’t by some metrics (player retention, original ‘vision’) anymore. Is it reasonable to say EQ1 is a failure? That sounds a bit crazy, but why can’t all MMOs be judged related to WoW and EVE, two titles that remain successful by all measures, and are as relevant today (if not far more so) than they were at release? If you loved EQ1 back in the day, would you not still love it today if it had been properly updated and kept relevant? Isn’t that a core feature of the genre; constant updates? “Getting old” shouldn’t be something that happens to successful MMOs, should it? And if indeed ‘getting old’ is acceptable, then after how long? A year, 5, 10? If a game is awesome for everyone who plays it for three months, and then everyone leaves, is that three months of awesomeness enough to call that MMO a success by player standards?
Ultimately what I hopefully have gotten across with this post is that when the words success or failure are used around an MMO, it’s usually more of a personal opinion or partial view than a definitive and unquestionable fact. Very few MMOs are all-around successful, while very few are also outright failures.
But it’s also horribly boring to always write in shades of gray, or have to pre-empt everything with “I don’t like this, but others do, so that’s cool too”. So with all of that said, SW:TOR blows, LotRO is a failure, WoW ‘accessibility’ was a horrible mistake (ok, that is a fact) and EVE is the greatest MMO eva!
Hardcore Casual 
You almost went as far as I wish you did. All that is missing is to structure a metric scale based on the definitions you laid out.
For instance: success = 2*suit success + 3*dev success + 3*player success + 2*time success.
Maybe such index should account for uncertainty aswell (80% for suit success, 40% for dev’s, etc). Thus we could determine an upper and lower bound rating of success for each MMO.
A scale for who though?
If its for a suit, does the suit care about anything BUT profit ratio? If its a scale for the player, isn’t it something like 90% player, maybe 10% dev? For a blogger? Hell failures are more blog ‘content’ than successful games, so what does that scale look like?
as the dedicated/self-appointed defender of lotro on your blog, I will have to point out that they are pushing out new content on a regular basis
huge zones, tons of quests
not gonna argue it is a success by anyh stretch,no one could, but the ‘skeleton-crew updating the cash-shop’ is simply a factual error, for once:)
they also came up with some great ideas, as far as I know AOE-looting was an industry first, mounted-combat also something rarely if ever seen in other mmo-s
I know it’s too little too late and it breaks my heart
i also know you not gonna change your mind on lotro and you shouldn’t either
could have been such an amazing game
The skeleton-crew line is like my “sell you the one ring” line, an exaggeration to make a point (LotRO doesn’t actually sell you the One Ring yet does it?). So yea, new content does hit LotRO, but that announcement of no new expansion, along with just the slower pace overall, still stands.
Mounted combat btw was done earlier, and much much better, in DF1, to just name one game quickly. (And doesn’t mounted combat in LotRO suck? That’s what I’ve read about it)
>LotRO doesn’t actually sell you the One Ring yet does it?
Nope, but I’m sure they’ll be available when they roll out flying Nazgûl mounts. ;)
Mounted combat is OK but the physics feel too amplified. At full speed on a mount you wind up circling the enemy a lot. It just makes the battles last longer.
Lotro changed strategy, unstead of an expansion decided for big updates. If they had put togethervthe two new gondor regions, plus the beorning and the BB in pelagir and called it expansion, it would be an expansion… …ok, lotro is not in a good place, playwr base is shrinking, but you do not need to bash all f2p with factual errors to get your point. Oh, and btw WoW also has a cash shop.
Two thoughts. One, I distinctly recall hearing that SWTOR had paid for itself and has been profitable for a couple of years. I don’t recall source and web search is tough on a phone.
But…. Two, suits too frequently define success not as “does it make money?” but as “does it make ALL the money?” As we’ve seen repeatedly, shareholders still delude themselves into expecting Money Hats from any MMG (or the latest Tomb Raider…), and anything less than millions in quarterly profits is perceived as a “failure” even when, by objective, rational measure, it’s not.
Would have to see a link to the first point, because I don’t recall ever seeing anything like that.
Point two; that’s why I put in the line about return on investment. Making ten bucks of profit when you had to spend 500m to get it, while still profit, is a pretty horrible way to invest 500m.
In defense of suits, there’s a reason that success is set higher than “make more money that it cost”. That’s because a successful game doesn’t just need to pay for itself, it has to pay for all the games that _don’t_ pay for themselves. If some other suit at the company has made some poor decisions, your game may need to be very successful to keep everyone employed. Think THQ.
WoW has been a big success in these terms because bliz could take their huge investment in Titan and throw it in the garbage when it wasn’t good enough. Eve is just barely a success because WoD nearly killed them.
Money hats are great and all, but Suit Success = the suit still has a job. That depends on the health of the company as a whole.
Totally on board with the suits/devs/players categorization, but you’ve lost me with the ‘age’ thing.
“A fourth factor, or perhaps wildcard, is time. How long is it fair to judge an MMO? For instance, EQ1 was a huge success all around in the first few years of its existence, while isn’t by some metrics (player retention, original ‘vision’) anymore. Is it reasonable to say EQ1 is a failure?”
No, by that standard Sandy Koufax’s career was a failure because it was too short, and I’m certainly not buying that. Mathematical models for success are nice and all, but here you’re leaving out things like “inspiring everyone”, “founding a genre” (even if all you do to found a genre is put graphics to a MUD and release it to the mainstream). By including these markers of societal and ideal success, we encapsulate the influence and meaning that leads to some flashes in the pan being remembered more clearly fifty years down the road than others who won the long race. While these different types of success are certainly correlated and linked, that doesn’t mean that we accomplish anything worth doing by putting them together in a formula with some made up coefficients. Success is too complicated a concept to be reduced to one number, unless one specifically is trying to talk about one singular aspect of success–which is why the rest of your post which breaks down success from multiple viewpoints works so well.
“Getting old” shouldn’t be something that happens to successful MMOs, should it? And if indeed ‘getting old’ is acceptable, then after how long?”
Should people die? Is getting old acceptable for your parents? I mean, clearly in a perfect world, neither people or MMO’s would die or get old, but clearly this isn’t a perfect world–so I don’t really understand where you’re going with this analogy. In my mind WoW and EVE are both suffering from diseases of senility right now, whereas nearer their beginning they were suffused with youth. The fact that they’re still making money and have big audiences is true, and nice; but the Rolling stones still have big audiences now, and this doesn’t prove that the Rolling Stones aren’t old now. To see that the Rolling stones are old, you have to look a bit deeper than just looking at the size of their audiences; but not that much deeper, because looking at the age of the audiences would do the trick nicely for not just the Rolling Stones, but probably for WoW and EVE too.
The music comparison is interesting, although I would extend it further back. While classical music can be seen a stupendous commercial success in historical aggregate, its purveyors are mostly struggling nowadays, and a young musician seeking fortune would be ill-advised to write symphonies. Nothing about its quality has diminished, but tastes, attention spans and entertainment alternatives have. In terms of popularity, it largely relies on the cultural space and prestige carved out in its heyday.
Public expectations change over time, decoupling popularity from objective quality – and incumbent advantage is a big deal.
I doubt that WAR was a success for suits. You cannot claim that any profit makes a success. If an investment of US$ 300,000,000 in an MMO makes US$ 300,000,001 in return the suits won’t commit suicide but they also won’t celebrate. To them a success is defined by the expected return.
Eve is not really a great sucess story: story..http://www.forbes.com/sites/erikkain/2014/04/17/eve-online-maker-posts-21m-loss-for-2013/
It is what it is, a niche eith some decent revenues.
For the thousandth time, EVE isn’t the only thing going on at CCP, especially in 2013.