Thanks for the article John. I can honestly say that I could not belive what I was reading in this Alertbox. I was in shock for a few hours and wondered if it was just me who thought that the figures in this article were insane.

In April 1999 Nielsen said "The cost of poor navigation and lack of design standards is even higher: at least ten million dollars per year in lost employee productivity for a company with 10,000 employees. World-wide the cost of bad intranet usability will grow to about $100 billion by the year 2001 unless better navigation systems are built and much stricter internal design standards enforced." - Alertbox April 1999

So by 2001 the cost of bad usability should have been $100 bilion. But we can now see (arccording to Nielsen) that in one year that cost has risen to $1.3 trillion.

In the same alertbox article Nielsen talks about this: "For example, Bay Networks invested $3 million in intranet usability and improved the design enough to save an estimated $10 million per year for its 7,000 users. It is very common to achieve this 10-to-3 payoff ratio (or better) in intranet usability projects."

Bay Networks were bought by Nortel Networks who then realised a lot more savings that had nothing to do with intranet usability:
"When he started the CIO job, Ricks told the trade publication, "the philosophy was (that) we needed flexibility to adjust business units up or down".

Since then, it's been down.A lot. Nortel's work force has been slashed from 94,500 at the end of 2000 to a target of 35,000 by the end of this year.

When Nielsen comes out with these kind of ridiculous figures, he is guilty of adding to the myth that computers will kick-start the world economy into a new sustained growth spurt.

Unfortunately, these developments are better explained by a very different interpretation: The failure of huge investments in information technology (about five trillion invested, about zero net return) has driven US businesses into a desperate gamble to avert ruin. They are trying to keep profits up by delivering poorer service (voice-menu receptionists, delayed installations) and lower quality (airline food and seat-room) using fewer underpaid and overworked people. Thomas K Landauer - Author of "The Trouble With Computers"

I have been working on a large scale (10,000+ employees) for the past five years and can honestly say that this Nielsen article will do a lot of damage to the usability cause in my organisation. If all the promises about cost savings and benefits for Content Management Systems, Customer Relationship Management, E-Training, Business to Business Commerce etc. had been realised to any degree then our costs per employee should be negative.

The Top Three Reasons Why Companies Have Intranets:

1) Because everyone else has got one.
2) Reduction in printed material (phone directories etc.)
3) Quicker communication with employees

Another study on Intranet Usability appears to concur with my list.
Measuring the ROIs of Intranets: Mission Possible?

With this article Nielsen is destroying his credibility with the Intranet people, who have to live in the real world and balance their promotion of Intranet Usability with the day to day difficulties of staying in business.

I will be attending the NNGroup Intranet Usability tutorial in London on Wednesday, and will see if I can raise some of these issues.

Posted by: Mac on November 11, 2002 04:39 AM

---

 

The figures that Jakob Neilson provides always sound like business plans for e-tailers on a financing round circa 1999

Posted by: James Tuddenham on November 11, 2002 09:09 AM

---

 

You know I am sure there is some truth in what he is saying…actually I know there is some truth in what he is saying, but coming from the other side of the web (the side where we pay you guys to do stuff and hope for an ROI) I just see JN grand standing for headlines.

He or NNg are either selling something or have a conference on… it gets good publicity for whatever it is they are spruiking. The only people that seem to suffer or cringe are those that work in the usability/HF industry or those that have enough gray matter to both walk and speak at the same time.

He can back up all his assertions and we can refute them. At the end of the day, those that have pumped millions into building intra, extra and internets need to hear this kind of stuff to somehow see a light at the end of the ROI sinkhole they bought into years ago.

Posted by: JB on November 11, 2002 11:43 AM

---

 

Bravo to John on a great article! I hope we can look forward to many more like this.

Referring to sections of the article:

Opportunity Costs
Yes, there are costs (and associated risks) that Nielsen chooses to ignore. Had he taken them into account, what would the numbers come out to be? Perhaps a net loss of $1 trillion?

Cost Savings Versus Revenue Generation
This is the bottom line. Savings don't mean anything if there's no income.

Diminishing Marginal Returns
This is a reocurring flaw in Nielsen's research and writing - he erroneously assumes the value of fixing each usability problem is the same. Perhaps because he does not know how to prioritize them?

Misapplication of Improvements
Much more could be written on this subject. This is where all the risks of implementing a solution come into play. Nielsen ignores the risks, or doesn't understand them.

Interaction Effects
Yes, Nielsen ignores the complexity of the processes that intranets affect...

Implementation Costs and Culture Shock
Go John! Are you saying that identifying problems does not solve them and the costs of identifying problems are often minimal compared to the costs of a solution? ;) Again, Nielsen has never demonstrated he understands this.

Posted by: Ron Zeno on November 11, 2002 01:13 PM

---

 

There's an old maxim: Work expands to fill the time available.

This means that if you free up time for your workers by improving usability, that freed time will get frittered away on busy-work, slacking off, gossip, etc. The only way you will realise the actual savings that JN proposes (or more accurately the corrected savings as pointed out above) would be if you either increase the workload or reduce the workforce. Unless the increased workload is coming from increased revenue activities (ie. more sales), you can expect a degree of worker rebellion.

Posted by: Eric Scheid on November 11, 2002 05:25 PM

---

 

Neilsen understands usability very well, but even someone with my undergraduate level of learning in statistics and economics can see that there are huge holes in this statement. John has summed things up very well, JN needs to stick to talking about what he knows about or he'll become a laughing stock. Which would be a shame, given that much of what he says makes sense.

Posted by: Alan Fisher on November 12, 2002 04:16 AM

---

 

IMO, the opportunity costs issue is a red herring. It's worth mentioning as a consideration for any company, but there's no way Nielsen could build that into an analysis, as it's bound to vary widely for companies, based on their industry's growth rates, unique opportunities for the company, cost of capital etc. There is much to complain about in the Alert Box, but asking Nielsen to incorporate opportunity costs into the analysis beyond a lip-service mention is unfair.

Posted by: Frank on November 12, 2002 01:20 PM

---

 

Alan, that was a nicely balanced statement. I sometimes forget that JN does sometimes hit the nail on the head.

Posted by: Lydia on November 12, 2002 03:06 PM

---

 

I asked a question about these figures at the NNGroup Usability Week on Wednesday. What I actually said was something like this:

When I first saw these ROI figures I was hoping they were a joke, but I realise you are taking them seriously. If I tried to justify spending on usability in my organisation using these figures I would be laughed out of the room.

The response from Jakob was something like this:

"Well yes, of course the figures aren't exact but does it matter if the ROI on usability is 700%, 1000% or 1200%"

He then moved onto the next question.

Posted by: Mac on November 15, 2002 07:30 AM

---

 

Wow, am I the only one who read this and thought it was a commercial? JN has given his great insight to us all freely for years but, does this NNg advertisement belong as an alertbox column? I can see using useit as a resource, he has invested plenty of time and money into it, but usually this kind of information is pushed over to the right hand side of the home page. I don't think anyone complained when he used useit to push NNg in that way, well all good things...

Posted by: Ryan on November 15, 2002 09:58 AM

---

 

"Well yes, of course the figures aren't exact but does it matter if the ROI on usability is 700%, 1000% or 1200%"

Or -1000%?

At what point does acting irresponsibly become unethical?

Posted by: Ron Zeno on November 15, 2002 10:22 AM

---

 

Austin Powers...Dr. Evil..."One Trillion Dollars!"

Posted by: Annoying Puff Cat on November 15, 2002 10:50 AM

---

 

Interesting article John. I think this needs to be promoted and discussed further -

"Usability is wonderful, but it can actually get in the way of other business needs. Too often we are told that usability is great, yet even more often, usability specialists, and usability gurus, completely disregard the reality of business. Usability cannot happen in a vacuum."

EMPATHY of other disciplines and seeing what the constraints are will assist usability folks make more headway in projects. Think we need more of this.

Posted by: daniel szuc on November 15, 2002 11:08 PM

---

 

"EMPATHY of other disciplines and seeing what the constraints are will assist usability folks make more headway in projects."

I agree. I think it is funny, almost to the point of laughing, that the usability community does such a poor job with empathy. Isn't a big chunk of usability about empathy? Understanding other people, what they want, what they need?

Posted by: John S. Rhodes on November 16, 2002 09:28 AM

---

 

You bet John!! You bet!! In fact I think thats what a lot of consulting is about ... in all forms. Understanding, listening and then addressing the issues.

Posted by: daniel szuc on November 16, 2002 09:49 AM

---