If you want to know when new articles go
online,
subscribe to the WebWord.com
Usability Newsletter!
Spanking
Jakob Nielsen
Article by John
S. Rhodes
Abstract
The purpose of this article is to critically review Jakob Nielsen's article,
Intranet Usability: The Trillion-Dollar
Question. In summary, Jakob Nielsen makes some fantastic claims about
intranet usability that must be weighed against other business needs and
constraints. For example, there might be better ways to spend money than
on usability, not all usability improvements are created equal, and it can be hard to apply
the changes dictated by a usability study. The criticisms can be applied
to many other usability articles.
Introduction
"The average mid-sized company could gain $5 million per year in employee productivity by improving its intranet design to the top quartile level of a cross-company intranet usability study. The return on investment? One thousand percent or more."
Jakob Nielsen,
Intranet Usability: The Trillion-Dollar Question
While I think
intranet usability is extremely important, I do not think we are being given the full picture of intranet productivity, savings, benefits, and costs. While I admit to saying that
usability can save a company, it is not everything. There are other things we should consider. At a
minimum in this case, we must consider that Jakob Nielsen is too myopic about the benefits of usability to the point of being just plain wrong.
Jakob Nielsen is selling us a dream.
Before I go on, let me clear the deck. I respect Nielsen's work. I envy his ability to capture the hearts and minds of
the press, and I thoroughly enjoy many of his books. He's one sharp cookie. Arguably, he's done more for usability than any other person I know. But, I don't think we should allow Nielsen to feed us all of his ideas without a touch of skepticism. That is what this article is about: Analyzing Jakob
Nielsen's most recent, outlandish, and unbelievable
claims.
Here are two
more quotes from Neilsen's article that fired me up:
"If we improved all the intranets in the world to the usability level achieved by the best 25% in our study, the world economy would save $311,294,070,513 per year for the sixteen test tasks alone."
"Assuming further usability improvements -- up to the very best level we found for each study task -- could save the world economy $1.3 trillion per year when we include estimated improvements in company-specific tasks."
I don't accept these claims. They don't wash with me. While Nielsen does use some logic in his article, he leaves out
many points, and adds so many assumptions. I feel that I have to speak up. In the rest of this article I will explain why
I think Jakob Nielsen is wrong. I'll also make it clear that usability is not a magic pill to cure all intranet ills.
Opportunity Costs
For some reason, Jakob
Nielsen is completely ignoring the opportunity costs associated with usability. In a nutshell,
an opportunity cost is what you give up in order to get something. Nielsen is assuming that in order to get the benefits of usability we must simply give up (or supply) money. Throw money at the problem and Nielsen thinks usability will improve.
While money can be used to improve usability, money can
be used for other things too. By spending money on usability there is the opportunity cost of not spending money on something else, such as security, stability, and uptime. While we could throw a load of money at the usability problems associated with an intranet, is that
always the best way to improve it? Unlikely, unless you totally
agree with Nielsen's ROI analyses.
To summarize, I
think that Nielsen assumes that the best use of money is to use it on usability.
Always and totally. That assumption is dangerous since there are so many other ways to spend money that might be more important to the business.
For example, we can get all caught up in the fantastic benefits of usability but
if an intranet isn't secure or isn't stable, what good is it?
Cost Savings Versus Revenue Generation
Nielsen makes an appeal to emotion with this quote:
"With all due respect for customer-facing design, the idea that it's a promotion to be transferred from the intranet to the dot-com team hurts company productivity."
My problem with this statement is that it really might be true that the dot-com team is more important than the intranet team. From a business perspective,
I would fully expect companies to have better external web sites than
intranets. If your internal web site isn't as usable as your external site, you are harming employee productivity more than customer productivity.
But tell me, what is wrong with treating your customers well?
An external site
can usually do a better job at helping a company drive sales. An intranet
will help you save money and improve productivity. The external site is good
for one thing, the intranet is good for another. You've got to balance
savings with revenues. For example, when was the last time a company
survived by cutting costs alone? Turning the above quote on its head, while
you might hurt employee productivity, you might improve sales because of
your external web site. It is about balancing cost savings against revenue;
intranet versus the external site.
Most of the employees I know treat customers with a great experience and they will go out of their way to
help them. Some companies will place more of an emphasis on employees than customers, but that is rare. In most cases, customers come before employees. So, where should they spend the money? On the intranet or the external web site?
Diminishing Marginal Returns
Some of the evidence provided by Nielsen leads me to think that all usability improvements are created equal. However, if you think about this idea for a moment, you will realize that some improvements are more important than other improvements.
Before I go on, let me explain the economic concept of diminishing marginal utility. Suppose you are hot and hungry for ice cream. You head to the ice cream shop and get a really big vanilla cone. Your first lick is fabulous. You are delighted. Your next couple of licks are good too, but not quite as good. By the time you are almost done with the cone, however, you are almost sick and you decide to throw
it away. What happened? With each lick of the ice cream cone, the benefits decreased. The entire experience is not the same from start to finish; each lick was worth a little bit less to you.
The same concept can often be applied in other ways. In the case of usability, you will
often get fewer and fewer returns as you apply usability changes. Eliminating your first major usability problems will provide great value, but
the last several fixes will provide only minimal value. Is that they best
way to spend money? Probably not. Hit the major problems and move on. Only
so much money should be applied to usability. In many cases, it makes sense
to go for the big impact changes only.
Misapplication of Improvements
Nielsen assumes that once people are given usability data that they will be able to make the right choices. He also assumes that development and code based on usability data will automatically lead to the grandiose improvements he outlines in his article. However, as any project manager can tell you,
going from requirements to a finished product is painful and never perfect.
Put in the current context, there is no way that a company would be able to perfectly improve the way that Nielsen implies. Benefits will almost certainly be less than expected.
Nielsen's estimates are based on the assumption of perfect implementation
and the correct application of improvements. I don't think that is
appropriate.
Interaction Effects
Unfortunately, Nielsen gives us the impression that usability improvements can be easily applied, and that they will always provide incredible benefits. Sorry folks, but that isn't always true.
There are situations where usability improvements directly conflict with business goals.
Here's a quick example. Let's assume that you want employees to fill out a benefits form online. They have to enter their employee IDs, their names, and their email addresses. Let's also assume that they need to accept special terms and agreements to apply the changes. Now, a usability test might indicate that all of this functionality belongs on one page and that splitting up the actions slows people down. To get some of Nielsen's usability benefits, you might have to consolidate everything. Seems great, right? However, what is missing is that other parts of the company need to impose restrictions on the form. For example, to meet ISO 9001 certification, the human resource department might need users to see the special terms and agreements on a separate page. As you can probably see, we simply can't get the usability benefits because another part of the company needs to impose a rule or restriction on the design of the form.
The point here is that you can't always get the benefits you expect. Usability is wonderful, but it can actually get in the way of other business needs. Too often we are told that usability is great, yet even more often, usability specialists, and usability gurus, completely disregard the reality of business. Usability cannot happen in a vacuum.
Implementation Costs and Culture Shock
Nielsen wants us to think
that ROI on usability is sky high, through the roof, and totally incredible. Take a look at this quote:
"To improve intranet quality, a company with 10,000 users would have to invest about $500,000 in usability. Thus, the return on investment for intranet usability ranges from a factor of 20 (for a company that starts out low and moves to average) to a factor of 10 (for a company that starts out average and moves to high)."
Despite these fantastic claims, I would argue that the cost of most usability changes is completely underestimated. His numbers seem to be based on savings, not implementation costs. In plain terms, the investment in usability is
much more than the cost of usability testing. It is also the cost of learning new interfaces, new documentation, and new code. It is unlikely that $500,000 would cover the costs associated with the
scale of recommendations given a Jakob Nielsen usability study.
I might be wrong
about Nielsen's estimates but I do know that change is painful. Radical changes are radically painful. Many employees would prefer to suffer with what they know than benefit from change. Never forget that improving intranet usability is also about changing company culture. As many people know,
changing corporate culture is like trying to push a bus through a garden hose.
Another implementation issue deals with practicality. Take a look at this quote:
"But don't take the lazy way out and just stick a PDF handbook on the intranet; give users other options for accessing the information as well. Search, navigation, and online reading are all enhanced when you convert content into well-designed intranet pages, each containing a meaningful chunk of information about a specific topic with cross-reference links to related material."
In many cases making this kind of change is, to put it bluntly, stupid. Of course it makes sense to give users other options. But what if Sally in Human Resources simply doesn't know how to make the change? Will she get training? Will the activity get outsourced? Will all
PDFs get converted? I hope you are starting to get the picture I am painting.
Making usability improvements can be very expensive, even if you can
when usability testing is affordable.
Conclusion
As I said above, I respect a lot of Jakob Nielsen's work. However, I'm terribly frustrated with some of his fantastic and unrealistic claims. While I certainly don't provide a nail-in-the-coffin rebuttal to Nielsen's article, I hope that I at least helped some people do some critical thinking.
Usability is wonderful, but it is not a magic bullet. It must be weighed against many other business needs. At a minimum,
before you drink from the fountain of usability, be sure that you understand all of the costs and benefits associated with it.
Think for yourself, and think critically.
Comments?
Please send them to me: john@webword.com
I want to know what you think about this article.
What next?
|