This seems to happen for me when I am working with something that relies on instructions for operation. During testing, people will slow down to read instructions and usually do the operation correctly; in reality no one reads the stuff, and the same error comes up time after time.

Case in point: on-screen notification for one program said they could not ship to addresses outside the United States, but the person could complete registration to be put on a mailing list for when this feature would be available. During testing, users saw this, but in reality we got several angry emails asking why they couldn't order anything.

Posted by: Lydia on April 11, 2003 04:14 PM

---

 

How do you know if you're doing proper user testing? Most of the "gurus" offering testing courses and materials don't know themselves.

Posted by: Ron Zeno on April 12, 2003 01:40 AM

---

 

lydia, you could add a feature to your testing that would really bug the subjects: count-down clock with only thirty seconds to scan the instructions. a nice loud buzzer (like my dryer) at the end of the time, please. electric shock is optional.

Posted by: Anita Rowland on April 12, 2003 04:42 PM

---

 

One thing I find time and time again is that users request interactive features such as discussion forums, blogs, bulletin boards, chat rooms. Usability tests show that people wanted to be able to move from information to ineraction. But these features never seem to get used, and post implementation testing always shows a lot of reluctance. I think this is because:
a) people request them because they are a 'nice idea', without really thinking about a practical application
b) because people will not automatically use features - they need to be compelled to use them, and an online community needs to develop for repeat usage

Posted by: on April 13, 2003 08:09 PM

---

 

users request interactive features

Irrelevent to usability. User requests do not make useful or usable products. Usability 101.

Usability tests show that people wanted to be able to move from information to ineraction.

Whose usability tests are those and are the results actually valid? Sounds like a gross overgeneralization.

Posted by: Ron Zeno on April 14, 2003 07:05 PM

---

 

users request interactive features

Irrelevent to usability. User requests do not make useful or usable products. Usability 101.

but why do they keep requesting the same thing? why are these requests translated into test findings (for example a user getting to a certain point and saying "I want to contribute to this content") what are the obstacles that stop these actions happening is real use

Usability tests show that people wanted to be able to move from information to ineraction.

Whose usability tests are those and are the results actually valid? Sounds like a gross overgeneralization.

Please remember that not all usability tests are carried out in a prescribed way by professionals - many users test, especially for smaller niche sites, happen more informally and can still produce some great qualitative analysis and leads for further investigation.

Posted by: on April 14, 2003 07:41 PM

---