If you want to know when new articles go
online,
subscribe to the WebWord.com
Usability Newsletter!
The
Perceived Information Architecture Test (PIA)
Article by John
S. Rhodes
Summary
The purpose of this article is
to describe WebWord's Perceived Information Architecture test. We explain
how the test is conducted, when it should be used, and what value it
provides.
Note: Our
articles are supported by paying clients. If you need help with your
usability projects please contact WebWord.
Background
Representations and Perceived Information Architecture
(PIA) generated a lot of email. Several people have asked for
details about the PIA test. They want to know how to administer the test and
when they should use it. A few people have been wise enough to ask
about the actual value of the PIA test. They want to know why they should
use the PIA test over something like card sorting. These kinds of questions
are excellent.
This kind of buzz is important because it means that people are willing to think beyond their current
toolboxes. It indicates that people are hungry for innovation and they want
to find better ways to understand how people think and act. Bravo!
How is the PIA Test Conducted?
Here is how it works. At the end of all other testing, and after a user
has been widely exposed to the web site or prototype, we provide the user
with the opportunity to illustrate the structure of the web site. We give
them a large sheet of easel paper and a lot of different colored pens and
markers. They do not have access to the web site or software at this time.
Getting started is entirely dependent on the users.
Some people love the ambiguity, some do not. Some users dive right in without any problem and other
users have no clue what they should or could
do. As I said above, we do not sit users down in front of the site. We thought about
doing that but it didn't make sense because the users would expect that they
should try to literally reproduce the structure, versus their interpretation
of the structure. More importantly, we thought they would just try to reproduce
pages versus the structure. That isn't the data we wanted. We
wanted to know more about the information architecture of the site (deep
structure), versus the visual design (surface structure).
We try to keep our instructions very open ended. We provide very little
structure. At the same time, we work very hard to encourage users to draw or write
out how they think the site is structured. We tell them that they can use
any drawing method that they want to describe the structure of the site. If
they don't understand, we use different words than structure, such as
organization or arrangement. We also try to get them to understand that we
are interested in how the pages relate to each other and how people get from
one page to the next.
Users are free to express their knowledge any way they want. They can use
boxes, words, labels, words, colors, or anything else they want to visually
display their knowledge. We strongly encourage users to speak out loud as
they are drawing. In fact, if they don't talk we prompt them with open
and generic questions.
If videotape is used, the test requires one tester and one user. We
have not seen the value of an extra observer in the room to record
information if videotape is rolling. However, if the tester is not using
videotape, capturing the rich data during the drawing process can be
daunting. Therefore, if videotape is not used, you will need a test
facilitator along with another usability professional to take notes.
When Should You Use the PIA Test?
The PIA test was designed to be used with a high fidelity prototype or
finished web site before the next redesign. If you are building a site from
scratch, this test will probably have limited value unless you have a highly
evolved prototype. We have only
used the PIA test using finished web sites, versus prototypes or mockups. At
this point in time, we are not sure if it would make sense to use this technique with a
low fidelity prototype.
The PIA Test is
useful when used in relation to card sorting tasks and other information
architecture tasks. We like to compare the clusters of concepts derived from
card sorting with the clusters of concepts in the PIA test. You can get an
idea of how users would structure information before they experienced the
site and then after. You can start to understand how their mental models for
the same information changed over time.
You should use the PIA test
when you want to literally show clients how users think. Clients don't care
much about users doing card sorting. Showing clients a card sorting session
is pretty dull. It isn't very sexy. However, showing videotapes
of the PIA test is very compelling. It is one of the only tests that helps
clients understand how users think about the underlying structure of a web
site. From our point of view, it is a testing method that can be used to
drive home the importance of information architecture. It provides sizzle to
the information architecture steak.
One final reason to use WebWord's PIA test is that it serves as a useful
memory task. In particular, it helps us find out what users remember about
the web site we are testing. This is a recall test unlike any other
you might give to user. We like it because it also taps into their visual
memory and semantic memory. Stated another way, we feel that it taps into concepts and
relations, not just words.
Value of WebWord's Perceived Information Architecture Test
Here is one key to the PIA test:
What users say about their drawings is at
least as important as the drawing itself. Remember, this is another channel
of expression for users. Some people are extremely visual and can express an
incredible amount of information that they would not otherwise be able to
express. It is a good idea to videotape the test because users like to point
to their illustrations. They make many interesting comments and they show
what they are thinking by
pointing out how information is related. They also point to things they want to
emphasize.
Surprisingly, the lack of structure of the PIA test is one of its main
strengths. At first, we were worried that users would not understand the task and that
their responses would be very generic. However, what we
found is that users generally cluster into one of three groups.
-
The first group tends to be very logical and visual. These users love the ambiguity and have
no problem jumping into
the task. This group of people tends to actually draw out
sitemaps (boxes, lines, arrows, etc.). This group is predominantly male.
- The second group is more verbal.
Not surprisingly they have a harder time literally drawing out an architecture. This second group of
users tends to write out lists or outlines. The more organized users in
this group tend to use headers and cluster the information, the
less organized folks in this group simply list out the core
concepts of the site (literally a list of words).
- The third group
tends to be very page oriented. That is, they like to draw
boxes, circles, squiggles, labels, and so forth. However, unlike
the first group, they tend to focus pages. They like to focus on the home page. They also tend to focus on the navigation
on the page along with key functionality (shopping cart, account
information, registration, etc.).
The PIA can be more effective
than card sorting for a few reasons.
Card sorting works fine and we have used it when it makes sense.
However, with our Perceived Information Architecture test, we not only
get an idea about what concepts are important and how they are clustered, but we also get an idea about how users think
the site we are studying is actually structured. In a way, users are trying to
build (or rebuild) the site
for us.
We also like that it allows users to express themselves
using a different modality. In plain terms we are saying that some
people can draw great pictures.
Some people are really good at remembering concepts. Some people are great at creating outlines. The PIA allows users to express themselves
in any or all of these ways. The task is just structured
enough to allow users to unleash a restrained creativity.
If you wanted to be strict
about what we have said, you might say that the PIA test is a form of card
sorting where the users have to create their own tools. This is true to some
degree. However, it really seems to be more than that because users are
forced to think visually. They also have to rely on their memories. So, we
are reluctant to say that this is simply another form of card sorting. It
provides a different kind of value.
The output is usually one sheet of easel paper.
We
generally don't get a high level of detail. Users are not generally
artists. However, the ideas that emerge are very salient and very important to users.
Note that in some cases we do get a high level of detail.
What You Can Do with the Data from the PIA Test?
How can this data be applied? That is, how can the data be used?
- If we see the same concepts coming up again and again, we
know we have found something important. This mirrors
card sorting.
- We can match the output to user behavior in normal
user testing. That is, we can match the PIA test outputs to
activities of users during testing to better understand how
users interact with navigational tools and cues. This is one
good reason to have videotaped sessions of user testing,
which we normally don't care about.
- In many cases, the PIA gets people to talk about
the structure of a web site and to point out problems that
they not able to articulate during testing. Note that this
kind of articulation and gesticulation would not really be possible with a card sorting task.
- A nice side effect of the PIA is that people jabber on
about what worked for them during testing, and what didn't. That is, they make comments about tasks that they didn't even
perform. For example, some users will draw the
home page and point out flaws and describe why they
would and would not click on certain items, or navigational
elements. While their actual drawing might be of little value,
their explanations and expressions are very valuable.
Conclusion
We are pretty sure that there
are other ways to apply the data from our PIA test. We are also going to
start investigating how we can integrate this testing procedure with other
tests. There might be ways to gather information about user behavior by
having users draw out their activity. We'll keep you posted.
Feedback? General Comments?
If you can think of ways to
extend this methodology, we would love to hear your ideas. If you use the
PIA test in your own research, would you please tell us how it worked for
you? We are very interested in tracking its use.
Send your email: john@webword.com
Thanks.
What next?
|