There’s a certain ethos in the UX community that goes like this: “You should test users in a focused way on the exact elements you want them to interact with. And through this focused testing you will receive great feedback. Complicated high fidelity prototypes make this difficult.”
This is the imaginary UX foe.
I’ve seen this both explicitly, in the form of blog posts and articles, as well as implicitly from being in the UX community for the past 3 years. Here is an example, via Digital Telepathy
“With so many things to do, it may be hard to focus. Clients and test subjects wander from tree to tree, getting lost in the beautiful forest you’ve created, making it hard to get focused feedback.”
This is in reference to nuanced, complicated prototypes that perfectly mimic how the final site will look and feel. I have news for you, if users are getting lost on a full fidelity version of your website, and can’t complete the tasks you give them, your site has problems. And dumbing down the testing is not the solution.
This ethos is among the most common in UX. The above article is even heavily referenced with senior UX practitioners offering advice to keep designs gray scale, and simple, for testing purposes. It’s so pervasive I’m quite certain that most people who read this post will vehemently disagree, perhaps even motivated to find well established authors who believe the old way of doing things is the best way.
Neverthless, I think this advice, while once a step forward for usability, is now a giant step backwards.
For starters, this ethos flies in the face of what the users themselves are saying. With every new download of Proto.io or InVision, users are voting with their wallets, and the vote is for more fidelity. Check out this new UI Kit that InVision recently released.
Why would a prototyping app release a gorgeous UI Kit? Aside from the fact that it’s great marketing, and will drives tons of users to their software, this is clearly where the future of prototyping is headed. As these new prototyping tools are becoming immensely popular. Put users face to face with what the site or app will actually look like, and test against a full fidelity interactive MVP.
Furthermore, we already have ample evidence that even the smallest of changes can have a huge impact on user behavior. And yet we’re comfortable putting users in front of grey boxes, with all of their attention fully on the task at hand, and think we’re going to receive relevant feedback on how they might engage with the site once it’s completely built, looks nothing like the current version, and with only a fraction of their attention span?
No one uses the internet the way a standard user test is setup.
This is the real UX foe.
Who cares if someone can get through a site navigation flow while they are 100% concentrating on it, and all of the other page elements have been stripped away or aren’t even built yet? You might as well test someone’s ability to do long division with a calculator in front of them.
People are incredibly distracted these days. More than half of the web’s traffic comes from mobile, and people are using mobile devices while they walk, while they drive (ugh), while they are watching TV or listening to music, or even while they are on another computer.
Users need to be testing actual websites, not stripped down wireframes and middling fidelity prototypes, and they need to do it in a way that somewhat resembles real life.
So what’s the solution?
Well I can tell you what the solution is not. The solution is not to find ways to make user testing easier. Spoon feeding users explicit questions with easy to use prototypes that only vaguely resemble the final product in an environment with no distractions is a surefire way to get useless data.
We need to find ways to UX test the same way psychologists have learned to trick research participants. You don’t tell a child that you are testing him to see how long he can go without eating a cookie. You find a pretext to have a child near some cookies, alone (i.e. not concerned with how their actions might be perceived), and then see what he does.
Personally I think UX practitioners need to find ways to irritate the people they are testing. Distract them while they use the site, give them impossible tasks to frustrate them, maybe even have them bounce between 2 different sites for maximum confusion. Users are doing this anyways! Your site needs to be so intuitive that half-angry half-distracted users can find their way around. And until you do, your site will never be as user friendly as it could be.