[UPDATED] I added the results of the quiz below. Check out those swanky graphs!

I’ve been re-reading one of my go-to UX books, Human Performance Engineering by Dr Robert Bailey.

Particularly, I’ve been focusing on the chapter about assessing the validity of test results. I mean, who doesn’t need a refresher on f testing? The chapter starts off with a bit about “Common Sense” design versus “Fact Finding” design. I asked Dr Bailey if I could reproduce the quiz for fun and he said yes.

So, here we go. Put on your UX vest, your Interaction Design belt, and your Usability britches. No need for your Graphic Design undies as you can see from the illustration below.

Consider the best design to be the one which causes the fewest burns and the least damage.

The Results

71 responses. 59% got it wrong.

The correct answer is #1. While this may not be the absolute “best design evar,” it was the best design of the original study. And for the record, I guessed #2, which is what prompted me to do this quiz in the first place.

Common sense is all well and good, but you shouldn’t rely on it solely. For the 40% that got it right, it either means you have good design sense or you are good at multiple choice guessing.

My takeaway: there’s nothing wrong with guessing, but we should all be testing more often.

* Illustration from Johns Hopkins University Press, © 1959, p. 7.

The premise: There’s all types of testing that can be done on a web app or site. Unit, Integration, System, etc. With that, there are many Web Developers who don’t have access to someone who focuses on Usability Testing.

We know Usability Testing uncovers many issues that are usually solvable, so instead of skipping it, why not put the power, so to speak, in the hand of someone who cares about quality and can run a few more tests?

I don’t see this as a way to avoid or ignore having a Usability Expert dive deep into the app or site, but more a way to get a good understanding of what issues the user will face with the app or site said Developer is… um.. developing, by putting some simple tools in their kits.

There are many ways to conduct a Usability Test and get good feedback. The main deciding factors being what you are testing and how solid you need your findings to be. The training I am working on would help Web Developers figure that out then walk them through how to do it. Ideally without much overhead or impact to schedule.

As I go about developing this content, I want to make sure I get the right stuff in, beyond the basics of getting decent data out of a usability session. That’s where you come in.

If you are a Web Developer or if you have strong opinions about this (in a constructive way), let me know in the comments below about what you’d expect to see, learn, walk away with after getting training on how to run a usability session.

Handouts

I need to start off with this mainly because I don’t want it to be overlooked. My main FAIL last night was forgetting to set context around the handouts I made available online. Let us remedy that now…

Both are meant as primers; documents to get you started assessing both what you need to design and did what you design work well. If you download these and decide, “Hey this will be handy,” remember these are not all-inclusive. You will likely find things that don’t apply to you, or things that seem missing. That’s okay.

Task Analysis. This document highlights a number of types of task analysis. I am using these by permission from my good friend Steve Fadden. He used it in a talk he gave at UPA a few years ago and I found it to be a good synopsis of the types of methods you can use to understand what your users need.

Each style of analysis has pros and cons, which are noted, and a decent description of what the method entails. It doesn’t give you a step by step process for carrying out each method, but like I said it’s a primer.

Heuristic Checklist. This is a list of items that should be used with a modicum of salt. They are rules of thumb that have worked for me in the realm of software development. Most of them can be applied to web applications and some can be applied to web sites. Take what applies to what you do and toss the rest. Add on to it if you need.

The Checklist can be used in two basic ways: solo and as a small group. Solo is just you going through an application and seeing to what extent each statement is true. A small group would proceed the same way, but typically would get together after each person has done it on their own to talk about the reasons behind the scores and typically to average or negotiate final scores and priorities.

If you have any questions about these documents or how to use them, please let me know.

The Talk

We estimated between 60 and 70 people came last night, which is great considering the weather wasn’t all that pleasant.

I tried to cover a lot in my hour and I think it went okay. If you attended, you can rate my talk. All constructive feedback is welcome. Seriously! I want to be more usable. :)

While the talk was titled Usability, I covered topics that typically get termed User Research, Business Analysis, Risk Analysis, Change Management, and oblique Simpsons references. While most people think of Usability Testing when they thing of Usability, I believe that most of the skills you learn to assess products or services can be applied to all of those other areas as well. It’s about figuring out What is trying to be done, then How to do it, then How Well that worked.

My motif was doing it right at the beginning will same you a lots of rework at the end. Which when written like that is a totally DUH statement. But I think it bears repeating, and repeating until eventually we all remember to do it on a regular basis.

Self-Assessment

Pro-tip #1: Always start by charming the audience…

It’s funny, but I did something I don’t normally do: as people were coming in and getting settled, I walked around and introduced myself to everyone, thanked them for coming, and gave them my card. Hopefully that made them feel welcome, but to be honest I did it for selfish reasons. I wanted to chill out and stop being so nervous!

I have Stage Fright. Always have. Even in my acting days. The good thing has always been that as soon as I begin my nervousness melts away (mostly). And last night was no exception. I used introducing myself to people as a way to begin before beginning. Plus it was a great way to make a connection with the audience even before that first slide went up.

Overall I think my talk was well structured, and I feel like my presentation style was professional but also “me.” I dropped one F-Bomb. And I’m okay with that. There were some things I flubbed. I had a couple of example stories I told that didn’t seem to come out as well during the talk as they did when I was practicing in my head earlier in the week.

I told the audience to ask questions as I talked. I like that. I get bored as a speaker just seeing eyeballs and only hearing my voice. But I need to remember that when answering a question there’s a difference between having an answer and having an opinion. I was asked a question about inserting Usability into the Agile (or the like) process. I don’t have a good answer for that, but I have an opinion. And I think that came across in my answer.

I liked that I remembered to pause as each new slide displayed. When there’s movement on a big screen the audience is not paying attention to you. I gave them a few seconds to take in the new slide and then started talking about it. Doing this also allowed me to collect my thoughts so it’s a double benefit really.

We broadcast the whole talk on Ustream. I wish I had been able to bring that aspect into the talk. If you watch online, please let me know how that worked so when we do it in the future we can make it better.

I made a mix tape for the talk which I played while people we settling. I liked the effect. It felt like a party with people mingling, eating food, and music playing in the background. I know this won’t work in every setting, but I think it worked well here.

What Did You Think

I really, really hope to get constructive feedback from everyone who attended. As an audience member you have a responsibility and opportunity to make speakers better by teling them what you liked and didn’t like. And I am totally open to that. So… comment here, send me an email, @ me on Twitter, rate me on SpeakerRate.

We were talking recently about the possibility when I mentioned I wasn’t sure exactly what I would talk about, beyond “something related to usability.” I suggested letting Refresh attendees should choose so Patrick set up a survey where you can help drive what the discussion will be about.

I encourage you to fill out the survey. If nothing else to keep me from rambling on like a Senator with a hankerin’ for a good filibuster.

The exact date is forthcoming, but it will be likely the first week of February on Tuesday, Wednesday, or Thursday. The exact day is something you can vote on in the survey, too.

My thanks to Patrick for inviting me to speak. I am looking forward to it and particularly looking forward to figuring out what I will be talking about. :)