The benefits to research of producing UI designs early in the software development process.

We were halfway through our first day of a joint application design meeting with the stakeholders. From across the table, a team of users, managers, and customers were firehosing us with requirements, business rules and processes, and task characteristics in response to questions from the developers and me. The details were numerous and sometimes seemingly contradictory. One developer leaned back in his chair and joked to me, “So, you got the UI design for that yet?” I looked down at my pages of scribbles. “I’ve got twelve designs,” I replied, “but I don’t know which is the right one.” That may have been an exaggeration. There was a good chance that none of them were right.

According to legend, in Chicago the advice for political participation is, “Vote early and often.” My advice for UI creation is, “Design early and often.” I see nothing wrong with starting to design on the first day you meet your users. In fact, I recommend it. That flies in the face of how many of us were taught to design. Many have been taught that designing a UI requires following The Process, a waterfall model of creation apparently inspired by the classic analysis – design – develop – test sequence followed by software engineers. To follow The Process (your Process may vary), you:

1. First conduct user research, doing contextual inquiries, card-sorts, user observations, archival research, and so forth. You gather all kinds of detailed information about your users: their tasks, their environment, their skills, their goals, their concerns, their dreams, their purpose in life, their favorite breed of dog. Compile all these data into a big honkin’ binder.
2. Then you analyze and aggregate the user data, constructing information inventories, task analyses, personas, business process flow charts, and so on. Stuff all these into another big honkin’ binder.
3. Now design your UI. Insight is bound to occur after pouring over all that analyzed and aggregated data. Document your design in yet another big honkin’ binder.
4. Develop a prototype of your design, where the first prototype may not be much more that the screen mockups you put in your design binder in Step 3.
5. Test your prototype on users in a usability test, collecting data with screen recorders, video recorders, key loggers, hit logs, and eye-tracking equipment.
6. Compile and analyze your usability test data, identifying problems in your design…. I think the results of all that deserve a big honkin’ binder of its own, don’t you?
7. Repeat Steps 3 through 6 until user performance achieves your usability targets.
8. Just kidding. Repeat Steps 3 through 6 until you run out of time and have to ship the product.

You understand that the “you” in the above is a plural “you.” Often different individuals are responsible for different steps, each throwing the results over the wall to the next person. But it works because you are all following The Process.

I learned The Process myself, but I don’t think I’ve ever actually done The Process. Partly this has been due to the various work environments I’ve been in, where time pressure and resources do not allow for such a protracted design process. But partly it has been due to my own personal design style. I prefer to be personally involved in all the steps of UI design: research, creation, and testing (except I don’t do coding anymore). As a researcher-designer, I can’t help but think about designs when I’m doing research, including on the first day of gathering information from the users. Furthermore, I find designing while researching to be helpful. Maybe it’ll be helpful for you too.

Benefits of Designing Early and Often

Knowing What to Ask the User

Old MacDonald might be old, but he’s always exploited the latest technology to run his farm. This year, he’s looking to upgrade the farm’s Enhanced Integrated Enterprise Information Organizer to get automatic data feeds from the chicken coop. Your job is to come up with the UI to display the new real-time data. MacDonald’s goal is to monitor the egg output of the chickens, collectively in order to forecast revenue and also for each chicken, as an indirect way to check her health and happiness. After a quick tour of the farm, you learn the following about the environment, task, and user.

• MacDonald spends his day in one of his Mobile Operations Offices, each being a tractor pimped out with three 24-inch displays with which he manages the farm activities through various windows of the Enhanced Integrated Enterprise Information Organizer that provide data from around the farm and support the dispatching of family and farmhands to complete chores.
• There are exactly 64 chickens housed in the chicken coop, a long building with nests on each side of a walkway. Egg output is measured by number of eggs passing down a chute from the chicken’s nest over a microswitch that increments a digital counter. Chutes from each chicken merge to deposit eggs in a centralize basket. Once a week, a farmhand records and resets the counters, but a new system will supply these data over the farm wireless network composed of Object Intelligent Network Controllers distributed around the farm.
• To forecast revenue, MacDonald needs a weekly report of total egg yield, which he enters into a spreadsheet for statistical analysis.

There. Start designing the UI.

Not much to go on, huh? Don’t worry about getting it right. Just put something down. One obvious design -the one closest to the current system, is a GUI equivalent of the chicken coop and its digital counters. A table can list all the chickens along with their egg output from the last seven days.

It’s a great design. Probably wrong, but still a great design because it immediately prompts questions for Old MacDonald to answer.

• How do you identify a chicken in the list?
• How do you sort the list?

In this way, early design can prompt user research.

Normally, we think of a design as something to present the user for review or testing. However, our first design, such that it is, is too primitive for that. Even the questions prompted by the design are inappropriate for the users just now. For example, asking MacDonald how he’d like to have his list sorted is too abstract, requiring him to imagine what it would be like to use a hypothetical design that’s little more than a scribble on a napkin. He won’t really know until he tries it.

Instead, you should translate the questions prompted by the design into something relevant to MacDonald’s current experiences. For example:

Your Question	Question Asked
How do you identify a chicken in the list?	If you were discussing the health or happiness of a chicken with a farmhand, how do you specify the chicken? By position in the coop? By name? Do they have names? Oh, with these cage-free chickens, does each chicken sit on the same nest every day?
How do you sort the list by default?	Do you get a sense the chicken might be getting sick, so start keeping an eye on her egg output? (That would imply you should sort the list by chicken identifier, so Old MacDonald can find a particular chicken quickly). Or do you first notice a drop in egg output and that prompts you to check on the chicken? (That would imply you sort by egg output, putting those with low output at the top of the list so they are spotted as soon as the window opens). Which comes first, the chicken or the egg?

Uncovering Requirements

Old MacDonald tells you that generally, a drop in egg output is usually the first sign of an unhappy chicken. Okay, so now you present him with your table of chickens sorted by egg output. He doesn’t dispute the value of making it easy to find the low-output chickens, but he doesn’t like that low output chickens from different physical places in the coop are shown together at the top of the list. He was hoping for something like he gets now from his farmhand who checks the egg counters every week: a map of the coop with the output for each nest written in. You had seen blank copies of these maps, but were told that a map was used to make the farmhand’s job easy -he could simply walk down the coop writing in the counter values from each side.

What’s up with Old MacDonald? Is he just resistant to change?

A few more questions and you come to understand that it’s important to know not only which chickens have low output but the pattern of low-output chickens in the coop.

• Two adjacent chickens with low output indicates the onset of a chicken epidemic that requires prompt quarantining to prevent disease from spreading through the whole coop.
• Emotional state is also related to low output, so low output among a few chickens in pecking range could mean they’re not getting along (or maybe, more alarmingly, plotting against Old MacDonald).
• A rain leak in the coop roof will tend to lower the output of chickens in inverse proportion to the straight-line distance to the leak, reflecting how wet the chickens get (getting rained on doesn’t make them sick, but it does piss them off; why they get as mad as, well, you know).
• An apparent drop in output along just one side of the coop could mean an egg-jam under the coop, where eggs get stuck when merging upstream of the basket, backing up over the chutes’ microswitches so that no more eggs are counted.

It seems Old MacDonald had a stealth requirement. It’s not enough for the system to inform him of the output of each chicken. The system also needs to assist in diagnosing the cause of any output drop. MacDonald didn’t think to make this an explicit requirement because he never imagined a design that would make that difficult. Until he saw yours.

Well, Old MacDonald could have saved you from wasting your time designing something that wouldn’t work if he just made it a requirement to display the egg output counts on a map, if that’s what he needs. But you don’t really want that. That level of detail in the requirements is tantamount to the user designing the system, which is rarely optimal. No, it’s better that you presented your simple first cut of the design and let it fail in order to ferret out the requirement. As far as users are concerned, the UI is the application. Written requirements and specs can sound complete, but users need something concrete to understand the implications of the requirements and specs. An early design provides this, and can thus trap requirements that slipped away before.

Now that you know that he needs prompt awareness of low output and the spatial patterns of such low outputs, you can design to best support both requirements. With your new understanding of the task, it’s apparent that the first step is to alert MacDonald to low output by a chicken; diagnosis is a second step. Thus, you could still have a the list of chickens sorted by egg output, but then you can have a link  for each chicken in the list that opens a map that’s zoomed-in on the chicken and his neighbors. In addition to the egg counts, you can graphically code their output to make any patterns apparent.

But wait. From MacDonald’s description of the diagnostic process, you learn that it’s not only important to know where a chicken’s output drops, but when as well. A chicken epidemic is distinguished from a chicken conspiracy in that for an epidemic, the output drop spreads over time as one chicken infects her neighbors, while for a conspiracy, the output drops for adjacent chickens all at once. With your new application getting daily egg output, rather than the weekly data the farmhand used to provide, your map can include a graphic representation of the history of egg output from each chicken to make such differences easier to spot. This is something Old MacDonald probably wouldn’t have thought of if he designed the map himself because he wouldn’t have thought of the implications of daily egg output feeds.

Focus for Usability Tests

You now need to design a graphic representation of each chicken’s daily output that makes it easy to see patterns in space and time. One way is to provide a mini-line graph with each chicken, like a sparkline. MacDonald assures you that it’s physiologically impossible for a chicken to lay more than one egg per day, so the sparkline can be design to emphasize the distinction between one egg per day and no eggs per day.

Another potentially more intuitive design is to shade-code egg output, with filled egg-shapes representing an egg laid that day, and open egg-shapes representing no egg.

Or maybe shape-coding: egg-shape for an egg, X for none.

Which is better for detecting patterns? That’s an empirical question. So, you create several maps for each type of code, each map encoding a different pattern of egg output corresponding to an underlying cause. You let MacDonald go through the maps and measure the accuracy and speed with which he can diagnose changes in egg output. The code with the best accuracy and speed is the code to use.

We’re taught that usability testing is central to user-centered design, but one of the problems with many usability tests is that they only tell you user performance, not what to design. However, what you did here is create a usability test to answer a specific design question. By making usability tests to specifically evaluate alternative designs or the assumptions underlying alternative designs, the results of the test tell you directly which design alternative to pursue. Open-ended no-agenda user observations and usability testing should be part of every product development. There is a lot of uncertainty in human behavior, so you should actively pursue serendipitous discoveries. However you make the most of your user research resources if most of your testing and observation seeks to answer specific design questions. Designing early and often means testing specific ideas early and often, making for efficient and focused research.

Making usability tests to evaluate specific designs or assumptions also tells you how much fidelity you need in the prototypes. For the question of which graphic code is better, you don’t need an interactive prototype. Static images drawn in Illustrator or PowerPoint will work just fine. However, some graphic design details may be important for seeing patterns, so you want the images to be pretty close to screenshots. Pencil-drawn sketches are not going to be sufficient.

Input for Backend Design

Turning now to the weekly report of total egg yield, that sounds like a pretty straightforward design, but consider how it might look. Each design alternative implies different questions you need to answer, leading you to better solutions. You could have a link for “Total Eggs, Past Week” on the home window which opens a dialog box displaying the number. That’s what Old MacDonald was envisioning, and he’s quite excited about the power that computers can provide for this task: no more tedious hand-summing the numbers from the map provided by the farmhand!

An alternative design would be to display the total in the window that lists of outputs individual chickens

That could save the user some time, but only if Old MacDonald combines checking on chicken health and happiness with revenue forecast. Does he? Should he? That’s a question for Old MacDonald.

Or maybe the best UI design is no UI design. Maybe the app can automatically insert the weekly egg output totals directly into MacDonald’s analysis spreadsheet. That’s a question for the backend designers. Can it be done? How much will it cost? Should MacDonald be given this option?

Often UI design impacts backend design. The introduction of automation, such as the mechanized data feed described here is one example. For tracking the output of individual chickens, you could also provide a feature that allows MacDonald to relate a bunch of low-output chickens together into one epidemic (or conspiracy), so that they can all be tracked together. This would also have back-end implications, requiring, among other things, changes to the database design. You want to know about these impacts early before the backend engineers get too far. UI design helps identify these impacts, so the earlier you start designing the better you’ll get along with the team members doing the backend.

Benchmark for Evaluation

Going for a walk alone on the farm, you start thinking about chicken health and happiness some more. Is the pattern of egg-output all there is to problem diagnosis? Surely once a problem is detected, a farmhand needs to go out to the coop to check it out. Maybe it would be better to have CCTV cameras in the coop, so MacDonald can look at any problem chickens himself. Click on a chicken on the map, and the camera zooms in on that chicken. Of course, that would mean new hardware, interfacing the hardware to the Object Integrated Network Controllers, storing video in the database, requiring bigger hard drives….

Such a blue-sky design is probably out of reach for MacDonald’s operation, at least for now. But such a design exercise done early can be useful for evaluating later more practical designs and pointing the way to doable solutions. For example, installing CCTV in the coop may not be in the budget, but it indicates a gap between egg output figures and the actual health and happiness of the chicken. Maybe the map window can include the cell phone number of the farmhand currently on coop duty, so MacDonald can easily alert him or her of chickens needing a closer look.

Blue-sky designs are useful even when you’re not the one doing the design. If your job as a usability specialist is instead to evaluate a product or competing products, a blue-sky design can help you decide what to look for and what to measure. If MacDonald is considering an off-the-shelf chicken monitoring system, you can compare it to your blue-sky design to see how it handles the same issues. How does it make the farmer aware of chicken problems? How does it help the farmer gather information on individual chickens? The blue-sky design, in effect, becomes a holistic instantiation of the underlying requirements of any system.

Fit with Discount Usability Engineering

But lets return to the farmhouse and get back to reality. You have a more immediate issue to resolve. While it’s true that usually a drop in  egg output is the first thing that alerts MacDonald to a problem, sometimes he does hear about a particular chicken and wants to check on her output. You need to decide how to support this without adding a lot of complexity given it’s a minor task. You could provide a Sort feature to re-order the chicken list by name, so MacDonald can sort and scroll to the chicken in question. Alternatively, you could provide a Find feature in the toolbar, where MacDonald types in a chicken’s name and the chicken list jumps to that chicken. Of course, any table should support both sorting and finding to provide basic flexibility for the user, but this is a small project with a limited budget and you are required to prioritize every feature.

So which is better to have? You’ve seen MacDonald use both Find features and Sort features in other apps, so neither should be difficult to learn. A quick KLM-GOMS calculation tells you sorting and scrolling will take at least 10 seconds (assuming the user reverts the chicken list to be sorted by output), while a Find toolbar control will take about 5 or 6 seconds, assuming the first five letters of a name is enough to uniquely identify the average chicken. So implementing Find is substantially faster than Sort. That’s a good enough answer for this issue. There’s no sense spending time on a lengthy usability test to try to get a more precise fix. If you can resolve usability issues on the cheap, do it.

While I don’t see a reason you can’t design early and often for major projects just to help yourself understand the design issues, designing early and often is particularly well-suited for discount usability engineering, and its reliance on fast design iterations, simple usability tests, and other fast means of evaluation such as cognitive walk-throughs and heuristic evaluation. It may seem that designing early and often means more work for you, but by focusing yourself on specific design issues, you avoid spending time researching a lot of stuff that ultimately makes a small impact on the final design. Designing early gives you something specific to analyze by whatever tool gives the most return for the effort.

Designing doesn’t have to mean big honkin’ binders, elaborate change control processes, and reams of test results. For many projects, you can achieve adequate usability without eye-trackers, video, screen recorders, and persona write-ups. In large projects, that stuff is mostly for you to communicate with others on the project, to convince clients, developers, and managers that you know what you’re doing and what should be done. If you are an accepted member of a small product team, you can make a light usability engineering footprint. Too much Process can discourage creativity, and designing early and often requires you to come up with multiple creative designs. If you make it hard to design, you’ll train yourself to stop doing it. For some of my projects, my usability engineering equipment, in it’s entirety, is the following.

A notepad is all you need to scribble down observations and comments from users and sketch out design alternatives. At some point, you’ll need to make UI prototypes on a computer, but when designing early and often, many of your designs never make it that far. You’ll need to communicate your designs to developers, but a simple wireframe is often good enough for projects using discount usability engineering.

Fit with Agile Development

Well before conducting the usability test, you have enough of your design done to give the developers something to work on. They can develop the backend storage to save the egg output history of each chicken and they can code up the basic layouts for the chicken list and the map windows, with empty space left for the graphics once you’ve determined what they are. Development can start while your user research is still underway.

This is another advantage of designing early and often: it fits well with agile methods of software development. You have a range of designs early on that you and a developer can peruse for commonalities that may be coded immediately. Details can be filled as your research continues and your designs converge. You may not know how to sort the list of chickens, but it’s pretty likely that there will be a list of chickens. The developers can create that right away and later add an ORDER BY clause to the associated stored procedure. Some of the developers’ work can serve as prototypes to further your research. As a member of an agile team, you can encourage the developers to develop first the features that are most diagnostic of how whole system should fit together.

Risks of Designing Early and Often

The primary risk of designing early and often is getting stuck too soon on a suboptimal design. Misguided notions about the user, task, or environment can become permanently imbedded in the UI, which wouldn’t have happened if you did all your research with an open mind beforehand.

The primary way to avoid this trap is through multiple parallel designs. You want to design not only early but often. Each time you make a design, ask yourself, how else could it be done? If you have a tabular layout, what would a page layout be like? If you’ve an object-centered UI structure, how would an task-centered structure work? What are the classes of objects that the user deals with? What designs do you get when you centered a window around each one? What happens when two classes are in the same window in a master-detail arrangement rather each being in a separate primary window?

For each alternative design, consider when each would be better that the other. For example, what needs to be true about the user, task, and environment for a page layout to be better than a tabular layout? For all your designs, determine the element that represents the biggest differences among the alternatives. What information do you need to decide on the element? That’s the most important research question to answer next. You don’t want your user research to prove a particular design so much as disprove all other competing designs you can think of. In this way, designing early and often encourages effective research rather than “research” that only sends you down the garden path to a pre-ordained conclusion.

It’s also important to continuously remember that any design does not exist for its own sake but to fulfill a certain user need. Treat your designs as a manifestation of a requirement that can have any number of appearances. For example, the chicken list is a manifestation of the need to be aware of drops in chicken output. That need could also be fulfilled with a list of alerts, where anytime a chicken’s output falls below a threshold, an alert appears on the list. Maybe the Chicken Alert system could send MacDonald an email when there’s an alert. A link in the email can open the app to display the map for diagnosis. The alert list would be for tracking on-going issues. Recognizing that a design is only as valuable as the need it fulfills will help you recognized better ideas when they come along.

Finally, you need to include an open-mind in your work despite doing the design early. That means more than having the right attitude, but also planning and conducting some research and tests with no specific agenda or issue to resolve. Just sit and watch and ask and try to understand the domain and the user.

Faster, Sharper Research

Ironically designing early and often in usability engineering is really more about doing research than design. By continuously thinking about multiple potential designs, your research provides a better value-to-cost ratio. It helps you straddle both the abstract concepts that characterize the user’s world and the concrete pixels-on-the-screen that’s your ultimate product.

If a real job were as simple as Old MacDonald’s chickens in the example today, designing early and often probably wouldn’t make much difference. The most rudimentary user research would probably uncover the requirement to support diagnosis, for example. But real jobs, even simple real jobs, are not as simple. As a usability specialist coming on the scene, there’s so much information than it helps to keep design in mind to focus the inquiry.

Summary Checklist

Problem: Getting the requirements and design issues for an application with limited research resources.

Potential Solution: Design early and often, creating multiple designs in parallel.

• Contrast alternative designs to identify research questions.
• Use feedback from designs to identify implicit requirements.
• Develop usability tests to evaluate competing designs and assumptions.
• Extract from promising UI designs any implications for backend design.
• Use blue-sky designs to stimulate and evaluate realistic alternatives.
• Use with discount usability and agile development.