Building the operational prototype was time consuming. It took more than twice as long as the paper prototype.
In the paper session, the burden of patience and imagination was on the user. I feel like paper prototypes will be good for illustrating concepts and refining them. The audience for the paper prototype should be a colleague who is either familiar with the process or has some background information about the operation you are trying to evaluate. Otherwise, I think too many things must be explained. For example, I had a discussion with a co-worker this morning about a design I was working on. Having a rough sketch from my tablet in the correct scale gave him a better idea of the idea I was trying to communicate so he could make constructive comments about it.
I think the operational prototype may be most helpful when I’m looking for actual user feedback on a particular operation. The test subjects can and should be far removed from the design process. The user can perform the test with a minimum amount of explanation.
Advantages of the Paper Prototype
- Rapid design, redesign, and refinement
- Low committment of time and attachment to design
- Lightweight, portable
Disadvantages of the Paper Prototype
- Inaccurate sense of the time required to perform an operation
- May not be useful in assessing the ergonomics of the operation
- Requires user to have knowledge of the problem domain; ie. what are we trying to assess? what needed to be left out? what exact operation must be performed?
- Requires a patient user
Advantages of the Operational Prototype
- Accurate indication of time required to complete operation
- Allows user to enter much more fully into the interaction (they can forget that you are there watching and simply concentrate on performing the task)
- May be very good for selling the concept beyond immediate colleagues
Disadvantages of the Operational Prototype
- Can be time consuming to produce (compared with paper prototype) and may develop a deeper personal attachment to the concept or idea. Perhaps delegating the task to another can help avoid this
- Requires greater attention to details of the interaction (again more time)
I think the operational prototype allowed me to really confirm that the operation took less time. It was clear to see (just based on the time of the videos alone) that it was a much shorter process.