Crowdsourcing Usability Testing VS IX Lab Usability Testing

IX Lab


Usability testing has become more and more important for user experience (UX) research and user interface (UI) design. However, conducting usability testing is expensive and time-consuming. For traditional usability testing such as lab testing, researchers have not only to spend time recruiting participants and host lab testing, but also have to provide equipment and testing rooms. Therefore, usability researchers are seeking an efficient way to conduct usability testing. Crowdsourcing is an emergent way for usability testing.


The project examined the usability of two popular health information systems (WebMD and MedlinePlus) to explore users’ perception of the two websites as well as the advantages and shortcomings of each websites’ design. I tested the two websites through two ways-user experience lab and Crowdsourcing platform-AMT(Amazon Mechanical Turk).

I asked participants to perform three tasks on each website and answer questions related to pros and cons for each websites. At the end of the test, I asked participants to choose one of the two websites to perform the fourth task. I analyzed the onscreen activities which were recorded by Morae software, and did quantitative analysis for time spent on tasks, error-free task completion, satisfaction of the answers, overall perception of each website, the website chosen for the fourth task and the users’ comparisons of the two websites. We also did qualitative analysis for the descriptive data.