Usability Testing for FreeDOS.org
April 30th, 2025
Purpose
This usability test focused on the ability of new users to find information from the FreeDOS website.
How quickly can users find the information they need?
Can users successfully find FreeDOS programs or files?
Can new users navigate the FreeDOS site?
How quickly can users find the information they need?
Can users successfully find FreeDOS programs or files?
Our specific research question was: How quickly can users receive and remember information on FreeDOS from the FreeDOS website without emailing Jim Hall?
Efficiency was measured by the time it took the test participants to complete the presented tasks. Critical and noncritical errors were counted, and the participants’ satisfaction was based on their comments and ratings, which they recorded after each scenario and after they completed the entire test.
Methods
Four methods of information gathering were used during the test.
The background questionnaires collected basic demographic information. Participants supplied information such as gender, age, education level, occupation, and experience with FreeDOS.
During the test, participants were asked to complete four ordinary tasks relevant to what the FreeDOS audience may search for on the website. After participants completed the scenarios, they were given post-task questions. All of the questions asked them to rate the ease or difficulty of the task.
When the test was over, the participants were given a debriefing interview to inquire about their impressions of the site and the testing experience in depth.
Afterwards, participants selected words from a set of 36 product reaction cards. Each participant selected five words that best described the website based on their experience using it.
Background
FreeDOS is a free, open-source Disk Operating System (DOS) program that is entering version 1.4. Users can use it to play classic DOS games, run legacy business software, or write new DOS programs. FreeDOS started in 1994, although DOS dates back further. Other operating systems used the name "DOS," but in this context, it means disk operating systems compatible with PC DOS. DOS and the many applications and games that ran on it remained popular throughout the 1980s and into the 1990s.
Jim Hall is the founder and project coordinator (equivalent to Executive Director) as well as the webmaster. He started FreeDOS while an undergraduate. He wasn't a fan of Windows 3.x, and when Microsoft announced in 1994 that the next version of Windows would replace DOS entirely, Jim launched a new development effort to create an open-source version of DOS. That is now FreeDOS. FreeDOS was most popular in the late 1990s and early 2000s, and it still has a small, but engaged, community today.
The usability test on FreeDOS involved observing five recruited participants completing common tasks on the FreeDOS website. Jim is planning to update the website and wants to ensure it is easy to use. He did a usability test a few years ago, which helped him update the website. Our usability test enabled him to understand where the “rough areas” of the website are (and what suggestions we had to improve them, based on the results in the usability tests).
Working in a group of three, we named our group DOS DOS Revolution, as a spin on the retro game “Dance Dance Revolution”. I was the only designer in this group; my two groupmates are writers. With my UX design knowledge, I identified user interface errors that my groupmates might be less likely to notice.
I crafted the presentation via Google Slides (above). Originally a blank format, I designed it in a similar style to the website: simple and a little retro. My groupmates wrote the majority of the document for the Usability Tests (below), while I visualized it in a presentation format. The whale appears as a mascot or motif on the website, so I made it a subtle animated progress bar that indicated where in the presentation we were.