Usability Evaluation

Usability Evaluation of Alzheimer’s Society Website

Summary

Conducted on behalf of the Alzheimer’s Society, this evaluation assesses the usability and user experience of the Alzheimer’s Society website. With millions of users per year, the effectiveness and ease of use of the Alzheimer’s Society website is vital to providing resources to people affected by dementia and supporting fundraising efforts. The purpose of this evaluation is to identify usability problems, defined as any aspect of the user interface that adversely impacts a user’s performance, in order to improve the website.

Remote moderated usability testing was conducted with eight participants, as it allowed for a focus on users’ behaviors and insight into users’ attitudes . Usability problems were documented according to their severity and informed four key recommendations for improving usability and user experience for the diverse community of users of the Alzheimer’s Society website.

Goals & Tasks

The following goals were informed by concerns stated in the brief. Tasks were carefully written to address the goals by collecting data on areas of interest of the site and supporting exploration to understand overall user experience. Success was defined for each task before beginning usability testing sessions.

GoalTaskTask is complete when…Task completion rate
1Explore how the home page informs users’ understanding of the Alzheimer’s Society and expectations of the site. Explore the home page but don’t click on anything just yet. Describe your impressions. In your own words, who is the Alzheimer’s Society? What do you expect to be able to find and do on this website? 
Participant has described their impressions and answered the questions in the task. 
100% (8/8)
2Determine how easily users can make donations to the Alzheimer’s Society.You’re interested in supporting the work the Alzheimer’s Society is doing. Give £100 to fund their cause.Participant has successfully gone through the donation form but before clicking “Complete £100 donation”.100% (8/8)
3Determine how easily users can find and sign up for a fundraising opportunity.You want to raise money for the Alzheimer’s Society another way. Find and register for an activity that you can partake in from home.Participant has successfully found an event and filled out the registration form (when applicable) but before clicking “Continue”.75% (6/8)
4Determine how easily carers can find information and resources related to increasing the independence of someone living with dementia.Find a few ideas for how you might help someone who has recently been diagnosed with dementia and is living alone to maintain their current lifestyle.Participant finds 2-3 different resources related to independence and feels satisfied. If a user finds less than 3 resources, prompt once to see if they can find more.100% (8/8)
5Determine how easily users can find and engage with the Alzheimer’s Society forums.Imagine you’re caring for someone living with dementia. It can be helpful to connect with other carers with similar experiences. Find out about other people’s experiences caring for someone living with dementia and how you would share your own online.Participant lands on a Dementia Talking Point forum page they deem relevant.75% (6/8)
6Understand and measure the overall user experience of the Alzheimer’s Society website.Conduct SUPR-Q questionnaire to measure overall user experience broken out into four parts: usability, credibility, loyalty, and appearance.Participant completes SUPR-Q questionnaire adapted from Sauro (2015).
Verb-based tasks were utilized to invite users to complete a specific action, strengthening evaluation of the website’s functionality across users. Tasks were worded to be appropriate for participants and, in order to gather unbiased data, avoided using words from the user interface or telling users where to go. Task scenarios were simple, generic, and relatable, enabling users to find information important to them, role playing where necessary. As dementia topics can be upsetting, tasks aimed to avoid emotional reactions, keeping the emphasis on website usability.

Participant Recruitment

As millions of people access the website each year, recruitment of participants focused primarily on experiences that might cause people to behave or feel differently when interacting with the website. Participants were asked about their level of experience making donations online, fundraising online, and with dementia topics, measured using a Likert familiarity scale. Participants were also asked how many hours per day on average they spend on the internet as a means to gauge their level of comfort navigating websites. In order to achieve the desired diversity in all segments of experience, internet usage, and demographics, eight participants were recruited for this study.

Recruiting a mix of demographics was also considered, and the participant pool consisted of four women, four men, four people between the ages of 20-40, and four people between the ages of 40-60.

Moderated Usability Testing Sessions

Because the aim of this evaluation is to improve the website’s usability and user experience, qualitative data from usability testing was emphasized. All usability tests with participants were conducted remotely over Zoom, which allowed for video, audio, and screen recording of participants’ interactions with the website. These methods of collecting raw data made in-depth analysis possible through which to address the goals stated above.

Participants were asked to think aloud as they carried out the tasks, a robust method for collecting qualitative data on why users take each action and what they are thinking and feeling about the website. As prompts that encourage users to think aloud can bias user behavior, precautions were taken to impartially incite users to share their thoughts when it was unclear what they were thinking, looking at, or doing. Follow-up questions were asked about any issues or noticeable impressions to ensure a complete understanding of usability problems and the user’s experience. 

Reporting Results

Qualitative data from usability testing was coded for usability problems, positive observations, and other feedback. Next, the positive observations and usability problems were summarized in a rainbow spreadsheet that for each, provided a description, context, severity rating, re-design recommendation if appropriate, which participants experienced the issue, and a telling quote from the usability testing sessions if applicable. Careful attention was paid to determining where the same usability problems appeared in different situations. Summary statistics were also calculated for each task and SUPR-Q results were analyzed in Excel. A record was kept of the pages visited by each participant in each task including page titles, URL’s, participants, and task number. The page titles match those in the transcripts and rainbow spreadsheet, thus making it easier for the Alzheimer’s Society to understand the range of pages looked at, see overlap with other evaluations, and address the usability problems. 

To respect confidentiality, specific results are not presented here. A more detailed report is available upon request.

CREATED FOR INM315 Evaluating Interactive systems (PRD2 A 2020/21) AS PART OF MSC HCID AT CITY, UNIVERSITY OF LONDON AND AWARDED A DISTINCTION.

More projects:

Portfolio

%d bloggers like this: