User Research (Iterative) Case Study
Usability Testing of the Digital Transformation Agency Beta Site
Overview
Vogl & Blake was commissioned by the Digital Transformation Agency to perform usability testing on the DTA beta site to ensure accessibility, engagement, and user friendliness. Usability testing was carried out in an agile environment using an iterative approach which allowed the multi-disciplinary team at the DTA to incrementally implement changes and solutions to the insights that arose from the testing sessions. The sessions consisted of 7 sprints over 14 weeks with 61 government service providers.
The Digital Transformation Agency
The Digital Transformation Agency (DTA) is a government organisation responsible for strategic and policy leadership on whole-of-government and shared information, as well as communications technology (ICT) investments and digital service delivery. They are dedicated to driving, advising, and securing the Australian Government’s investment in Australia’s digital future.
The Project
Objective
Between May and September 2018, Vogl & Blake facilitated qualitative UX research to test the Digital Transformation Agency’s beta site for usability, engagement, functionality, findability, design, information architecture, content, language, and understandability with diverse government service providers. Testing additionally focused on the experiences of people with visual impairments, including those who do and do not use assistive technology to optimise accessibility and inclusivity.
Approach
Vogl & Blake worked in collaboration with the DTA team to recruit participants, design and facilitate sessions, analyse and share findings, as well as making recommendations to service design and for future testing sessions. DTA team members were present at usability testing sessions which offered a comprehensive and contextual understanding of results and recommendations.
An iterative approach best suited this project as it allowed flexibility throughout the process which was essential for success in this agile environment and collaborative structure. Actionable insights developed by Vogl & Blake were implemented by the DTA team between each sprint, enabling redesign and augmentation of subsequent sessions to re-test relevant changes and optimise future results.
The Process
Participants
Government service providers representing DTA’s typical user base were recruited for research sessions held in various convenient locations for participants (e.g. Canberra or Sydney DTA, or service provider’s workplace). Participants were recruited to include experienced users as well as newcomers who were unfamiliar with the site in order to diversify perspectives. Specific audiences were recruited for specific sections (e.g. service providers working in procurement to test findability/content in procurement section on the site).
Methods
UX research methods used in this project included:
Providing scenarios
Closed card sorts
Testing mockups
Outcomes
The site was tested on both mobile and desktop devices with 61 government service providers across 7 sprints and offered numerous leverageable learnings throughout the 14-week process. Changes were implemented and retested after each sprint, helping the DTA team refine elements of the site to optimise the user experience.
Usability testing sessions with participants who had vision impairments revealed accessibility issues that were unidentified in other types of accessibility testing, highlighting the necessity of specified testing and design. Particularly notable was the heterogeneous experiences of those with visual impairments, who looked at and used the website in different ways, with additional variation depending on assistive technologies (or lack thereof).
