MODEL DEVELOPMENT

Designing and Usability Testing New ODR Communication Models

We worked with an inclusive designer and an accessibility evaluator to design new communication models for courts to adapt to fit their needs.

To help courts communicate more effectively with parties, we created court communication materials that addressed the barriers we had uncovered in OPEN focus groups. The materials integrate the expertise of our inclusive design partner and a leading accessibility evaluator, the feedback of 20 usability test participants and secondary research on effective practices for communicating with individuals with low literacy and low digital technology literacy.

Models for Effective Court Communications about ODR

The models we created follow a simple workflow. By scaffolding information about ODR, we were able to make tasks easy to understand and follow. Below are short descriptions of the materials, listed in order within the workflow.

For more detailed information about the models and how they were created and tested, check out the Models on the Court Resources page. For a demonstration of the models, check out this brief video.

Notice to Defendant of Mandatory Online Dispute Resolution

A document that accompanies the defendant's

court summons. It provides basic information about ODR, steps the recipient needs to take to get started using ODR, and resources available to them.

ODR Explainer Video

A short explainer video that describes and illustrates how the ODR process works, as well as step-by-step instructions on how to register for ODR.

ODR Home Page Prototype

The main landing webpage that explains how ODR works, how user data is protected, and contact information for help.

ODR Self-Help Guide for Defendants

A separate webpage that includes seven steps for users to follow to prepare for ODR as a defendant.

ODR Account Registration Webpages

A series of short webpages that allow users to register for ODR using the information provided to them.

How We Created the Models

Based on what we learned from the OPEN Phase 1 focus groups, we created models that were designed to enhance trust, be readily understood and easily navigated, particularly for people with low literacy and low digital technology literacy. To this end, we partnered with an inclusive designer. To ensure that the materials were also accessible to individuals with disabilities, we also partnered with a leading accessibility evaluator to conduct a comprehensive review of all materials.

Once we had initial, working versions of the models, we held a first round of usability testing to obtain feedback from real world users. Between usability testing sessions, we and our design partner implemented needed changes as identified through the usability testing. We then worked with the accessibility evaluator to identify changes needed to comply with Web Content Accessibility Guidelines (WCAG 2.1). We implemented those changes and then re-tested the near-final models in a second round of usability testing; the second set of testers identified only a few minor changes to be made. The final models thus reflect this iterative and collaborative approach with our design partner, accessibility partner, and 20 usability test participants.

The purpose of the usability tests was to obtain real world feedback on the models from individuals whose backgrounds are similar to those of self-represented litigants, particularly those with low incomes and low literacy.

Usability Test Recruitment and Participants

We conducted usability tests in-person across the country, including two sites we had previously visited for focus groups: a rural town in New Hampshire with a primarily white population, and Baltimore, Maryland, where we worked with Black participants. During the first round of tests, we also obtained feedback in the Santa Fe area of New Mexico with a large Latino population.

During the second and final round of tests, we worked with majority Black participants in Philadelphia, PA.
Of the 20 total participants, three indicated that they had less than a high school diploma, twelve had a high school diploma or equivalent and five had some college or trade school.
Participants ranged in age from 27 to 67. In terms of income, fifteen participants made less than $25,000 and five made between $25,001 and $50,000.

How We Tested the Models

During the usability tests, we asked participants to navigate the materials as if they were real.

To help put them in the shoes of a person who would actually use the materials, we provided them a scenario in which Jane Roe, a defendant, is facing a small claims case filed by a former business partner, John Doe.

Each test was conducted with individual testers and was scheduled for 90 minutes.
We asked testers to complete a series of tasks using the models, without assistance from the researchers.
These tasks included reviewing the notice and accompanying court summons, navigating to the court ODR website, reading the information about ODR, watching the informational video, using the interactive guide, and registering for an account.
While completing tasks, testers voiced their thoughts out loud so we could understand why they made certain choices.
The researchers took notes on their behaviors and commentary and asked follow-up questions to understand their experience in greater depth.
We also asked how confident they felt about using the materials at multiple points during the test.

Once participants completed the final task, we then asked them to fill out a survey rating the materials and their experience using them. We then conducted interviews to ask about their understanding of ODR, what changes they would make to the models, which materials were most effective at helping them to understand ODR, and general questions on how they usually complete tasks on the internet.