Tools for Detecting Bias in Website Designs

How can we support designers in knowing how their designs may be perceived by a diverse audience?

Here are some of the tools we have built:

Augury: Augury provides design feedback by quantifying the aesthetics of any given website and instantly predicting visual appeal for a specific demographic and geographic target group. Augury's core functionality is based on three artifacts: (1) algorithms that compute a set of 30 image metrics, (2) website annotations derived from three LabintheWild experiments with more than 60,000 participants (ratings on websites' perceived colorfulness, visual complexity, and visual appeal), and (3) predictive models of colorfulness, visual complexity, and appeal. Augury also includes our system Juxtapose, which provides design inspiration via a library of suitable example websites from various countries (Nordhoff et al., 2018). %We introduced our computational models at CHI 2013 (Reinecke et al., 2013, Honorable Mention Award). We additionally contributed an open-source dataset with almost 40,000 participants (which has now grown to more than 60,000 participants, or 3.6 million ratings), including peak estimates and models of visual appeal for diverse demographic and geographic groups, presented at CHI 2014 (Reinecke and Gajos, 2014).
Several companies have used Augury, including a US-based automotive company and a globally operating broadband and telecommunications provider. Its design suggestions have lead to redesigns that increased conversion rates by up to 37%. The system, awarded the Madrona Prize 2017 for the research project with the best commercialization potential, has been commercialized at Augury Design Inc.

Colorcheck: Colorcheck predicts which colors on a given user interface or image will be distinguishable depending on a users' demographics and vision abilities, and depending on situational lighting conditions, such as varied combinations of natural and artificial lighting, glossy screens, and monitor settings (Reinecke, Flatla, Brooks, 2016). The system uses models derived from the data of more than 30,000 participants who completed a color vision test on LabintheWild in realistic usage situations, such as outside in bright sunlight or indoors with a low monitor brightness. Designers can use ColorCheck to easily identify the areas of a user interface or image that contain problematic color pairs and recolor them accordingly.

Publications

Katharina Reinecke, David Flatla, and Christopher Brooks, "Enabling Designers to Foresee Which Colors Users Cannot See", Human Factors in Computing Systems (CHI), 2016. Acceptance: 23.4%    Best Paper Award - Top 1%    PDF

Lane Harrison, Katharina Reinecke, and Remco Chang, "Infographic aesthetics: Designing for the first impression", Human Factors in Computing Systems (CHI), 2015. 4 pages (Note). Acceptance: 18%    PDF

Katharina Reinecke and Krzysztof Gajos, "Quantifying Visual Preferences Around the World", Human Factors in Computing Systems (CHI), 2014.
Acceptance: 22.8%    PDF

Katharina Reinecke, Tom Yeh, Luke Miratrix, Rahmatri Mardiko, Yuechen Zhao, Jenny Liu, and Krzysztof Z. Gajos, "Predicting Users' First Impressions of Website Aesthetics With a Quantification of Perceived Visual Complexity and Colorfulness", Human Factors in Computing Systems (CHI), 2013.
Acceptance: 20%   Honorable Mention - Top 5%    PDF