A New Tool for Testing Your Design Concepts Ethically

A New Tool for Testing Your Design Concepts Ethically

Design research at scale requires ethics at scale
Jeremy Raider
Andrea Rabinelli
Molly Dauphin
read time:

How many people can you reach online with $5, a few hours, and limited advertising expertise? During a recent early-stage concept test, we found an answer: more than 2,000. Our goal was to get our prototype into the world prior to launching it to gain a better understanding of how the concept would be received in the market.

Early concept testing is essential to the design process. Without it, a seemingly great idea could turn out to be a flop, totally misaligned with consumers’ needs. Remember “New Coke”—a new version of the classic soda that replaced the original? Although the company had run tests, they only focused on validating the new taste; they neglected to see the potential impact of changing an iconic brand. It’s one of the most well-known missed opportunities by a company many know and trust.

These days, though, getting feedback is easier than ever. Thanks to social media ads, email campaigns, and digital surveys, we can reach thousands of people—and fast. Our colleagues at IDEO have even built a tool called Shape that allows designers to collect quantitative and qualitative feedback from prospective users in just a couple of days. But with powerful tools like these comes great responsibility. Now that we have the ability to test ideas at scale, we need to adapt our old-school rules of design ethics to account for new methods.

Ethical considerations have always been at the heart of our work. We even provide new IDEOers with a book called The Little Book of Design Research Ethics, which outlines three principles—responsibility, respect, and honesty—that guide us through the design research process with the end-user top of mind. However, as the technology we use continues to evolve, so must the way we apply these principles.

Here's what those three principles mean in practice:

  • Responsibility: We act to protect people’s current and future interests. We make participants aware of how we will use the information they share, and we take carefully considered steps to safeguard their information. Our priority is making sure our research is not harmful to people in any way.
  • Respect: We honor participants’ limits and value their comfort—because participants are people, not subjects. We strive to be considerate of their cultural expectations and sensitivities at play.
  • Honesty: We’re truthful and timely in communication, and we want to help participants make informed choices about what they share, when they share it, and how they share it. Consent is key, and every action taken should be with the user in mind. Our goals and motives must remain as transparent as possible.

Recently, we sat down to interview people with a chronic illness to gauge their interest in a daily caretaking service to take pressure off of families and themselves. In person, we were able to make it clear that the service was still just a prototype—it wasn’t something they could actually access. But what if we went the digital route and ran social media ads for the service? What if we unintentionally misled a large group of people to believe that a service that could improve their quality of life existed when it was just a prototype? The consequence could have been huge.

With that in mind, our group of design researchers have created a guide to ethical concept testing to evaluate design concepts before launching a test, to make sure that they’re fair and beneficial for all involved. Think about it like a list that pushes you to consider the impact of sharing your concept before it goes live.

A few notes before we dive in: If you’re a business, keep in mind that under GDPR—the EU’s General Data Protection Regulation, which went into effect in May 2018—there might be some formal checks you need to do even before you start to use targeted ads. It may be worth checking with your legal team before moving forward. Lastly, while this guide specifically deals with digital concept testing, we encourage you to adapt it for any concept testing at scale.

In addition to going through the plan provided above, we always recommend running your concept test with a small number of people prior to going live so you can catch any flaws that you might not have noticed. After all, you're only human.

This tool is a first step in molding our design ethics to new tools, but it’s just the beginning of a long, collective conversation about the ways they will need to adapt over time. While $5 can get you started with concept testing, that scale creates more considerations for responsible designers to make. For now though, we want to leave you with our community’s top resources for learning more about ethical design.

No items found.
Jeremy Raider
Jeremy is a research-driven problem solver. He's fascinated with uncovering insights hidden within our behaviors, thoughts, and environments.
Andrea Rabinelli
Andrea is a human-centered data scientist who embraces design as a way to give data a purpose. In his free time, you can find him jamming with his guitar, both in and out of the office.
Molly Dauphin
Molly is a designer and illustrator working out of Brooklyn, NY who spends weekends bike riding and decluttering for fun. She was Jeremy’s junior prom date in high school.

Subscribe to the IDEO newsletter

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.