How many people can you reach online with $5, a few hours, and limited advertising expertise? During a recent early-stage concept test, we found an answer: more than 2,000. Our goal was to get our prototype into the world prior to launching it to gain a better understanding of how the concept would be received in the market.
Early concept testing is essential to the design process. Without it, a seemingly great idea could turn out to be a flop, totally misaligned with consumers’ needs. Remember “New Coke”—a new version of the classic soda that replaced the original? Although the company had run tests, they only focused on validating the new taste; they neglected to see the potential impact of changing an iconic brand. It’s one of the most well-known missed opportunities by a company many know and trust.
These days, though, getting feedback is easier than ever. Thanks to social media ads, email campaigns, and digital surveys, we can reach thousands of people—and fast. Our colleagues at IDEO have even built a tool called Shape that allows designers to collect quantitative and qualitative feedback from prospective users in just a couple of days. But with powerful tools like these comes great responsibility. Now that we have the ability to test ideas at scale, we need to adapt our old-school rules of design ethics to account for new methods.
Ethical considerations have always been at the heart of our work. We even provide new IDEOers with a book called The Little Book of Design Research Ethics, which outlines three principles—responsibility, respect, and honesty—that guide us through the design research process with the end-user top of mind. However, as the technology we use continues to evolve, so must the way we apply these principles.
Here's what those three principles mean in practice:
Recently, we sat down to interview people with a chronic illness to gauge their interest in a daily caretaking service to take pressure off of families and themselves. In person, we were able to make it clear that the service was still just a prototype—it wasn’t something they could actually access. But what if we went the digital route and ran social media ads for the service? What if we unintentionally misled a large group of people to believe that a service that could improve their quality of life existed when it was just a prototype? The consequence could have been huge.
With that in mind, our group of design researchers have created a guide to ethical concept testing to evaluate design concepts before launching a test, to make sure that they’re fair and beneficial for all involved. Think about it like a list that pushes you to consider the impact of sharing your concept before it goes live.
A few notes before we dive in: If you’re a business, keep in mind that under GDPR—the EU’s General Data Protection Regulation, which went into effect in May 2018—there might be some formal checks you need to do even before you start to use targeted ads. It may be worth checking with your legal team before moving forward. Lastly, while this guide specifically deals with digital concept testing, we encourage you to adapt it for any concept testing at scale.
Download a printable PDF of the tool here.
In addition to going through the plan provided above, we always recommend running your concept test with a small number of people prior to going live so you can catch any flaws that you might not have noticed. After all, you're only human.
This tool is a first step in molding our design ethics to new tools, but it’s just the beginning of a long, collective conversation about the ways they will need to adapt over time. While $5 can get you started with concept testing, that scale creates more considerations for responsible designers to make. For now though, we want to leave you with our community’s top resources for learning more about ethical design.