Midea, one of China’s largest appliance manufacturers, wanted to identify growth opportunities in the domestic consumer market, while focusing on people’s cultural beliefs around health and well-being. During the course of the project, IDEO found that Midea could play a key role in building a healthier nation.

The team set out to design potential future products for Tong (“through, whole, and to understand”) that are built around the basic themes of eating, breathing, and being: an easy-to-use, hassle-free herbal tea dispenser, an air balancer that improves and balances the quality of air in the home, and a body-balancing mobile phone app that encourages users to rebalance their body condition where needed, through movements created with the Chinese philosophies of well-being in mind.

The Tong project was featured extensively in the press, and opened further conversations with Midea about creating new value and extending into new markets.

Put food on the table or keep the furnace running on a frigid day? It’s an unfortunate choice many low-income families in Flint, Michigan, face.

To help customers in Flint and other Michigan towns stay on top of their utility bills and better understand the relationship between their energy consumption and the resulting costs, Consumers Energy partnered with IDEO and utility assistance provider The Heat and Warmth Fund (THAW) to create an innovative pilot billing program called Clear Control in late 2013.

The pilot was designed with three major human-centered changes to the traditional utility experience:

• Instead of monthly bills, participants received two bills per month. Shorter billing periods allowed for smaller, more frequent payments, helping customers match available funds with their ability to pay.
• Daily text updates on energy usage and bill amounts kept consumption top of mind.
• Consumers Energy offered personalized, in-home audits to help customers make low-cost, high-impact changes to their energy bills.

A year later, the results of the pilot are energizing. Clear Control participants accumulated no new debt, while the control group owed $300; pilot customers used 20 percent less electricity, reducing their bills by $200 per year; and 95 percent of participants paid their bills in full and on time, compared with 35 percent in the control group. The utility company simultaneously benefited by avoiding arrearages, late-payment fees, and shutoffs. Consumers plans to incorporate Clear Control lessons into its overall energy assistance program design and development moving forward.

"It’s no longer ‘me’ against ‘them’—now it’s ‘us,’" said one Clear Control participant.

In 1980, Apple asked IDEO to develop a mouse for their radical new computer, the Lisa. Previous attempts at mouse design, by Douglas Englebart and Xerox PARC, yielded results that were too expensive and hard to make. The Apple mouse needed to be more reliable and less than 10 percent of the cost of the earlier versions.

To start, the design team created a cheaper, much-improved mechanism that would operate the mouse, and found that a complex, plastic "ribcage" would hold the pieces together. The team similarly tested and refined the mouse’s other key components, from the audible and tactile click of the button to the rubberized coating on the ball. A record turntable spun for days, logging “mouse miles” to check the reliability of the electromechanical assembly.

The resulting mouse proved mechanically and economically sound and was changed only slightly when adapted for use with the first Macintosh computer. The basic mechanism design is used in virtually all mechanical mouses produced to date.

Food is the heart of most cultures. In China, one of the oldest cultures in the world, food is central to family gatherings and celebrations. It’s what binds people, a tangible way of showing love and respect. But in recent years, the Chinese food industry has suffered from concerns around safety.

Two siblings, Harn and Anmao Sun, set out on an ambitious venture: to create a trusted source of food in China and, by setting an example, to instigate a movement around conscious farming practices. The American-born entrepreneurs launched Hunter Gatherer, a restaurant and retail food space built around a transparent supply chain, from farm to table to grocery shelves. The venture is based on growing and serving what they call real food: The ingredients are chemical-free and come from known and trusted sources—food they’d eat themselves. The purpose of the venture is simple: to serve people real food that is their right.

Having studied and worked in the US, Harn and Anmao grew accustomed to browsing farmers markets on Saturday mornings or picking up a burrito at Chipotle. But when they returned to China, they were unsure about the safety of the food they were eating.

“In China, you wouldn’t go to a food stand, because you’d be concerned about the ingredients,” Anmao said. “You can’t take the safety for granted.”

Hunter Gatherer’s very existence challenges bedrock assumptions about how food is produced in China. A long and depressing list of scandals—milk contaminated with melamine, pork produced from diseased pigs, used cooking oil repackaged and sold as new—has rocked the public trust.

“In China, agriculture is going through a huge change,” Anmao said. “In the US, agriculture operations are regulated. Farmers’ practices are regulated. If they use pesticides, there are limits because these farmers understand that once a certain threshold is hit, additional pesticides will not yield more crops. But in China, pesticide use came about 30 years after the US and Europe, and because of a lack of full understanding, farmers in China incorrectly operate under the assumption that more pesticides means more crop yields. Also, with over 250 million farmers in China, it becomes immensely difficult to properly regulate everyone.”

New Ventures

Harn and Anmao wanted to restore trust in the simple act of eating in China, and they came to IDEO to build a scalable business based on that vision. This was essentially a startup, a shared commitment between IDEO and Hunter Gatherer.

“I’ve lived in Shanghai for the past seven years, and as a relatively active and health-conscious person, I’ve had to invest quite a bit of time in finding food that I trust,” said Charles Hayes, managing director of IDEO Shanghai. “That investment has been both in the time it takes to research the sources of ingredients but also a literal investment in paying a high price for imported food just for that extra peace of mind.”

Anmao and Harn took up residence in the IDEO Shanghai offices to incubate their business. Together with IDEO designers, they tested packaging, cooked countless meals, and found ways to introduce unfamiliar ingredients to Chinese consumers. They discussed different concepts of lunch and dinner, and how notions of freshness in Chinese culture might affect cold-storage systems. In IDEO’s prototyping kitchen, the teams tested all manner of different foods, flavors, colors, mixes, and price points, and they brought consumers in to try them out. IDEO designers, along with the founding team, also worked on the brand and positioning of what would eventually become the Hunter Gatherer enterprise—its location, human resources, and marketing strategies.

As the restaurant concept came together, finding the right food sources had to be addressed. Traditional agriculture couldn’t be trusted; organic certifications were available on the black market, and safety protocols around chemical use were nonexistent. Anmao and Harn decided to invest in their own farms. They met with farmers, asked questions about their methods, and walked in the fields to see for themselves. Eventually they settled on two plots of land outside Shanghai and began developing more than 80 different crops to be used in the restaurant.

From Seed to Table

When it came time to find the right space for the first location, Anmao and Harn discovered an open, airy storefront on Anfu Road in Shanghai’s tree-lined former French Concession area. They designed and built a retail space and kitchen, and along with IDEO designers, they trained the staff and organized workflows. The staff set the tables with Mason jars filled with flowers, hung photographs of the farmers who grew the food, and put up a sign that showcases the percentage of vegetables coming directly from the Hunter Gatherer farms, already more than 70 percent in less than a year after opening.

Customers can shop for fresh vegetables, locally sourced and imported canned and jarred goods, bins of cereal, freshly made smoothies, artisan sauces, and exotic chocolates. Upstairs, they can feast on delicious dishes in the homey restaurant.

Reviews have been glowing. One customer raved about the machine that grinds fresh nuts. Another loved the open space, the “good vibes,” and the refreshing cucumber water. Young people go there on dates, and families return for the delicious menu variety.

The convivial space has blossomed into more than just a restaurant and grocery store. It’s become a gathering place of sorts, both for the community and for partners who share the same vision. Farmers and food purveyors are invited to showcase their products at seasonal events and to mingle with customers and share recipes and stories.

“Hunter Gatherer has come to symbolize a kind of lifestyle the Chinese people have been looking for,” said Eugene Lin, an IDEO business designer. “A place they can enjoy hanging out with friends, as well as a movement and mission they can stand behind.”

A Digital Future

This is just the beginning of the venture. The plan is to open 40 more locations throughout China. But the vision for Hunter Gatherer’s growth is bigger than brick-and-mortar restaurants—where the business will truly scale is in the digital space. Eventually, the plan is to launch a Hunter Gatherer online grocery store from which anyone can buy food grown from trusted sources.

“We have to create trust with the consumer. It’s the most difficult and most valuable thing to do in China, especially with food,” Anmao said. “Our brick-and-mortar locations will establish that increased trust. Then we can go online and grow the idea to the rest of the country.”

Will Hunter Gatherer have a hand in changing the way that the world’s most populous nation thinks about food? Its principles are already spreading, from neighboring farmers adopting more sustainable planting methods, to a growing market for products like wine.

But the true sign of success will be almost invisible—the moment when an increase in demand will result in safe food for everyone and when real food becomes commonplace.

As a data scientist, this idea got my gears turning: What if I could use machine learning to build a laugh-detecting algorithm?

Prototyping is a great way to explore how emerging technologies will play a role in our connected future. But instead of focusing on how we will teach our machines to sense human behavior, I’m interested in designing what we will do with the information our devices are learning. In a world filled with machines that learn and interact with us, it’s important that they are able to respond to all parts of our humanity—including the things that make us laugh.

I wanted to see what kind of laugh detection prototype I could build in a few weeks. Here’s how I did it:

1. Building the algorithm

To build the laugh-detecting algorithm, I knew that I wanted to use deep learning, so I turned to Keras, a programming library which makes creating neural networks fast and simple. To train the laugh detector, I needed to use sequences of audio as the input and predict whether or not laughter would result as the output. Fortunately, I found Audioset, a collection of over two million 10-second audio clips with labels from over 6,000 categories. Using a subset of the Audioset collection that is half laughter and half speech, I was able to get a laughter detection model running with 87 percent accuracy within a few days.

After creating a training dataset, I got started on the machine learning challenge. The Audioset data does not contain the extracted raw audio, only a 128-length feature vector for each second of input, which is created using a convolutional network operating on the raw audio spectrogram. These feature vectors will be the input for the machine learning algorithm, and the output will be a binary label of whether the input contained laughter. Since I am working with a relatively large dataset, I wanted to use some deep learning methods, so I turned to keras, an API that makes creating and training neural networks extremely simple, while still leveraging the computational efficiency of optimized tensor computation libraries like TensorFlow and Theano.

Since the model input is sequential, I wanted to try a recurrent neural network first. I started with a single layer LSTM model that quickly converged to 87% accuracy. Learning from the code in a very similar project, I found that applying batch normalization to the LSTM input was very important for getting the model to converge. One of the headaches of deep learning is that seemingly trivial details like this can have a large impact on convergence. I was happy with the performance of my first LSTM model, but I decided to try out a couple more options before moving on. I tried a 3-layer LSTM (because if one layer is good, three must be better), then I tried a simple logistic regression model to see if the fancy RNN architecture made a difference. The models performed with 88% and 86% accuracy, respectively, which showed me that, in this case, more layers did not equate to more power.

Ultimately, I ended up choosing the single layer LSTM, even though it was larger than the logistic regression model, because it was able to handle variable length input. The training data was all sequences that were 10 seconds long, but I wanted the laughter detector to be able to respond quicker than that. While the logistic regression could only operate on input that was the same length as the training data, the LSTM model can still operate on any input length, as long as it is split into one-second chunks.

To take the trained model and run it as a live laugh detector, I put together a python script that pulled together a few different steps. First, the audio is captured from the microphone using pyaudio and chunked into three-second clips. (The length of these clips is an adjustable parameter.) The raw audio is then fed into the pre-trained vggish network and converted to a sequence of feature vectors with the same parameters I used for creating the Audioset training data. The sequence is then fed into the trained LSTM model, and the laughter prediction score is returned and written to a timestamped .csv file. By default, the raw audio is then discarded, and only the score is saved. That way, there are no concerns about storing or transmitting anyone’s private conversations. By running the audio capture and processing in separate threads, I was able to run this continuously without any lag on my laptop.

2. Designing the experience

Once I had the ability to measure laughter, I had to tackle the real design challenge: How do we implement the laugh detector in project spaces without making it seem obnoxious or creepy?

I enlisted the help of a few smart colleagues to approach the challenge from multiple angles: a software designer experienced in designing for play, an environments designer to explore how the detector would show up in a project space, and a communication designer who’s written on the science of laughter.

We had a brainstorm around what we might be able to do with a laughter detector, and the ideas poured in. Perhaps we could use laughter as a weekly indicator of morale (similar to Bhutan’s gross national happiness), or host a competition between project teams for who laughed the most? If we could determine when people are laughing, maybe we could figure out why they are laughing, too. We could determine when laughter is genuine or just polite, which jokes are the funniest, or even measure who is laughed at the most.

Some of the more wild ideas included a gif of an IDEO Chicago director Jon Wettersten (below) who laughs along with us, or an alert that would play videos of clowns if we went too long without laughing.

Ultimately, we wanted to select ideas that would give teams insight into their laughter, but not be too intrusive or prescriptive.

Information systems shouldn’t overwhelm users—particularly when that information concerns people’s moods. There are times when it’s appropriate to not be laughing; you don’t want your smartwatch telling you to laugh more during your grandma’s funeral. And if we were to build a device that listens for laughter, we'd need to make sure that we communicate and maintain people’s expectations of privacy.

With all this in mind, we narrowed it down to two concepts that we would take forward into the prototyping phase.

3. Hatching prototypes

The first thing we made was a simple information dashboard built in Dash. We referred to this design as a “FitBit for laughter,” as it was meant to help teams set goals and measure their progress towards those goals.

This raised the the question, what if you could track your emotional health in the same way that you tracked your physical health? What kind of goals would you set? What would your “laughter workout” look like?

To start, we measured laughs by time of day over a period of 24 hours, laughs per day over the past week, and how close teams are to achieving a daily “laugh quota.”

The second prototype used the output of the laugh detector to control the color and intensity of a Hue lightbulb. When there is a lot of laughter in a room, the light glows a bright yellow to mirror the warmth that laughter brings to a space. When there is less laughter, it transitions to a dim blue to help teams stay calm and focused. In this way, the environment responds to the activities and emotion of its inhabitants.

What if the smart systems we create could react not only to a user’s presence or interaction, but adapt to the user’s mood? How might we design smart systems or spaces that are emotionally responsive?

4. User testing

We wanted to see how people would react to the laugh-bulb, so we got a few colleagues to sit down and test it out. The original plan was to play funny videos to stimulate laughter, but as soon as people sat down, they started joking so much, the videos were unnecessary.

Laughter is a reinforcing behavior; when you laugh, the people around you laugh too, creating a virtuous cycle that can leave you laughing until you’re sore. Seeing the light grow brighter as the room fills with laughter adds energy to that cycle. The group that came in to test the laugh-bulb ended up laughing so much that you could hear them throughout the studio.

WATCH: Testing the laugh-bulb prototype.

Next up, I’d love to put these prototypes in front of more people to see whether quantifying laughter can affect their mood. And I'm left wondering, which emotional or mental health signals could be measured and recorded next?

As more aspects of our lives are recorded, we are going to be able to quantify and improve our lives in ways we can’t yet imagine. Let’s design these systems not just to make us smarter, but to make us more joyful, too.

Play goes hand in hand with innovation. I learned this from Brendan Boyle, director of IDEO's Play Lab, who says that if you measure the amount of laughter in a project space, the teams who chuckle most are also the most successful.

As soon as the rain stopped, we got on our bikes to survey the damage. We saw the flowers on the horizon: droopy and sad-looking. Then, suddenly, they started illuminating, one after the other—opening up, blooming. To this day, we still don’t know how they came to life. But the sight was exactly what we had envisioned. Better even.

That was 2014: the official start of FoldHaus Art Collective. Since then, our art installations have become bigger and more audacious. Shrumen Lumen, five massive, origami mushrooms, followed in 2016 (they’re now on exhibit at the Smithsonian American Art Museum), and we most recently built a five-story tall, geodesic sphere covered with 42 origami shells called RadiaLumia.

Each project takes months to design, prototype, and build. Which means I often get asked: Why do you spend your nights and weekends doing this when you have a fulfilling day job as a creative leader?

For me, it means that I get to work with a bunch of other creative types who pitch in to build something beautiful that’s bigger than what any one of us could do on our own. I get to work with my hands in a way that’s rare in my day job, and every project stretches my own sense of what’s possible.

And because FoldHaus projects are always a group effort, we need to broaden the question: Why do any of us—between five and 30 volunteers contribute to our projects on any given weekend—give up sleeping in to solder? Or skip the beach to code LED patterns? As one of the leaders of the group, I have learned a few things about what it takes to build a strong community and take a bold vision from a Post-it to the Playa.

Here are the 5 principles that have helped this team pull off increasingly ambitious projects:

1. Set an audacious—and contagious—vision

Drawing in a team requires not only that you love your idea, but that others do, too. After all, if you’re asking them to volunteer their time, they’ve got to be pulled in week after week, when they could easily be at the taco truck.

Blumen Lumen was a hit because everybody loves flowers. With the Shrumen Lumen, it took some convincing—many worried about associations with 'shrooms at Burning Man. This year, I made sure to test out the RadiaLumia concept with a few key members of the team, so I knew I had broad support. The buy-in worked, and has yielded our biggest team of volunteers by far.

The bottom line: Pressure-test your idea early—before you get too attached to it—so others will want to join up and help you realize the vision.

2. Stand on your own shoulders

Leading people to a grand vision is impressive only if you can actually pull it off. Ladder up from where you (and your team) actually are to where you want to go. And, acknowledge that it may not all happen at once.

Even before our Blumen Lumen, we built three shade structures for Burning Man. One was a disaster, but we learned from it. Now, we ensure that we are leveraging all we mastered the previous year to create the next installation. People are drawn to us because of our bold vision, but they stay with us because they know we have what it takes to make it happen.

The bottom line: Learn from previous outings that you can get over the mountain, so you can ask people to stretch and meet a new challenge.

Folding the origami pieces looks easier than it is.

3. Make it accessible

Part of the fun of leading a creative team is building camaraderie and community. But that can be constrained if a project requires too much specialized knowledge. Ensuring that part of the project is achievable by those without technical or unique skills is key to drawing a crowd when you need it.

On all of our projects, a big part of the effort is folding the plastic origami pieces. That requires pure manual labor, and anyone can be trained to do it, so we hosted all-hands-on-deck build parties every weekend leading up to Burning Man.

The bottom line: If you want to draw a crowd, create easy ways for people to participate without specialized expertise.

4. Leave room for individual passions

When people are motivated to do something, get out of their way and let them go for it. Given the freedom, teammates will create something together that’s more amazing than what a single creative leader could pull off. Set the vision, then let people self-select into groups to get it done.

With RadiaLumia, no one person knows everything that’s going on at every given moment, which is kind of scary but also amazing, because it’s coming together despite—or because—of that!

When we shared the vision at the kickoff meeting in February, there were a bunch of people who’d only just met. Within minutes, they were already troubleshooting light placement. By the end of the afternoon, they’d demonstrated a cool light show—on day one! We now have several self-directed teams that are loosely connected, but working independently.

The bottom line: If you create a compelling enough vision and rough guardrails, you’ll create the conditions for people to self-organize in a way that will surprise you.

5. Embrace ambiguity

The beauty of large-scale creative endeavors is in all of the unknowns. Try to remain flexible about who realizes the vision (and how).

Each year there are moments of extreme uncertainty. Weeks before installing the Shrumen Lumen at Burning Man, the origami kept getting stuck when we tried to pull the stems over the steel substructure that would anchor them to the ground. This was completely unexpected and took two weekends to debug. (Spoiler alert: We fixed it, but not without panic!). Troubleshooting is part of the process.

The bottom line: Expect the unexpected, and look for the beauty and lessons in it rather than the difficulty.

One of the most rewarding parts of FoldHaus is that while I may set the vision, once we pull off a communal project, the whole team feels immense ownership and pride. Being able to share both the process and the result is what makes the late nights and desert rainstorms worth it.

See RadiaLumia come to life at Burning Man 2018.

Also: We’d love to hear about your ambitious team efforts, at Burning Man or elsewhere, using the hashtag #MakerArt.

Special thanks to Amy Bonsall for her massive help crafting this story.

‍

We’d been working 16-hour shifts to install our art piece, a series of 10 larger-than-life, undulating, mechanical origami flowers. It was our first big build at Burning Man, and it took way longer than we expected to set up. After days, the flowers still weren’t working. Then came a torrential rainstorm.

Direct feedback that teaches rather than chides requires trust, and trust takes time.

Unlike humans, AI craves to know when it’s wrong. The entire premise behind machine learning is that it learns from failure or success. For example, we may feel that weather algorithms always get it wrong, but they can only improve by comparing their forecast to what actually happened. It’s therefore our job to tell the system when it’s wrong and acknowledge when it gets it right—but that requires a level of trust.

We can often tell if our friends or partners are unhappy with our actions, even if they don’t directly tell us. We can read their facial expressions, subtle changes in their tone of voice, and body language. Machines, however, don’t have that kind of intuition. Until we’re all walking around with some kind of lightweight EEG machine strapped to our head, machines will need to find proxies for how to sense human emotions, desires, and preferences. The obvious way to do this is to just to ask us, but that is manufacturing a transactional relationship, not a trusting one.

All I’m asking AI is for a little respect

A good example of an AI asking for trust rather than earning it can be found in the personal styling services that are popping up. They are doing some amazing things in integrating data science into every component of their business to drive growth. But my experience put me in the awkward position of feeling like I needed to explain myself and share sensitive personal details with someone I’d just met.

Signing up for the subscription service begins with a long and detailed survey about body type and style preferences. I spent a good 30 minutes thoughtfully answering myriad questions to create my style profile, and yet I returned everything that they sent me in the first box. None of it jived with my personal style, and only one thing fit.

I had put the time into creating a relationship with the AI—training it with my intimate information, such as how my body was shaped—and that process had given me a false sense of mutual understanding. If it had just taken a look at my Instagram profile and sent me a box, I would have given it permission to get it all wrong. But its surprisingly clunky and unnatural way of “sensing” my preferences for clothes set an unrealistically high expectation of what was to follow. Questions about slightly nuanced styles of plaid let me to believe that it could understand I like simple patterns—and then it sent me a shirt better fit for my grandfather.

By asking me to train a system before we had established a relationship of trust, the AI had breached it before it had even formed. When I’m in a trusting relationship, I can forgive someone for an error. But this felt different—like I’d been duped into a false sense of intimacy. I’m going to try one more box, but I have a strong hunch I’ll be cancelling my subscription soon.

AI is a relationship

While rare, there are a few AI with whom I’m in a trusting relationship of mutual respect. I relish my moments of training them because they respect my time and provide me with value as I train them—even when the AI is mostly wrong. I’ve come to truly value those moments when they tell me I’m wrong, because they do so in a way that deeply respects my position as a human and their position as a machine.

The format of the playlist is likewise brilliant: I don’t need to rate the songs but simply listen to them or skip them after I’ve heard a bit, and songs that I love get added to my own playlists or saved to my phone. Over time, it learns my preferences and can dish me up even better tunes. There’s no artificiality to training Spotify, and each and every time I train it, there’s value back to me as the user. Most importantly, Spotify has earned my trust through helping me discover a slew of new artists and, in doing so, earned its right to be wrong. (I don’t really love St. Vincent, but I understand why Spotify thinks I would.)

But we’re not the only one doing the teaching. When it comes to “training” humans, Waze and Google Maps are setting the standard. Their features are clearly based on a deep understanding of the common human emotional conditions that we endure during traffic: stress, anxiety, impatience, and a deep temptation to bail on our algorithmically recommended route in favor of our own “shortcuts” (which are almost never shorter).

Maybe the most brilliant bit of human-centered design inside of these navigation apps is how they highlight alternative routes and how much longer they’d take. Instead of having to blindly trust the route the AI has chosen, it considers its own possible fallibility and assures us by showing us the other routes we might be considering. Thinking of bypassing the highway? That’ll take another four minutes. Want to cut through that residential area? Here’s some construction of which you might not be aware.

In this way, Waze and Google Maps serve as a sort of angel on our shoulder, counseling us away from our inner traffic-hating demons. They’re not “AI-splaining” or chiding us for considering alternate routes, but rather giving us the necessary information to make the right decision. They’re rooted in a deep respect of human agency.

At IDEO, we prefer to think of AI as Augmented Intelligence rather than Artificial Intelligence. Taking a human-centered approach to building relationships between AI and humans compels us to meet humans on their terms, building relationships of trust and respect, and always remembering that intelligent systems must exist in service of humanity, not the other way around.

Beautiful, human-centered, human-AI relationships are about understanding human beings and our wonderful, weird intricacies and inconsistencies. They’re about designing the experience of AI around humanity, rather than the other way around. AI can only function well if it learns from its mistakes, and that means establishing trusting relationships with humans so they’ll feel comfortable saying when it’s wrong.

This article was originally published in Quartz Ideas.

Illustrations by Cassandra Fountaine.

‍

It’s awkward to correct a stranger when they’re wrong. How did you feel when Miguel from IT, whom you’ve only met once, told you that “learnings” wasn’t a real word? Or when Robin lectured you about the correct pronunciation of “macaron” at the office holiday party?

But, for the most part, that relationship is passive. And that begs the question, what if we could teach our machines more actively? Could we school them on very specific things that are important to us? That thought became the perfect setup to build a little experiment involving selfies.

A self-portrait or a selfie is actually more than just one picture. It’s one picture out of endless tries which all look pretty much the same ... except that they don’t. At least not to the person taking the self-portrait. That person’s individual aesthetic, packed with subtle and subjective nuances, must be captured at exactly the right moment. Adding to the challenge, it's hard to explain to someone else the way you squint your eyes when you nail your smile or how you purse your lips to look roguish.

Enter Brainchild: a machine learning–based camera that can be taught individual preferences and make them accessible as a function. You can train the camera to understand how you like to see yourself without going through 100 iterations. And most importantly, it allows you to share your sensibility with somebody else.

Brainchild is just a prototype for now, but I'm hoping others will be inspired to build on the idea.

Beautiful, baby, beautiful

Machines are a billion times faster in quantitative tasks than we are. For a long time, the problem was one of not understanding quality. But that has changed with the improvement of so-called deep learning techniques—a subset of AI or machine learning.

By teaching Brainchild how you would like to be portrayed, it can assess the quality of what it sees in real time, and give you haptic feedback when it perceives you looking your best, so you can simply take that one picture.

It also allows others taking a snapshot of you to see you as if through your own eyes. Taking portraits is a very intimate art. Rather than taking the human out of the loop, Brainchild augments this human-to-human interaction by making it more collaborative.

A feel-good feedback loop

The intervals between interacting with machine-learning products and experiencing their learnings are often very long. There are many technical challenges, but more accessibility and immediate feedback would help us teach more actively and develop a better intuition for them. In return, machines could learn better and become more personal.

Brainchild is designed to provide immediate feedback. You teach it how you like to be portrayed, instead of letting it guess, and every picture you take based on its haptic recommendation serves as feedback on how well it learned and how well you taught it. (Plus, it's private. Brainchild works offline and stores all of your information locally, so there's no danger of it being shared.)

Smile!

How does the Brainchild prototype work? There are countless technical parameters we could talk about, but in the most simple terms, it makes use of a so-called transfer learning process based on a fine-tuned convolutional neural network—a technique frequently used in computer vision.

Brainchild needs to be taught five fundamentals:

1. What a human face looks like
2. How to learn about new faces
3. How to isolate the face in a picture
4. The way the portrait subject looks normally
5. The way the portrait subject looks when they deem themselves looking their best

The first two points require huge amounts of coding, data, number crunching, and time. That’s what consumed most of my time (next to working on the wrong file for a day!).

The third point addresses the fact that when you take a portrait there is always something behind you. To focus only on you without distraction, Brainchild extrudes your face from the background. That is something it learns before you use it.

Then you become the teacher. In order to learn how you look your best, it needs to know how you look normally. So, you feed it a few examples of both, either by taking new snapshots, or by using pictures stored on your phone.

To help Brainchild differentiate between the two picture sets you simply twist the front plate, switching between two learning modes. The serious smiley means every picture you take will teach Brainchild how you look “normally”. Switching to the happy smiley means every picture you take will teach it what it looks like when you look your best. This is how you get to know each other.

As soon as the camera spots you in a way that matches what you taught it about looking good, it will gently vibrate and light up the camera’s trigger button. The vibration and brightness increase the closer your match with your own ideal.

Brainchild doesn’t snap the photo for you, it just signals you at the optimal moment to do so. You can also hand the camera to someone else to capture you in the way you like to be captured. And the more you teach it, the more accurate it becomes.

It might seem mundane to teach a camera how to take a better selfie, but the technology has already inspired one of our clients to rethink parts of a high precision medical device that could dramatically reduce treatment costs for patients, and another software engineer to work on a playful version of FaceID.

I myself am now working on a body pose add-on for fashion photography, and am curious to see all of the other ways we might actively teach our machines and help them augment us in more personal ways.

Instagram: @brain_children
Concept and Technology: Jochen Maria Weber
Visual Design: Tiffany Yuan
Industrial Design: Leo Marzolf
Special thanks: Tobias Toft

We teach our machines every day. Voice assistants learn how to talk to us based on what we say to them. Navigation apps guide us based on the routes we take. Without millions of teachers like you, our machines would be much dumber.

I help organizations cut through complexity by shaping how they show up, through brand strategy, identity, and storytelling.

Working at the intersection of brand and business, I bring an editorial eye and a bias toward making—turning ideas into tangible experiences that people can understand, trust, and believe in.

My work spans government, healthcare, financial services, media, and consumer goods, from launching Gen Z-focused ventures to building innovation labs and reimagining legacy brands. I relish moments of ambiguity and enjoy translating across teams, perspectives, and priorities to move ideas forward.

I started my career in editorial and digital design, shaping cross-platform experiences and identity systems in publishing and agency environments. That foundation still shapes how I work today: I’m detail-oriented, collaborative, and overreliant on the Oxford comma. I hold a BFA in Visual Communication from the School of the Art Institute of Chicago.

For 25 years, I've asked the same question: Who does this design exclude—and what are we going to do about it?

My work spans human-centered strategy, inclusive design, and qualitative research, with clients ranging from Microsoft, Google, and Verizon to the National Science Foundation, and San Francisco Unified School District.

Before IDEO, I taught elementary school in East Palo Alto and spent a decade doing design work with social service organizations—where I learned firsthand what it costs people when systems are built without them in mind.

I am currently writing User Error, a nonfiction book about digital access and the design decisions behind it. I live in Oakland, California.

‍

My expertise spans brand, communication, and product design across tech, education, the arts, and social impact.

I believe great work demands both high-level vision and obsessive attention to detail, and only happens through collaboration.

I bring an inclusive, hands-on approach and a deep understanding of business, which enables me to consistently deliver excellence while always asking why.

Before joining IDEO, I worked at the brand firms Pentagram and VSA Partners. I began my career as a designer, leading teams at the School of the Art Institute of Chicago and the Milwaukee Art Museum. I hold an MFA in graphic design from Maryland Institute College of Art and a BFA in communication design from the Milwaukee Institute of Art & Design, and have taught at some of the best design schools in the US.

I am responsible for IDEO’s long-term success in China and working with clients to use design as a tool to enable growth. Specifically, I have helped Chinese companies design holistic brand solutions through the development of their products, communication, services, and innovation teams, and I have helped multinationals expand their presence and influence in China.

I advise global leaders on developing China-led innovation and capabilities.

In over 15 years in IDEO Shanghai, I have worked on projects that use design to elevate the quality of the experience of healthcare products and services, streamline processes that increase productivity, create spaces and programs that promote and enable inclusive communities, and build next generation mobility solutions that are planet-positive.

Before joining IDEO, I worked at Philips Electronics and the Electrolux Group in Italy, the Netherlands, and Singapore on a number of breakthrough commercial products. I am a member of the Young President Organization in Shanghai.