Featured Product
This Week in Quality Digest Live
Innovation Features
Rupa Mahanti
Understanding data decay
AI monitors real-time data from the physical system
Katie Rapp
The future of manufacturing is about making processes more efficient
Ray Hein
It’s time to lean in to smart technology to help close the skills gap
Adam Zewe
The chip could enable lower-cost devices that perform better and use less hardware

More Features

Innovation News
Ultrasonic flaw detector now has B/C scan capability, improved connectivity, and an app to aid inspection
Tapping tooz for AR/VR competence center
Provides opportunities to deepen leadership capabilities
ASI Construction partners with end users to deliver solutions to production operations
New technology can reduce pollution, bolster energy storage
KCP25C grade with KENGold coating sets new standard for wear and productivity in steel turning
Resistant to high-pressure environments, and their 3/8-in. diameter size fits tight spaces
Algorithms protect data created and transmitted by IoT and other small electronics

More News

Maria Guadalupe


Turn the Office Into a Lab

Randomized controlled trials help firms find out what’s going wrong (and right) in their organizations

Published: Wednesday, March 4, 2020 - 13:03

When the 2019 Nobel Prize in Economics was announced, it was an exceptional moment because in addition to celebrating contributions to economic science, a methodology was also recognized. The laureates Abhijit Banerjee, Esther Duflo, and Michael Kremer are known for their work in randomized controlled trials (RCTs), which develop evidence-based recommendations for public policy and organizations in the fight against global poverty.

Work with RCTs has led to the identification and implementation of impactful poverty-reduction policies in areas such as skills training and agriculture. But as a methodology, RCTs are not only for the developing world. They have been used for decades in pharmaceutical companies’ clinical trials, and in this age of data, they hold enormous promise to identify business solutions and support evidence-based management.

This methodology can challenge widely held assumptions about what gets results. Finding out what we think works but in fact is a waste of resources is one more reason to adopt RCTs more systematically. This year’s Nobel winners concluded after many trials that microfinance is not the solution to poverty that many had thought and hoped, and that in many cases, people with access to micro-credit (the treatment group) did not fare better than those who didn’t (the control group).

The purpose of the RCT Lab at INSEAD is to help bring this methodology to firms by connecting companies to INSEAD faculty and supporting the joint research (funded by the Abu Dhabi Department of Education and Knowledge). Recently, we combined two RCT conferences into a single workshop. It was a collaboration of Field Days: Experiments Outside the Laboratory and INSEAD Randomized Controlled Trials Days. Robert Dur of Erasmus University at Rotterdam and I were pleased to host leading economists and other academics conducting field experiments, who came together to present and discuss their work.

RCTs: From job search to food and health

Researchers at the conference covered a wide range of topics.

A number of papers focused on job search and matching firms to employees. For example, INSEAD’s Dylan Glover discussed his joint work on how the French public-employment service stimulated firms’ labor demand by reducing their recruitment costs. Starting in 2015, the researchers analyzed a program that included new free services for small and medium enterprises (SMEs), notably the preselection of job candidates. They found this program increased the hiring of registered job seekers by 10 percent, leading to 18 additional workdays created by treated firms, on average.

Instead of the traditional focus on policies that assist job seekers, this study shows that firm-based, active, labor-market policies may have significant added value in weak labor markets. In a country of relatively high unemployment like France, it is key to find solutions that work, such as this one.

Also in the French context, Pierre Chandon (INSEAD) and other researchers used a large RCT in 60 French supermarkets to establish if a simplified nutrition label placed on the front of packages in four food categories could lead consumers to purchase healthier foods over a three-month period. Evaluating the effectiveness of four predesigned labels, the one with the largest impact translated into a 2.6 percent improvement in the average nutritional quality of consumers’ shopping baskets. “Because food preferences are deeply ingrained, and because most people eat for pleasure, not nutrition, simply providing nutrition information can only lead to small improvements on what people eat,” he said. “That’s why randomized controlled trials on a large scale and over a significant amount of time are indispensable in obtaining precise and reliable measures of the effects of nutrition information.”

Selection and delegation in organizations

Oriana Bandiera, one of our keynote speakers and this year’s winner of the Yrjo Jahnsson Award (which often points the way toward the Nobel), presented her work on public-sector workers and the importance of employee selection for effective bureaucracies: “A well-functioning bureaucracy requires having the right people and retaining them,” she says.

Motivation in employment—i.e., does the worker do the job for love or for cash—is a problem that Bandiera has considered in several articles over the years. One paper, about Zambia’s Ministry of Health recruiting nurses for rural positions, shows how dual motivation is a possibility.

The ministry recognized that there were two types of motivation to join a career in nursing: those who are interested in a good wage, and those with a “good heart.” They worried that if they emphasized career goals in their recruiting, they would attract self-interested nurses rather than pro-social, caring nurses. But they needed more than caring nurses; they needed skilled caring nurses. So this gave rise to a project where Bandiera and her co-authors tested what the effect of emphasizing the career aspects of the job did to the pool of applicants, and followed these applicants over time as they were hired by the organization.

The findings show that, in line with common wisdom, offering career incentives encourages selfish applicants, but at the same time it encourages talented applicants. As the latter are also more pro-social and more likely to be selected, we end up with better workers.

In an ongoing experiment with Michael Best, Adnan Khan, and Andrea Prat, Bandiera studies how the allocation of authority in a bureaucracy affects corruption: Should one grant more decision power to those lower down in the organization? The field setting for this study was the public procurement service in Pakistan. After procurement officers find the necessary chairs, paper, etc., these purchases must be approved by the accountant general, an onerous task involving a lot of documentation.

To study how the allocation of authority impacts layers of an organization, the researchers compared the effect of monetary incentives with that of a change in decision power. During the first year, buyers were granted incentives (bonuses) based on their overall purchasing savings. In the second year, they were granted autonomy: freedom from filling in all the forms and abiding by certain rules.

The researchers found that granting authority at a lower level generated more savings. Prices paid by procurement officers went down 9 percent when the buyers had autonomy, and down only 3 percent when they had incentives to find good bargains. This challenges the conventional wisdom that money is always the biggest motivator: Firms can improve performance by redesigning their hierarchies. In addition, this experiment has had substantial impact on the organization as a whole. The total savings from the better design that the researchers helped identify are $70 million per year.

RCTs and big data: research at Uber

Incentives—a little push to buy something or spend more money on a service—also featured in the second keynote. John List of the University of Chicago is also the former chief economist at Uber. He now holds the same role at Lyft. While at Uber, List and his team had access to reams of excellent, nationwide data. He shared results about studies conducted around apologies (and incentives) and tipping.

In a first experiment, Uber tested the effect of apologies for bad rides on the company’s client-retention rates. Looking at 3 percent of the worst rides (late pickups, too much time spent in the car, etc.), Uber representatives emailed three different kinds of apologies to riders. One with a direct apology, one said they were “idiots” for causing the rough journey, and another promised the riders that they would never experience such a terrible ride again. In addition to the apology emails, they tried another set of emails with the same types of apology combined with a voucher for money off their next journey, the only kind of compensation for disgruntled riders. Refunding fares is too expensive for the firm. What the researchers found is that an apology on its own won’t get the rider back in an Uber the way an incentive (voucher) does. But List said that a well-timed promotion—like after a bad ride—is better than just any promotion.

Another incentive, tipping, was something Uber drivers were after for a long time but Travis Kalanick was opposed to on principle. After the #DeleteUber campaign ratcheted up, Uber’s founder finally agreed to a trial. During a two-month period in 2017, an experiment that generated data from 40 million trips began. List and his co-authors found that 59 percent of riders never tip, and only 1 percent always tips. Women drivers receive 10 to 12 percent more than men drivers, regardless of the rider’s gender. Driver earnings, however, were flat, and the trip quality didn’t improve with tipping, List explained.

Data as the new oil

During the two-day workshop, we saw a range of questions and solutions arising from RCTs. At this moment, the methodology is in focus not only due to the recent Nobel prize, but also because we live in the age of data.

As List pointed out, “Data is now the most valuable resource in the world. It’s no longer oil. But much like oil needs a refinery, data also needs a refiner, and that’s where academics come in. They lend expertise, they understand how to think about causality, they understand how to generate new data to make causal claims and to figure out underlying mechanisms.”

First published Jan. 24, 2020, on INSEAD’s Knowledge blog.


About The Author

Maria Guadalupe’s picture

Maria Guadalupe

Maria Guadalupe is an associate professor in the Economics and Political Science department at INSEAD, the academic director of the INSEAD Randomized Control Trials (RCT) Lab, and a member of the European Competitiveness Initiative at the school.