By: HAI Stanford University
Date: July 2, 2021
Clinical drug trials compare a treatment with a placebo and aim to determine the best course of action for patients. Given enough participants, such randomized control trials are the gold standard for determining causality: If the group receiving the drug improves more than the group receiving the placebo, it’s safe to assume that the better outcome is the result of the drug. With the knowledge gained from the experiment, drug makers can confidently roll out a drug to thousands or even millions of patients.
But this traditional approach to conducting experiments has a major downside: The group receiving the less beneficial treatment inevitably loses out.
The stakes are quite different in a drug trial than when testing versions of a company webpage or advertisement, but the essential problem of any A/B testing is the same. “We gain knowledge at the expense of this opportunity cost,” explains Mohsen Bayati, an associate professor at the Stanford Graduate School of Business. And the opportunity cost only increases with the number of different variants being tested — as, for example, in a chemotherapy drug trial, where each combination of drugs requires its own treatment group.