By: Benedict Evans
September 15, 2019
- Machine learning finds patterns in data. ‘AI Bias’ means that it might find the wrong patterns – a system for spotting skin cancer might be paying more attention to whether the photo was taken in a doctor’s office. ML doesn’t ‘understand’ anything – it just looks for patterns in numbers, and if the sample data isn’t representative, the output won’t be either. Meanwhile, the mechanics of ML might make this hard to spot.
- The most obvious and immediately concerning place that this issue can come up is in human diversity, and there are plenty of reasons why data about people might come with embedded biases. But it’s misleading, or incomplete, to think that this is only about people – exactly the same issues will come up if you’re trying to spot a flood in a warehouse or a failing gas turbine. One system might be biased around different skin pigmentation, and another might be biased against Siemens sensors.
- Such issues are not new or unique to machine learning – all complex organizations make bad assumptions and it’s always hard to work out how a decision was taken. The answer is to build tools and processes to check, and to educate the users – make sure people don’t just ‘do what the AI says’. Machine learning is much better at doing certain things than people, just as a dog is much better at finding drugs than people, but you wouldn’t convict someone on a dog’s evidence. And dogs are much more intelligent than any machine learning.