Could an AI doctor be less biased than a human one?

Last week, our Symptom Checker was accused of gender bias for showing different matching conditions for an identical set of symptoms in a male vs a female test case. This blog explains why that seemingly odd result is in fact correct. It also discusses the problem of bias in medicine and what we're doing to help root it out at Babylon.

What is gender bias?

Gender bias is ‘prejudice in action or treatment against a person on the basis of their sex’ and ‘discrimination on the basis of gender'¹. This bias affects women across the world.

Gender bias may be unintentional, but it very much exists and is harmful. Also, gender bias doesn’t only apply to women. It applies to all of us - whether male, female, transgender or gender nonconforming.

Not all gender differences are down to bias

Last week Babylon was accused of gender bias when our Symptom Checker said the most probable cause of a 59-year-old female smoker’s short and uncomplicated history of central chest pain (with a background of previous panic attacks) was - a panic attack. Meanwhile for a male with the same symptoms and characteristics it suggested that there might be something wrong with his heart.

First of all, our Symptom Checker is absolutely not a diagnostic tool and we are very clear about this. Second, it’s not designed to highlight an unlikely and worst-case scenario. That would make it both inaccurate and inappropriate. The purpose of our Symptom Checker is to provide medical information about conditions which match the entered symptoms.

We know that it looked bad - ‘There she goes, being all hysterical again…Have a cup of tea, dear”, as she fades away silently in the corner… But we can explain it. Bear with me.

Women get a poor deal in healthcare

For many years, women have been prejudiced against when it comes to cardiac care. They’re less likely to be offered the right treatment after a heart attack and more likely to die as a result of that^2,3. Women’s complaints of symptoms such as pain are frequently put down to emotional rather than physical causes and, for many health conditions, men are investigated and treated more extensively than women by doctors ⁴.

One particular study showed that if a woman were to collapse in public due to a heart attack, she’d be less likely to receive CPR from passersby than a male⁵. Oh yes and...older women are less likely to be admitted to intensive care for life-saving treatment than older men with the same severity of illness⁶.

Whether this is because heart attacks are incorrectly believed to be a “male thing”, because pain in women is frequently thought of as psychological in origin, or because women are regarded as the “lesser sex” or even because (in the case of avoiding CPR) – “those darn breasts just get in the way”, gender bias is an unfair obstacle to receiving good healthcare.

We live in a world that’s roughly 50% female (and in Babylon’s AI department two-thirds of our doctors are female). It’s not really our style to minimize the health concerns of women. So why did the Symptom Checker appear to act in a gender biased way?

The stats and facts

Our Symptom Checker works on the basis of probability. It makes calculations based on lots of epidemiological data from a huge number of research studies. It then comes up with hundreds of possible matching conditions, from which it displays those which most closely match the symptoms entered.

The risk of coronary heart disease (CHD) in women is well known to be different from that of men. Women, on average, develop CHD 15 years later than men⁷. So, a woman aged 59 has a risk of a 44 year old man, all other things being equal.

When it comes to anxiety and depression, studies show that women are almost twice as likely as men to suffer from anxiety disorders⁸ and more likely than men to have common mental health problems such as depression.⁹

Zebras or horses?

Frequently, as GPs in the UK, we are reminded, “If you hear hoofbeats, don’t think zebras. It’s probably horses.”

Unlike a human doctor, AI won’t be swayed by the most recent ‘similar’ three cases of chest pain it saw in men that week. It won’t repeatedly think back to the one-off case that resulted in a heart attack without the person exhibiting the usual symptoms and as a result become hypervigilant, suggesting everyone queues up outside A&E. Basically, it is far less likely to shout “zebras!”, unless things are also looking a little black, white or stripy too.

It also has no opinions on sex and gender. It doesn’t look at the data it receives through a veil of prejudice. In many ways, you could argue that an ‘AI doctor’ is the only type of doctor that isn’t clouded by the unintentional force that is gender bias…or is it?

Stamping out gender bias

Of course, AI is not perfect. Both it and we are constantly learning. AI is as great as the data that goes into it and despite the fact our data scientists, researchers, epidemiologists and doctors are all specially trained for this role – sometimes the research that provides medical data can exhibit gender bias.

That said, we’re super careful and aware of this. And as a result, we have a team that’s working specially to address a wide variety of bias found in healthcare research.

The great thing about our Symptom Checker is that we can update its decision-making according to new research, testing and feedback. This is an incredible advantage and allows us to make quick and effective improvements should we come across any research bias, whether it be gender related or otherwise.

Yes, we’ve got plenty of work to do, but we’re stubbornly optimistic that we can make some headway in stamping out gender bias and go on to lead the way for access to prejudice-free healthcare worldwide.

When it comes to hearing feedback and concerns about our Symptom Checker we're all ears. If there's something you'd like to flag, please fill out this form and our team will look into it right away and get back to you with any updates.

References

1. Alspach, J. (2012). Is There Gender Bias in Critical Care? Critical Care Nurse, 32(6), pp.8-14.

2. Alabas, O., Gale, C., Hall, M., Rutherford, M., Szummer, K., Lawesson, S., Alfredsson, J., Lindahl, B. and Jernberg, T. (2017). Sex Differences in Treatments, Relative Survival, and Excess Mortality Following Acute Myocardial Infarction: National Cohort Study Using the SWEDEHEART Registry. Journal of the American Heart Association, 6(12).

3. Jarman, A., Mumma, B., Perman, S., Kotini-Shah, P. and McGregor, A. (2019). When the Female Heart Stops: Sex and Gender Differences in Out-of-Hospital Cardiac Arrest Epidemiology and Resuscitation. Clinical Therapeutics, 41(6), pp.1013-1019.

4. Hamberg, K. (2008). Gender Bias in Medicine. Women's Health, 4(3), pp.237-243.

5. Blom, M., Oving, I., Berdowski, J., van Valkengoed, I., Bardai, A. and Tan, H. (2019). Women have lower chances than men to be resuscitated and survive out-of-hospital cardiac arrest. European Heart Journal.

6. Fowler RA, Sabur N, Li P et al (2007). Sex- and age-based differences in the delivery and outcomes of critical care. CMAJ 177, 1513–1519.

7. Bhf.org.uk. (2019). Women have heart attacks too. [online] Available at: https://www.bhf.org.uk/informa... [Accessed 16 Sep. 2019].

8. Remes, O., Brayne, C., van der Linde, R. and Lafortune, L. (2016). A systematic review of reviews on the prevalence of anxiety disorders in adult populations. Brain and Behavior, 6(7), p.e00497.

9. Webarchive.nationalarchives.gov.uk. (2014). Mental Health & Wellbeing in England. [online] Available at: https://webarchive.nationalarc... [Accessed 16 Sep. 2019].