Would you trust a computer with your life?

I was speaking at the Digital Health Forum in London this week, with a younger and more digitally native audience than I’m usually in front of.

Lots of good questions, but one of the most interesting debates was on the future for AI, artificial intelligence, in medicine.  I’d be a fool to speculate on how the scene will look in ten, or even five years time.  But let’s ask the question now, would you trust a computer with your life?

When I talk to GPs about online access, the most frequent fear is “How will this affect our insurance?”.  Yet one of the most common questions is, “How does this deal with red flag symptoms?”

If you think about it, the two are inextricably linked.  A computer which sounds an alarm in any form is offering an implied diagnosis of some sort.  It has crossed a line and is now a medical device.  Regulations and yes, insurance implications follow.

The patient and the supplier then have to trust that the computer will always act on “red flags”.  It leads to an algorithm which becomes so conservative that humans have to be employed to filter its decisions.  This is the route taken by NHS 111, which the LMC conference in May labelled “a barrier between patients and their care”, calling for it to be scrapped.  31% of ambulance call outs in England are now generated by this algorithm.

I’ve found one of the most helpful guides in the debate is Nicholas Carr’s “The Glass Cage”.  Looking at automation in several spheres including medicine, he points out the dangers of relying on a technology which displays a frightening lack of common sense.  This is not to be a Luddite, but show that we have choices between human and machine centred automation.  “The goal is to divide roles and responsibilities in a way that not only capitalises on the computer’s speed and precision, but also keeps workers engaged, active and alert”.

With this as the design intent we can get the computer to do what it’s good at, in the case of askmyGP taking a history - to the point where a clinician can take over and take decisions.

My view is that technology must be the servant, but must help us with the presenting problem:  too much demand for too few GPs.  Not even HEE believes that a net 5,000 increase in GPs is possible.  We have to enable our existing GPs to be more productive, to save time, to give the right care to the right patients.

Dr Sara Cowell


PS  Feedback on the webinar, “Understanding Demand, the key to better service and lower stress” has been amazing and sometimes humbling.  One GP wrote, “a great insight and vision of Primary Care in the next few years and a potential solution to the GP workforce crisis.”

If you would like a link to the recorded webinar please let me know.

Planning a meeting for your locality, federation, board or CCG?  Perhaps I can come and speak in person.

One response to “Would you trust a computer with your life?”

  1. Good points Harry. I’m now reading Nicholas Carr’s book on your recommendation. Very relevant for us at Operon.

Leave a Reply