AN AI chatbot could help students pass an exam to qualify as a doctor, experts claim.
ChatGPT is an online robot that responds to questions like a human and has already been used by more than 1million people since its launch last November.
GettyResearchers found ChatGPT is able to pass the US Medical Licensing Exam, a test taken by medical students before they can practise[/caption]
Now, researchers found it is also able to pass the US Medical Licensing Exam, a test taken by students before they can practise.
The test usually takes years of study but ChatGPT scored around the 60 per cent pass mark and could be interacted with “like a colleague”, the team said.
Study author Dr Tiffany Kung, of Harvard University, said the bot’s performance “marks a notable milestone” in how developed AI could become.
The study, published in PLOS Digital Health, tested ChatGPT on 350 questions from three exams from the June 2022 USMLE.
The bot scored between 52 per cent and 75 per cent across the three exams.
It achieved a higher grade than PubMedGPT, a rival AI model that is trained exclusively on biomedical literature.
Dr Kung, who also works for tech start-up AnsibleHealth, said: “Reaching the passing score for this notoriously difficult expert exam marks a notable milestone in clinical AI maturation.”
She added: “ChatGPT contributed substantially to the writing of our manuscript.
“We interacted with ChatGPT much like a colleague, asking it to synthesise, simplify, and offer counterpoints to drafts in progress.
“All of the co-authors valued ChatGPT’s input.”
Dr Stuart Armstrong, Co-Founder and Chief Researcher of Aligned AI, said: “This is an impressive performance, and we should expect to see more such successes in AI in the future.”
But some experts believe the bot’s success in the exams says more about the tests themselves than they do the technology’s medical capabilities.
Professor Peter Bannister of the Institution of Engineering and Technology, said: “ChatGPT demonstrates an impressive ability to generate logical content in numerous settings.
“These results serve to highlight the limitations of written tests as the only way of assessing performance in complex and multi-disciplinary professions such as medicine.
“More generally this research underlines the need to base technology solutions on the full scope of the challenge.
“In this case [the tech needs to] provide comprehensive, in-person clinical care to patients from a wide range of populations.”
When asked by our Sun reporter to diagnose a lump, ChatGPT sensibly declined and told us to visit a real doctor.
The bot said: “It’s important to keep in mind that it’s never a good idea to self-diagnose, as it may be a serious condition that requires medical attention.”
It comes as experts predict AI’s could soon dominate medical advice being dished out online and even replace human doctors.
Google has developed a similar software called Med-PaLM, which learns from medical text and the conversations it has with users.
The technology performed the same as real doctors when giving medical advice, independent experts say.
Dr Simon McCallum, of Victoria University of Wellington, New Zealand, said: “Society is about to change.
“Instead of warning about the hypochondria of randomly searching the internet for symptoms, we may soon get our medical advice from Doctor Google or Nurse Bing.”