Think twice before turning to Dr AI

Metro North Health Digital Metro North Director Dr Jason Brown

In 2025, more than half of Australians used generative AI platforms in their daily lives, with many using tools like ChatGPT and Google Gemini to self-diagnose medical conditions.

While there are many benefits to using AI, it is never a replacement for professional medical advice and there are some key considerations people must be aware of before turning to Dr AI.

Metro North Chief Digital Health Officer Dr Jason Brown said bias is one of the key risks when using AI to self-diagnose health conditions.

“AI models are trained on historic data, which may under or over present different race and demographic groups,” he said.

“The answers people are receiving can therefore be skewed and inaccurate, and these incorrect answers will continue to be propagated on a large scale due to limited or non-existent human oversight,” he said.

“There is also a lack of regulation of AI platforms and most services do not go through standard TGA/FDA processes, mainly due to a regulation lag where innovation outpaces regulatory frameworks.”

For many years, Google has been the go-to online platform for people to source health advice, but with the rise of AI, Dr Brown said there are some considerations people need to make.

“Google encourages critical thinking due to a mix of results that requires people to evaluate and cross-reference them,” he said.

“Conversely, AI often delivers a confident, authoritative response which mimics an experienced professional.

“People are more likely to act on AI suggestions without verification, potentially resulting in delays to treatment or harmful self-treatment.”

Additionally, people should use caution if choosing to upload medical test results to an AI platform.

“Once data has been uploaded to AI, it can be assumed that it is no longer private,” Dr Brown said.

“The metadata contained in results may be used to expose personal data to potentially bad cyber actors, putting individuals at risk of having their identify stolen.

“I tend to say, never upload anything unless you would be happy to have it displayed on a public billboard.”

With all things considered, Dr Brown said with the right knowledge, AI can be a valuable tool in the health space.

“AI can be a great place to start if you are experiencing health concerns. It can provide education on non-urgent symptoms and help you prepare for appointments by providing examples of questions to ask your doctor or health professional,” he said.

“Just ensure you are using reputable tools and are aware of the limitations and bias.”