While AI can offer immense potential to improve patient outcomes and streamline medical processes, system flaws can pose risks to ethics, security and data accuracy.
Dr Gao Yujia, assistant group CTO of the National University Health System, urged caution in AI implementation in healthcare in the keynote session, “Unveiling the Dark Side of AI,” at HIMSS24 APAC.
“While AI has undoubtedly enhanced patient care, it’s crucial to acknowledge the potential pitfalls,” he emphasised.
A primary concern is the risk of bias in AI algorithms. Dr Gao cited an incident during COVID-19 where an AI algorithm, trained on biased data, disproportionately down-prioritised people of colour who had the same conditions as other patients.
It is critical, Dr Gao stressed, that AI systems are trained on diverse and representative datasets.
“Similar to a ‘garbage in, garbage out’ scenario, AI can only be as accurate as the data that is fed to it. Some AI algorithms fail to perform as expected because they were not trained on relevant data or were overfitted to specific datasets.”
Beyond data quality, Dr Gao also raised concerns about the ethical implications of AI in healthcare.
“We use AI to help us organise our thoughts, but that same intent can be used maliciously. For example, AI-generated deepfakes have been used to impersonate healthcare workers for ransomware attacks. Fraudulent medical research is also a possibility because of how convincing AI-generated content can be.”
Asked if all is “doom and gloom” for AI in healthcare, Dr Gao disagreed.
“A more immersive and instantaneous digital world is inevitable, but what protects us [from AI risks] is maintaining a balance between human agency and AI’s utility.”
According to Dr Gao, ethical frameworks, transparency and human oversight are key factors in harnessing the power of AI while mitigating its potential harms.
He advised healthcare professionals to neither fear AI nor be fearless of AI.
“We should take a step back to understand, appreciate, understand and plan how to integrate AI into healthcare in our society.”