Wendy J. Nilsen
Deputy Division Director at NSF
Title: Next in Smart Health
Abstract: As technology dramatically changes much of our world, much of the processes in healthcare remain reliant on manual analysis and human judgement. While this non-automated approach is often justified in terms of being based on best practice, it ignores the risk of having providers with a range of expertise and abilities using ever increasing amounts of data to address life-threatening issues. A non-automated approach also does not address the staffing shortages that have only become more pronounced since the pandemic. To address these issues and optimize processes will require new technologies that bring insights into large multimodal data sources, but also that produce results that are valid, reliable and trustworthy. This in turn requires teams of researchers who can cross the cultural and language boundaries that arise between biomedical, healthcare, computing and engineering researchers , as well as build trust between these diverse groups. This talk will focus on some of the recent advances in the field and address ways that we can move these areas forward faster.
Curators' Distinguished Professor, AAAS Fellow, AIMBE Fellow, LAS, IDSI, IPG
Department of Electrical Engineering and Computer Science, University of Missouriasd
Title: Prompt-based learning for biomedical images and text
Abstract: Foundation models, trained on large-scale data of images and natural languages, offer unprecedented opportunities for a wide range of applications. The potential of these models is further magnified when combined with prompt-based learning, allowing for the achievement of state-of-the-art (SOTA) performance even with a small number of labeled data. This talk focuses on the biomedical applications of two foundation models: ChatGPT and the Segment Anything Model (SAM). As the volume of the literature continues to grow exponentially, manual curation methods cannot extract the embedded knowledge efficiently. In response, we developed a pathway curation pipeline that synergizes image understanding and text mining techniques for deciphering biological knowledge. This pipeline employs SAM, contrastive learning, and Siamese networks to identify key attributes of pathway entities and their relationships. The integration of ChatGPT's predictive capabilities for gene interactions has proven useful in enhancing the extraction of pathway information. To optimize ChatGPT's responses, a novel iterative prompt refinement strategy was applied, in which the efficacy of these prompts was evaluated using metrics such as F1 score, precision, and recall, and subsequently, the evaluation results were fed into ChatGPT to suggest better prompts. The prompts were further refined using Tree-of-Thought iterations. We also applied prompt-based learning for SAM-based protein identification from cryo-Electron Microscopy (cryo-EM) images. The outcomes of our studies underscore the potential utilities of prompt-based learning for efficient biomedical data analyses and predictions.
Fullgraf Professor, Washington University in St. Louis
Founding Director, AIM Institute, Washington University in St. Louis
Editor-in-Chief, ACM Transactions on Cyber-Physical Systems
Title: Personalized Predictions of Clinical Outcomes and Treatment Response with Wearables and Machine Learning
Abstract: Driven by growing adoption of wearable devices and advancements in artificial intelligence, Internet of Medical Things (IoMT) have emerged as new clinical instruments for precision medicine. Wearable devices enable unobtrusive monitoring of patients in their daily lives. To realize their potential in precision medicine, we need to develop machine learning models capable of extracting reliable clinical information from noisy and incomplete wearable data. Furthermore, the ML approaches need to scale effectively across a wide range of sample sizes. They should provide robust predictions in the presence of moderate amounts of data, while improving predictive power when large amounts of data become available. Finally, to support precision medicine, ML models need to make personalized prediction of not only the clinical outcomes of a patient but also their response to a specific treatment so that the treatment can be tailored for the patient. This talk will present our recent efforts to tackle these challenges in three clinical studies using Fitbit wristbands. 1) We built a robust feature engineering and ML pipeline tailored for wearable studies with limited sample sizes. We demonstrated the effectiveness of the pipeline in predicting post-operative complications and hospital readmissions in a prospective clinical trial of patients undergoing pancreatic surgery. 2) We developed WearNet, an end-to-end deep learning model for detecting mental disorders using Fitbit data. WearNet effectively enhanced predictive performance by exploiting a large public dataset including 8,996 participants and 1,247 diagnosed with mental disorders. 3) We explored multi-task ML to predict individualized treatment response to depression therapy based on wearable data collected from a randomized controlled trial (RCT) for patients undergoing behavioral therapy. Results from the clinical studies demonstrated the effectiveness of ML in predicting clinical outcomes in the presence of noise and missing data collected by wearables. Furthermore, ML provides a promising approach to predict individualized treatment response by leveraging data collected in RCTs. We will conclude the talk by highlighting the opportunities and directions in advancing IoMT as vital instruments for precision medicine.