Skip to main content

COGnition Seminar Series

Navigating Complexity: Evaluating AI Performance on Real-World Biomedical Tasks

Thursday, October 10, 2024

  • Zhiyong Lu, PhD FACMI, FIAHSI

    Dr. Zhiyong Lu is Senior Investigator with tenure at the NIH Intramural Research Program, leading research in biomedical text and image processing, information retrieval, and AI/machine learning. In his role as Deputy Director for Literature Search at National Center of Biotechnology Information (NCBI), Dr. Lu oversees the overall R&D efforts to improve literature search and information access in resources like PubMed and LitCovid that are used by millions worldwide on a daily basis. Additionally, Dr. Lu holds an Adjunct Professor position with the Department of Computer Science at the University of Illinois Urbana-Champaign (UIUC).

Towards Conversational Diagnostic AI – Challenges in evaluating AMIE, an AI agent for diagnostic dialog

May 2, 2024

  • Anil Palepu

    Anil Palepu (Google Research) presents AMIE (Articulate Medical Intelligence Explorer), a large language model (LLM)--based AI system optimized for diagnostic dialogue. Anil will describe AMIE, its remarkable diagnostic accuracy and human interaction skills, and the challenges in evaluating its performance. The extensive testing included 149 case scenarios from clinical providers, 20 PCPs for comparison with AMIE, and evaluations by specialist physicians and patient actors.