Notice: Function _load_textdomain_just_in_time was called incorrectly. Translation loading for the acf domain was triggered too early. This is usually an indicator for some code in the plugin or theme running too early. Translations should be loaded at the init action or later. Please see Debugging in WordPress for more information. (This message was added in version 6.7.0.) in /opt/bitnami/wordpress/wp-includes/functions.php on line 6131

Notice: Function _load_textdomain_just_in_time was called incorrectly. Translation loading for the filebird domain was triggered too early. This is usually an indicator for some code in the plugin or theme running too early. Translations should be loaded at the init action or later. Please see Debugging in WordPress for more information. (This message was added in version 6.7.0.) in /opt/bitnami/wordpress/wp-includes/functions.php on line 6131

Notice: Function acf_get_value was called incorrectly. Advanced Custom Fields - We've detected one or more calls to retrieve ACF field values before ACF has been initialized. This is not supported and can result in malformed or missing data. Learn how to fix this. Please see Debugging in WordPress for more information. (This message was added in version 5.11.1.) in /opt/bitnami/wordpress/wp-includes/functions.php on line 6131

Deprecated: preg_replace(): Passing null to parameter #3 ($subject) of type array|string is deprecated in /opt/bitnami/wordpress/wp-includes/kses.php on line 2018

CLIP Colloquium: How can we enable LLM auditing?

Home    /     Events

Research Talks/Events

Date/Time: Wednesday, February 18, 2026 11:00 am - 12:00 pm

Location: IRB 5105

Contact: Wei Ai

Abstract: Oversight and auditing of AI systems is becoming increasingly difficult as people use systems in a wide variety of ways, with instructions expressed in natural language prompts. We can no longer use readily quantifiable metrics like accuracy or statistical parity to understand model performance and potential impacts. Instead, we need ways of conducting open-ended analyses of models and usage data that do not infringe on user privacy. In this talk, I will discuss ways we are working towards these goals, beginning with an in-depth analysis of LLM usage in a specific domain: AI for querying astronomy literature. While manual analysis of usage data and follow-up interviews with astronomers offer an in-depth look at how astronomers interacted with an LLM-powered system, manual evaluation does not scale to the large volume of usage data in other contexts. Thus, I will next discuss methods for automated inductive coding, which offer more scalability, and finally, leveraging synthetic data to enable increased oversight of model usage and development without compromising privacy.

Bio: Anjalie Field is an Assistant Professor in the Computer Science Department at Johns Hopkins University. She is also affiliated with the Center for Language and Speech Processing (CLSP) and the Data Science and AI Institute. Her research focuses on the ethics and social science aspects of natural language processing, which includes developing models to address societal issues like discrimination and propaganda, as well as critically assessing and improving ethics in AI pipelines. Her work has been published in NLP and interdisciplinary venues, like ACL and PNAS, and in 2024 she was named an AI2050 Early Career Fellow by Schmidt Futures. Prior to joining JHU, she was a postdoctoral researcher at Stanford, and she completed her PhD at the Language Technologies Institute at Carnegie Mellon University.

Speaker(s): Anjalie Fields