Sara Kingsley is a PhD Researcher at Carnegie Mellon University’s School of Computer Science, where she is advised by Dr. Beibei Li and Dr. Jeffrey P. Bigham. She is also an AI Security Researcher at CMU's Software Engineering Institute.
Sara is an expert in designing quantitative computational social science, econometric or data science methods to operationalize and measure various safety and security risks of transformer (e.g. generative AI) and machine learning models, especially AI/ML models that are embedded into various information Systems, including social media and search engines, and that perform critical functions of workplaces and labor markets. More specifically, she is an expert in defining and operationalizing constructs (e.g. an abstract idea or concept) for different risks in order to measure their potential impact on AI systems and the economy. She operationalizes the measurement of these risk constructs by transferring econometric and data science methods to the analysis of AI system data. To collect or generate AI system data, she uses social science tools like surveys, and also builds software and machine learning systems, including transformer models, to automate or augment the measurement of risk constructs in AI evaluations. Her dissertation work focuses on designing evaluations of Embedded AI Information Systems to identify and measure the risks of AI for workplaces and labor markets, focusing on the context of job advertising and job search technologies specifically.
During her Ph.D., Sara held multiple AI safety and security roles, including as a member of OpenAI's Red Team Network, and as a software engineer and expert on Meta's AI Foundation model safety team. As a summer research intern for the Berkman Klein Center at Harvard Law School, she also helped organize the first Generative AI red teaming competition at Defcon. Later, she built on this experience as a data science consultant for helping Humane Intelligence, helping HI design its first bias bounty competition, and by building software to automatically evaluate the competition submissions and to generate synthetic evaluation datasets.
Before starting her PhD, Sara previously was appointed to serve in the U.S. Department of Labor as a Legislative Officer for the Office of Congressional and Intergovernmental Affairs (OCIA). In this role, Sara was responsible for communications between the U.S. Department of Labor and the U.S. Senate and U.S. House of Representatives. She also designed an award-winning information management system for her agency, resulting in recognition and an award from the U.S. Secretary of Labor. After her time serving in the federal government, Sara worked for Microsoft Corporation where she conducted research on online labor markets, resulting in peer-reviewed articles, and for the Chief Economist, and also where she conducted economics research on the impact of worldwide subsea and terrestrial fiberoptic, Wi-Fi and experimental network deployments utilizing various spectrum-sharing or software-defined radio technology; in this role, she also conducted research on mergers & acquisitions, and the future of work with respect to the impact of AI on the labor market.
Beyond AI evaluation research and labor economics, Sara is an avid train travel enthusiast, skiing hobbyist, avid book reader, and former learner of various human languages including Farsi, Arabic, Spanish, and some German (she can still read a little in these languages but alas is not very proficient anymore!)