Sorelle Friedler

Assistant Professor of Computer Science

Sorelle Friedler is an Assistant Professor of Computer Science at Haverford College and an Affiliate at the Data & Society Research Institute. Her research focuses on the fairness and interpretability of machine learning algorithms, with applications from criminal justice to materials discovery.

Sorelle is a Co-Founder and Executive Committee Member of the ACM Conference on Fairness, Accountability, and Transparency (FAT*) as well as a former Program Committee Co-Chair of FAT* and FAT/ML. She has received a Mozilla grant, Fellowship, and NSF grant for her work on preventing discrimination in machine learning. Her work on this topic has been featured in IEEE Spectrum, Gizmodo, and NBC News and she has been interviewed about algorithmic fairness by the Guardian, Bloomberg, and NPR.

Sorelle is the recipient, along with chemistry professors Josh Schrier and Alex Norquist, of a DARPA contract and two NSF Grants to apply data mining techniques to materials chemistry data to speed up materials discovery, using interpretable machine learning techniques to inform scientific hypotheses. One paper on this work was featured on the cover of Nature and was covered by The Wall Street Journal and Scientific American.

Before Haverford, Sorelle was a software engineer at Alphabet (formerly Google), where she worked in the X lab and in search infrastructure. She holds a Ph.D. in Computer Science from the University of Maryland, College Park, and a B.A. from Swarthmore College.

CV (pdf) Grants Papers Press Teaching

Grants

Mozilla Responsible Computer Science Challenge (2019 - 2020): Responsible Problem Solving: Focusing on the societal consequences of design choices in data structures and algorithms. Suresh Venkatasubramanian, Sorelle Friedler, and Seny Kamara. $150,000 (Haverford portion: $29,524).

LinkedIn Data Access Award (2018 - 2020): Gaining access to hard-to-reach and disadvantaged populations via controlled interventions in the economic graph. Suresh Venkatasubramanian, danah boyd, and Sorelle Friedler. Non-monetary data access grant.

DARPA Synergistic Discovery and Design (SD2) (2018 - 2021): TA2+TA3: Discovering Reactions and Uncovering Mechanisms of Hybrid Organohalide Perovskite Formation. Joshua Schrier, Sorelle Friedler, and Alexander Norquist. $3,604,943.

NSF DMR-1709351 (2017 - 2020): CDS&E: D3SC: The Dark Reaction Project: A machine-learning approach to exploring structural diversity in solid state synthesis. Joshua Schrier, Sorelle Friedler, and Alexander Norquist. $645,288.

NSF IIS-1633387 (2016 - 2019): BIGDATA: Collaborative Research: F: Algorithmic Fairness: A Systemic and Foundational Treatment of Nondiscriminatory Data Mining. Suresh Venkatasubramanian, danah boyd, and Sorelle Friedler. $953,432 (Haverford portion: $172,742).

Knight News Challenge Prototype Fund (2016): Could your data discriminate? Sorelle Friedler, Wilneida Negron, Surya Mattu, Suresh Venkatasubramanian. $35,000.

Data & Society Research Institute Fellow (2015 - 2016): Preventing Discrimination in Machine Learning: from theory to law and policy. $10,000.

NSF DMR-1307801 (2013 - 2016): The Dark Reaction Project: a machine learning approach to materials discovery. Joshua Schrier, Alexander Norquist, and Sorelle Friedler. $299,998.

Papers

Journal Papers

Xiwen Jia, Oscar Huang, Allyson Lynch, Matthew Danielson, Immaculate Lang’at, Alexander Milder, Aaron Ruby, Hao Wang, Sorelle A. Friedler, Alexander J. Norquist, and Joshua Schrier. Anthropogenic biases in chemical reaction data hinder exploratory inorganic synthesis. Nature, 573: 251 - 255, Sept. 12, 2019. [link]

Harry Levin and Sorelle A. Friedler. Automated Congressional Redistricting. ACM Journal of Experimental Algorithmics, 24(1): 1-10, 2019. [PDF | link | code]

Philip Adler, Casey Falk, Sorelle A. Friedler, Tionney Nix, Gabriel Rybeck, Carlos Scheidegger, Brandon Smith, and Suresh Venkatasubramanian. Auditing Black-box Models for Indirect Influence. Knowledge and Information Systems, 54(1): 95-122, 2018. [PDF | link | code]

Paul Raccuglia, Katherine C. Elbert, Philip D. F. Adler, Casey Falk, Malia B. Wenny, Aurelio Mollo, Matthias Zeller, Sorelle A. Friedler, Joshua Schrier, and Alexander J. Norquist. Machine-learning-assisted materials discovery using failed experiments. Nature, 533: 73 - 76, May 5, 2016. [PDF | link | project site]

Sorelle A. Friedler and David M. Mount. A Sensor-Based Framework for Kinetic Data Compression. Computational Geometry: Theory and Applications, 48(3): 147 - 168, March 2015. (doi: 10.1016/j.comgeo.2014.09.002) [PDF | link]

Sorelle A. Friedler and David M. Mount. Approximation algorithm for the kinetic robust k-center problem. Computational Geometry: Theory and Applications, 2010. (doi: 10.1016/j.comgeo.2010.01.001). [PDF (preprint) | link]

Sorelle A. Friedler, Yee Lin Tan, Nir J. Peer, and Ben Shneiderman. Enabling teachers to explore grade patterns to identify individual needs and promote fairer student assessment. Computers & Education, 51(4):1467-1485, December 2008. [PDF (preprint) | link] [code and help videos]

Peer-reviewed Conference Proceedings

Charles Marx, Richard Phillips, Sorelle A. Friedler, Carlos Scheidegger, and Suresh Venkatasubramanian. Disentangling Influence: Using disentangled representations to audit model predictions. In Neural Information Processing Systems (NeurIPS), 2019. [preprint]

Benjamin Fish, Ashkan Bashardoust, danah boyd, Sorelle Friedler, Carlos Scheidegger and Suresh Venkatasubramanian. Gaps in Information Access in Social Networks. In The Web Conference (WWW), 2019. [PDF]

Mohsen Abbasi, Sorelle A. Friedler, Carlos Scheidegger, and Suresh Venkatasubramanian. Fairness in representation: Quantifying stereotyping as a representational harm. In SIAM International Conference on Data Mining (SDM), 2019. [PDF]

Sorelle A. Friedler, Carlos Scheidegger, Suresh Venkatasubramanian, Sonam Choudhary, Evan P. Hamilton, and Derek Roth. A comparative study of fairness-enhancing interventions in machine learning. In Proceedings of the Conference on Fairness, Accountability, and Transparency (FAT*), 2019. [PDF | code]

Andrew Selbst, danah boyd, Sorelle A. Friedler, Suresh Venkatasubramanian, and Janet A. Vertesi. Fairness and Abstraction in Sociotechnical Systems. In Proceedings of the Conference on Fairness, Accountability, and Transparency (FAT*), 2019. [PDF | link]

Danielle Ensign, Sorelle A. Friedler, Scott Neville, Carlos Scheidegger, Suresh Venkatasubramanian. Decision Making with Limited Feedback: Error bounds for Recidivism Prediction and Predictive Policing. In Algorithmic Learning Theory (ALT) 2018. [PDF | link]

Danielle Ensign, Sorelle A. Friedler, Scott Neville, Carlos Scheidegger and Suresh Venkatasubramanian. Runaway Feedback Loops in Predictive Policing. In Proceedings of the Conference on Fairness, Accountability, and Transparency (FAT*), 2018. [PDF | link]

Richard L. Phillips, Kyu Hyun Chang, and Sorelle A. Friedler. Interpretable Active Learning. In Proceedings of the Conference on Fairness, Accountability, and Transparency (FAT*), 2018. [PDF | link]

Philip Adler, Casey Falk, Sorelle A. Friedler, Gabriel Rybeck, Carlos Scheidegger, Brandon Smith, and Suresh Venkatasubramanian. Auditing Black-box Models for Indirect Influence. In Proceedings of the IEEE International Conference on Data Mining (ICDM), 2016. [PDF | code]

F. Betul Atalay, Sorelle A. Friedler, and Dianna Xu. Convex hull for probabilistic points. In Technical Papers of the 29th Conference on Graphics, Patterns and Images (SIBGRAPI '16), 2016. [PDF]

Michael Feldman, Sorelle A. Friedler, John Moeller, Carlos Scheidegger, and Suresh Venkatasubramanian. Certifying and Removing Disparate Impact. Proceedings of the 21st ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2015. [PDF | code]

Sorelle A. Friedler and David M. Mount. Spatio-temporal range searching over compressed kinetic sensor data. In Proc. of the European Symposium on Algorithms (ESA), pages 386-397, 2010. [PDF (preprint) | link] [TR]
     2nd Workshop on Massive Data Algorithmics, 2010 [PDF]
     Fall Workshop on Computational Geometry, 2009 [PDF]

Sorelle A. Friedler and David M. Mount. Compressing kinetic data from sensor networks. In Proc. of the 5th International Workshop on Algorithmic Aspects of Wireless Sensor Networks (AlgoSensors), pages 191 - 202, 2009. [PDF (preprint) | link] [TR]

Workshop Papers and Technical Reports

Dylan Slack, Sorelle Friedler and Emile Givental. Fairness Warnings. NeurIPS Workshop on Human-Centric Machine Learning, 2019. [link]

Dylan Slack, Sorelle Friedler and Emile Givental. Fair Meta-Learning: Learning How to Learn Fairly. NeurIPS Workshop on Human-Centric Machine Learning, 2019. [link]

Kadan Lottick, Silvia Susai, Sorelle Friedler, and Jonathan Wilson. Energy Usage Reports: Environmental awareness as part of algorithmic accountability. NeurIPS Workshop on Tackling Climate Change with Machine Learning, 2019.

Charles Marx, Richard Phillips, Sorelle A. Friedler, Carlos Scheidegger, and Suresh Venkatasubramanian. Disentangling Influence: Using disentangled representations to audit model predictions. arXiv:1906.08652, Jun. 20, 2019. [link]

Dylan Slack, Sorelle A. Friedler, Chitradeep Dutta Roy, and Carlos Scheidegger. Assessing the Local Interpretability of Machine Learning Models. NeurIPS Workshop on Human-Centric Machine Learning, 2019. [link]

Danielle Ensign, Sorelle A. Friedler, Scott Neville, Carlos Scheidegger and Suresh Venkatasubramanian. Runaway Feedback Loops in Predictive Policing. Presented as a talk at the Fairness, Accountability, and Transparency in Machine Learning Workshop, Aug. 14, 2017. [link]

Danielle Ensign, Sorelle Friedler, Scott Neville, Carlos Scheidegger and Suresh Venkatasubramanian. Decision Making with Limited Feedback: Error bounds for Recidivism Prediction and Predictive Policing. Presented as a poster at the Fairness, Accountability, and Transparency in Machine Learning Workshop, Aug. 14, 2017. [PDF]

Richard L. Phillips, Kyu Hyun Chang, and Sorelle A. Friedler. Interpretable Active Learning. Presented at the ICML Workshop on Human Interpretability in Machine Learning, Aug. 10, 2017.

Sorelle A. Friedler, Carlos Scheidegger, and Suresh Venkatasubramanian. On the (im)possibility of fairness. arXiv:1609.07236, Sept. 23, 2016. [link]

Nicholas Diakopoulos, Sorelle Friedler, Marcelo Arenas, Solon Barocas, Michael Hay, Bill Howe, HV Jagadish, Kris Unsworth, Arnaud Sahuguet, Suresh Venkatasubramanian, Christo Wilson, Cong Yu, and Bendert Zevenbergen. Principles for accountable algorithms and a social impact statement for algorithms. Dagstuhl working group write-up. July, 2016. [ PDF | link]

Ifeoma Ajunwa, Sorelle Friedler, Carlos E. Scheidegger, and Suresh Venkatasubramanian. Hiring by Algorithm: Predicting and Preventing Disparate Impact. Presented at the Yale Law School Information Society Project conference Unlocking the Black Box: The Promise and Limits of Algorithmic Accountability in the Professions, Apr. 2, 2016. [PDF]

Philip Adler, Casey Falk, Sorelle A. Friedler, Gabriel Rybeck, Carlos Scheidegger, Brandon Smith, Suresh Venkatasubramanian. Auditing Black-box Models by Obscuring Features. arXiv:1602.07043. [link]

Michael Feldman, Sorelle A. Friedler, John Moeller, Carlos Scheidegger, and Suresh Venkatasubramanian. Certifying and Removing Disparate Impact. Presented at the Fairness, Accountability, and Transparency in Machine Learning Workshop, Dec. 12, 2014. [link]

F. Betul Atalay, Sorelle A. Friedler, and Dianna Xu. Probabilistic Kinetic Data Structures. Presented at the Fall Workshop on Computational Geometry, Oct. 25, 2013. [PDF | link]

Sorelle A. Friedler and David M. Mount. Realistic compression of kinetic sensor data. Technical Report CS-TR-4959, University of Maryland, College Park, 2010. [PDF | TR]

Thesis

Sorelle A. Friedler. Geometric Algorithms for Objects in Motion. Dissertation committee: Prof. David Mount (chair), Prof. William Gasarch, Prof. Samir Khuller, Prof. Steven Selden, Prof. Amitabh Varshney. Defense date: July 30, 2010. [PDF] [presentation]

Patents

Mohammed Waleed Kadous, Isaac Richard Taylor, Cedric Dupont, Brian Patrick Williams, Sorelle Alaina Friedler. Permissions based on wireless network data. US 20130244684 A1. Publication date: Sep. 19, 2013.

Sorelle Alaina Friedler, Mohammed Waleed Kadous, Andrew Lookingbill. Position indication controls for device locations. US 20130131973 A1 (also WO 2013078125 A1). Publication date: May 23, 2013.

Press

Related to Machine-learning-assisted materials discovery using failed experiments:

Adam Marcus and Ivan Oransky. What scientists could learn from startups. The Week and STAT, May 12, 2016.

Daniela Hernandez. Why Machines Should Learn From Failures. The Wall Street Journal, May 6, 2016.

Jordana Cepelewicz. Lab Failures Turn to Gold in Search for New Materials. Scientific American, May 6, 2016.

Philip Ball. Computer gleans chemical insight from lab notebook failures. Nature News, May 4, 2016.

Related to Certifying and removing disparate impact:

Lauren J. Young. Computer Scientists Find Bias in Algorithms. IEEE Spectrum, August 21, 2015.

Julianne Pepitone. Can Resume-Reviewing Software Be As Biased As Human Hiring Managers? NBC News, August 17, 2015.

Kiona Smith-Strickland. Computer Programs Can Be as Biased as Humans. Gizmodo, August 16, 2015.

Background on Algorithmic Fairness:

Sam Levin. A beauty contest was judged by AI and the robots didn't like dark skin. The Guardian, September 8, 2016.

David Ingold and Spencer Soper. Amazon Doesn't Consider the Race of Its Customers. Should It? Bloomberg, April 21, 2016.

Rose Eveleth. The Inherent Bias of Facial Recognition. Motherboard, March 21, 2016.

Laura Sydell. Can Computer Programs be Racist and Sexist? NPR, March 15, 2016.

Lauren Kirchner. When big data becomes bad data. ProPublica, September 2, 2015.

Hal Hodson. No one in control: The algorithms that run our lives. New Scientist, February 4, 2015.

Regularly Taught Classes

Haverford

CS 104: Topics in Introductory Programming
CS 106: Introduction to Data Structures
CS 340: Analysis of Algorithms
CS 399: Senior Thesis

Past Classes

Haverford

CS 101: Fluency with Information Technology
CS 105: Introduction to Computer Science
CS 207: Data Science and Visualization
CS 395: Mobile Development for Social Change

University of Maryland, College Park

Design and Analysis of Computer Algorithms, Summer 2009
Organization of Programming Languages, Summer 2007
Computer Organization (TA), Spring 2006
Introduction to Low-Level Programming Concepts (TA), Fall 2005