Interpretability in Data-Centric ML #

Q1: Why do we need interpretable machine learning? #

• For debugging and validation of models.
• To allow human review and oversight of decisions.
• To improve usability by aligning models with human intuition, past experience, and values.

Q2: When is interpretability particularly important? #

• When the problem formulation is incomplete.
• When the model’s predictions have associated risks.
• When humans are involved in the decision-making loop.

Q3: What are interpretable features? #

• Features that are most useful, understandable, and meaningful to the user.

Q4: How can interpretable features help performance? #

• Lead to more efficient training.
• Improve model generalization.
• Reduce vulnerability to adversarial examples.
• The perceived interpretability-performance tradeoff is mostly a myth.

Q5: What qualities make features interpretable? #

• Readability
• Understandability
• Relevance
• Abstraction when necessary

Q6: How do we get interpretable features? #

• Involving users directly in the feature design process.
• Using interpretable feature transformations.
• Generating new interpretable features through crowd-sourcing and algorithms.

Q7: What are examples of methods for interpretable feature creation? #

• Collaborative feature engineering with domain experts.
• Flock: clustering crowd-generated feature descriptions.
• Ballet: allowing feature engineering with simple feedback loops.
• Pyreal: structured feature transformations for explanations.
• Mind the Gap Model (MGM): groups features using AND/OR logical structures.

Q8: What was observed in the Child Welfare case study? #

• Confusing or irrelevant features can hinder usability and trust.
• Clear, meaningful features helped screeners better interpret model recommendations.

Q9: What is the role of explanation algorithms in interpretability? #

• They help diagnose flawed features or data by revealing what the model actually uses.

Q10: What are the final conclusions about interpretable features? #

• ML models are only as interpretable as their features.
• Interpretable features are central for transparent, human-centered ML.
• Effective feature engineering must involve human collaboration, thoughtful transformations, and systematic generation methods.

References #

• Sibyl: Challenges of ML in Child Welfare (Zytek et al., 2021)
• Stop Explaining Black-Box Models (Rudin, 2019)
• Adversarial Examples Are Not Bugs (Ilyas et al., 2019)
• Flock: Crowd-Machine Learning Classifiers (Cheng & Bernstein, 2015)
• Ballet: Collaborative Feature Engineering (Smith et al., 2021)
• Pyreal Project
• Mind the Gap Model (Kim et al., 2015)