Interpretability of machine learning models


In this research, we investigate the effect of cognitive biases on human understanding of machine learning models, focusing on inductively learnt rules. The interim results are presented in:

Short summary

Cognitive biases that demonstrated in our crowdsroucing experiments with association rules include base rate neglect (Kahneman and Tversky, 1973) and insensitivity to sample size (Tversky and Kahneman, 1974). These biases make the user focus on the confidence of the rule and neglect its support. As follows from our review of 20 cognitive biases possibly affecting interpretability of rules, number of debiasing techniques have been proposed in psychology.

Some of these, such as frequency formats (Gigerenzer and Hoffrage, 1995), require only changes in user interfaces that present machine learning results, others imply introduction of "interpretability biases" into learning algorithms. A commonly adopted assumption is that shorter models are more interpretable. We review evidence for and against the use of the Occam's razor principle as an optimization criterion in machine learning algorithms.

With the QCBA algorithm, we attempt to improve the interpretability of models generated by CBA by reducing the size of the generated models both in tems of the rule count and the number of rules.