Glossary · Technology

Precision and recall

Precision is the share of cases a model flags that are truly positive; recall is the share of truly positive cases the model flags. Improving one usually costs the other.

Field: Technology
Examined in: CBA-AIP, CBA-DAP
Learning objectives: 2

Where the CBA Standards examine it

Artificial Intelligence ProfessionalCBA-AIP

3.1Calculate accuracy, precision, recall, F1 and error rates from a small confusion matrix or result table.Model Development and Evaluation

Data Analytics ProfessionalCBA-DAP

6.4Calculate and interpret basic evaluation measures, including forecast error measures and classification accuracy, precision and recall from a small confusion matrix.Introduction to Predictive Methods

In the specimen paper

CBA-AIP · 01AI and Machine Learning Foundations

Kestrel Bank in Kenya reviews mobile money transfers flagged by its model. In one week the model flagged 400 transfers, of which 80 were later confirmed fraudulent, and a further 120 confirmed fraudulent transfers were never flagged. What are precision and recall for that week?

Answer

Precision 20%, recall 40%.

Why that is the answer

Precision and recall answer two different questions and each carries its own denominator. Precision asks what share of the work you created was worth doing, so it divides the 80 true positives by the 400 transfers flagged, giving 20%. Recall asks what share of the fraud actually present you found, so its denominator is the 200 confirmed frauds in the week, 80 caught and 120 missed, giving 40%. Name the denominator before you divide, because every wrong option here is a denominator that was never named.

The CBA-AIP specimen paper

Confusion matrixTechnology: A table comparing a classification model’s predictions with actual outcomes, showing true and false positives and negatives.
ClassificationTechnology: A machine learning task that assigns inputs to categories, such as flagging a transaction as fraudulent or not.