PAPIs Latam 2019

This talk covers two ML problems Stripe faced (and solved!) in building its credit card fraud detection system:

- Addressing class imbalance in training data: legitimate transactions greatly outnumber fraudulent ones, and models trained on imbalanced data often have low precision
- Performing counterfactual evaluation to measure performance in production and obtain unbiased training data when the ML system itself is constantly changing outcomes (if we block a transaction we think is fraudulent, we don’t actually know if we got it right—we have no way to observe the transaction’s ultimate outcome)