When predictive models mislead careful teams
Most harmful forecast errors do not look wild. They look tidy, confident, and slightly more convenient than reality.
Careful teams still get misled when the model answers a different question than the decision requires. A churn score optimised for recall may flood account managers with false alarms. A demand model trained on promotions may understate baseline need when the calendar goes quiet.
Proxy targets that drift
If your predictive decision system optimises for clicks while the business needs retained margin, the model will “succeed” while the organisation loses. Revisit the target whenever incentives or product mix shift — not only when accuracy metrics dip.
Leakage dressed as accuracy
Features that include post-decision information produce beautiful backtests and useless live performance. In class we run a deliberate “leak hunt” on sample notebooks so learners feel the embarrassment in a safe room rather than in a board pack.
Override culture without feedback
Human overrides are healthy when they are logged. Unlogged overrides teach the organisation nothing and slowly train people to ignore the model entirely. Capture the reason in the same place as the forecast, then review overrides monthly for patterns.
Business analytics for predictive decision systems is as much about governance of trust as it is about algorithms. Teach both, or the careful team will still walk into avoidable surprises.