"What is model drift in machine learning"

"Model drift is the gradual degradation of an AI model's accuracy over time as the real-world data it encounters shifts away from the data it was trained on. A fraud detection model trained in 2022 may miss new fraud patterns in 2025 — without drift monitoring, you won't know until customers complain."

What Is Model Drift and How Do You Catch It Before It Costs You

Model drift is when your AI model gets worse over time — not because the model changed, but because the world did. The data your model sees in production gradually diverges from the data it was trained on, and its predictions become less accurate. Slowly, silently, and expensively.

If you have a machine learning model in production — for fraud detection, demand forecasting, customer churn prediction, content recommendation — drift is happening. The question is whether you’re monitoring for it.

How It Works

AI models learn patterns from historical data. A fraud detection model trained on 2023 transaction data learned what fraud looked like in 2023. But fraud tactics evolve. Customer behavior changes. Market conditions shift. The patterns the model learned become less representative of what it actually encounters.

There are two types of drift:

Data drift (covariate shift). The distribution of input data changes. Maybe your customer base shifted demographics, or a new product line brought in different purchase patterns, or seasonal patterns deviated from historical norms. The model is seeing inputs that look different from what it trained on.

Concept drift. The relationship between inputs and outcomes changes. What constituted “good” customer behavior in 2023 might look different in 2025. The features haven’t changed, but what they predict has.

Both types are dangerous. Both are invisible without monitoring.

When a Business Should Care

Any time you have a model making decisions that affect revenue, customer experience, or risk:

Fraud detection. Fraud evolves constantly. A model that catches 95% of fraud at deployment might catch 70% six months later as attackers adapt their tactics. The gap costs you money every day it goes undetected.

Demand forecasting. Supply chain models trained on pre-pandemic data famously failed in 2020. That’s an extreme example, but subtler shifts happen constantly — a competitor enters the market, a product goes viral on social media, consumer preferences evolve.

Customer churn prediction. The signals that predict churn change as your product evolves, your pricing changes, and your customer base matures. A churn model that doesn’t account for these shifts will miss at-risk customers and flag healthy ones.

Credit and risk scoring. Economic conditions change. A credit model trained during stable economic times may perform poorly during a downturn — right when accurate risk assessment matters most.

What to Watch Out For

Monitor input distributions, not just accuracy. By the time accuracy drops noticeably, drift has been happening for a while. Track the statistical distribution of your input features over time. When they start shifting significantly from training data distributions, investigate — even if accuracy metrics look fine so far.

Set up automated alerts. Don’t rely on manual reviews to catch drift. Tools like Evidently, WhyLabs, and Fiddler can continuously monitor for distribution shifts and alert your team when thresholds are crossed. If you’re using a managed ML platform (SageMaker, Vertex AI), drift monitoring is typically available as a feature.

Plan your retraining cadence. Most production models need periodic retraining — the question is how often. Some models need weekly retraining (high-frequency trading, real-time recommendations), others quarterly (customer segmentation, demand planning). Your monitoring data will tell you the right cadence.

Don’t confuse model drift with bad data. Sometimes what looks like drift is actually a data pipeline issue — a broken ETL, a schema change, or missing features. Before concluding that your model has drifted, verify that the data feeding it is clean and complete.

LLM applications drift too. If you’re using a large language model through an API, the model provider can update the model without notice. Your prompts were optimized for a specific model version. When the model changes, your outputs may degrade. This is a form of drift that’s entirely outside your control — which is why LLMOps monitoring matters.

The Verdict

Model drift is inevitable. Every model in production is drifting — the only variable is speed and impact. The companies that get value from AI long-term aren’t the ones that build the best models. They’re the ones that monitor and maintain their models systematically.

If you have a model in production and you can’t answer the question “how has this model’s accuracy changed in the last 90 days,” that’s your first action item.