You Retrained the Model. What Exactly Is Approved for Production?

The production approval should cover a versioned deployment bundle—not just the retrained model file. That bundle ties the exact artifact to its preprocessing, evaluation evidence, decision threshold, target runtime, monitoring signals and rollback owner. If one of those pieces changes, the approved behavior may change even when the weights do not.

This matters in manufacturing because a prediction is only one step in a physical decision. A vision score can trigger manual review, reject a part or stop a line. The threshold, camera crop, runtime and fallback route therefore belong to the release decision.

Why the model version is not enough

The open textbook Introduction to Machine Learning Systems describes deployment as a systems-engineering handoff: models, dependencies and configuration should be treated as one deployable unit. It also separates three interfaces that can fail independently—data to model, model to infrastructure, and production to monitoring.

A model registry helps identify versions and source runs. It does not, by itself, prove that production is using the approved image transform, threshold or hardware path. The Model Cards paper makes the same distinction from a documentation angle: model version, intended use, preprocessing, evaluation data and decision thresholds all matter when judging suitability.

Put five gates around the release

1. Exact artifact: record the model version, checksum, code commit and source experiment or training run.
2. Reproducible context: fix the data version, evaluation split and preprocessing contract, including crop, resize, normalization and feature definitions.
3. Operating policy: approve the decision threshold, abstention rule, human-review route and action authority. A threshold is a business-control setting, not merely a model parameter.
4. Target runtime: test the container, libraries, drivers, hardware and serving mode under the actual latency and throughput requirement. The site’s batch-versus-real-time guide shows why the decision deadline belongs in this choice.
5. Operating controls: name the monitored signals, alert trigger, responsible owner and deployable previous version.

Worked example: the weights stayed the same

Consider a fictional seal-inspection model. On a synthetic acceptance set of 1,000 packs containing 40 confirmed defects, threshold 0.62 catches 36 defects and routes 90 packs to review. Threshold 0.78 catches 30 defects and routes 45 packs to review. The model weights are identical, but the operating consequence is not.

Now add a tighter crop and an optimized runtime. Either change can alter scores or latency. The release record should therefore identify the approved threshold, transform and runtime together, then attach the acceptance results and review capacity used to approve them. The numbers above are teaching data, not a benchmark.

After release, infrastructure uptime is not enough. A model can remain available while its relationship to the process changes. The concept-drift lesson explains why outcome-linked monitoring matters, while the feature-importance guide shows why a diagnostic change alone should not trigger retraining.

What this gate does—and does not prove

The five-gate bundle is an editorial operating template, not a universal standard or certification. It improves traceability and makes a rollback decision executable. It does not guarantee safety, accuracy, regulatory compliance or business value. Higher-risk workflows may need independent validation, security review, formal change control and additional domain-specific evidence.

NIST AI RMF 1.0 supports the underlying control logic: it calls for documented intended use and human oversight, a decision on whether deployment should proceed, assigned responsibility to disengage systems with inconsistent outcomes, and post-deployment monitoring that includes override, incident response, recovery and change management.

Three takeaways

• Approve a reproducible deployment bundle, not an isolated model file.
• Treat preprocessing, threshold and runtime changes as behavior changes that require evidence.
• Do not release until monitoring, rollback triggers and ownership are explicit.

Sources

• Vijay Janapa Reddi and contributors, Introduction to Machine Learning Systems, Volume I, “Machine Learning Operations”, open textbook chapter, accessed October 19, 2026.
• Margaret Mitchell et al., “Model Cards for Model Reporting”, 2019 conference paper; arXiv version first submitted October 10, 2018.
• National Institute of Standards and Technology, Artificial Intelligence Risk Management Framework (AI RMF 1.0), NIST AI 100-1, January 2023.
• MLflow, “ML Model Registry” documentation, accessed October 19, 2026.

Next action

Take the model currently awaiting production approval and write one version identifier beside each of the five gates. If a gate has no evidence or named owner, hold the canary until that gap is closed and re-tested as part of the same bundle.