Assure the deployed system, not the model in isolation
A model card and benchmark profile are useful inputs, but an organisation deploys a system: model, prompts, retrieval, tools, permissions, workflow, users and controls. The same model can behave very differently across two operational settings.
Re-assurance should therefore focus on the behaviour and consequences of the assembled system. The central question is whether evidence supporting the previous deployment decision still applies after the change.
Define change triggers before the upgrade arrives
Teams make better decisions when reassessment criteria are part of the original release process. Otherwise, a vendor update becomes an urgent judgement call made under delivery pressure.
- A new foundation model or major model version.
- Material changes to prompts, tools, data sources, permissions or workflow position.
- A vendor change affecting retention, hosting, safety behaviour or service terms.
- Expansion into new users, decisions, data classes or operating regions.
- A significant incident, recurring near miss or drift in monitored behaviour.
Use a tiered response to change
Not every change requires a full repeat of every test. The response should reflect what changed, the uncertainty it introduces and the consequence if previous assumptions no longer hold.
A low-consequence wording adjustment may justify a focused regression check. A new model controlling tools in a consequential workflow may require renewed scenario testing, authority review, human-intervention testing and a fresh deployment decision.
The scope of re-assurance should follow the changed assumption, not the size of the software diff.
Keep a stable behavioural test set
A useful assurance set reflects the actual operating context: representative tasks, boundary cases, adversarial or ambiguous inputs, tool failures, missing information and scenarios where escalation is required.
Running a stable core of tests across versions makes changes visible. New tests should be added when incidents, user behaviour or emerging capabilities reveal assumptions the previous set did not cover.
Record the deployment decision
The final output is not simply a test score. Decision-makers need a concise record of the change, evidence reviewed, exceptions found, controls relied upon, residual uncertainty, approving owner and next review trigger.
That record creates continuity when models move quickly. It allows the organisation to explain why the system remained in service, which evidence supported that choice and what would cause the choice to be revisited.