Write the serving contract
Before deployment, define the request schema, feature freshness, response format, deadline, expected throughput, and behavior when a dependency fails. Include authentication and authorization, data handling, and the model and preprocessing versions. A model server may receive already-preprocessed features, so budget for feature lookup and transformations as well as inference. Choose batch when results can be generated before a known deadline; use online serving when the application needs a response during the request.
A checkout risk decision has a 200 ms end-to-end budget. Feature lookup takes 40 ms, networking 20 ms, inference 90 ms, and application processing 30 ms in a representative measurement. How much remains? Answer: 20 ms. Tail latencies cannot generally be added from individual percentiles to obtain the end-to-end percentile; measure the whole request under load.
Validate the artifact and environment
Record the registered model version, container image, preprocessing code, dependencies, and evaluation data. Test a representative request end to end, including missing features, malformed input, timeouts, authorization failures, and realistic concurrency. Compare offline and serving transformations on the same examples. Load-test the actual deployment shape and set capacity using measured latency and cost; autoscaling does not eliminate cold-start or sudden-spike constraints.
Create a release checklist for a classifier. Include one input contract test, one parity test, one security test, and one load test. Specify the version identifiers you would retain for rollback.
Choose the rollout and define gates
A canary sends a limited share of eligible traffic to the candidate before increasing exposure. Define comparable traffic assignment, observation windows, minimum sample sizes, and rollback thresholds in advance. Monitor error rate, timeouts, tail latency, resource saturation, prediction distributions, and business outcomes. When labels arrive late, operational health provides early evidence but does not prove predictive quality. Shadow traffic can compare outputs without using the candidate response for decisions; it still needs privacy controls and enough capacity.
Propose a 5%, 25%, then 100% rollout for a low-risk classifier. Write one immediate rollback rule for errors and one later quality review after labels arrive. Explain why five minutes of low traffic may be insufficient evidence.
Know the endpoint and rollback limits
A rollback routes traffic back to a known-good version and restores compatible preprocessing and feature dependencies. Practice it before an incident, and decide who can stop a rollout. Google Cloud supports multiple serving options with different networking and traffic-management capabilities. For example, the documented private services access endpoint does not itself support traffic splitting; separate endpoints and a suitable routing layer may be needed. Verify the current documentation for the exact endpoint type before adopting a rollout design.
A new candidate increases the endpoint error rate, while predictive labels take two weeks to mature. Decide what to do immediately, what to preserve for investigation, and how to compare quality later without exposing all users.
A private services access endpoint must gradually route traffic between two models. What is the appropriate first design check?
Keep exploring
- Machine learning on Google Cloud best practices ↗
- Private services access endpoint limitations ↗
- AI and ML reliability guidance ↗
Use the official certification page and its linked resources to validate your coverage. For guided labs, explore the Google Skills learning path. Revisit the practical exercise and explain your decision in your own words before moving on.