Separate signals from conclusions
Data drift means an input distribution has changed. Training-serving skew means the data or processing used for serving differs from training. Concept drift concerns a change in the relationship between inputs and outcomes. Monitor operational reliability alongside model quality, and use delayed labels when available to check whether a shift actually affects performance.
A retailer launches in a new region and input distributions change. What additional evidence would you collect before deciding to retrain?
Evaluate generative systems continuously
Define a representative evaluation set and a rubric before iterating on prompts, retrieval, or model versions. Track grounding, correctness, safety, and task completion. Automated evaluators can help scale review, but calibrate them against human judgments. Include adversarial and unanswerable requests, and test again when the retrieved content or prompt changes.
Design a regression set for a policy assistant, including an outdated document, an ambiguous request, and a malicious instruction inside a retrieved document.
Build layered protections
Restrict data access, minimize sensitive information, validate tool arguments, and monitor suspicious activity. Treat model output and retrieved text as untrusted input to downstream systems. A safety filter is one layer, not a complete security boundary. Review quality across relevant user groups and create an escalation path for harmful or uncertain outcomes.
Map the path of a sensitive support request through your system. Identify where access checks, redaction, logging controls, and human review belong.
The distribution of an input feature changes, but labels have not arrived yet. What can you conclude?
Keep exploring
Use the official certification page and its linked resources to validate your coverage. For guided labs, explore the Google Skills learning path. Revisit the practical exercise and explain your decision in your own words before moving on.