Verify the measurement first
Before changing the model, verify the target, positive class, metric, split, and evaluation code. Check that labels are correct and mature, transformations use only training statistics, and examples or entities have not leaked across splits. Compare against a simple baseline on the same population. A high score can be a symptom of leakage; a low score may reflect a labeling or evaluation mismatch rather than a weak algorithm.
A fraud model scores almost perfectly offline. One input is the chargeback outcome recorded after the transaction. Explain why the feature is suspicious and how to rebuild the evaluation using only information available at prediction time.
Read training and validation curves carefully
When training loss continues to improve while validation loss rises, overfitting is one plausible explanation. Check the split and distribution before deciding: a genuine time shift or data-quality issue can also produce a gap. When both losses remain poor, investigate model capacity, features, optimization, label noise, and the baseline. Very unstable or exploding loss warrants checks for non-finite values, outliers, and learning-rate or numerical issues. These patterns suggest hypotheses; they are not a unique diagnosis.
Training loss drops from 0.50 to 0.10 while validation loss drops to 0.30 and then rises to 0.45. Suggest two explanations and a controlled experiment for each. Explain why the final test set should not be used to pick an epoch.
Run one informative experiment at a time
Record a hypothesis, expected signal, cost, and comparison before changing a feature, regularization setting, model family, or training duration. Use the same validation cohort and metric where possible. Repeat close comparisons when randomness is material, and inspect performance across relevant slices. Early stopping can select a point based on validation behavior, but repeated experimentation can overfit the validation set itself; preserve an untouched test set for a final assessment.
Propose an experiment to test whether regularization improves a model that fits training data much better than validation data. State the fixed baseline, what changes, the decision metric, and the limits of the conclusion.
Trace offline-to-production differences
If offline performance is good but production quality is poor, compare the serving population, feature availability and transformations, missingness, model version, decision threshold, and label collection. Verify the same raw example through both paths where privacy permits. Investigate operational failures and feedback loops as well as input drift. An input shift is a warning signal; measure outcome quality when representative labels mature before concluding that retraining is the remedy.
A feature is measured in dollars during training but cents during online serving. Describe a parity test that would expose the mismatch, an immediate mitigation, and a regression check for future releases.
Training loss decreases while validation loss rises after several epochs. What is the best initial interpretation?
Keep exploring
Use the official certification page and its linked resources to validate your coverage. For guided labs, explore the Google Skills learning path. Revisit the practical exercise and explain your decision in your own words before moving on.