Calculate error on a shared holdout
For actual values 10, 20, and 30 and predictions 12, 16, and 36, the signed errors using prediction minus actual are 2, -4, and 6. The absolute errors are 2, 4, and 6, so mean absolute error (MAE) is 4 units. The squared errors are 4, 16, and 36, so mean squared error (MSE) is 56/3, about 18.67 squared units. Root mean squared error (RMSE) is about 4.32 units. Squaring gives larger errors more influence.
Recalculate all three metrics. If the last prediction changes from 36 to 40, calculate the new MAE and MSE: 16/3 and 40, respectively. Explain why the large error changes MSE more sharply.
Choose a metric for the decision
MAE is expressed in the target’s units and describes average absolute error. RMSE is also in target units but emphasizes large misses more strongly. Neither tells you whether errors are usually overestimates or underestimates, nor whether the operational cost is asymmetric. Percentage errors can be unstable or undefined when actual values are zero or near zero. Define the business loss explicitly when underprediction and overprediction have different consequences.
A hospital staffing forecast underestimates demand more often than it overestimates it. Propose a metric or cost calculation that distinguishes those directions, and explain why MAE alone cannot do so.
Inspect residuals and slices
Plot signed residuals against prediction time, predicted value, important features, and relevant groups. Patterns may reveal missing seasonality, bias, changing variance, or data problems. An aggregate score can conceal a severe error for a small but important segment. Check sample size and uncertainty, compare with a simple baseline, and investigate whether the evaluation population matches the intended use. Avoid claiming a feature causes an error pattern without further evidence.
A model has similar overall MAE in two regions but consistently underpredicts one region and overpredicts the other. What additional summaries would expose this pattern?
Interpret R² and product metrics carefully
R² compares squared errors with a mean-based reference on the evaluated data. It can be negative on held-out data when predictions are worse than that reference; a high R² is not proof that errors meet a business tolerance. For supported regression models, BigQuery ML’s ML.EVALUATE reports MAE, MSE, median absolute error, R², and other metrics. Provide an appropriate held-out dataset and review the function’s model-specific behavior.
A model has high R² because the target varies widely, but its typical error exceeds the staffing tolerance. Explain which additional metric and decision constraint you would present to the service owner.
Actual values are 10, 20, and 30; predictions are 12, 16, and 36. What is MAE?
Keep exploring
Use the official certification page and its linked resources to validate your coverage. For guided labs, explore the Google Skills learning path. Revisit the practical exercise and explain your decision in your own words before moving on.