Machine Learning Practice Test

Section A: Multiple Choice

• A1 - B. High bias, low variance means the model is too simple, so it underfits and does poorly on both training and test data.
• A2 - C. The test set must be untouched during training and model selection so it gives an unbiased estimate of generalization to new data.
• A3 - B. Each of the k iterations trains on k-1 folds and validates on the held-out fold; scores are then averaged, giving more stable estimates than a single split.
• A4 - B. Recall = TP / (TP + FN), the fraction of actual positives that the model correctly identifies. (A is precision; D is accuracy.)
• A5 - B. F1 is the harmonic mean of precision and recall, 2·(P·R)/(P+R), which penalizes a large imbalance between the two.
• A6 - B. ROC plots TPR vs. FPR over all thresholds; AUC summarizes performance across thresholds (0.5 is random, 1.0 is perfect).
• A7 - B. Low training error with much higher validation error is the classic overfitting signature; both-high means underfitting.
• A8 - A. L1's penalty has a constant-magnitude gradient that can push coefficients to exactly zero, yielding sparse, feature-selecting models; L2 shrinks but rarely zeroes them.
• A9 - B. Parameters move opposite to the gradient (the descent direction) by a step proportional to the learning rate, reducing the loss.
• A10 - A. Trees split on thresholds and naturally model interactions and non-linearities; a plain linear model assumes a linear relationship (in log-odds for logistic regression).
• A11 - B. k-means is unsupervised clustering (no labels); k-NN is supervised and predicts from the labels of the nearest training points.
• A12 - B. Fit the scaler on training data only and reuse those statistics on validation/test; fitting on the full data or the test set leaks test information.

B1. Choosing a metric for an imbalanced problem

Accuracy is misleading because the negative (non-fraud) class dominates: predicting "not fraud" for everything scores 99.5% while catching zero fraud, so accuracy rewards ignoring the rare class entirely. Better metrics focus on the positive class: precision and recall (and their combination, the F1 score), and the precision-recall curve / area under it, which is more informative than ROC-AUC under heavy imbalance. The right balance depends on costs: if missed fraud is expensive, prioritize recall; if false alarms are costly, weight precision. Confusion-matrix-based metrics make the rare-class errors explicit in a way accuracy hides.

B2. Diagnosing and fixing overfitting

The large gap between high training accuracy (99%) and much lower validation accuracy (72%) indicates overfitting: the model has memorized training-specific patterns and noise that do not generalize. Concrete fixes include: (1) reduce model complexity, e.g., lower the tree depth, reduce the number of estimators, or increase the minimum samples per leaf; (2) add regularization (e.g., L1/L2 penalties, or boosting parameters such as a smaller learning rate with early stopping); (3) gather more training data or use data augmentation; (4) remove noisy or leaky features and apply feature selection; and (5) use cross-validation to tune hyperparameters rather than fitting to a single split. Early stopping on the validation loss is especially effective for boosted models.

B3. Picking an algorithm for a task

For medium-sized tabular data with mixed numeric and categorical features, a gradient-boosted decision tree ensemble (e.g., XGBoost, LightGBM, or scikit-learn's HistGradientBoosting) is a strong default: it handles non-linearities and feature interactions, is relatively robust to feature scaling and monotonic transforms, and provides feature-importance estimates to indicate which features matter. A regularized logistic regression is a reasonable, more interpretable baseline if a linear decision boundary is adequate and you encode categoricals appropriately. Tree ensembles typically outperform deep neural networks on small-to-medium tabular datasets like this, so they are the pragmatic first choice.

B4. Interpreting a confusion matrix

With TP = 80, FN = 20, FP = 100, TN = 800: Precision = TP / (TP + FP) = 80 / 180 ≈ 0.44; Recall = TP / (TP + FN) = 80 / 100 = 0.80; Accuracy = (TP + TN) / 1000 = 880 / 1000 = 0.88. The model catches 80% of true disease cases (decent recall) but only 44% of its positive predictions are correct, so it raises many false alarms (100 FP). In a medical screening context the 20 false negatives are the most dangerous errors, since they are missed diseases; the 100 false positives mainly cause extra follow-up testing. Whether this tradeoff is acceptable depends on the cost of a missed diagnosis versus the cost of unnecessary follow-up, and the threshold could be lowered to raise recall further at the expense of precision.

B5. Explaining a data leakage scenario

This is target leakage. The feature "number of follow-up visits in the 60 days after discharge" is recorded after the prediction point and is a direct consequence of the outcome being predicted: patients who are readmitted typically have more post-discharge contact. The model therefore "sees the future" and uses information that would not be available at the moment a real prediction must be made (at or before discharge). Validation accuracy looks near-perfect because the feature is essentially a proxy for the label, but the model would fail in production where that information does not yet exist. The fix is to only use features available strictly before the prediction time and to define a clear cutoff so no post-outcome data enters the feature set.

B6. Supervised vs. unsupervised framing

This is an unsupervised problem because there are no predefined labels; the goal is to discover structure (natural customer segments) rather than predict a known target. A clustering algorithm such as k-means is suitable (with features scaled first, since k-means uses Euclidean distance), or hierarchical clustering / DBSCAN if cluster shapes or counts are unknown. Because there is no ground truth, usefulness is judged by internal validity metrics (e.g., silhouette score), the stability of clusters across runs or subsamples, and, most importantly, whether the resulting segments are interpretable and actionable for distinct marketing campaigns. The number of clusters k is typically chosen using methods like the elbow or silhouette analysis combined with business judgment.