K=5 is too small for imbalanced data and should be set to at least 50

95% accuracy is always excellent regardless of class distribution

The model may simply predict class A for everything; 95% accuracy matches the naive majority-class baseline, so the model may have learned nothing useful

KNN cannot handle imbalanced datasets at all

0.0

0.25

1.0

0.5

Eager learner

Semi-supervised learner

Lazy learner

Unsupervised learner

In practice, Entropy and Gini rarely produce significantly different trees; Gini is computationally cheaper since it avoids logarithm calculations

Entropy should only be used for multi-class problems; Gini only for binary

Gini always produces deeper trees than Entropy

Entropy is always superior because it comes from information theory

The Information Gain of both features is doubled

Both features will have identical Information Gain; the tree may arbitrarily choose either one, but the tree structure and predictions are unaffected

The tree will fail to build because it detects collinear features

The tree will use both features, creating a deeper and more complex model

Underfitting — the model needs more neighbours to handle more features

Curse of dimensionality — in high dimensions, distances become nearly uniform, making nearest neighbours meaningless

Feature leakage — the random features contain target information

Overfitting — KNN memorised the 200 new features

No boundary is formed because K=1 is invalid

A smooth curve similar to logistic regression

A Voronoi tessellation where each training point owns a region

A single straight line dividing the feature space

Try max_depth = 3, 5, 10 on the training data and pick the one with the highest training accuracy

Hold out a final test set, then use k-fold cross-validation on the remaining data to evaluate max_depth values from 1 to 30; select the depth with the best average CV score, then report performance on the held-out test set

Use the full dataset with a single 50/50 split for each depth value

Use cross-validation to test max_depth values and report the best CV score as the final performance estimate

Prediction time decreases significantly

The classifier will always predict the majority class

The decision boundary becomes more complex

The classifier overfits the training data