Tree A is better because fully grown trees always capture more information

Tree A is likely overfitting, and restricting depth (pre-pruning) improved generalisation in Tree B

The difference is due to random variation and not meaningful

Tree B is underfitting because it's less complex

ID3

KNN

C4.5

CART

Red — because its total weight (1.0) exceeds Blue's total weight (0.75)

Blue — because it has 2 votes out of 3 neighbours

Blue — because the average distance to Blue points is shorter

It's a tie and cannot be resolved

Euclidean Distance

Hamming Distance

Manhattan Distance

Cosine Similarity

Entropy always prefers more balanced splits, so it would choose the first option

Gini prefers the first split because it minimises the maximum impurity across children

Gini and Entropy always produce identical split rankings

Both criteria would prefer the split that produces the pure [100%, 0%] child, but their relative scores for the two splits may differ slightly

It creates leaf nodes that perfectly memorise training data, including noise

It always produces linear decision boundaries

It requires normalised features to function correctly

It uses too few features at each split

Information Gain Ratio

Entropy

Variance Reduction

Gini Impurity

Use a Decision Tree for initial rule-based triage, then route uncertain cases (near the decision boundary) to KNN for refined local classification

Train both on separate random halves of the data and average their predictions

Use KNN's output as a feature for the Decision Tree

Replace the Decision Tree's leaf predictions with KNN predictions computed over the full dataset

The tree is underfitting — the constraints are too aggressive, preventing it from learning the data's complexity

The tree is overfitting to the training set

The dataset is too noisy for any classifier

Feature scaling was not applied