Handwritten Letter CNN

Twenty-six handwritten letters as 28 × 28 grayscale images, and a model that should tell them apart. It is the simplest project here, so the interesting part isn’t whether a CNN can do it. It is how much of the final score each decision earned, and whether the mistakes that remain are the model’s fault or the handwriting’s.

The file had 99,040 rows, 9,465 of them exact duplicates. Following the duplicates led somewhere more useful: almost all of them carried labels −1 and −2, which aren’t letters. Viewed as images, those 10,240 rows were blank, uniform or corrupted with a few stray marks, and they also accounted for every blank image in the file.

Removing the two invalid labels removed the duplicates and the blanks with them, leaving 88,800 rows across 26 balanced classes, with only 2 duplicates left.

Each change was a controlled experiment on the validation set, chosen on macro F1 so a weak letter can’t hide behind strong ones. The test set was used once, at the end.

• Always predict the most common letter

  Validation macro F1 0.003

• Dense network on flattened pixels

  Validation macro F1 0.881

• First CNN, keeping the 2D layout

  Validation macro F1 0.911

• Tuned batch size and epochs

  Validation macro F1 0.930

• A third convolution block (32, 64, 128 filters)

  Validation macro F1 0.937

• Dropout 0.7 before the dense layer

  Validation macro F1 0.946

• Batch normalisation; augmentation

  Validation macro F1 Tested, not kept

Dropout of 0.7 is high, and it was treated as an underfitting risk, but it came out of a sweep rather than a default, and L2 regularisation alone topped out at 0.925. On the held-out test set the final model scored 94.95% accuracy and 0.949 macro F1.

Before training anything, the average image of each class suggested which letters would be confused: I and L, G and Q, O and Q, U and V, M and N. The final confusion matrix confirms most of that list. The biggest remaining errors are I with L, G with Q, and U with V, pairs that are often ambiguous to a person too.

Feature maps and misclassified examples are in the notebook as clues to what the network responds to, not as proof of why it decided anything.

• Follow the duplicates to the cause

  Dropping 9,465 duplicates would have tidied the symptom. Tracing them to two labels that weren’t letters removed corrupted images the model would otherwise have been trained and tested on.

• Beat a real baseline, not a strawman

  A dense network already reaches 0.881. Showing it first makes the CNN’s gain honest: convolution adds three points of macro F1, not ninety.

• One change per experiment

  Protocol, capacity, regularisation, batch normalisation and augmentation were tested in order, each against the previous best. Every line in the final model can be traced to the experiment that justified it.

• Predict the errors before seeing them

  Listing likely confusions from the class averages before training turns the confusion matrix into a check of a stated expectation, rather than a story written after the fact.

• A letter classifier, not OCR: it sees one centred letter at a time.
• The remaining errors are on genuinely ambiguous handwriting, where a better model may not be possible from the image alone.
• The dataset’s source and licence weren’t given, so it isn’t in the repository.