swageConcept & eval
The mechanism
Every prediction runs the same seven-step pipeline, live, in your browser — the exact code trainer and browser share (src/lib/normalize.ts) so they can never quietly drift apart:
MediaPipe's Hand Landmarker outputs 21 (x, y, z) points per hand. A left hand is mirrored onto a canonical right hand, the wrist becomes the coordinate origin, the whole hand is scaled so the wrist-to-middle-knuckle distance is 1 (invariant to hand size and camera distance), and the hand is rotated so that same reference point lands at a fixed angle (invariant to how the hand is tilted in-plane — see Limitations for what this does not correct). The resulting 63 numbers feed a tiny neural network — Dense(48, relu) then Dense(24, softmax), about 4,200 parameters, <50KB committed as JSON — that outputs a letter and a confidence. A held-stable prediction (SPEC.md §7.1: 6 of the last 8 frames agreeing at ≥0.9 confidence) fires once, shown in amber above.
Split policy
| Split | Source | Purpose |
|---|---|---|
| train | asl-now community pool (MIT), J/Z folders excluded | model fitting |
| val | 1 self-collected signer, never in train | threshold tuning, early stopping |
| test | 1 different self-collected signer, never in train or val | the published numbers, touched once |
That is the design. What this build actually has: the asl-now pool's own signer composition is unverified — no such field exists in that dataset, so it is used for training only, never for published val/test numbers (matches the design above). The self-collected val/test signers do not exist yet in this repo — see the provisional banner above. The numbers below come from a random, file-level split of the asl-now pool itself, train=1312 / val=281 / test=281 samples — real numbers, honestly captioned, not the design's actual claim.
Per-letter precision / recall / F1
Test-set accuracy: 95.7% (281 samples). Ship bar is ≥70% on a genuine per-signer test set (SPEC.md §3.4) — not met, because the split above is provisional.
| Letter | Precision | Recall | F1 | Support |
|---|---|---|---|---|
| A | 1 | 1 | 1 | 10 |
| B | 0.8333 | 1 | 0.9091 | 10 |
| C | 1 | 1 | 1 | 8 |
| D | 1 | 0.8182 | 0.9 | 11 |
| E | 0.9 | 1 | 0.9474 | 9 |
| F | 1 | 0.8889 | 0.9412 | 9 |
| G | 0.8667 | 0.9286 | 0.8966 | 14 |
| H | 1 | 1 | 1 | 10 |
| I | 1 | 1 | 1 | 10 |
| K | 1 | 0.9 | 0.9474 | 10 |
| L | 1 | 1 | 1 | 13 |
| M | 1 | 0.9231 | 0.96 | 13 |
| N | 1 | 0.9231 | 0.96 | 13 |
| O | 1 | 1 | 1 | 15 |
| P | 1 | 1 | 1 | 11 |
| Q | 0.9375 | 1 | 0.9677 | 15 |
| R | 1 | 0.8182 | 0.9 | 11 |
| S | 0.8571 | 1 | 0.9231 | 12 |
| T | 0.9091 | 1 | 0.9524 | 10 |
| U | 0.9412 | 1 | 0.9697 | 16 |
| V | 0.8571 | 1 | 0.9231 | 12 |
| W | 1 | 1 | 1 | 11 |
| X | 1 | 0.875 | 0.9333 | 16 |
| Y | 1 | 0.9167 | 0.9565 | 12 |
Confusion matrix
Rows are the true letter, columns the predicted letter. The diagonal (correct predictions) is shaded; off-diagonal cells above 40% of this matrix's highest confusion count are highlighted in amber.
| ↓true \ pred→ | A | B | C | D | E | F | G | H | I | K | L | M | N | O | P | Q | R | S | T | U | V | W | X | Y |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| A | 10 | |||||||||||||||||||||||
| B | 10 | |||||||||||||||||||||||
| C | 8 | |||||||||||||||||||||||
| D | 1 | 9 | 1 | |||||||||||||||||||||
| E | 9 | |||||||||||||||||||||||
| F | 1 | 8 | ||||||||||||||||||||||
| G | 13 | 1 | ||||||||||||||||||||||
| H | 10 | |||||||||||||||||||||||
| I | 10 | |||||||||||||||||||||||
| K | 9 | 1 | ||||||||||||||||||||||
| L | 13 | |||||||||||||||||||||||
| M | 12 | 1 | ||||||||||||||||||||||
| N | 12 | 1 | ||||||||||||||||||||||
| O | 15 | |||||||||||||||||||||||
| P | 11 | |||||||||||||||||||||||
| Q | 15 | |||||||||||||||||||||||
| R | 9 | 1 | 1 | |||||||||||||||||||||
| S | 12 | |||||||||||||||||||||||
| T | 10 | |||||||||||||||||||||||
| U | 16 | |||||||||||||||||||||||
| V | 12 | |||||||||||||||||||||||
| W | 11 | |||||||||||||||||||||||
| X | 2 | 14 | ||||||||||||||||||||||
| Y | 1 | 11 |
Most confused pairs
Ranked by raw confusion count in the matrix above — this is exactly what Drill mode's "confusable" bias draws from (try it):
- X → predicted as G
- D → predicted as B
- D → predicted as E
- F → predicted as B
- G → predicted as Q
- K → predicted as V
No J, no Z — by design, not by omission
24 classes, not 26: A B C D E F G H I K L M N O P Q R S T U V W X Y. Both excluded letters require traced motion a single held frame cannot see. More on Limitations.