Frequency Data vs Word Grades
Borgmann project · 29 October 2026

Can the frequency data predict a word's grade?

We tested three things: whether the word-frequency database predicts Eric's calibration scores, whether it predicts the scores in the consolidated crossword list, and whether that crossword list predicts Eric's scores.

Short answer: no. Frequency data ranks Eric's list only weakly (correlation about 0.4) and does no better than "accept everything" at a 50 cut. On the crossword list it runs backwards: the more common a string, the lower its score. The crossword list's score says nothing about Eric's (correlation 0.02); only being on the list at all helps, by about 8 points.

How it was tested

Each scored item was looked up in word_frequency.db (21 million entries). Its row gives 21 features: counts in Google Web, Google Books (2009, 2012, 2020), Usenet and OpenSubtitles; membership of the canonical list, Wiktionary, BNC, Broda, XWord Info and Gooch's BIGDICT; Gooch's dictionary codes (OED, Chambers, Web2, OSPD, place, phrase); and the database's English probability. A ridge regression maps these to the 0-100 score. Every prediction is made by a model that never saw that item (leave-one-out for Eric's list, 10-fold for the crossword list). The baseline predicts the average score.

1. Eric's calibration list from the frequency data

167 items; 131 have a database row. The other 36, mostly spaced phrases such as bit the bullet and Nobel Peace Prize, cannot be scored at all.

MeasureModelBaseline
Rank correlation with Eric0.43—
Variance explained (R²)0.200
Mean absolute error6.57.4
Right at the 50 cut64%66% (accept all)
Right at the 60 cut79%76% (reject all)

The best single signals: membership of the canonical list (+0.38), Usenet count (+0.37), number of sources (+0.36). The full model barely improves on any one of them. Its predictions are compressed: about 52 for items Eric scored 40-49 and about 57 for 70-79.

Largest missesEricModel
bigamy9061
werewolf8663
offboarding7048
liveblog1756
rajbansi2653

Weak ranking, no use as an accept/reject gate.

2. The crossword list's scores from the frequency data

The consolidated crossword list (Broda + XWord Info, 556,686 entries); 530,230 have a database row. Broda and XWord Info membership were left out, since they are the source of the score.

MeasureAll entriesWithout the default 50s
Entries530,230216,658
Rank correlation0.310.38
Variance explained (R²)0.090.20
Mean absolute error (baseline)7.6 (7.9)11.6 (12.5)
Separating 50+ from below (AUC)0.750.83

56% of the list sits at exactly 50, which looks like a default, so the second column leaves those out. Every frequency feature correlates negatively with the score (total count −0.20, Google Books 2020 −0.21): the lowest-scored entries are common words editors reject (raped, bugger) and the highest are lively phrases the corpora barely contain. Predictions stay between 61 and 64 for every true band from 50 to 100.

Frequency can weakly flag bad fill. It cannot rank good fill.

3. Eric's calibration list from the crossword list

67 of the 167 items are on the crossword list.

ItemsEric's mean
On the crossword list6756.5
Not on it10048.7

Among the 67, the correlation between the list's score and Eric's is +0.02: none. His mean stays between 51 and 60 in every band of list score.

WordEricCrossword list
bigamy9050
werewolf8650
fatberg4575
snowclone4070
liveblog1764

Adding the crossword list to the frequency model changes nothing (rank correlation 0.35 vs 0.36, R² 0.16 vs 0.15, on all 167 items).

Membership of the crossword list is worth about 8 points. Its score is noise against Eric's.

Limits

131 and 167 items make the results on Eric's list noisy. The regularisation strength was chosen on the same held-out error that is reported, which flatters the models slightly. The crossword list's scores rate crossword fill, which is a different target from Eric's wordplay grades. Tools: freqmodel.cpp and xwjoin.cpp in the wordlist project.