Compound Words on the Web
How to find closed compounds that dictionaries miss, and what 104,006 files of Common Crawl, FineWeb and FineWeb-Edu show. 14 November 2026, updated 16 November 2026.
A compound word can be written three ways: solid (lifecycle), open (life cycle) or hyphenated (life-cycle). Dictionaries usually record one or two of the forms. Writers use all three, and the web holds enough text to measure how often each form is used. The question behind this work is the one asked in The Legitimacy of AIRVENT: when is a solid form such as airvent a word in the language, even though no dictionary lists it?
The method
Finding compounds means finding pairs of words that are sometimes written as one. The method has three parts.
- Make the candidate list from both directions. Take every two-word headword in the dictionaries (life cycle, gold mine). Then split every solid word into two words that are each in a dictionary or word list, following Eric Chaikin's technique of splitting words. The solid words come from the dictionaries, the word lists, and words already seen on at least 3 web sites. The splitting finds compounds that no dictionary records in any form (back link, skill set, breath work). Each part must have at least 2 letters, except a single a (alot). The result is 1,554,646 candidate pairs.
- Count all three forms in the same text, under the same rules. Only English prose lines count: lines that end like sentences and contain enough common function words. A line repeated in one file is counted once. A pair counts as open when its two words stand next to each other with only white space between them, and as hyphenated when a hyphen joins them inside one token. Punctuation between them ("life, cycle") breaks the pair. Each form is counted in occurrences and in distinct web sites (domains). The site count is the better measure, because one site that repeats a phrase thousands of times counts once.
- Compare the forms. For each candidate, set the solid, open and hyphenated counts side by side. The form on the most sites is the one writers prefer. Then compare it with what the dictionaries list.
Raw counts need four filters before they can be read:
- Run-ons of function words. ofthe, alot, makesure and eachother are frequent solid forms, but they are typing errors, not compounds. Pairs with a function word as either part are left out.
- Word breaks. thro ugh appears on about 100 sites because pages break through across a line. A candidate needs its open and hyphenated forms together on at least 1% of the sites that use it.
- Names. Angeles, Donald and WordPress split into words but are names. Forms written in lower case less often than airvent itself (38.9% of the time) are left out, so airvent sets the cutoff.
- Chance meetings. An open count also includes places where the two words merely meet ("they want to be come true" is rare, but "any one of them" is common). The solid form is the reliable signal. The open count measures how often the phrase is written apart.
One part of the method does not work as it stands: splitting a solid word into three or more listed words. The word lists contain so many short, obscure words that nearly every long word splits. Datasets becomes "dat as ets" and shockwaves becomes "sho ckw aves". Splits of this kind produce 1.3 million false candidates. A split into many parts is worth keeping only when the web also writes those parts apart, as with state of the art and state-of-the-art. A second counting pass is now measuring the open and hyphenated forms of 12.1 million phrases of three to five words. Those phrases come from three sources:
- the dictionaries' multi-word headwords;
- 110,535 spaced entries of a word-frequency database;
- every way of splitting a solid web word into 3 to 5 listed words.
The data so far
| Measure | Common Crawl | FineWeb | FineWeb-Edu | Total |
|---|---|---|---|---|
| Files | 100,010 | 1,586 | 2,410 | 104,006 |
| Web pages | 371,160,957 | 1,487,969,118 | 1,525,223,056 | 3,384,353,131 |
| Prose lines counted | 6,233,716,050 | 12,895,491,329 | 18,024,979,396 | 37,154,186,775 |
| Words | 238,639,287,679 | 611,845,307,989 | 944,198,402,777 | 1,794,682,998,445 |
| Occurrences of candidate pairs in the text | 70,104,417,580 | 189,229,263,170 | 288,598,545,827 | 547,932,226,577 |
| Distinct words and pairs on 3 or more sites | 35,869,645 | |||
| Two-word compounds on 3 or more sites in some form | 725,612 | |||
| The same, after the filters above | 171,812 |
The 171,812 filtered compounds divide as follows by the form writers prefer and the forms the dictionaries list. The dictionaries checked are Webster's Third, the Collegiate 10th and 11th, the OED, Webster's 1913, the Learner's and Medical dictionaries, two thesauruses and others. The word lists checked are ENABLE, Collins Scrabble Words 2024 and the NASPA Word List 2023.
| Dictionaries and lists give | Solid preferred | Open preferred | Hyphenated preferred |
|---|---|---|---|
| No form at all | 25,602 | 16,742 | 5,695 |
| Open or hyphenated only | 546 | 53,495 | 4,167 |
| Solid only | 37,116 | 15,579 | 7,205 |
| Solid and another form | 2,499 | 2,150 | 1,016 |
| All | 65,763 | 87,966 | 18,083 |
Two rows show where the dictionaries and the web disagree:
- 26,148 compounds are written solid on most sites, yet no dictionary or list gives the solid form: the first row's 25,602 plus the second row's 546.
- The "solid only" row holds 22,784 compounds that the lists give only solid but writers mostly write open or hyphenated (15,579 plus 7,205). Some of these open counts are chance meetings of two words ("felt like" against the Scrabble word feltlike), so this row needs reading entry by entry.
Counting every compound whose solid form appears on at least 3 sites but is in no dictionary or list, whatever form is preferred, there are 82,377. Of these, 59,050 are solid on at least 100 sites and 21,618 on at least 1,000. 32,029 of the 82,377 are written solid on most of their sites. Airvent is among the 82,377 but not among those: it is solid on 1,153 sites, open on 55,217 and hyphenated on 2,067.
| Solid | Solid | Open | Hyphenated | Solid share of sites | Listed form |
|---|---|---|---|---|---|
| dataset | 16,684,853 | 7,892,496 | 110,187 | 58% | none |
| plugins | 8,552,611 | 61,380 | 1,887,658 | 69% | none |
| runtime | 6,690,539 | 1,329,690 | 931,086 | 59% | none |
| sportsbook | 4,183,245 | 189,416 | 2,750 | 85% | none |
| backlinks | 3,702,412 | 217,771 | 37,133 | 83% | none |
| cardstock | 2,035,939 | 1,080,805 | 29,436 | 59% | none |
| keychain | 1,021,415 | 283,097 | 18,202 | 73% | open |
| rulebook | 838,372 | 487,432 | 20,780 | 63% | open |
| breathwork | 827,778 | 204,966 | 22,934 | 67% | none |
| goldmine | 787,396 | 1,562,580 | 26,354 | 50% | open, hyphenated |
| walkability | 703,691 | 2,316 | 6,359 | 97% | none |
| Solid | Solid | Open | Hyphenated |
|---|---|---|---|
| ecofriendly | 227,490 | 934,544 | 23,253,113 |
| nonrefundable | 367,499 | 211,419 | 2,956,838 |
| noninvasive | 2,270,078 | 164,939 | 6,979,861 |
| nontoxic | 1,271,955 | 282,371 | 6,075,136 |
| followups | 47,435 | 231,486 | 2,051,237 |
| longtime | 10,461,046 | 66,137,843 | 6,360,237 |
| Compound | Solid | Open | Hyphenated |
|---|---|---|---|
| long term | 1,014,661 | 42,976,083 | 168,967,196 |
| well being | 27,546,835 | 9,481,135 | 81,243,407 |
| health care | 155,483,373 | 96,631,219 | 5,320,506 |
| non profit | 25,846,652 | 611,981 | 20,978,991 |
| web site | 434,126,807 | 38,173,749 | 1,388,273 |
Still to come
- FineWeb (1,586 files so far) and FineWeb-Edu (2,410 files) are now counted, and the figures above include them.
- Phrases of three to five words are still being counted. That pass will give the first reliable count of multi-part compounds.
- When it is done, its counts will be added to this article.