Which words are the most useful to know?
-
- Rookie
- Posts: 53
- Joined: Sun Apr 17, 2011 1:42 pm
- Location: Bletchley
Which words are the most useful to know?
I did an analysis on which words would be most useful to know for standard letters rounds.
You can find the results here:
https://docs.google.com/spreadsheets/d/ ... ue&sd=true
Download the file as an Excel spreadsheet and feel free to play around - you can use it as a study tool or just for reference. Hope you fellow word geeks find it useful!
You can find the results here:
https://docs.google.com/spreadsheets/d/ ... ue&sd=true
Download the file as an Excel spreadsheet and feel free to play around - you can use it as a study tool or just for reference. Hope you fellow word geeks find it useful!
Last edited by Robert Foster on Sun Apr 02, 2023 7:52 pm, edited 2 times in total.
-
- Series 74 Champion
- Posts: 146
- Joined: Fri Mar 20, 2009 10:59 pm
Re: Which words are the most useful to know?
This is really great!
Re: Which words are the most useful to know?
Rob, this is brilliant. Thank you so much.Robert Foster wrote: ↑Mon Jul 09, 2018 9:10 pm I did an analysis on which words would be most useful to learn for each of 3, 4, and 5 vowel picks in standard letters rounds.
You can find the results here:
http://bit.ly/2KXffJw
Download the file as an Excel spreadsheet (don't try and open it as a Google Sheet) and feel free to play around - you can use it as a study tool or just for reference. Hope you fellow word geeks find it useful!
- Jon O'Neill
- Ginger Ninja
- Posts: 4545
- Joined: Tue Jan 22, 2008 12:45 am
- Location: London, UK
Re: Which words are the most useful to know?
Yeah, this is great. Nice work.
Only thing I would suggest as an improvement is to do some face-up shuffling in the simulated rounds (or weight against rounds with double or triple letters). It seems to me that the most useful words seem a bit skewed towards double-letters compared to https://www.apterous.org/wordstats.php?variant=0&dic=0
Only thing I would suggest as an improvement is to do some face-up shuffling in the simulated rounds (or weight against rounds with double or triple letters). It seems to me that the most useful words seem a bit skewed towards double-letters compared to https://www.apterous.org/wordstats.php?variant=0&dic=0
-
- Rookie
- Posts: 53
- Joined: Sun Apr 17, 2011 1:42 pm
- Location: Bletchley
Re: Which words are the most useful to know?
Thanks guys!
Good point - I always assumed the shuffling was totally random like at a co:event but this looks not to be the case. I uploaded a second version which decreases the chance of two of the same letter being drawn successively: http://bit.ly/2KXffJwJon O'Neill wrote: ↑Wed Jul 11, 2018 12:33 pm Only thing I would suggest as an improvement is to do some face-up shuffling in the simulated rounds (or weight against rounds with double or triple letters). It seems to me that the most useful words seem a bit skewed towards double-letters compared to https://www.apterous.org/wordstats.php?variant=0&dic=0
-
- Rookie
- Posts: 53
- Joined: Sun Apr 17, 2011 1:42 pm
- Location: Bletchley
Re: Which words are the most useful to know?
Latest version is here! You have to download it because the file won't show up in preview. Make sure to enable editing and macros if prompted.
https://bit.ly/2Qykkto
https://bit.ly/2Qykkto
Re: Which words are the most useful to know?
Great work Rob!
- Matt Morrison
- Post-apocalypse
- Posts: 7822
- Joined: Wed Oct 22, 2008 2:27 pm
- Location: London
- Contact:
-
- Rookie
- Posts: 53
- Joined: Sun Apr 17, 2011 1:42 pm
- Location: Bletchley
Re: Which words are the most useful to know?
Latest version of my word usefulness doc following the Nov 2020 update:
https://drive.google.com/drive/folders/ ... OBoD4LcS_4
https://drive.google.com/drive/folders/ ... OBoD4LcS_4
Re: Which words are the most useful to know?
Beautiful Rob. Thanks!
Re: Which words are the most useful to know?
Lovely jubblyRobert Foster wrote: ↑Thu Nov 26, 2020 9:54 pm Latest version of my word usefulness doc following the Nov 2020 update:
https://drive.google.com/drive/folders/ ... OBoD4LcS_4
-
- Rookie
- Posts: 53
- Joined: Sun Apr 17, 2011 1:42 pm
- Location: Bletchley
Re: Which words are the most useful to know?
Updated version thanks to suggestions from Callum and Fiona. A few changes:
- The logic behind the word that appears in 'Single Word' column has been changed - for words with multiple anagrams, it should state the most useful to learn of these. (Roughly the logic is "prefer words that take useful letters on the end like S and D; if still a tie, prefer words which are still words after removing the last letter; if still a tie, prefer the word that is most frequent in real life usage.
- An extra column has been added listing the letters that the Single Word takes on the end to form other words (so 'drs' for BREATHE)
- Default picking ratios have been updated to closer reflect that of apterous. This has resulted in a different ordering of the most useful words. If you preferred the original listing you can just change your picking preferences back to (1/3, 1/3, 1/3)
- The logic behind the word that appears in 'Single Word' column has been changed - for words with multiple anagrams, it should state the most useful to learn of these. (Roughly the logic is "prefer words that take useful letters on the end like S and D; if still a tie, prefer words which are still words after removing the last letter; if still a tie, prefer the word that is most frequent in real life usage.
- An extra column has been added listing the letters that the Single Word takes on the end to form other words (so 'drs' for BREATHE)
- Default picking ratios have been updated to closer reflect that of apterous. This has resulted in a different ordering of the most useful words. If you preferred the original listing you can just change your picking preferences back to (1/3, 1/3, 1/3)
- Jon O'Neill
- Ginger Ninja
- Posts: 4545
- Joined: Tue Jan 22, 2008 12:45 am
- Location: London, UK
Re: Which words are the most useful to know?
I've said it before and I'll say it again. This really is good.
Re: Which words are the most useful to know?
Brilliant - thanks Rob!Robert Foster wrote: ↑Fri Nov 27, 2020 5:57 pm Updated version thanks to suggestions from Callum and Fiona. A few changes:
- The logic behind the word that appears in 'Single Word' column has been changed - for words with multiple anagrams, it should state the most useful to learn of these. (Roughly the logic is "prefer words that take useful letters on the end like S and D; if still a tie, prefer words which are still words after removing the last letter; if still a tie, prefer the word that is most frequent in real life usage.
- An extra column has been added listing the letters that the Single Word takes on the end to form other words (so 'drs' for BREATHE)
- Default picking ratios have been updated to closer reflect that of apterous. This has resulted in a different ordering of the most useful words. If you preferred the original listing you can just change your picking preferences back to (1/3, 1/3, 1/3)
- Charlie Reams
- Site Admin
- Posts: 9494
- Joined: Fri Jan 11, 2008 2:33 pm
- Location: Cambridge
- Contact:
Re: Which words are the most useful to know?
This is very cool! Any chance you could share it in some generic form (like CSV) for those of us who don't have Excel?
-
- Rookie
- Posts: 53
- Joined: Sun Apr 17, 2011 1:42 pm
- Location: Bletchley
Re: Which words are the most useful to know?
Cheers! I've uploaded the CSVs for each tab now.
https://drive.google.com/drive/folders/ ... OBoD4LcS_4
https://drive.google.com/drive/folders/ ... OBoD4LcS_4
Re: Which words are the most useful to know?
Lovelier jubblierRobert Foster wrote: ↑Sun Nov 29, 2020 6:45 pm Cheers! I've uploaded the CSVs for each tab now.
https://drive.google.com/drive/folders/ ... OBoD4LcS_4
- Charlie Reams
- Site Admin
- Posts: 9494
- Joined: Fri Jan 11, 2008 2:33 pm
- Location: Cambridge
- Contact:
Re: Which words are the most useful to know?
Thank you so much!
I'm really enjoying the collaborative improvement of these lists. One thing I wonder is how to capture the relative impact of spotting/missing words of different length. In particular, knowing 9s might be much more important than it seems because there's a 36-point swing at stake (versus 16 or less for all shorter words). To investigate this, I pulled some data from recent human-vs-human apterous games (about 2 million rounds total) on the average net impact in terms of max length:
(To give an example of what this table means, if the max in a letters round is 8 and you miss it then you expect your opponent will gain about 1.75 points over you on average, whereas if you spot it then you expect to gain about 2.98 points over them. Obviously these are just averages and could look quite different depending on the skill level of the players and the specific selection involved.)
The main thing this shows very clearly is that knowing nines is much more important than it might seem in terms of frequency. Maybe one way to incorporate this information into the ranking is to say that one extra 9 is worth 10.79 (the spread from +7.13 to -3.66) versus, say, 4.73 for an eight, and weight all the frequency data accordingly.
Anyway this is great so let me know if there's any other data I can pull that would help.
I'm really enjoying the collaborative improvement of these lists. One thing I wonder is how to capture the relative impact of spotting/missing words of different length. In particular, knowing 9s might be much more important than it seems because there's a 36-point swing at stake (versus 16 or less for all shorter words). To investigate this, I pulled some data from recent human-vs-human apterous games (about 2 million rounds total) on the average net impact in terms of max length:
Code: Select all
Length Expected net points for...
Missing Spotting
4 -1.57 +0.70
5 -1.76 +1.20
6 -1.88 +1.62
7 -1.96 +2.07
8 -1.75 +2.98
9 -3.66 +7.13
The main thing this shows very clearly is that knowing nines is much more important than it might seem in terms of frequency. Maybe one way to incorporate this information into the ranking is to say that one extra 9 is worth 10.79 (the spread from +7.13 to -3.66) versus, say, 4.73 for an eight, and weight all the frequency data accordingly.
Anyway this is great so let me know if there's any other data I can pull that would help.
Re: Which words are the most useful to know?
I think a really interesting (but possibly difficult to calculate) piece of data from apterous would be to see a list of words most frequently missed that would have change the outcome of a game from loss to win if other rounds in the game stayed the same.Charlie Reams wrote: ↑Sun Nov 29, 2020 9:16 pm I pulled some data from recent human-vs-human apterous games
e.g. if Bob loses 99 - 100 to Alice, and RD 13 was:
Selection: AEIORDNTS
Bob: STAINED
Alice: STRAINED
Then any 8s or 9s from that selection get another mark next to them on the list.
I guess you'd calculate if as follows:
- Find all games with a margin (M) of < 26*
- In those games, find all of the rounds where a user did not get a letters max
- For each word available in that round that is better than what the loser declared, calculatethe swing (S) for that word
- For each word where S > M, add that word to the list
- *26 because the largest possible swing in a letters round is +18 pts for you, -8 for the opponent.
-
- Rookie
- Posts: 53
- Joined: Sun Apr 17, 2011 1:42 pm
- Location: Bletchley
Re: Which words are the most useful to know?
Nice sleuthing Charlie, I may take up your offer of juicy apto data! I should have mentioned that in my doc, the words are already weighted according to how many points they score (x18 for a 9, x8 for an 8 etc).
It turns out that my crude weightings are actually pretty close to the ones generated by your method (I've inflated yours so they can be compared more easily with mine):
So perhaps 9s in my list are slightly underrated, but not by a massive amount.
I've added a readme that explains a few other things about the doc.
https://drive.google.com/drive/folders/ ... OBoD4LcS_4
It turns out that my crude weightings are actually pretty close to the ones generated by your method (I've inflated yours so they can be compared more easily with mine):
Code: Select all
points... weightings...
length lost gained swing Charlie's Rob's
4 -1.57 +0.7 2.27 3.85 4
5 -1.76 +1.2 2.96 5.02 5
6 -1.88 +1.62 3.5 5.94 6
7 -1.96 +2.07 4.03 6.84 7
8 -1.75 +2.98 4.73 8.03 8
9 -3.66 +7.13 10.79 18.31 18
I've added a readme that explains a few other things about the doc.
https://drive.google.com/drive/folders/ ... OBoD4LcS_4
Last edited by Robert Foster on Tue Dec 01, 2020 9:50 am, edited 1 time in total.
- Charlie Reams
- Site Admin
- Posts: 9494
- Joined: Fri Jan 11, 2008 2:33 pm
- Location: Cambridge
- Contact:
Re: Which words are the most useful to know?
Super interesting! So I think that's a good argument for mostly studying to hit maxes and not worrying about the "secondary effect" of words that will probably score even when they aren't the max, which means your lists are probably even more useful than I thought.Robert Foster wrote: ↑Mon Nov 30, 2020 11:41 amSo perhaps 9s in my list are slightly underrated, but not by a massive amount.Code: Select all
points... weightings... length lost gained swing Charlie's Rob's 4 -1.57 +0.7 2.27 3.85 4 5 -1.76 +1.2 2.96 5.02 5 6 -1.88 +1.62 3.5 5.94 6 7 -1.96 +2.07 4.03 6.84 7 8 -1.75 +2.98 4.73 8.03 8 9 -3.66 +7.13 10.79 18.31 18
I've added a readme that explains a few other things about the doc.
https://drive.google.com/drive/folders/ ... OBoD4LcS_4
- L'oisleatch McGraw
- Devotee
- Posts: 930
- Joined: Sun Dec 13, 2015 2:46 am
- Location: Waterford
- Contact:
Re: Which words are the most useful to know?
If I sort the document by Word length (i.e. Column D), so that it lists all the 9s first, the the 8s etc; the data in columns O and R disappears. Is there a way to sort it that way without that happening?
![Arrow :arrow:](./images/smilies/icon_arrow.gif)
![Arrow :arrow:](./images/smilies/icon_arrow.gif)
![Arrow :arrow:](./images/smilies/icon_arrow.gif)
-
- Rookie
- Posts: 53
- Joined: Sun Apr 17, 2011 1:42 pm
- Location: Bletchley
Re: Which words are the most useful to know?
Sorting by length seems to work OK for me. I did:
Highlight columns B-R (or columns B-Q in the latest version) > Sort & Filter > Custom Sort > Tick the 'My data has headers' box > Column D Largest to Smallest.
Also 'Takes' information only shows for words of 8 letters and below, so if you have all of the 9s at the top of the list then you won't see anything until you scroll down to the 8s.
Highlight columns B-R (or columns B-Q in the latest version) > Sort & Filter > Custom Sort > Tick the 'My data has headers' box > Column D Largest to Smallest.
Also 'Takes' information only shows for words of 8 letters and below, so if you have all of the 9s at the top of the list then you won't see anything until you scroll down to the 8s.
- L'oisleatch McGraw
- Devotee
- Posts: 930
- Joined: Sun Dec 13, 2015 2:46 am
- Location: Waterford
- Contact:
Re: Which words are the most useful to know?
Thank you. I am not familiar with excel. I didn't tick the relevant box... and may have done other things wrong, not sure. But it is working now.
By the way, I don't know if this is any use to you for future editions, but what I did was included Column A when resorting the data, then added a new column showing "rank by letter count"... and in that numbered all 31583 nine letters words by usefulness, then all 28617 eights, and so forth. And the benefit of having the rank numbers in column A, is you can compare the relative usefulness of the 1000th best nine with the 1000th best eight etc.
Handy for someone who likes to learn say, the first 100 most useful nines, then the first 100 eights, then 7s, and then go back and look at the next 100 nines... etc.
Thanks again for the help.
By the way, I don't know if this is any use to you for future editions, but what I did was included Column A when resorting the data, then added a new column showing "rank by letter count"... and in that numbered all 31583 nine letters words by usefulness, then all 28617 eights, and so forth. And the benefit of having the rank numbers in column A, is you can compare the relative usefulness of the 1000th best nine with the 1000th best eight etc.
Handy for someone who likes to learn say, the first 100 most useful nines, then the first 100 eights, then 7s, and then go back and look at the next 100 nines... etc.
Thanks again for the help.
![Arrow :arrow:](./images/smilies/icon_arrow.gif)
![Arrow :arrow:](./images/smilies/icon_arrow.gif)
![Arrow :arrow:](./images/smilies/icon_arrow.gif)
- L'oisleatch McGraw
- Devotee
- Posts: 930
- Joined: Sun Dec 13, 2015 2:46 am
- Location: Waterford
- Contact:
Re: Which words are the most useful to know?
I spent this evening getting to grips with your word list... and I have some observations to make. What I am about to write may not be of great relevance, but in case I have stumbled upon something useful that could help you, I am sharing.
Back in 2017 I made my own nerd list in preparation for going on telly. Now, this was in no way as statistically sound or thorough. It was more a patchwork quilt drawn from many sources including gut feeling, experience of commonly occurring words on Apto and CD, Apto's 'most missed lists etc, with a little bit of input from Paul Erdunast's word playability list (he hates when I call it "The NERDUNAST", but... how could you not?!
) It would have had a LOT more input from Paul's, but it was very close to my TV appearance when I got that list, so had to make some very rushed adjustments to include at least some of his work...
Anyhow, my list is about words that a mid-level Apterite, without natural anagramming flair, cannot readily see in a selection... and it gives crutches (falseagrams / stemmers) to make them more visible. So this would mean some words (very few) that are easy to spot (like RELATIONS) are not on the list at all.
In all, the list contains 77 six letter words, 327 seven letter words, 465 eight letter words, and 370 nine letter words, rated roughly according to perceived usefulness.
I've been cross referencing my old list with yours (I've only looked at 9-letter words so far), and in doing so, found a few outliers.
Firstly there are the words I had listed as high importance, that were lower down on yours:-
WORD ........................ EOIN's APPROX RANK ........................ ROB's APPROX RANK
BOTANISES ........................ 1st - 100th range ........................ 800th - 900th range
CORTISONE ........................ 1st - 100th range ........................ 600th - 700th range
GINORITES ........................ 1st - 100th range ........................ 300th - 400th range
ARTINITES ........................ 1st - 100th range ........................ 400th - 500th range
AMNESTIES ........................ 1st - 100th range ........................ 800th - 900th range
DRIPSTONE ........................ 1st - 100th range ........................ 300th - 400th range
TORSIONAL ........................ 1st - 100th range ........................ 400th - 500th range
RESONATOR ........................ 1st - 100th range ........................ 500th - 600th range
DREARIEST ........................ 1st - 100th range ........................ 300th - 400th range
RESTORALS ........................ 1st - 100th range ........................ Outside Top 2000 nines
BANDOLIER ........................ 100th - 200th range ........................ 500th - 600th range
IGNORABLE ........................ 100th - 200th range ........................ 1000th - 1100th range
DEERHOUND ........................ 100th - 200th range ........................ Outside Top 2000 nines
SEALSTONE ........................ 100th - 200th range ........................ 600th - 700th range
LIMESTONE ........................ 100th - 200th range ........................ 400th - 500th range
DEMOTIONS ........................ 100th - 200th range ........................ 700th - 800th range
ADELASTER ........................ 100th - 200th range ........................ 600th - 700th range
SERMONISE ........................ 100th - 200th range ........................ 1100th - 1200th range
Then there are the words of high importance (top 400 nines) on your list that were absent from mine:-
WORD ............................................ ROB's APPROX RANK
OPERATICS ...................................... 106th most important nine
CREMATION ...................................... 156th most important nine
MACONITES ...................................... 191st most important nine
ADEMPTION ...................................... 230th most important nine
PERISOMAL ...................................... 264th most important nine
OSMICATED ...................................... 307th most important nine
PEDICATOR ...................................... 317th most important nine
PROLAMINE ...................................... 328th most important nine
DECIMATOR ...................................... 331st most important nine
METAPORES ...................................... 340th most important nine
PREMIATES ...................................... 341st most important nine
AIRPHONES ...................................... 342nd most important nine
HARMONISE ...................................... 343rd most important nine
PANTIHOSE ...................................... 353rd most important nine
ATROPHIES ...................................... 354th most important nine
BROMINATE ...................................... 384th most important nine
IMPOSTURE ...................................... 385th most important nine
PINAFORES ...................................... 389th most important nine
NORMATIVE ...................................... 392nd most important nine
ABREPTION ...................................... 399th most important nine
From this quick and dirty analysis, I wonder is there something a little askew in the current playability algorithm? It feels like certain words with repeated common letters (e.g. Botanises, Amnesties, Restorals, etc) are more likely to appear in a selection, than words containing more than one of the less common letters, but without any repeated letters (e.g. Operatics, Ademption, Decimator, etc)
What's your take? Do you think there may be something in this?
--------------------------------------------------------------------------------------
TL;DR
Perhaps combinations of repeated common letters (e.g. oo, ee, ss, rr) are more likely to appear in a nine letter word than a combination of two less common letters, but with no repeats in the selection (e.g. mb, ph, cp, cv)?
Back in 2017 I made my own nerd list in preparation for going on telly. Now, this was in no way as statistically sound or thorough. It was more a patchwork quilt drawn from many sources including gut feeling, experience of commonly occurring words on Apto and CD, Apto's 'most missed lists etc, with a little bit of input from Paul Erdunast's word playability list (he hates when I call it "The NERDUNAST", but... how could you not?!
![Mr. Green :mrgreen:](./images/smilies/icon_mrgreen.gif)
Anyhow, my list is about words that a mid-level Apterite, without natural anagramming flair, cannot readily see in a selection... and it gives crutches (falseagrams / stemmers) to make them more visible. So this would mean some words (very few) that are easy to spot (like RELATIONS) are not on the list at all.
In all, the list contains 77 six letter words, 327 seven letter words, 465 eight letter words, and 370 nine letter words, rated roughly according to perceived usefulness.
I've been cross referencing my old list with yours (I've only looked at 9-letter words so far), and in doing so, found a few outliers.
Firstly there are the words I had listed as high importance, that were lower down on yours:-
WORD ........................ EOIN's APPROX RANK ........................ ROB's APPROX RANK
BOTANISES ........................ 1st - 100th range ........................ 800th - 900th range
CORTISONE ........................ 1st - 100th range ........................ 600th - 700th range
GINORITES ........................ 1st - 100th range ........................ 300th - 400th range
ARTINITES ........................ 1st - 100th range ........................ 400th - 500th range
AMNESTIES ........................ 1st - 100th range ........................ 800th - 900th range
DRIPSTONE ........................ 1st - 100th range ........................ 300th - 400th range
TORSIONAL ........................ 1st - 100th range ........................ 400th - 500th range
RESONATOR ........................ 1st - 100th range ........................ 500th - 600th range
DREARIEST ........................ 1st - 100th range ........................ 300th - 400th range
RESTORALS ........................ 1st - 100th range ........................ Outside Top 2000 nines
BANDOLIER ........................ 100th - 200th range ........................ 500th - 600th range
IGNORABLE ........................ 100th - 200th range ........................ 1000th - 1100th range
DEERHOUND ........................ 100th - 200th range ........................ Outside Top 2000 nines
SEALSTONE ........................ 100th - 200th range ........................ 600th - 700th range
LIMESTONE ........................ 100th - 200th range ........................ 400th - 500th range
DEMOTIONS ........................ 100th - 200th range ........................ 700th - 800th range
ADELASTER ........................ 100th - 200th range ........................ 600th - 700th range
SERMONISE ........................ 100th - 200th range ........................ 1100th - 1200th range
Then there are the words of high importance (top 400 nines) on your list that were absent from mine:-
WORD ............................................ ROB's APPROX RANK
OPERATICS ...................................... 106th most important nine
CREMATION ...................................... 156th most important nine
MACONITES ...................................... 191st most important nine
ADEMPTION ...................................... 230th most important nine
PERISOMAL ...................................... 264th most important nine
OSMICATED ...................................... 307th most important nine
PEDICATOR ...................................... 317th most important nine
PROLAMINE ...................................... 328th most important nine
DECIMATOR ...................................... 331st most important nine
METAPORES ...................................... 340th most important nine
PREMIATES ...................................... 341st most important nine
AIRPHONES ...................................... 342nd most important nine
HARMONISE ...................................... 343rd most important nine
PANTIHOSE ...................................... 353rd most important nine
ATROPHIES ...................................... 354th most important nine
BROMINATE ...................................... 384th most important nine
IMPOSTURE ...................................... 385th most important nine
PINAFORES ...................................... 389th most important nine
NORMATIVE ...................................... 392nd most important nine
ABREPTION ...................................... 399th most important nine
From this quick and dirty analysis, I wonder is there something a little askew in the current playability algorithm? It feels like certain words with repeated common letters (e.g. Botanises, Amnesties, Restorals, etc) are more likely to appear in a selection, than words containing more than one of the less common letters, but without any repeated letters (e.g. Operatics, Ademption, Decimator, etc)
What's your take? Do you think there may be something in this?
--------------------------------------------------------------------------------------
TL;DR
Perhaps combinations of repeated common letters (e.g. oo, ee, ss, rr) are more likely to appear in a nine letter word than a combination of two less common letters, but with no repeats in the selection (e.g. mb, ph, cp, cv)?
![Arrow :arrow:](./images/smilies/icon_arrow.gif)
![Arrow :arrow:](./images/smilies/icon_arrow.gif)
![Arrow :arrow:](./images/smilies/icon_arrow.gif)
- L'oisleatch McGraw
- Devotee
- Posts: 930
- Joined: Sun Dec 13, 2015 2:46 am
- Location: Waterford
- Contact:
Re: Which words are the most useful to know?
Doing a little more nerding around this...
I listed 20 (more problematic imo) nine-letter words from both our lists in order of playability, and then checked how often each word had appeared on Countdown, and on Apterous. I then listed 20 nine-letter words in order of playability that were in the Top 100 of both of our lists, as a control sample.
--------------------------------
If a word appeared 4 or more times on CD, and 160 or more times on Apto, it is coloured RED on the lists. (aka best!)
If a word appeared 4 or more times on CD, but 159 or fewer times on Apto, it is coloured GREEN on the lists. (aka next best)
If a word appeared 3 or fewer times on CD, but 160 or more times on Apto, it is coloured BLUE on the lists. (aka 3rd best!)
If a word appeared 3 or fewer times on CD, and 159 or fewer times on Apto, it is coloured BLACK on the lists. (aka worst!)
--------------------------------
First off, 20 common words that are in my Top 400, but absent from Rob's
![Image](https://scontent.fdub4-1.fna.fbcdn.net/v/t1.0-9/141457260_4241950955822272_7955095611804725721_o.jpg?_nc_cat=109&ccb=2&_nc_sid=8024bb&_nc_ohc=X8p95P6i7K4AX_Fp7D7&_nc_ht=scontent.fdub4-1.fna&oh=5dc630828827e9be826673d0dfb92f29&oe=6032C156)
Next up, 20 common words that are in Rob's top 400, but absent from mine.
![Image](https://scontent.fdub4-1.fna.fbcdn.net/v/t1.0-9/141427024_4241950962488938_3113957347912294189_o.jpg?_nc_cat=103&ccb=2&_nc_sid=8024bb&_nc_ohc=hLeAykPBEj4AX_bLkIY&_nc_ht=scontent.fdub4-1.fna&oh=4246f4b1138a1bd3021656b41b324c28&oe=60320BD0)
Finally, 20 of the most common nine-letter words imaginable! These are Top 100 on both of the lists.
![Image](https://scontent.fdub4-1.fna.fbcdn.net/v/t1.0-9/141482167_4241950959155605_4114522034875422178_o.jpg?_nc_cat=102&ccb=2&_nc_sid=8024bb&_nc_ohc=W5O1q7N_hZcAX8YmIvh&_nc_ht=scontent.fdub4-1.fna&oh=b5d6db43cb34102403e6461575dd268b&oe=6033FE1F)
What do these lists tell us?
--------------------------------
My take:-
All the RED words belong in the top 400 most playable 9s.
Some of the black and blue words are new additions to the dictionary and cannot be fairly judged in this context.
Some of the black and blue words don't belong in the top 400.
The words coloured GREEN are generally far more important than the words coloured BLUE, when determining which words to prioritise.
I listed 20 (more problematic imo) nine-letter words from both our lists in order of playability, and then checked how often each word had appeared on Countdown, and on Apterous. I then listed 20 nine-letter words in order of playability that were in the Top 100 of both of our lists, as a control sample.
--------------------------------
If a word appeared 4 or more times on CD, and 160 or more times on Apto, it is coloured RED on the lists. (aka best!)
If a word appeared 4 or more times on CD, but 159 or fewer times on Apto, it is coloured GREEN on the lists. (aka next best)
If a word appeared 3 or fewer times on CD, but 160 or more times on Apto, it is coloured BLUE on the lists. (aka 3rd best!)
If a word appeared 3 or fewer times on CD, and 159 or fewer times on Apto, it is coloured BLACK on the lists. (aka worst!)
--------------------------------
First off, 20 common words that are in my Top 400, but absent from Rob's
![Image](https://scontent.fdub4-1.fna.fbcdn.net/v/t1.0-9/141457260_4241950955822272_7955095611804725721_o.jpg?_nc_cat=109&ccb=2&_nc_sid=8024bb&_nc_ohc=X8p95P6i7K4AX_Fp7D7&_nc_ht=scontent.fdub4-1.fna&oh=5dc630828827e9be826673d0dfb92f29&oe=6032C156)
Next up, 20 common words that are in Rob's top 400, but absent from mine.
![Image](https://scontent.fdub4-1.fna.fbcdn.net/v/t1.0-9/141427024_4241950962488938_3113957347912294189_o.jpg?_nc_cat=103&ccb=2&_nc_sid=8024bb&_nc_ohc=hLeAykPBEj4AX_bLkIY&_nc_ht=scontent.fdub4-1.fna&oh=4246f4b1138a1bd3021656b41b324c28&oe=60320BD0)
Finally, 20 of the most common nine-letter words imaginable! These are Top 100 on both of the lists.
![Image](https://scontent.fdub4-1.fna.fbcdn.net/v/t1.0-9/141482167_4241950959155605_4114522034875422178_o.jpg?_nc_cat=102&ccb=2&_nc_sid=8024bb&_nc_ohc=W5O1q7N_hZcAX8YmIvh&_nc_ht=scontent.fdub4-1.fna&oh=b5d6db43cb34102403e6461575dd268b&oe=6033FE1F)
What do these lists tell us?
--------------------------------
My take:-
All the RED words belong in the top 400 most playable 9s.
Some of the black and blue words are new additions to the dictionary and cannot be fairly judged in this context.
Some of the black and blue words don't belong in the top 400.
The words coloured GREEN are generally far more important than the words coloured BLUE, when determining which words to prioritise.
![Arrow :arrow:](./images/smilies/icon_arrow.gif)
![Arrow :arrow:](./images/smilies/icon_arrow.gif)
![Arrow :arrow:](./images/smilies/icon_arrow.gif)
- Jon O'Neill
- Ginger Ninja
- Posts: 4545
- Joined: Tue Jan 22, 2008 12:45 am
- Location: London, UK
Re: Which words are the most useful to know?
This is all good stuff.
Splitting hairs but one caveat is that some of the words would not have been in the dictionary for as long as others. While there is no "date entered dictionary" thing on Lexplorer, you could work out appearances/day since the first appearance.
That's not perfect because the volume of games has fluctuated over time. And also newly appearing words will suffer in the Lexplorer count because they wouldn't have been spotted as much early on as people learned the word.
Splitting hairs but one caveat is that some of the words would not have been in the dictionary for as long as others. While there is no "date entered dictionary" thing on Lexplorer, you could work out appearances/day since the first appearance.
That's not perfect because the volume of games has fluctuated over time. And also newly appearing words will suffer in the Lexplorer count because they wouldn't have been spotted as much early on as people learned the word.
-
- Post-apocalypse
- Posts: 13258
- Joined: Mon Jan 21, 2008 10:37 pm
Re: Which words are the most useful to know?
Wouldn't all maxes appear in the Lexplorer count?
- Jon O'Neill
- Ginger Ninja
- Posts: 4545
- Joined: Tue Jan 22, 2008 12:45 am
- Location: London, UK
Re: Which words are the most useful to know?
Yes, but twice if both players get it.
- L'oisleatch McGraw
- Devotee
- Posts: 930
- Joined: Sun Dec 13, 2015 2:46 am
- Location: Waterford
- Contact:
Re: Which words are the most useful to know?
Everything I've written in the last few days on this thread is about splitting hairs. This is a near perfect resource, and fair play to Rob (and anyone else involved) for making it. Is it perfect? No. Will it ever be perfect? No. Why be an arsehole and point that out? Because I think there is potential to bring it a lot closer to perfection, otherwise there'd be no point in posting nitpicky comments.
I think the main thing I've picked up on in the above analyses is that, while the original list may have been a bit skewed towards favouring double-letter words... the new version seems skewed a bit too much against them, perhaps in particular EE, AA, and OO (based on an observational hunch rather than anything solid).
------------------------------------------------------------
Whilst striving toward perfection, you'd have to wonder about some of the following:
1. Do we have anyone on the inside (i.e. Countdown Team) who has contributed any data (re: shuffling, actual letters distributions, how often the letters distribution is tweaked if ever, etc)?
2. If not, how is the letters distribution data derived? Statistical analysis from the wiki?
3. What would Countdown Team make of all of this? Would they view it favourably, or frown upon it, or not give a crap? If the Apterisation of the Series Finals is something that gets on their tits, then surely a list like this would be similarly annoying, as it further polarises the top Apterites from the rest of the field.
4. How stable is the letters distribution on CD? Will it be tweaked in future? How regular would such tweaks be?
5. How invested is Rob (and whoever helps) in this list? Would a new version be released every time there is a dictionary update, or perhaps a new version once a year... or are there no plans to do future updates?
------------------------------------------------------------
The reason I am so interested in this list all of a sudden, is I'm about to launch an ongoing project to upload videos with falseagrams and stemmers to YouTube. Each video will be titled something along the lines of "Essential 9s: Vol1", "Essential 7s: Vol7" etc, and will contain 100 words per episode... and I was hoping to base the "essentialness" of the words on Rob's playability index.
![Arrow :arrow:](./images/smilies/icon_arrow.gif)
![Arrow :arrow:](./images/smilies/icon_arrow.gif)
![Arrow :arrow:](./images/smilies/icon_arrow.gif)
Re: Which words are the most useful to know?
Graeme posted an interesting thread a while back (which I can't find) where he did a statistical anaylsis on how likely it is that the letters tiles are just shuffled face down like a deck of cards with no tweaks. He found that there was something like a 0.01% change that no face up tweaking was happening.L'oisleatch McGraw wrote: ↑Sun Jan 24, 2021 3:42 am 1. Do we have anyone on the inside (i.e. Countdown Team) who has contributed any data (re: shuffling, actual letters distributions, how often the letters distribution is tweaked if ever, etc)?
IIRC members of the Countdown Team posted to in that thread assure that nothing likeface up tweaking was happening as you can see here.
I also did some of my own simulations a few months ago and posted the findings which you may find interesting
I can't speak for them, but Countdown is an (albeit quite niche) competitive intellectual pursuit. Just like Chess, Scrabble, Maths Olympiads, Quizzes etc there will always be people who take it seriously enough to go the extra mile and beat the competition by practising better than anybody else.L'oisleatch McGraw wrote: ↑Sun Jan 24, 2021 3:42 am 3. What would Countdown Team make of all of this? Would they view it favourably, or frown upon it, or not give a crap? If the Apterisation of the Series Finals is something that gets on their tits, then surely a list like this would be similarly annoying, as it further polarises the top Apterites from the rest of the field.
- Before apterous it was people like Julian Fell who were probably absolutely hammering their hand held electronic copy of the game to get in loads more practise than you could by watching the show, and then studying the paper dictionary to pick out high probability words
- Then when apterous came out you had Kirk Bevins who was there at the birth and was the first person to really put in the hours on apterous (and anahack) to get himself head and shoulders above the rest.
- Now you have players who study from probability based lists, and who do their own coding to help themselves pick out the stuff they want to be studying. Rob (best letters player at the moment) is an example of somebody who does that, and he is notable better than the next best on apterous (who I assume are people who fall more into the Bevins breed of smashing apterous hard)
Yes, I believe Rob does update it when there are new releases.L'oisleatch McGraw wrote: ↑Sun Jan 24, 2021 3:42 am 5. How invested is Rob (and whoever helps) in this list? Would a new version be released every time there is a dictionary update, or perhaps a new version once a year... or are there no plans to do future updates?
Re: Which words are the most useful to know?
1) You're not being an arseholeL'oisleatch McGraw wrote: ↑Sun Jan 24, 2021 3:42 am Everything I've written in the last few days on this thread is about splitting hairs. This is a near perfect resource, and fair play to Rob (and anyone else involved) for making it. Is it perfect? No. Will it ever be perfect? No. Why be an arsehole and point that out? Because I think there is potential to bring it a lot closer to perfection, otherwise there'd be no point in posting nitpicky comments.
2) You are right that the list probably has room for improvements. However, I think those improvements would be very very marginal, and probably only make a difference for players at Rob's level. Best thing to do if you want to get better is just gobble up the list starting from the top.
-
- Enthusiast
- Posts: 266
- Joined: Sun Apr 07, 2013 11:30 pm
Re: Which words are the most useful to know?
Just a quick note from me to say that probabilities are really weird and unintuitive, and in fact the lack of repeated letters at the top of Rob's list is entirely expected from a statistical point of view, assuming no shenanigans in the shuffling (which is probably a dud assumption but never mind).
A single instance of a repeated letter in a selection, all other things being equal, roughly halves (!) the probability of that selection appearing.
A single instance of a repeated letter in a selection, all other things being equal, roughly halves (!) the probability of that selection appearing.
-
- Rookie
- Posts: 53
- Joined: Sun Apr 17, 2011 1:42 pm
- Location: Bletchley
Re: Which words are the most useful to know?
I would always encourage any suggestions as to how to make my list more accurate! So thanks for taking the time to make these insights.
My general philosophy was to make a list that's as objectively correct as possible, with no artificial tweaks applied after looking at the list to reinforce what 'should' be at the top according to people's preconceptions. You can't get a lot more generalised than 'look at a lot of (75 million) Countdown rounds and count how many times the words come up' so this is the basis of the approach I used.
Using this method, there are (I think) only three things that could significantly change the result of the simulation:
- The letters distribution. I borrowed the distribution that Graeme suggested on the Ask Graeme thread, which he arrived at from looking at frequency counts of letters over a recent series. Here it is FWIW: [B:2, C:3, D:6, F:2, G:4, H:2, J:1, K:1, L:5, M:4, N:8, P:4, Q:1, R:9, S:9, T:9, V:2, W:2, X:1, Y:1, Z:1] [A:15, E:20, I:13, O:13, U:7]. While the CD letters distribution does appear to fluctuate slightly over time, it's nothing major.
- The weightings of letter picks. I designed my list to be customisable in this respect because picking impacts word usefulness enormously; there's no point in learning a lot of 5V nines if you and your opponent barely ever pick it, and vice versa. The default is [5V:10%, 4V:60%, 3V:30%] (although your doc might be working from an older version which used equal weightings for all three vowel picks?). I took this ratio from apterous picking data.
- The shuffling algorithm. As Jack mentioned, the show (and apterous, almost certainly) takes steps to ensure that the same letters are less likely to appear consecutively in the deck. The method I used to emulate this is: "draw a letter, if it's identical to the last letter, discard it and draw again. Keep whatever this new letter is, even if it's identical again". Shuffling is the aspect which I have looked into the least though. If Charlie would be willing to share the exact letters distribution used on apterous or the shuffling algorithm with me, I'd happily re-run my simulations with these parameters. I imagine this is apterous secret sauce though. I think my method is a good approximation otherwise.
Answers to a couple of other questions that cropped up:
- It's just me that works on this
- I update the lexicon and run a new simulation whenever apterous publishes its dictionary changes.
My general philosophy was to make a list that's as objectively correct as possible, with no artificial tweaks applied after looking at the list to reinforce what 'should' be at the top according to people's preconceptions. You can't get a lot more generalised than 'look at a lot of (75 million) Countdown rounds and count how many times the words come up' so this is the basis of the approach I used.
Using this method, there are (I think) only three things that could significantly change the result of the simulation:
- The letters distribution. I borrowed the distribution that Graeme suggested on the Ask Graeme thread, which he arrived at from looking at frequency counts of letters over a recent series. Here it is FWIW: [B:2, C:3, D:6, F:2, G:4, H:2, J:1, K:1, L:5, M:4, N:8, P:4, Q:1, R:9, S:9, T:9, V:2, W:2, X:1, Y:1, Z:1] [A:15, E:20, I:13, O:13, U:7]. While the CD letters distribution does appear to fluctuate slightly over time, it's nothing major.
- The weightings of letter picks. I designed my list to be customisable in this respect because picking impacts word usefulness enormously; there's no point in learning a lot of 5V nines if you and your opponent barely ever pick it, and vice versa. The default is [5V:10%, 4V:60%, 3V:30%] (although your doc might be working from an older version which used equal weightings for all three vowel picks?). I took this ratio from apterous picking data.
- The shuffling algorithm. As Jack mentioned, the show (and apterous, almost certainly) takes steps to ensure that the same letters are less likely to appear consecutively in the deck. The method I used to emulate this is: "draw a letter, if it's identical to the last letter, discard it and draw again. Keep whatever this new letter is, even if it's identical again". Shuffling is the aspect which I have looked into the least though. If Charlie would be willing to share the exact letters distribution used on apterous or the shuffling algorithm with me, I'd happily re-run my simulations with these parameters. I imagine this is apterous secret sauce though. I think my method is a good approximation otherwise.
Answers to a couple of other questions that cropped up:
- It's just me that works on this
- I update the lexicon and run a new simulation whenever apterous publishes its dictionary changes.
-
- Post-apocalypse
- Posts: 13258
- Joined: Mon Jan 21, 2008 10:37 pm
Re: Which words are the most useful to know?
The shuffling shenanigans seem to be geared towards making it even less likely you'll get repeated letters. So while it may be a dud assumption that there are no shenanigans, the shenanigans push things further in the direction you're talking about.Sam Cappleman-Lynes wrote: ↑Sun Jan 24, 2021 12:39 pm Just a quick note from me to say that probabilities are really weird and unintuitive, and in fact the lack of repeated letters at the top of Rob's list is entirely expected from a statistical point of view, assuming no shenanigans in the shuffling (which is probably a dud assumption but never mind).
A single instance of a repeated letter in a selection, all other things being equal, roughly halves (!) the probability of that selection appearing.
-
- Newbie
- Posts: 27
- Joined: Tue Jan 12, 2021 6:02 pm
Re: Which words are the most useful to know?
Anyone knows how the letters are shuffled on Countdown?, surely they don't just pick random letters up or we would end up with 3 Zs 2 Qu etc!
- L'oisleatch McGraw
- Devotee
- Posts: 930
- Joined: Sun Dec 13, 2015 2:46 am
- Location: Waterford
- Contact:
Re: Which words are the most useful to know?
Well, the replies that have come in since, have me convinced that it is currently as near perfect as it needs to be... So I can put to bed any lingering reservations and power ahead with the YouTube project with full gusto.Robert Foster wrote: ↑Sun Jan 24, 2021 4:16 pm - It's just me that works on this
- I update the lexicon and run a new simulation whenever Apterous publishes its dictionary changes.
To conclude, you are a fucking superstar imo.
![Smile :)](./images/smilies/icon_e_smile.gif)
Thanks for the resource and for being so willing to hear opinions, make tweaks etc.
------------------------------------------------------------
[Edited to answer this question: "although your doc might be working from an older version which used equal weightings for all three vowel picks?". No, when you mentioned there was a new version, I got the latest (V4?) before making the 3 lists above.]
![Arrow :arrow:](./images/smilies/icon_arrow.gif)
![Arrow :arrow:](./images/smilies/icon_arrow.gif)
![Arrow :arrow:](./images/smilies/icon_arrow.gif)
-
- Enthusiast
- Posts: 266
- Joined: Sun Apr 07, 2013 11:30 pm
Re: Which words are the most useful to know?
I had a game in which 5 C's were drawn, and I think someone has also mentioned to me a game with 6 C's, which is double what you use in your simulations!Robert Foster wrote: ↑Sun Jan 24, 2021 4:16 pm While the CD letters distribution does appear to fluctuate slightly over time, it's nothing major.
For my own lists I wrote code that would scrape wiki data for the latest series and come up with a letters distruibution based on that, but I must have just written that as a one-off throwaway because I can't seem to find it any more. Otherwise, I would have offered to share it.
- Charlie Reams
- Site Admin
- Posts: 9494
- Joined: Fri Jan 11, 2008 2:33 pm
- Location: Cambridge
- Contact:
Re: Which words are the most useful to know?
The apterous algorithm is not a huge trade secret, but it is intended to shuffle a deck for use in a full game rather than a particular round, so certain things would shake out differently. (For example, apterous builds the deck for the entire game in one pass so duplicate aversion would span rounds, e.g. if the last vowel of one round is A, it's slightly less likely that the first vowel of the next round is A.) It's definitely less stringent at avoiding doubles and trebles than your approach and has a bunch of other tweaks, but my approach is also a best-guess so it's perfectly possible that yours is a better simulation of Countdown itself.Robert Foster wrote: ↑Sun Jan 24, 2021 4:16 pm - The shuffling algorithm. As Jack mentioned, the show (and apterous, almost certainly) takes steps to ensure that the same letters are less likely to appear consecutively in the deck. The method I used to emulate this is: "draw a letter, if it's identical to the last letter, discard it and draw again. Keep whatever this new letter is, even if it's identical again". Shuffling is the aspect which I have looked into the least though. If Charlie would be willing to share the exact letters distribution used on apterous or the shuffling algorithm with me, I'd happily re-run my simulations with these parameters. I imagine this is apterous secret sauce though. I think my method is a good approximation otherwise.
If you're interested, I'd be happy to give you a big file of all the selections from the last N years (since whenever I last changed the algorithm) and you could use those as raw data, rather than trying to simulate all the vagaries of letter picking, de-dupe etc.
Do you simulate the full 11 rounds every time? This is probably a very marginal effect, but one of the consequences of active de-duplication is that you will tend to push common letters to later in the game (once the dupes start to "stack up" at the end of the deck and become harder to avoid), which shifts the distribution slightly.
-
- Rookie
- Posts: 53
- Joined: Sun Apr 17, 2011 1:42 pm
- Location: Bletchley
Re: Which words are the most useful to know?
My sim generates every round in isolation - the consonant and vowel decks are reshuffled after every round, and duplicate aversion doesn't span rounds. So it's possible for a Q to come up in several consecutive rounds, unlike TV Countdown, but the 'stacking up' effect that you mention is avoided.
Would love to run some apterous rounds through my sim - I've sent you an aptomail!
Would love to run some apterous rounds through my sim - I've sent you an aptomail!
Re: Which words are the most useful to know?
Would be super interested to see if the simulator generates a significantly different word ordering when letters are generated more like Charlie describesRobert Foster wrote: ↑Sun Jan 24, 2021 9:07 pm My sim generates every round in isolation - the consonant and vowel decks are reshuffled after every round, and duplicate aversion doesn't span rounds. So it's possible for a Q to come up in several consecutive rounds, unlike TV Countdown, but the 'stacking up' effect that you mention is avoided.
Would love to run some apterous rounds through my sim - I've sent you an aptomail!
Re: Which words are the most useful to know?
Also Rob, how many rounds do you simulate and how long does it take to run.
-
- Rookie
- Posts: 53
- Joined: Sun Apr 17, 2011 1:42 pm
- Location: Bletchley
Re: Which words are the most useful to know?
About an hour for 25 million random rounds. The plan is to run my usefulness sim on 1) every round in TV Countdown history 2) a bunch of apterous rounds and 3) randomly generated rounds, as I have been using for my doc. Then we can draw some sweet sweet comparisons.
- L'oisleatch McGraw
- Devotee
- Posts: 930
- Joined: Sun Dec 13, 2015 2:46 am
- Location: Waterford
- Contact:
Re: Which words are the most useful to know?
I understand that the best way for someone to learn these would be to go through the list from top to bottom in whatever sorts of chunks seem manageable ...however not everyone likes to do it that way. If you were to learn the words in chunks of 100 by letter count (so "Essential nines Vol. 1" being the 100 most useful nines, "Essential sevens Vol. 3" being the 201st-300th most useful sevens, etc.)
If someone were to make a series based on that system, you then have to wonder which ones to concentrate on first, and here's where things get weird.
Looking at the word rank, here's where the 100th most useful of the nines, eights, sevens, and sixes rank overall:-
100th nine is 1536th
100th eight is 301st (outranks all but the 1st 10 nines)
100th seven is 169th (outranks all but the 1st 5 nines)
100th six is 926th (outranks all but the 1st 54 nines)
According to that, if you are looking to maximise value, you should learn all of:
-The first 100 sixes (1 volume)
-The first 700 sevens (7 volumes)
-The first 500 eights. (5 volumes)
before getting to the 100 best nines!
-----------------------------------------------
Now, because the nines are worth double points in the game (and also harder to see in selections than sevens), I would have prioritised the first 100 nines as being the most important of those lists to learn, and then moved on to a volume or two of sevens and eights, before coming to volume 2 of the nines...
Using this system, surely "Essential nines Vol. 1" should not be ranked as the 14th most important volume in the series? Seems off.
If someone were to make a series based on that system, you then have to wonder which ones to concentrate on first, and here's where things get weird.
Looking at the word rank, here's where the 100th most useful of the nines, eights, sevens, and sixes rank overall:-
100th nine is 1536th
100th eight is 301st (outranks all but the 1st 10 nines)
100th seven is 169th (outranks all but the 1st 5 nines)
100th six is 926th (outranks all but the 1st 54 nines)
According to that, if you are looking to maximise value, you should learn all of:
-The first 100 sixes (1 volume)
-The first 700 sevens (7 volumes)
-The first 500 eights. (5 volumes)
before getting to the 100 best nines!
-----------------------------------------------
Now, because the nines are worth double points in the game (and also harder to see in selections than sevens), I would have prioritised the first 100 nines as being the most important of those lists to learn, and then moved on to a volume or two of sevens and eights, before coming to volume 2 of the nines...
Using this system, surely "Essential nines Vol. 1" should not be ranked as the 14th most important volume in the series? Seems off.
![Arrow :arrow:](./images/smilies/icon_arrow.gif)
![Arrow :arrow:](./images/smilies/icon_arrow.gif)
![Arrow :arrow:](./images/smilies/icon_arrow.gif)
-
- Rookie
- Posts: 53
- Joined: Sun Apr 17, 2011 1:42 pm
- Location: Bletchley
Re: Which words are the most useful to know?
This sounds about right. Generally, sevens and eights are much more useful than nines because:L'oisleatch McGraw wrote: ↑Tue Jan 26, 2021 2:37 am According to that, if you are looking to maximise value, you should learn all of:
-The first 100 sixes (1 volume)
-The first 700 sevens (7 volumes)
-The first 500 eights. (5 volumes)
before getting to the 100 best nines!
- Nines only come up in 7% of rounds, compared to 34% where the max is eight and 41% where the max is seven (according to apto)
- A single eight can be the most useful word in many different rounds. For example GODETIAS is being logged as the most useful word in the following rounds [GODETIASA, GODETIASB, GODETIASC... etc.] (All except GODETIASR, where it is outstripped in usefulness by ASTEROID, and GODETIASM/GODETIASP where DOGMATISE/PAGODITES are available). This effect is magnified further for the most useful sixes and sevens, whereas you don't get this effect at all for nines.
- You can play sevens and eights even when they're not the max, although my doc doesn't account for this
Words are already being weighted by how many points they score (x18 for nines, x8 for eights etc.). I don't factor in anything like how easy a word is to spot. You can practise from the list however you like of course, but I still maintain that the best order in which to learn the list is, um, the order of the listL'oisleatch McGraw wrote: ↑Tue Jan 26, 2021 2:37 am Now, because the nines are worth double points in the game (and also harder to see in selections than sevens), I would have prioritised the first 100 nines as being the most important of those lists to learn, and then moved on to a volume or two of sevens and eights, before coming to volume 2 of the nines...
![Wink ;)](./images/smilies/icon_e_wink.gif)
- L'oisleatch McGraw
- Devotee
- Posts: 930
- Joined: Sun Dec 13, 2015 2:46 am
- Location: Waterford
- Contact:
Re: Which words are the most useful to know?
Yep, understood. All good points.Robert Foster wrote: ↑Tue Jan 26, 2021 9:40 am - Nines only come up in 7% of rounds, compared to 34% where the max is eight and 41% where the max is seven (according to apto)
- A single eight can be the most useful word in many different rounds.
- You can play sevens and eights even when they're not the max, although my doc doesn't account for this
Haha, of course...Robert Foster wrote: ↑Tue Jan 26, 2021 9:40 am You can practise from the list however you like of course, but I still maintain that the best order in which to learn the list is, um, the order of the list![]()
![Laughing :lol:](./images/smilies/icon_lol.gif)
I am going to make those videos with the volumes of "Essential nines" " Essential eights" etc... and they will be based on your list, because it is obviously deadly. But -full disclosure- I will be putting "Essential nines: Vol1" earlier in the series than 14th. It just feels right (even though that feeling may be rooted in stupidity. I am a simple organism with simple ways.)
I play a few opponents regularly on Apto, some of whom I expect to be tough, and others who I think I have a fair chance against. One opponent in particular stands out... the lad who uses the name Matthew Rutherford. We're quite evenly matched, but there was a time (around two months ago) when he was rated 100-200 rating points below me, yet I feared him above all...
![Confused :?](./images/smilies/icon_e_confused.gif)
Another thing happened... around an hour ago actually, that made me sit up and take note. If you're familiar with some of the ancient Greek myths, you'll remember they placed great faith in portents and omens... well, I'm like that... superstitious. (e.g. I waited till S77 to go on CD, because I felt that was a lucky number.) Well, I was catching up on the latest episode of Countdown, and still wondering whether to do "Essential nines Vol:1" as the 14th or 1st episode in the series, when DICENTRAS came up as a nine on the show. I have taken that as a sign from the Gods to trust my gut. I know it makes little sense from a statistical point of view... but perhaps in a few months something WILL crop up that changes the algorithm of the CD usefulness list to fall more in line with favouring nine letter words and certain duplicate common letters.
One way or another... because I am going to base these videos on the list you've made, I felt I owed you a full explanation of why I am going to deviate a little bit from the recommended study order. Please do not feel any need to respond to this strange ramble. I've already been annoying enough. But now that the explanation is out of my system... I will shut up, hopefully!
![Smile :)](./images/smilies/icon_e_smile.gif)
![Arrow :arrow:](./images/smilies/icon_arrow.gif)
![Arrow :arrow:](./images/smilies/icon_arrow.gif)
![Arrow :arrow:](./images/smilies/icon_arrow.gif)
-
- Rookie
- Posts: 71
- Joined: Sun Feb 10, 2019 9:41 am
Re: Which words are the most useful to know?
This is an interesting point about the relative value of 9's. To my mind, they do seem to be worth more than the raw score of 18 (as used in Rob's spreadsheet) would indicate. In a tight game, getting the 9 that your opponent does not will frequently be the difference between winning and losing the game - not only from the large score increase, but the psychological effect that can have on your opponent to suddenly be 18 points worse off. That just isn't the same if you lose a round to a 7 or 8 letter word.
I remember Sam CL commented recently that he'd learnt almost as many 9's as other words (?), so a far higher proportion than Rob's list would indicate, suggesting he probably agrees this theory about the value of 9's, as I think many others do too. But then, Sam lost the final due to three 8-letter words that Luke spotted - not a 9 in sight... (But then perhaps another day if a rare 9 had come up, maybe Sam would have got a huge benefit)
I remember Sam CL commented recently that he'd learnt almost as many 9's as other words (?), so a far higher proportion than Rob's list would indicate, suggesting he probably agrees this theory about the value of 9's, as I think many others do too. But then, Sam lost the final due to three 8-letter words that Luke spotted - not a 9 in sight... (But then perhaps another day if a rare 9 had come up, maybe Sam would have got a huge benefit)
-
- Enthusiast
- Posts: 266
- Joined: Sun Apr 07, 2013 11:30 pm
Re: Which words are the most useful to know?
My point about the relative value of 9s 1) depends on the exact structure of the game and 2) is not based in actual evidence.
But roughly the idea is that if you miss a 9 your opponent spots, you need to beat them in (probably) two subsequent rounds AND beat them to the conundrum to win. Whereas if you miss an 8 you can still win just by beating them to the conundrum.
But roughly the idea is that if you miss a 9 your opponent spots, you need to beat them in (probably) two subsequent rounds AND beat them to the conundrum to win. Whereas if you miss an 8 you can still win just by beating them to the conundrum.
- Charlie Reams
- Site Admin
- Posts: 9494
- Joined: Fri Jan 11, 2008 2:33 pm
- Location: Cambridge
- Contact:
Re: Which words are the most useful to know?
A few relevant stats from apterous (human-vs-human 15 rounders):
Presumably it would be possible to answer the 9-undervaluing question definitively with a simulation of a series of games between two players, one who follows Rob's list exactly and another who prioritizes 9s more heavily, but I suspect the effect is pretty small if it exists at all.
- A player who beats their opponent with a 9 goes on to win the game 87% of the time.
- A player who beats their opponent with a 8 goes on to win the game 73% of the time.
- A player who beats their opponent with a 7 goes on to win the game 71% of the time.
- A player who beats their opponent with a 6 goes on to win the game 67% of the time.
- A player who wins on the letters rounds overall wins the game 84% of the time.
Presumably it would be possible to answer the 9-undervaluing question definitively with a simulation of a series of games between two players, one who follows Rob's list exactly and another who prioritizes 9s more heavily, but I suspect the effect is pretty small if it exists at all.
-
- Rookie
- Posts: 53
- Joined: Sun Apr 17, 2011 1:42 pm
- Location: Bletchley
Re: Which words are the most useful to know?
Factors like letters distribution and the shuffling algorithm heavily influence what the most useful words will be. So I wanted to compare what the results of my simulation would look like given different Countdown 'environments' as input. Hopefully this can shed some like on things like whether apterous and the rounds used to generate my usefulness document are a good representation of rounds on TV Countdown, or what kinds of word are over/under-represented in my usefulness document. For simplicity I only considered 4-vowel rounds in each environment.
The four environments I looked at were:
- TV Countdown - For input, I used every 4-vowel round in a main series of TV Countdown from the start of Series 46 (the beginning of the 15-round era) up to the end of Series 81 (Haughton vs. Wing). Over 25,000 rounds.
- Apterous - For input, I used every 4-vowel round ever played on apterous in the Normal, Blind, Dogfight and Instant variants. Does not include rounds from the Duel, Super Gym and Classic Games. Over 3 million rounds (many thanks again to Charlie who gave me permission to use this data!)
- Rob's spreadsheet rounds - A sample of randomly generated 4-vowel rounds where I try to recreate 'face-up' shuffling by redrawing a letter if it's identical to the one drawn before. This is the same technique that I used to generate the rounds for my usefulness document. Over 3 million rounds.
- Co:event style rounds - A sample of randomly generated 4-vowel rounds where there is no attempt to recreate face-up shuffling, just like drawing letters from a board game at a Lincoln-style co-event. Over 3 million rounds.
I generated the top 500 most useful words for each environment.
1) This table shows how many words there are of each length among the top 500 most useful words.
Observations
- Nines come up on TV Countdown more than any of the other environments - only slightly more frequently than apterous, but much more frequently than in Rob's spreadsheet rounds. There is not a single nine among the top 500 co-event words.
- The distribution of word lengths among the top 500 is remarkably similar between TV Countdown and apterous. So I would say apterous does a fantastic job at simulating TV Countdown rounds - the shuffling algorithm and letter distribution must be close to that used on the show.
2) This table shows how many words there are among the top 500 which contain at least one pair of duplicate consonants or one pair of duplicate vowels. (e.g. RESONATE contains a duplicate vowel, LORINER contains a duplicate consonant and NEONATE contains both)
- There is a massive gap between the co:event figures and all of the others - you are far more likely to encounter duplicate letters in a round with purely random shuffling (bear this in mind next time you get four I's in a round at a co:event!). So the face-up shuffling steps taken on apterous and TV Countdown make a significant difference to gameplay.
- TV Countdown avoids duplicate letters in rounds most stringently, closely followed by apterous. Rob's spreadsheet rounds fall a bit shorter of the mark, though still nowhere near the level of purely random shuffling.
From what I've seen here it looks like my usefulness spreadsheet does undervalue 9s, as suspected by a few people in the thread (even when ignoring things like match-winning potential and psychological factors). Also my spreadsheet still overvalues words with repeated letters, despite the measures I took to avoid duplicate letters in rounds. Apterous looks like a much more faithful simulation of the kinds of round that would appear on Countdown, so going forward I think I'll use the apto rounds as raw input to generate my usefulness spreadsheet instead of my own randomly generated rounds.
The four environments I looked at were:
- TV Countdown - For input, I used every 4-vowel round in a main series of TV Countdown from the start of Series 46 (the beginning of the 15-round era) up to the end of Series 81 (Haughton vs. Wing). Over 25,000 rounds.
- Apterous - For input, I used every 4-vowel round ever played on apterous in the Normal, Blind, Dogfight and Instant variants. Does not include rounds from the Duel, Super Gym and Classic Games. Over 3 million rounds (many thanks again to Charlie who gave me permission to use this data!)
- Rob's spreadsheet rounds - A sample of randomly generated 4-vowel rounds where I try to recreate 'face-up' shuffling by redrawing a letter if it's identical to the one drawn before. This is the same technique that I used to generate the rounds for my usefulness document. Over 3 million rounds.
- Co:event style rounds - A sample of randomly generated 4-vowel rounds where there is no attempt to recreate face-up shuffling, just like drawing letters from a board game at a Lincoln-style co-event. Over 3 million rounds.
I generated the top 500 most useful words for each environment.
1) This table shows how many words there are of each length among the top 500 most useful words.
Code: Select all
Length TV apto rob co:event
6s 15 12 34 85
7s 178 186 233 269
8s 274 273 227 143
9s 33 29 6 0
- Nines come up on TV Countdown more than any of the other environments - only slightly more frequently than apterous, but much more frequently than in Rob's spreadsheet rounds. There is not a single nine among the top 500 co-event words.
- The distribution of word lengths among the top 500 is remarkably similar between TV Countdown and apterous. So I would say apterous does a fantastic job at simulating TV Countdown rounds - the shuffling algorithm and letter distribution must be close to that used on the show.
2) This table shows how many words there are among the top 500 which contain at least one pair of duplicate consonants or one pair of duplicate vowels. (e.g. RESONATE contains a duplicate vowel, LORINER contains a duplicate consonant and NEONATE contains both)
Code: Select all
TV apto rob co:event
duplicate V 159 166 208 337
duplicate C 24 31 42 133
- TV Countdown avoids duplicate letters in rounds most stringently, closely followed by apterous. Rob's spreadsheet rounds fall a bit shorter of the mark, though still nowhere near the level of purely random shuffling.
From what I've seen here it looks like my usefulness spreadsheet does undervalue 9s, as suspected by a few people in the thread (even when ignoring things like match-winning potential and psychological factors). Also my spreadsheet still overvalues words with repeated letters, despite the measures I took to avoid duplicate letters in rounds. Apterous looks like a much more faithful simulation of the kinds of round that would appear on Countdown, so going forward I think I'll use the apto rounds as raw input to generate my usefulness spreadsheet instead of my own randomly generated rounds.
-
- Post-apocalypse
- Posts: 13258
- Joined: Mon Jan 21, 2008 10:37 pm
Re: Which words are the most useful to know?
It's a good job you found this out before applying to go on the show to be honest. Imagine going on only to find out you'd completely misjudged your training programme and you ended up losing your first (and only) game with about 5 or 6 points.Robert Foster wrote: ↑Sun Jan 31, 2021 10:48 pm
From what I've seen here it looks like my usefulness spreadsheet does undervalue 9s, as suspected by a few people in the thread (even when ignoring things like match-winning potential and psychological factors). Also my spreadsheet still overvalues words with repeated letters, despite the measures I took to avoid duplicate letters in rounds. Apterous looks like a much more faithful simulation of the kinds of round that would appear on Countdown, so going forward I think I'll use the apto rounds as raw input to generate my usefulness spreadsheet instead of my own randomly generated rounds.
- L'oisleatch McGraw
- Devotee
- Posts: 930
- Joined: Sun Dec 13, 2015 2:46 am
- Location: Waterford
- Contact:
Re: Which words are the most useful to know?
Madness!Robert Foster wrote: ↑Sun Jan 31, 2021 10:48 pm From what I've seen here it looks like my usefulness spreadsheet does undervalue 9s, as suspected by a few people in the thread (even when ignoring things like match-winning potential and psychological factors). Also my spreadsheet still overvalues words with repeated letters, despite the measures I took to avoid duplicate letters in rounds.
That analysis is very cool, and the results (though the repeated letters thing was something I had doubted) speak for themselves.
One thing is certain: we are living through a very exciting time to be a CD obsessive.
What someone needs to do next (hopefully someone else, coz Rob deserves a break, and a fkn medal!) is to do this same analysis per series since S46... or if that is too fussy, per year, or per 3 years or per 5 years since S46... to see how consistent the letters selections have been over time. Has the magic formula changed at any point, or does evidence suggest the same letters distribution and shuffling algorithm have been used since 15Rers were introduced?Robert Foster wrote: ↑Sun Jan 31, 2021 10:48 pm - TV Countdown - For input, I used every 4-vowel round in a main series of TV Countdown from the start of Series 46 (the beginning of the 15-round era) up to the end of Series 81 (Haughton vs. Wing). Over 25,000 rounds.
Gavin Chipper wrote: ↑Sun Jan 31, 2021 10:59 pm Imagine going on only to find out you'd completely misjudged your training programme and you ended up losing your first (and only) game with about 5 or 6 points.
![Laughing :lol:](./images/smilies/icon_lol.gif)
![Laughing :lol:](./images/smilies/icon_lol.gif)
![Laughing :lol:](./images/smilies/icon_lol.gif)
![Arrow :arrow:](./images/smilies/icon_arrow.gif)
![Arrow :arrow:](./images/smilies/icon_arrow.gif)
![Arrow :arrow:](./images/smilies/icon_arrow.gif)
-
- Acolyte
- Posts: 123
- Joined: Sun Dec 22, 2019 10:41 pm
- Location: Birmingham's Eastern Fleapit
Re: Which words are the most useful to know?
Not always, LM! https://www.apterous.org/viewgame.php?game=2779426I play a few opponents regularly on Apto, some of whom I expect to be tough, and others who I think I have a fair chance against. One opponent in particular stands out... the lad who uses the name Matthew Rutherford. We're quite evenly matched, but there was a time (around two months ago) when he was rated 100-200 rating points below me, yet I feared him above all...because I knew he was VERY good at nines. If there was a nine, he was probably going to see it. I lost more than my fair share of games due to his quirky nine-spotting skills. Based on that experience, I think a good grounding in basic nines can make quite a difference, not only in performance, but in how a player is perceived by his peers... psychological factors.
![Smile :)](./images/smilies/icon_e_smile.gif)
I didn't think I spotted nines more often usual...My current letters max rate is just over 40%, and I do miss a corker quite often. Maybe more 9s come up in the selection when I play you? It's mostly likely luck of the draw-I'm not that good! Either or, glad to know my letters game at least puts the shits up someone
![Wink ;)](./images/smilies/icon_e_wink.gif)
The Vicar of Dudley*
*(Not ordained, nor do I live Dudley. Godspeed!)
*(Not ordained, nor do I live Dudley. Godspeed!)
-
- Series 74 Champion
- Posts: 146
- Joined: Fri Mar 20, 2009 10:59 pm
Re: Which words are the most useful to know?
Sorry to bump. I've worked with Rob using his spreadsheet previously (which is imo one of the best resources for letters round learning). A few observations, mostly on the usefulness of 9s, which I'd suggest should be higher than is currently valued:
9s importance
There are a few reasons why I think people have an instinct that learning 9s is more useful than lists of words by probability suggests - even with absolute points (6, 7, 8, 18) factored in. They are reflective of what others have put in but not entirely the same:
1. It's the value of the points for winning a letters round relative to the average margin of victory, rather than absolute number of points, or number of points relative to the average score, that is the appropriate comparison. 18 points seems even larger when you compare it relative to the average margin of victory. If say the average margin of victory of a Countdown game is 25 points, getting 18 points is a pretty huge deal compared to that, than 8 points. This is somewhat reflected in Charlie's really helpful stat regarding how often a player who spots a 9 goes on to win the game.
2. This effect is magnified in closer games. I don't speak for everyone, but certainly many people learn words to win the big games (series finals etc). These are likely to be tighter than average, as the opponent is better. If you're much stronger (or weaker) than your opponent, such that you'll probably beat (or lose to) them by over 40 points in any event, it doesn't much matter if you spot a 9 that they don't. It matters a lot more in games you expect to win or lose by 15 points or fewer on average. I would be willing to bet that in games in Apterous where each player is 1500 rated or higher, the player that spots a 9 goes onto win at a significantly higher percentage than 87%. I'd make the same bet for players within 200 rating points of each other.
3. Sam is right that the 'two letters rounds + conundrum' value of a 9 is an important factor raising the value of a 9, independently of the above two points. If you could easily get back the points of a 9, it wouldn't matter so much that 9s take up a sizeable chunk of the average margin of victory if you could get those points back quickly. But because that's not possible, their value is even higher.
So I'd advocate awarding a much higher relative value of 9s than is currently awarded on the spreadsheet. My instinct would say that in the top 1,000 words, there should be around 200 9s. But I wouldn't be that fussed if the final total is 150 or 250 (250 might be a bit much, though).
Repeated letters
I remember my similar (but less helpful than Rob's) spreadsheet overvalued repeated letters massively. I had to manually nix them. It's good to see the data on the face-up shuffling which seems to confirm this. It's also helpful to see the data on the relative frequency of nines.
Rob, you do brilliant work.
9s importance
There are a few reasons why I think people have an instinct that learning 9s is more useful than lists of words by probability suggests - even with absolute points (6, 7, 8, 18) factored in. They are reflective of what others have put in but not entirely the same:
1. It's the value of the points for winning a letters round relative to the average margin of victory, rather than absolute number of points, or number of points relative to the average score, that is the appropriate comparison. 18 points seems even larger when you compare it relative to the average margin of victory. If say the average margin of victory of a Countdown game is 25 points, getting 18 points is a pretty huge deal compared to that, than 8 points. This is somewhat reflected in Charlie's really helpful stat regarding how often a player who spots a 9 goes on to win the game.
2. This effect is magnified in closer games. I don't speak for everyone, but certainly many people learn words to win the big games (series finals etc). These are likely to be tighter than average, as the opponent is better. If you're much stronger (or weaker) than your opponent, such that you'll probably beat (or lose to) them by over 40 points in any event, it doesn't much matter if you spot a 9 that they don't. It matters a lot more in games you expect to win or lose by 15 points or fewer on average. I would be willing to bet that in games in Apterous where each player is 1500 rated or higher, the player that spots a 9 goes onto win at a significantly higher percentage than 87%. I'd make the same bet for players within 200 rating points of each other.
3. Sam is right that the 'two letters rounds + conundrum' value of a 9 is an important factor raising the value of a 9, independently of the above two points. If you could easily get back the points of a 9, it wouldn't matter so much that 9s take up a sizeable chunk of the average margin of victory if you could get those points back quickly. But because that's not possible, their value is even higher.
So I'd advocate awarding a much higher relative value of 9s than is currently awarded on the spreadsheet. My instinct would say that in the top 1,000 words, there should be around 200 9s. But I wouldn't be that fussed if the final total is 150 or 250 (250 might be a bit much, though).
Repeated letters
I remember my similar (but less helpful than Rob's) spreadsheet overvalued repeated letters massively. I had to manually nix them. It's good to see the data on the face-up shuffling which seems to confirm this. It's also helpful to see the data on the relative frequency of nines.
Rob, you do brilliant work.
Re: Which words are the most useful to know?
A fun way to settle the 9s debate would be to write some bots which have different lexicons and make them have a tournament against each other
-
- Series 74 Champion
- Posts: 146
- Joined: Fri Mar 20, 2009 10:59 pm
Re: Which words are the most useful to know?
On reflection, I think this bet would be a loser on the basis that the games that would be cut out are the ones where a much stronger player wins by like 50 and spots a 9, but the 9 spot didn't make a difference to the outcome. The point though is that the more important games are exactly the ones where a 9 is more likely to be determinative of victory, and therefore are the ones worth learning the 9s for.Paul Erdunast wrote: ↑Mon Mar 01, 2021 2:09 pm I would be willing to bet that in games in Apterous where each player is 1500 rated or higher, the player that spots a 9 goes onto win at a significantly higher percentage than 87%. I'd make the same bet for players within 200 rating points of each other.
- L'oisleatch McGraw
- Devotee
- Posts: 930
- Joined: Sun Dec 13, 2015 2:46 am
- Location: Waterford
- Contact:
Re: Which words are the most useful to know?
You mention conundrums... If you look at CD holistically (rather than focusing on letters rounds in isolation) 9s become even more important again, as every conundrum that will be set (and often be decisive) will be a 9 letter word. More incentive to learn the 9s first.Paul Erdunast wrote: ↑Mon Mar 01, 2021 2:09 pm 3. Sam is right that the 'two letters rounds + conundrum' value of a 9 is an important factor raising the value of a 9.
Well... that cannot be emphasised enough!
But Paul, so do you.
I imagine I'm not the only player who owes you a debt of gratitude. Your list fortuitously came out in 2017 just as I was preparing for TV. It helped focus the last minute study... and -maybe the 8 wins would have happened anyway- but I would not fancy switching my life into the alternate dimension where I went on CD without reaping the benefits of your word list, to find out for certain.
![Arrow :arrow:](./images/smilies/icon_arrow.gif)
![Arrow :arrow:](./images/smilies/icon_arrow.gif)
![Arrow :arrow:](./images/smilies/icon_arrow.gif)
-
- Rookie
- Posts: 53
- Joined: Sun Apr 17, 2011 1:42 pm
- Location: Bletchley
Re: Which words are the most useful to know?
New version with the March dictionary update:
https://drive.google.com/drive/folders/ ... OBoD4LcS_4
https://drive.google.com/drive/folders/ ... OBoD4LcS_4
- L'oisleatch McGraw
- Devotee
- Posts: 930
- Joined: Sun Dec 13, 2015 2:46 am
- Location: Waterford
- Contact:
Re: Which words are the most useful to know?
Rob, a stat I think would be really useful to help people decide how deep into this list they should go is a cumulative table/graph of that shows how often the most common max in a round is below a certain threshold.
For example (numbers completely made up) I imagine something like
Top 1000: 50%
Top 2000: 60%
Top 3000: 65%
Top 4000: 67%
Top 5000: 69%
Top 10000: 75%
Top 15000: ...
(So if a person knew only the top 4000 word sand had perfect recall/spotting ability you'd expect their letters max rate to be 67% if the above data was true)
For example (numbers completely made up) I imagine something like
Top 1000: 50%
Top 2000: 60%
Top 3000: 65%
Top 4000: 67%
Top 5000: 69%
Top 10000: 75%
Top 15000: ...
(So if a person knew only the top 4000 word sand had perfect recall/spotting ability you'd expect their letters max rate to be 67% if the above data was true)