c4countdown

I've been doing a somewhat interesting investigation using Quackle in French. I hope to replicate it in English when I have the time (or more realistically, patience!). Basically I got Quackle's championship player to play against Quackle's championship player. If both players are identical and use exactly the same criteria to choose their moves, at the end of the game the difference due to what I call the "random chance variable" as I think luck is a subjective word.

The results are definitely interesting. Quackle B beat Quackle A 21-19 with just 10 points of spread. But if you ignore the negative numbers, it comes out as 3894 which is 97.35 points a game. The most notable game was Quackle A beating Quackle B by 356 points. When you've got the best AI Scrabble player in the world and it scores half as much as its opponent, it's fair to say there's some chance in that. Oh the median was 90.50.

This doesn't really surprise me to be honest. From analysing my own games on the ISC, you can quite lose five games in a row and not play worse than when you win five games in a row. I wouldn't have said as high as 97.35 points per game though. It explains a lot when you have two players with the same rating why one can beat the other by 150 points then the next game its the other way around.

I was doing some notes to go with the results and it's basically turned into a full essay now, in French obviously. And in my conclusion I've put in a lot of factors, i.e. human player don't play like computers, human players can often have large differences in ability whereas if you play a computer against itself, it should have about the same average score. If you put a computer with a 380 average and 480 average against each other, you'd expect the stronger one to win a lot more than 50% - more like 80%. I just think if you compared this to chess, with this difference in ability the stronger player could win 100 in a row whereas in Scrabble that's not possible - if it were Countdown you could do it. I've won every game of Countdown I've ever played against my girlfriend but she can beat me one game in 4 at Scrabble. However in my conclusion I've made it clear that the main reason someone loses a Scrabble game is because they miss more words than their opponent.

Martin

I don't really know much about Scrabble but I enjoy the odd casual game. By "Quackle A" you are presumably referring to the player who goes first? I noticed you haven't mentioned this in your post - is it considered an advantage or disadvantage in Scrabble? Or doesn't it matter in the long run?

Well no I think it's random starts, it certainly wasn't one player starting every game. It's statistically an advantage to go first for a few reasons. Firstly you can have one more go than your opponent. Imagine you've played 12 moves each, player one plays all his tiles on his 13th move, player two doesn't get a 13th move. Secondly you can dictate the shape of the game, you could choose something like BUG which blocks the board a bit or something like LAZIER which opens it. Thirdly if you have terrible letters you can just change and go second, it's like going second but with better letters.

Martin

I don't play Scrabble that much, but I honestly didn't think it would've mattered much if you played 1st or 2nd as long as you get good letters and can play a few bingoes too.

How is it different from luck in Countdown?

The letters game on Countdown is like a non-infinite version of Scrabble.

Why does it seem like everything in UK culture is different from everything in UK culture within reason?

Jason Larsen wrote:How is it different from luck in Countdown?

There's no luck in Countdown* in this way because both players work with the same set of letters.

(* sometimes there may be luck involved in which player is asked to declare first)

Anyway, I think it's pertinent in your analysis Martin to record whether the first or second player wins, I can't believe you ignored that. Go back and start again

Jason Larsen wrote:Why does it seem like everything in UK culture is different from everything in UK culture within reason?

I really can't answer that

Corby wrote:Anyway, I think it's pertinent in your analysis Martin to record whether the first or second player wins, I can't believe you ignored that. Go back and start again

Exactly - you might find your "random chance variable" is greatly reduced if you take this into account.

But, I think if you're smart enough, you can do very well at Countdown. It's just your luck.

Also, there was a TV version of Scrabble on 25 years ago. It was hosted by Chuck Woolery, who hosted the revived version of Lingo 6 years ago (I was told you did get that in the UK,) and he was the original host of Wheel of Fortune here in the US. Scrabble just ran its course.

This remains a topic close to your heart, I see. I applaud your efforts to bring in some statistical basis.

Martin Gardner wrote:The results are definitely interesting. Quackle B beat Quackle A 21-19 with just 10 points of spread. But if you ignore the negative numbers, it comes out as 3894 which is 97.35 points a game. The most notable game was Quackle A beating Quackle B by 356 points. When you've got the best AI Scrabble player in the world and it scores half as much as its opponent, it's fair to say there's some chance in that. Oh the median was 90.50.

What are you trying to measure? The mean winning margin? 40 games doesn't seem that many. It might be informative to describe the full distribution of winning margins - how many games are close? How many games are still winnable in the pre-endgame? How many games are total whitewashes?

Also what's the correlation between blanks, bonuses and winning percentages/margins?

If you put a computer with a 380 average and 480 average against each other, you'd expect the stronger one to win a lot more than 50% - more like 80%.

Where did you get that statistic from?

Jason Larsen wrote:Also, there was a TV version of Scrabble on 25 years ago. It was hosted by Chuck Woolery, who hosted the revived version of Lingo 6 years ago (I was told you did get that in the UK,) and he was the original host of Wheel of Fortune here in the US. Scrabble just ran its course.

We have had Lingo in the UK and we should have it back .

Also we had a UK version of TV Scrabble, where some minor won a holiday in one of the series .

I know, with Martin Walker, Martin Daniels, Toby Anstis and Eamonn Holmes.

Well you're wrong and right. If you're trying to measure "luck" (ok I did say random chance variable) then things like one player starting first, one player having both blanks, those are things that happen in real tournaments and online games. The main argument on the forum has been that spread is not very important - what matters is winning games, not spread and 21-19 is a very fair result. I suppose I could do it again and have player A as always the player that goes first, no matter which bot it is. Oh and I can do more sims but I'm currently doing it in English with SOWPODS - after 10 the average spread is about 110 points a game which is more than in French, which is the opposite of what I expected to see. And frankly I'm not trying to find an accurate figure, I'm rather hoping that someone will agree / disagree strongly enough to redo the simulation and see if the results are the same.

The conclusion however does fit in with my personal experience. Only a weak correlation between "playing well" relative to one's own standards and winning games. If I win 6 games in a row it's generally not because I'm missing fewer words than I do at other times, it's just me having better letters than my opponent.

Martin

In fairness no I do like the game and I'm not really biased one way or the other. In French with have http://en.wikipedia.org/wiki/Duplicate_Scrabble which is more popular by about a 15:1 ratio (21 000 rated players in duplicate, 1500 players in match play) and match play has a reputation for being nearly all luck, which I think is totally unfair. For a start Pierre-Olivier Georget has won the national title 3 times in 5 years, and finished second one year as well. There's no doubt that a player who plays very well will usually beat a player who's weaker but certainly if you took two players with different ratings, what difference would you expect? Maybe over 100 games one player would win 60 and the other 40. But if it's just 5 games, one player might win all five. It's a bit like tossing a biassed coin. Head might come up 60 times out of 100 but tails might come up for the first five tosses. On the other hand I know Hervé Bohbot who Soo and Craig will know a little bit about, will rubbish anything that says match play contains luck without giving any arguments to the contrary.

But no it doesn't really matter and that fact is I enjoy the game and I like the thinking element. I like to work out the probabilites as I go along, I like the tactics, I like the friendliness of the game. I mean most game have some luck element, in Texas hold 'em there's an enormous amount of luck (a quote from the 'Devilfish' on the channel 5 poker said "60% luck") and bridge I suppose as well, there's a lot of luck, but it doesn't stop people playing. Plus I've never heard of anyone playing for $10 000 over a game of Scrabble. Plus duplicate is good for those who like it, but it's also slow (about 2 hrs for a game) and can be intimidating as well. The average score for a duplicate game should be about 950, and I can imagine a lot of players could only get 500 or 600. It's gonna be a bit depressing if the average score is 80% and you're doing 60%, but if it's a match play game you can play against someone else with the same average score and have a very entertaining game. So no, I'm not advocating people don't play because there's a significant luck (chance, randomness, whatever) element to it.

Martin

That's right!

If you know English very well, you're good at the game and if you're lucky, you pick the right letters.

Jason Larsen wrote:That's right!

If you know English very well, you're good at the game and if you're lucky, you pick the right letters.

Not only English. There are quite a few foreign words you can have too.

I thought foreign words weren't allowed.

Jason Larsen wrote:I thought foreign words weren't allowed.

http://www.absp.org.uk/words/languages.html

Here is all the words, in different languages that are allowed in Scrabble .

I know from one of the computer simulations I just did, the only two words I knew the meaning of in the whole games were REPULSE and EX! All of the other ones were obscure. I know in one of the games it played MINDFUCK^ which I thought was hilarious, although I don't know exactly what it means it's not too hard to make a guess!

Martin

I was talking about Countdown, I'm sorry.

Jason Larsen wrote:I was talking about Countdown, I'm sorry.

My bad, I thought your reply after Martin's post was to do with his Scrabble post (to do with the luck involved) etc.

Martin Gardner wrote:I know from one of the computer simulations I just did, the only two words I knew the meaning of in the whole games were REPULSE and EX! All of the other ones were obscure. I know in one of the games it played MINDFUCK^ which I thought was hilarious, although I don't know exactly what it means it's not too hard to make a guess!

Never watched Total Recall?

Anyway yea I've always been against best of 5 finals in Scrabble. As I've said before I would much rather the player's record from the main part of the competition was carried over to the best of 5 bit. There's enough luck in Scrabble as it is without exaggerating it via a shoddy, amateurish format.

That's ok, Joseph.

Foreign words aren't allowed on Countdown, I take it.

Jason Larsen wrote:That's ok, Joseph.

Foreign words aren't allowed on Countdown, I take it.

I think its only American-spelling words that arent allowed. There will be French words, Spanish words etc that will be allowed, but probably just not as many as you can have in Scrabble.

Of course, we've borrowed words from other languages.

For Jason, you can have any word in the current Oxford Dictionary of English as long as it follows Countdown rules - i.e. no proper nouns, no abbreviations, no words with hyphens or commas in (like re-elect or wasn't, although reelect might be ok I don't know). It doesn't matter where the word's from as long as it is considered to be used in English by the dictionary writers. So passé is fine for example, or premiere. And you can have proper nouns if they're also common nouns too. Wales for example is a country, but it's also the plural of wale!

My favourite beef about film premiere is that it's a masculin noun in French and therefore should be film premier!

Martin

So, Countdown is not too hard after all.

Jason Larsen wrote:So, Countdown is not too hard after all.

To play? IMO - At home yes (very easy to play), in the stuido no (especially when nerves kick in).

Martin Gardner wrote:My favourite beef about film premiere is that it's a masculin noun in French and therefore should be film premier!

Martin

I'm confused as to how you pronounce it. Is it pruh-mear or prem-yair?

I've practiced many times, honestly.

But, I'm surprised nobody is aware of and has mentioned the most realistic way of playing Countdown at home (Hint: It involves my least favorite host of Countdown.) But, I enjoyed it nonetheless.

dinos_the_chemist wrote:

Martin Gardner wrote:My favourite beef about film premiere is that it's a masculin noun in French and therefore should be film premier!

Martin

I'm confused as to how you pronounce it. Is it pruh-mear or prem-yair?

I think American is pruh-mear and British is prem-yair. Could be wrong.

Ginger Jono wrote:

dinos_the_chemist wrote:

Martin Gardner wrote:My favourite beef about film premiere is that it's a masculin noun in French and therefore should be film premier!

Martin

I'm confused as to how you pronounce it. Is it pruh-mear or prem-yair?

I think American is pruh-mear and British is prem-yair. Could be wrong.

What about "Prem-ee-air"?

Gevin-Gavin wrote:What about "Prem-ee-air"?

I just assumed the difference between that and prem-yair was negligible. Then it becomes more dependent on accent, I would've thought.

Craig Beevers wrote:Anyway yea I've always been against best of 5 finals in Scrabble. As I've said before I would much rather the player's record from the main part of the competition was carried over to the best of 5 bit. There's enough luck in Scrabble as it is without exaggerating it via a shoddy, amateurish format.

How does it work exactly? So are you saying that if you've won three more games than your opponent in the preliminary rounds, then they would have to win 4-1 against you to force a draw? I know nothing about how it works, but I would say by doing that you might as well do away with the final. Just have a league. Maybe to hurry things up, once everyone has played each other once, you can slice of the bottom half and everyone else carries on (keeping all previous scores). Then maybe reduce the players again depending on how many you start with, but never down to just two unless you are going to start from scratch because it comes across as a bit unsatisfactory.

Basically there are too many players for everyone to play everyone. Say it's a massive tournament you play 16 games against the others players, so maybe 16 players out of 40 or something. Players - just like at Countdown competitions - are sorted by wins first, and spread second rather than just points scored*. In the World Championships and some national championships the players ranked 1st and 2nd play a best of five final. In the 2005 World Championship for example It was Adam Logan and Pakorn N (spelling?). who played a best of 5 final. Adam had won 20 out of 24 and Pakorn 18 out of 24. Adam actually won, but had Pakorn won the best of 5 3-2 then overall he would have won fewer games. I know in the Spanish world championship once they used the system Craig describes - a best of 3 final with the results simply added on to the previous games, rather than starting afresh.

*spread = total of players score minus total of the opponents' scores in the games they played directly against each other.

Martin

Have now got to 40 simulations using SOWPODS (British English) the results are :


Quackle A 20 - 20 Quackle B
Mean score : 436
Median score : 439
Total spread : +369 for Quackle A
Total spread, ignoring negative numbers : 3447
Mean spread : 86.19
Median spread : 79
Biggest spread : Quackle A won by 262 points
Smallest spread : Quackle A won by 2 points

So basically if you accept my hypothesis, about 86 points a game of spread is down to the variation in the letters selections. I'll try and get both the French and English simulations up to 100 games by the end of the Easter Holidays, which is 24 days from now.

Martin

Martin Gardner wrote:Basically there are too many players for everyone to play everyone. Say it's a massive tournament you play 16 games against the others players, so maybe 16 players out of 40 or something. Players - just like at Countdown competitions - are sorted by wins first, and spread second rather than just points scored*. In the World Championships and some national championships the players ranked 1st and 2nd play a best of five final. In the 2005 World Championship for example It was Adam Logan and Pakorn N (spelling?). who played a best of 5 final. Adam had won 20 out of 24 and Pakorn 18 out of 24. Adam actually won, but had Pakorn won the best of 5 3-2 then overall he would have won fewer games. I know in the Spanish world championship once they used the system Craig describes - a best of 3 final with the results simply added on to the previous games, rather than starting afresh.

*spread = total of players score minus total of the opponents' scores in the games they played directly against each other.

Martin

How are players sorted if they haven't played against each other but have equal wins? Overall "goal difference"?

As I've said before I don't really see the point in a "final" if it's just a continuation of the previous league - you might already know the winner before the start! I can see the point in reducing the number of players in the league as time goes on, so that the best players will come up against each other more often, while keeping all old results. It's just that I'd stop before two. Although if it did go down to two and previous results counted, I wouldn't call it a final - just the extreme extension of cutting the lower players out of the league.

Martin Gardner wrote: Quackle A 20 - 20 Quackle B
Mean score : 436
Median score : 439
Total spread : +369 for Quackle A
Total spread, ignoring negative numbers : 3447
Mean spread : 86.19
Median spread : 79
Biggest spread : Quackle A won by 262 points
Smallest spread : Quackle A won by 2 points

Publish the full scores and we'll talk some analysis.

Would it not be informative to keep track of (I appreciate this can't be done retrospectively) the letters each player had during the game?

Also, how good are the good scrabble bots? Do they just look for the best individual word they can make? Or are they more tactical (not setting up triple words and so on)?

I apologise if this doesn't work

Martin



Quackle A Quackle B Spread Spread Winners Nb. wins
1 416 429 -13 13 B 1
2 355 475 -120 120 B 2
3 604 384 220 220 A 1
4 413 440 -27 27 B 3
5 309 466 -157 157 B 4
6 576 321 255 255 A 2
7 329 468 -139 139 B 5
8 487 389 98 98 A 3
9 451 459 -8 8 B 6
10 456 391 65 65 A 4
11 510 467 43 43 A 5
12 526 438 88 88 A 6
13 397 395 2 2 A 7
14 498 482 16 16 A 8
15 424 574 -150 150 B 7
16 477 398 79 79 A 9
17 412 464 -52 52 B 8
18 343 464 -121 121 B 9
19 322 476 -154 154 B 10
20 543 440 103 103 A 10
21 327 465 -138 138 B 11
22 425 474 -49 49 B 12
23 386 409 -23 23 B 13
24 451 445 6 6 A 11
25 389 436 -47 47 B 14
26 606 344 262 262 A 12
27 487 443 44 44 A 13
28 451 361 90 90 A 14
29 425 461 -36 36 B 15
30 374 473 -99 99 B 16
31 458 370 88 88 A 15
32 475 345 130 130 A 16
33 412 490 -78 78 B 17
34 403 448 -45 45 B 18
35 519 362 157 157 A 17
36 437 371 66 66 A 18
37 511 571 -60 60 B 19
38 477 385 92 92 A 19
39 384 407 -23 23 B 20
40 393 389 4 4 A 20

Michael Wallace wrote:Would it not be informative to keep track of (I appreciate this can't be done retrospectively) the letters each player had during the game?

Also, how good are the good scrabble bots? Do they just look for the best individual word they can make? Or are they more tactical (not setting up triple words and so on)?

I can't think what I'd do with a list of all the letters. The computer player Quackle basically picks a shortlist of moves and them simulates them all to the end of the game, and comes up with a winning percentage for each move. The idea is that if the bot only makes optimal moves, the only factor that can account for the difference in the two scores in the "random chance variable" in the letters. It's obviously not watertight but it's not a bad start either.

Martin

Michael Wallace wrote:Also, how good are the good scrabble bots? Do they just look for the best individual word they can make? Or are they more tactical (not setting up triple words and so on)?

Quackle beat all-comers at last year's Man vs Machine tournament in Toronto, and continues to improve. It's pretty good

In fairness the best way to take luck out of Scrabble would be to get rid of the 50 point bonus when you play all the letters. It works the same way as the 18 point bonus in Countdown. You get some games, not every game far from it, where player A picks up 3 bingoes in 4 moves and player B has maybe two bingoes possible in the whole game. There is such a thing in the French rules as a "game without 50 point bonuses" although I've never actually tried it yet. It'd change the game a lot because people tend to "fish" for bingoes when they don't have anything good, so letters liek AEEINRT would go down in value and letters like K, J and Z would go up in value.

I wouldn't like to play that sort of Scrabble every day, but if like Countdown you were to look at the "maximum" available to each player, it would be a lot more even if you disallowed 50 point bonuses.

Martin

Martin Gardner wrote:In fairness the best way to take luck out of Scrabble would be to get rid of the 50 point bonus when you play all the letters. It works the same way as the 18 point bonus in Countdown. You get some games, not every game far from it, where player A picks up 3 bingoes in 4 moves and player B has maybe two bingoes possible in the whole game. There is such a thing in the French rules as a "game without 50 point bonuses" although I've never actually tried it yet. It'd change the game a lot because people tend to "fish" for bingoes when they don't have anything good, so letters liek AEEINRT would go down in value and letters like K, J and Z would go up in value.

I wouldn't like to play that sort of Scrabble every day, but if like Countdown you were to look at the "maximum" available to each player, it would be a lot more even if you disallowed 50 point bonuses.

Martin

While that would probably reduce the luck element it would 'shift' a lot of it onto picking the big tiles and the like.

Yes I think you're right.

Martin

Martin Gardner wrote: Quackle A Quackle B Spread Spread Winners Nb. wins
1 416 429 -13 13 B 1
2 355 475 -120 120 B 2
3 604 384 220 220 A 1
4 413 440 -27 27 B 3
5 309 466 -157 157 B 4
6 576 321 255 255 A 2
7 329 468 -139 139 B 5
8 487 389 98 98 A 3
9 451 459 -8 8 B 6
10 456 391 65 65 A 4
11 510 467 43 43 A 5
12 526 438 88 88 A 6
13 397 395 2 2 A 7
14 498 482 16 16 A 8
15 424 574 -150 150 B 7
16 477 398 79 79 A 9
17 412 464 -52 52 B 8
18 343 464 -121 121 B 9
19 322 476 -154 154 B 10
20 543 440 103 103 A 10
21 327 465 -138 138 B 11
22 425 474 -49 49 B 12
23 386 409 -23 23 B 13
24 451 445 6 6 A 11
25 389 436 -47 47 B 14
26 606 344 262 262 A 12
27 487 443 44 44 A 13
28 451 361 90 90 A 14
29 425 461 -36 36 B 15
30 374 473 -99 99 B 16
31 458 370 88 88 A 15
32 475 345 130 130 A 16
33 412 490 -78 78 B 17
34 403 448 -45 45 B 18
35 519 362 157 157 A 17
36 437 371 66 66 A 18
37 511 571 -60 60 B 19
38 477 385 92 92 A 19
39 384 407 -23 23 B 20
40 393 389 4 4 A 20

What does basically all of this mean?

Before I feed this into my Statsomatic 3000 (by which I mean, play with it in R), could you confirm who went first in each game?

Martin Gardner wrote:In fairness the best way to take luck out of Scrabble would be to get rid of the 50 point bonus when you play all the letters. It works the same way as the 18 point bonus in Countdown. You get some games, not every game far from it, where player A picks up 3 bingoes in 4 moves and player B has maybe two bingoes possible in the whole game. There is such a thing in the French rules as a "game without 50 point bonuses" although I've never actually tried it yet. It'd change the game a lot because people tend to "fish" for bingoes when they don't have anything good, so letters liek AEEINRT would go down in value and letters like K, J and Z would go up in value.

I wouldn't like to play that sort of Scrabble every day, but if like Countdown you were to look at the "maximum" available to each player, it would be a lot more even if you disallowed 50 point bonuses.

Martin

Could you run Quackle games under the different scoring systems? It would be interesting to see if it made for closer games. Or if you had bots of differing ability, it would be interesting to see if the different scoring system produces a more consistent winner (you could run the same tests using "Gevin" scoring on Countdown). You never know, maybe that's what Charlie is doing when the bots are playing each other on Apterous!

Gavin Chipper wrote:You never know, maybe that's what Charlie is doing when the bots are playing each other on Apterous!

That's just to give all the bots a full rating in every combination of variant and dictionary.

I think people might be overestimating the luck element. Pakorn won 13 games in a row during the Worlds - even if only 75% of games are winnable (as suggested on the other Scrabble thread on here) the chance of winning all 13 with faultless play is 0.75 to the power of 13=0.0237=2.37%. Even if 85% of games are winnable, the chance is 0.1209=12.9%, still very small indeed.

Martin Smith wrote:I think people might be overestimating the luck element. Pakorn won 13 games in a row during the Worlds - even if only 75% of games are winnable (as suggested on the other Scrabble thread on here) the chance of winning all 13 with faultless play is 0.75 to the power of 13=0.0237=2.37%. Even if 85% of games are winnable, the chance is 0.1209=12.9%, still very small indeed.

Dude, 12.9% is about 1 in 7, which is not very small indeed, especially when you consider that 1) it was a long tournament so Pakorn had plenty of opportunities for a streak 2) many other players with skill comparable to his did not enjoy such a streak 3) Pakorn himself has played in many other tournaments without such a streak. Given all that it would be rather miraculous if no one had won 13 on the trot, and in fact it suggests that your %age win is rather too high.

The basic fact is that 50% of games are winnable against perfect opposition (slightly fewer if we account for draws). If your actual rate is much higher than that, it merely suggests that your opposition is not very good. It doesn't tell you much about the nature of Scrabble itself.