Rachel v Carol

Clive Brooker · Post by **Clive Brooker** » Mon Apr 13, 2009 6:19 pm

This is an attempt to perform an objective statistical comparison of the respective performances of Rachel and Carol at the Countdown numbers game.

For Rachel, I have used data from her first 13 weeks (January 12th – April 10th). For Carol I have “randomly” picked the corresponding 13 weeks from 2008 (January 14th – April 11th. This equates to 61 programmes for each.

The relevant numbers games are those where neither contestant could provide an exact solution although one was possible. For Carol, over the selected period this gives 57 games, for Rachel 56.

For each of these games, a tariff is obtained from William Tunstall-Pedoe’s well-known online solver. This solver calculates a difficulty rating from 1% to 99% based on a number of criteria which can be viewed in WTP’s FAQ.

An individual’s “performance index” (hereafter PI) is defined as

PI = Σ (tariff * (solved= ‘YES’) - (100 – tariff) * (solved=’NO’)) / N

Where N is the number of games in the summation.

A PI of zero could thus be obtained by solving 4 in 5 20% games, or alternatively 1 in 5 80% games. A PI of 30 could be obtained with 100% success at 30% games or 50% success at 80% games.

Note that the maximum PI obtainable is the arithmetic mean tariff across all games.

Now the results…..

Carol solved 38 out of 57 games (66.7%)
Rachel solved 35 out of 56 games (62.5%)

The average tariff (= maximum PI) for Carol’s games was 57.1
The average tariff (= maximum PI) for Rachel’s games was 63.2

Carol’s PI = 23.8
Rachel’s PI = 25.7

So even without making any allowance for Rachel’s initial nervousness, her performances to date seem to be at least a match for Carol’s towards the end of her career.

Excluding the first three weeks (COC), Rachel’s stats become:

Solved 33 out of 49 ( 67.3%)
Average tariff 62.9
PI = 30.3

which looks formidably good.

During the selected period, Carol does seem to have the edge in the more difficult games (7/13 vs 6/19 for 80+ games) but is let down by more failures in easier games. Interestingly, Rachel has only missed two sub 50% games, both early in COC.

This is, of course, all subject to the accuracy of my work and the data. The main area of concern I have is with late solutions not being recorded as such. In particular, on February 2nd Rachel has twice been credited in CountdownWiki with high tariff solutions which were given late. In preparing the data I have generally taken the daily reviews to the most accurate source and Rachel’s results have not been inflated by solutions known to have been late.

I hope you found this interesting – I certainly did. When I started I hoped I was going to show that Carol is still the best….

Clive.

Michael Wallace · Post by **Michael Wallace** » Mon Apr 13, 2009 6:24 pm

Clive Brooker wrote:An individual’s “performance index” (hereafter PI) is defined as

PI = Σ (tariff * (solved= ‘YES’) + (100 – tariff) * (solved=’NO’)) / N

Where N is the number of games in the summation.

Perhaps I am not understanding this, but doesn't this mean your PI goes up if you miss a solution? (and indeed, goes up by a huge number if you miss a really easy one - should that + be a -?)

Clive Brooker · Post by **Clive Brooker** » Mon Apr 13, 2009 6:40 pm

Michael Wallace wrote:
Clive Brooker wrote:An individual’s “performance index” (hereafter PI) is defined as

PI = Σ (tariff * (solved= ‘YES’) + (100 – tariff) * (solved=’NO’)) / N

Where N is the number of games in the summation.
Perhaps I am not understanding this, but doesn't this mean your PI goes up if you miss a solution? (and indeed, goes up by a huge number if you miss a really easy one - should that + be a -?)

You understand perfectly -thanks. Edited.

Martin Bishop · Post by **Martin Bishop** » Mon Apr 13, 2009 7:11 pm

So, to make this absolutely clear, high score = good, low score = bad.

Michael Wallace · Post by **Michael Wallace** » Mon Apr 13, 2009 7:58 pm

I've been thinking about this, and I reckon it'd be fun (once there's more data) to plot histograms of difficulty for both solved and not solved games. Summary statistics are nice, but I reckon there's sufficient subjectivity in what people think makes someone good (is someone who consistently gets 75% difficulty ones, but is rubbish at 90% ones better or worse than someone who is amazing at the really hard ones, but (somehow) rubbish at the easier ones, for instance).

Jon O'Neill · Post by **Jon O'Neill** » Mon Apr 13, 2009 8:10 pm

Really good post. Puts to bed all the crap the papers were coming out with about her early doors.

Woo!

Charlie Reams · Post by **Charlie Reams** » Mon Apr 13, 2009 8:42 pm

Interesting, nice one. A few comments:

- WTP's difficulty-o-meter involves some pretty arbitrary constants and I don't entirely agree with it's evaluation. For example, if you have a target of 601 and it's trivial to make 600 (e.g. 6x100) then a human will typically spend most of the time trying to make 1 from the other 4 numbers, so they'll probably solve it even if that final step is quite hard. However WTP will classify this is as a hard solution because it has no sense of top-down tactics, it looks at each method as a bottom-up problem of evaluating a particular arithmetic expression (more like they have on DCeDL.)

- The difference might well reflect the fact that Carol preferred to factorise the target and make the factors, which is a "fragile" method; if you can't make even one of the factors, you're nowhere. Rachel prefers the split multiplication, which is a very robust method but occasionally misses really spectacular solutions. This would explain why Carol did better on hard ones even though Rachel was better overall.

- There was some feeling that Carol may have become weaker towards the end of her career. Have you considered looking at another block from a couple of years ago? (Series 49 is probably the earliest when we have an adequately long block of recaps.)

- Was there any correlation with the selection of 6 small/4 large/whatever?

Simon Myers · Post by **Simon Myers** » Mon Apr 13, 2009 9:15 pm

Intriguing post, it's something I'd thought about doing myself once or twice, but never bothered to carry it out.

I'm interested in the answers to Charlie's questions too. Although the data are quite sparse, it might be interesting to look at the first 28 of Rachel's numbers games versus the second set of 28 - see to what extent nerves and inexperience may have had their effects early on.

Andy Thomson · Post by **Andy Thomson** » Mon Apr 13, 2009 9:35 pm

I can't pretend to understand any of this high-falutin' flim-flam but my gut feeling is that, over the last few years Carol became increasingly deficient in working out answers that even The Missus and I could get whereas Rachel has got over her initial nerves and is now pretty masterful at getting the answers.

Also sprach the Simpleton!

Clive Brooker · Post by **Clive Brooker** » Mon Apr 13, 2009 9:36 pm

Charlie Reams wrote:Interesting, nice one. A few comments:

- WTP's difficulty-o-meter involves some pretty arbitrary constants and I don't entirely agree with it's evaluation. For example, if you have a target of 601 and it's trivial to make 600 (e.g. 6x100) then a human will typically spend most of the time trying to make 1 from the other 4 numbers, so they'll probably solve it even if that final step is quite hard. However WTP will classify this is as a hard solution because it has no sense of top-down tactics, it looks at each method as a bottom-up problem of evaluating a particular arithmetic expression (more like they have on DCeDL.)

- The difference might well reflect the fact that Carol preferred to factorise the target and make the factors, which is a "fragile" method; if you can't make even one of the factors, you're nowhere. Rachel prefers the split multiplication, which is a very robust method but occasionally misses really spectacular solutions. This would explain why Carol did better on hard ones even though Rachel was better overall.

- There was some feeling that Carol may have become weaker towards the end of her career. Have you considered looking at another block from a couple of years ago? (Series 49 is probably the earliest when we have an adequately long block of recaps.)

- Was there any correlation with the selection of 6 small/4 large/whatever?

All those points/questions had occurred to me in some form.

WTP rating: if there's something else I should use that would be easy enough to do.
As a matter of interest, the "easiest" problem in my study was rated at 8%, and was a 4 large. I didn't think it was that easy although I did get it.

Split Multiplication/Factors: interesting idea. The data is still a bit thin to draw strong conclusions along the lines you've suggested. I've always thought both are/were pretty adept at both methods.

Regarding Carol's performance over time, clearly there could be many factors. For example, after RW's death she took more overall responsibility and possibly didn't give the same priority to the numbers. I didn't notice any fall-off after the start of last year's rumblings but it might be interesting to look at the evidence.

Different selections - this wasn't part of my original plan but I can extend easily enough. Again, the data may be a little sparse for Rachel.

Since the reaction has been pretty positive, I think I shall try repeating the study with a suitable block from the RW era - as well as continuing to monitor Rachel.

Charlie Reams · Post by **Charlie Reams** » Mon Apr 13, 2009 9:58 pm

Other newbies take note: ^ This is how to be cool.

Kai Laddiman · Post by **Kai Laddiman** » Mon Apr 13, 2009 10:11 pm

Charlie Reams wrote:Other newbies take note: ^ This is how to be cool.

^ Do I win?

Clive Brooker · Post by **Clive Brooker** » Mon Apr 13, 2009 10:13 pm

Simon Myers wrote:...Although the data are quite sparse, it might be interesting to look at the first 28 of Rachel's numbers games versus the second set of 28 - see to what extent nerves and inexperience may have had their effects early on.

The numbers are as follows:

1st 28 referrals (to 20/2)

Solved 14 (50%)
Average rating 61.9
PI 11.9

2nd 28 referrals (23/2 onwards)

Solved 21 (75%)
Average rating 64.0
PI 39.5

The 7 2nd half failures were rated 88, 88, 88, 61, 66, 89 and 89.

Charlie Reams · Post by **Charlie Reams** » Mon Apr 13, 2009 10:22 pm

Clive Brooker wrote:
Simon Myers wrote:...Although the data are quite sparse, it might be interesting to look at the first 28 of Rachel's numbers games versus the second set of 28 - see to what extent nerves and inexperience may have had their effects early on.
The numbers are as follows:

1st 28 referrals (to 20/2)

Solved 14 (50%)
Average rating 61.9
PI 11.9

2nd 28 referrals (23/2 onwards)

Solved 21 (75%)
Average rating 64.0
PI 39.5

The 7 2nd half failures were rated 88, 88, 88, 61, 66, 89 and 89.

So a pretty clear improvement. Hopefully the uptrend will continue. Just out of interest, can we see the 61 and 66 games?

Paul Howe · Post by **Paul Howe** » Mon Apr 13, 2009 10:39 pm

Andy Thomson wrote:I can't pretend to understand any of this high-falutin' flim-flam but my gut feeling is that, over the last few years Carol became increasingly deficient in working out answers that even The Missus and I could get whereas Rachel has got over her initial nerves and is now pretty masterful at getting the answers.

I reckon Carol was probably coasting a bit over the last few series, I know I'd be tempted to slack off after after 20+ years of solving numbers games. She still came up with diamond solutions on a regular basis though.

And Rachel has been excellent, a worthy successor.

John Bosley · Post by **John Bosley** » Tue Apr 14, 2009 5:10 am

I don't count the first weeks because they were not normal contestants; they were octochamps and so on and they only left Rachel the impossible ones to do.

Clive Brooker · Post by **Clive Brooker** » Tue Apr 14, 2009 5:50 am

John Bosley wrote:I don't count the first weeks because they were not normal contestants; they were octochamps and so on and they only left Rachel the impossible ones to do.

I can see why you might think that, but it's not true. The only real sitter Rachel has ever missed was 25*10+75+2=327 which came in her 4th programme, the COC contest between.....

Edit: Also, the whole point of producing an index score is to give high credits for solving tough games and correspondingly small debits for missing them.

Clive Brooker · Post by **Clive Brooker** » Tue Apr 14, 2009 6:17 am

Charlie Reams wrote:...can we see the 61 and 66 games?

61 : 25 9 5 2 6 5 T 654 (March 23rd, pink dress)
66 : 25 2 8 9 6 7 T 944 (April 3rd)

Adam Dexter · Post by **Adam Dexter** » Tue Apr 14, 2009 10:22 am

Kai Laddiman wrote:
Charlie Reams wrote:Other newbies take note: ^ This is how to be cool.
^ Do I win?

No, because in no paradigm are you a newbie

PS... ^

Kai Laddiman · Post by **Kai Laddiman** » Tue Apr 14, 2009 10:52 am

Clive Brooker wrote:
John Bosley wrote:I don't count the first weeks because they were not normal contestants; they were octochamps and so on and they only left Rachel the impossible ones to do.
I can see why you might think that, but it's not true. The only real sitter Rachel has ever missed was 25*10+75+2=327 which came in her 4th programme, the COC contest between.....

John Bosley · Post by **John Bosley** » Tue Apr 14, 2009 3:50 pm

Who is

Kai? Me or Clive?

I am too old and out of my depth on this forum, but I was talking about the 'feel' of it - not the statistical analysis.

Kai Laddiman · Post by **Kai Laddiman** » Tue Apr 14, 2009 4:04 pm

John Bosley wrote:Who is Kai? Me or Clive?

I am too old and out of my depth on this forum, but I was talking about the 'feel' of it - not the statistical analysis.

Clive.

BTW IMO IIRC AFAIK http://crosswordtools.com/numbers-game/ is pretty good.

Clive Brooker · Post by **Clive Brooker** » Tue Apr 14, 2009 5:43 pm

Kai Laddiman wrote:
John Bosley wrote:Who is Kai? Me or Clive?

I am too old and out of my depth on this forum, but I was talking about the 'feel' of it - not the statistical analysis.
Clive.

BTW IMO IIRC AFAIK http://crosswordtools.com/numbers-game/ is pretty good.

I'm also old and clueless, so you'll have to educate me. Am I supplosed to retract "sitter"? I confess I didn't check whose game it was before posting!

Adam Dexter · Post by **Adam Dexter** » Tue Apr 14, 2009 6:04 pm

Kai Laddiman wrote:
John Bosley wrote:Who is Kai? Me or Clive?

I am too old and out of my depth on this forum, but I was talking about the 'feel' of it - not the statistical analysis.
Clive.

BTW IMO IIRC AFAIK http://crosswordtools.com/numbers-game/ is pretty good.

So typical of a tweenager.

By the way, in case you thought that was a different language John, I'll translate

"By the way" "In my opinion" "If I recall correctly" (I had to look that up) "As far as I know"

Kai Laddiman · Post by **Kai Laddiman** » Wed Apr 15, 2009 7:06 pm

Adam Dexter wrote:
Kai Laddiman wrote:
John Bosley wrote:Who is Kai? Me or Clive?

I am too old and out of my depth on this forum, but I was talking about the 'feel' of it - not the statistical analysis.
Clive.

BTW IMO IIRC AFAIK http://crosswordtools.com/numbers-game/ is pretty good.
So typical of a tweenager.

By the way, in case you thought that was a different language John, I'll translate

"By the way" "In my opinion" "If I recall correctly" (I had to look that up) "As far as I know"

I wouldn't trust that. Being taught like this is like being taught by Kirk; the teacher looks stuff he doesn't know up on the Internet.

Kirk Bevins · Post by **Kirk Bevins** » Wed Apr 15, 2009 10:46 pm

Kai Laddiman wrote:
I wouldn't trust that. Being taught like this is like being taught by Kirk; the teacher looks stuff he doesn't know up on the Internet.

Oi. I object to this.

Joyce Phillips · Post by **Joyce Phillips** » Thu Apr 16, 2009 6:27 pm

My impression is that Rachel is gaining in confidence and is getting most - if not all - of the possible solutions. She exudes a lot of charm and doesn't cackle in the background, which Carol was prone to do.

Jeff Clayton · Post by **Jeff Clayton** » Sun Apr 19, 2009 9:35 pm

So the contestants can't find the numbers solution, and Mr. Stelling hands the floor to Rachel. She hasn't got it either.

Fine, but I feel that she doesn't so much explain why she's missed, suggest her thought processes, provide stimulating airtime etc as suddenly discover a terrible itch on the back of her head and offer a lame excuse.

Rabbit. Headlights.

Jeff

D Eadie · Post by **D Eadie** » Mon Apr 20, 2009 9:10 am

Jeff Clayton wrote:So the contestants can't find the numbers solution, and Mr. Stelling hands the floor to Rachel. She hasn't got it either.

Fine, but I feel that she doesn't so much explain why she's missed, suggest her thought processes, provide stimulating airtime etc as suddenly discover a terrible itch on the back of her head and offer a lame excuse.

Rabbit. Headlights.

Jeff

Post of the year. That's most uninformed, ignorant and defenceless post of the year.

PS - 'times' is STILL a verb.

Jeff Clayton · Post by **Jeff Clayton** » Mon Apr 20, 2009 10:04 pm

985 from six small on Monday - I went the same way as Junaid with 123x8 for 984.

"No, couldn't get anywhere near it I'm afraid."

Afraid of what?

PS I'll overlook your own little grammatical error.

Dinos Sfyris · Post by **Dinos Sfyris** » Tue Apr 21, 2009 10:00 am

Jeff Clayton wrote:985 from six small on Monday - I went the same way as Junaid with 123x8 for 984.

"No, couldn't get anywhere near it I'm afraid."

Actually Jeff, she said IIRC that she couldn't get much closer than that. It was impossible to solve. Its completely possible she got 984, but why would we want to hear a solution for 1 away?

Derek Hazell · Post by **Derek Hazell** » Tue Apr 21, 2009 10:18 am

Dinos Sfyris wrote:Its completely possible she got 984, but why would we want to hear a solution for 1 away?

Yes, she probably remembers how Jeff still makes her do it if she is 1 away, so thought it was better to be a little mysterious.

D Eadie · Post by **D Eadie** » Tue Apr 21, 2009 10:52 am

Rachel doesn't know the answer is impossible until i tell her during the commercial break, so until that time she keeps on trying to find the answer, hence not always ideal to show a 'one away' solution and fill the white board up, when there is still time to try and solve it exactly.

Simple really.

Debbi Flack · Post by **Debbi Flack** » Tue Apr 21, 2009 3:28 pm

I, like most others on here, think she does a fabulous job. She is really relaxing into the role now and going from strength to strength. Excellent choice, Mr Eadie!

Clive Brooker · Post by **Clive Brooker** » Tue Apr 21, 2009 9:14 pm

Debbi Flack wrote:I, like most others on here, think she does a fabulous job. She is really relaxing into the role now and going from strength to strength. Excellent choice, Mr Eadie!

And the stats do seem to back this up.

I've looked at Carol in series 51 (early 2004), and extended all three blocks of data to 14 weeks to balance Rachel's career to date. I've also corrected a logical flaw whereby solutions provided by someone other than the arithmetician or the contestants were assumed to imply failure by the arithmetician. Rather irritatingly there was one reviewer in 2004 who never indicated whether the solution was given by Carol or the reviewer, so those games have to be ignored.

The summary stats are now:

Rachel has solved 39/61 (63.9%)
Carol (S58) solved 41/60 (68.3%)
Carol (S51) solved 36/60 (60.0%)

Rachel's average tariff 63.3
Carol's average tariff (S58) 57.9
Carol's average tariff (S51) 61.1

Rachel's index score 27.2
Carol's index score (S58) 26.2
Carol's index score (S51) 21.1

Rachel's index score post COC is now 31.6

So Rachel is still ahead, and Carol was less consistent a few years ago than recently. But a really bad week from Rachel could still even things up.

The data remain too sparse to see any obvious pattern from the selection type. So far Rachel has 100% success at 3 large and 0% success at 4 large (that surprised me too) and this is obviously not indicative of her true ability. She's had far more 6 small games to deal with and is at least matching Carol's success on these.

Derek Hazell · Post by **Derek Hazell** » Sat Oct 31, 2009 8:20 pm

Obviously faked, seeing as they've apparently never met.
Plus technical stuff about image quality, which I am sure Alec could explain better.

Alec Rivers · Post by **Alec Rivers** » Sat Oct 31, 2009 9:01 pm

Derek Hazell wrote:Obviously faked, seeing as they've apparently never met.
Plus technical stuff about image quality, which I am sure Alec could explain better.

The main giveaway is antialiasing – look at Carol's edges.

I've done a 1-min bodge-job to improve it slightly and make it a little more convincing:

Phil Reynolds · Post by **Phil Reynolds** » Sun Nov 01, 2009 10:43 am

Derek Hazell wrote:Obviously faked

Indeed - whoever did it couldn't even be bothered to make Carol's "shadow" fall in the same direction as Rachel's.

Sue Sanders · Post by **Sue Sanders** » Sun Nov 01, 2009 10:57 am

You've got that smudge application on your photosuite haven't you? I use it to reduce my bingo wings and narrow my waist!!! The white of the letters visible behind Carol's arm needs blueing out.

I have been thinking recently how Rachel has really found her feet now, not just in the actual maths but in terms of being relaxed and her personality shining through now. It must be helped by the fact that Jeff has stopped being so revoltingly lascivious, but I still have to say, I don't see it as a see-saw. My estimation of Rachel has come up without my estimation of Carol having to go down.

Alec Rivers · Post by **Alec Rivers** » Sun Nov 01, 2009 12:30 pm

Sue Sanders wrote:You've got that smudge application on your photosuite haven't you?

It's the Retouch tool on Paint Shop Pro (type: Soften, size: 5, opacity: 21, step: 1, density: 100, shape: Round), since you ask.

Kai Laddiman · Post by **Kai Laddiman** » Sun Nov 01, 2009 1:56 pm

Is Rachel touching Carol's bum? That's just taking it too far...

Dinos Sfyris · Post by **Dinos Sfyris** » Mon Nov 02, 2009 9:12 pm

Also the VOWEL box sign between Carol's arm and her body is completely disproportionate and geographically skew to the CONSONANT one

Derek Hazell · Post by **Derek Hazell** » Mon Nov 02, 2009 9:21 pm

Dinos Sfyris wrote:Also the VOWEL box sign between Carol's arm and her body is completely disproportionate and geographically skew to the CONSONANT one

All those carrots have done wonders for your eyesight, Dinos!

Every cloud.

Andy Clews · Post by **Andy Clews** » Tue Nov 24, 2009 6:20 pm

Sue Sanders wrote: I have been thinking recently how Rachel has really found her feet now, not just in the actual maths but in terms of being relaxed and her personality shining through now.

(Back after a long silence!)

I agree. Rachel's self-confidence has really come to the fore now, and I've been quite taken aback by how skillful she can be on the numbers board. She used to look rather lanky and ungainly but she seems to have filled out nicely now, looks less likely to topple off her high heels, wears some very nice clothes and in general scrubs up really rather well. I wonder how long it will be before she starts to branch off into other TV work, a la Carol (unless I've been asleep and she already has)? I really can't see her doing just the Countdown gig for long. She looks like she could be a pretty good presenter in her own right.

Alec Rivers · Post by **Alec Rivers** » Tue Nov 24, 2009 6:35 pm

Andy Clews wrote:She looks like she could be a pretty good presenter in her own right.

I agree, and I'd love to see her get a presenter's job. If I was a producer, I'd give her one.

Charlie Reams · Post by **Charlie Reams** » Tue Nov 24, 2009 6:42 pm

Alec Rivers wrote:
Andy Clews wrote:She looks like she could be a pretty good presenter in her own right.
I agree, and I'd love to see her get a presenter's job. If I was a producer, I'd give her one.

I really can't imagine why she's reluctant to join the forum.

Alec Rivers · Post by **Alec Rivers** » Tue Nov 24, 2009 7:39 pm

Charlie Reams wrote:
Alec Rivers wrote:
Andy Clews wrote:She looks like she could be a pretty good presenter in her own right.
I agree, and I'd love to see her get a presenter's job. If I was a producer, I'd give her one.
I really can't imagine why she's reluctant to join the forum.

Whatever happened to plausible deniability?