FOCAL scoring system

Discussion and announcements relating to unofficial Countdown competitions, held online or in real life. Observation, discussion, reflection, and other stuff ending in -ion.
Post Reply
Tim Down
Rookie
Posts: 98
Joined: Fri Feb 12, 2016 9:45 am

FOCAL scoring system

Post by Tim Down » Sun Aug 12, 2018 11:02 pm

I’d like to propose a new scoring system for FOCAL standings from 2019 onwards.

The current system works fine when there are similar attendances at each event. However, a particularly large event such as CO:LON, which has roughly double the number of players of an average event, carries so many points that it unjustifiably dwarfs a good result in a smaller event. For example, in 2018 when CO:LON was attended by 67 players and CO:Leeds by 33 players, first place at Leeds scored the same number of FOCAL points as 35th place at London. Given the quality of field at Leeds, it’s difficult to argue that these represent similar levels of achievement. I do think it’s reasonable to award a few more points for the larger events, however, so my proposal is as follows:

Each event has a maximum number of points (call it N) that can be scored that is determined and published at the start of the FOCAL year. The winner is awarded the maximum number of points (N) and everyone else’s points are distributed evenly between 0 and N based on their position. For example, if an event is worth 500 points and has 25 players, the winner gets 500 points, second gets 480, third 460 and so on down to 25th who gets 20.

The rest is a just a matter of determining how many points to award for each event. I suggest using numbers significantly higher than the number of players at the event to reduce the effect of rounding.

I’ve created a web page showing the current and proposed scoring systems, applied to the 2018 results so far. It lets you tweak the points available for each event. In my example, there are three tiers of events: tier 1, worth 600 points (COLIN, CO:LON, .co.mk), tier 2 worth 500 points (COLIN Hangover, CoBris, Co:Leeds, CoCam) and tier 3 worth 400 points (CoNuT, Co:Wat). I’ve based this on attendance in 2018. It means that larger events are still worth more than smaller ones but to a lesser degree than at present.

http://timdown.co.uk/code/countdown/focal.html

Notes

  • I couldn’t find a list of placings from all events so mine are calculated from http://greem.co.uk/cgi-bin/coevents/event.py. My numbers don’t exactly match the most recently published standings (of which I only have the top 8) so I’ve clearly got something wrong somewhere. They’re close though. (EDIT: Graeme has now kindly furnished me with accurate placings so I'll update the page soon.)
  • My own position is higher in my proposed scoring system. However, this isn’t a big surprise because what prompted me to look into this was my dismay at how much ground I’d lost after being unable to attend CO:LON. In any case, I’m not proposing changing the scoring for 2018 so I wouldn’t benefit anyway.

User avatar
Graeme Cole
Series 65 Champion
Posts: 1528
Joined: Tue Jul 06, 2010 9:59 pm

Re: FOCAL scoring system

Post by Graeme Cole » Mon Aug 13, 2018 12:32 am

I agree that the current scoring system does reward the top positions at smaller events less than it should compared to the middling positions at large events.

However, I'm not convinced that the points value of an event (number of points given to the winner) needs to be decided in advance of the event. All that means is that someone has to pluck a number out of the air based on expected turnout many months in advance, and/or some subjective view of how "prestigious" the event is. Why guess? I think it's worth keeping the direct link between the number of players and the number of points available at an event.

However, there's no reason why it has to be a linear relationship, as it is now.

You could come up with particular curve on a graph, something of the form y=ax^k, where 0 < k < 1, and a is some constant multiplier to give the y-value the desired scale. (If anyone is having trouble visualising what that actually looks like, it's a curve that starts at zero, goes up sharply to start with, then levels off. It's always going up, but at a progressively shallower gradient.)

The parameters of the curve could be chosen such that, say, the points value of a 40-person event was significantly more than the points value of a 20-person event, but the value of a 60-person event was not-so-significantly more than that of a 40-person event. So winning a larger event would still score more than winning a smaller one, but FOCAL scores would no longer be so heavily skewed by results (or non-attendance) at larger events.

One other possible avenue of exploration is to de-linearise the relationship between finishing position and points awarded - give the first placed player (say) 500 points, the second-placed player 300, the third 200, then work linearly down from there, with the intention that first place at a small event would outscore most middling positions at larger events. (Edit: did Gevin suggest something like this a while ago?) However, I haven't run any numbers on this so I have no idea whether it'll produce results that make sense. For that reason I'm hesitant to suggest that as an "improvement" based on my own vague 1am guesses when I know that Zarte did a lot of work reviewing simulations of various scoring systems on previous years to come up with the system we've got now. One problem I can already see with it is it might make the top players even harder to catch up with.

User avatar
Thomas Carey
Kiloposter
Posts: 1017
Joined: Sun Jan 09, 2011 4:17 pm
Location: North-West of Bradford
Contact:

Re: FOCAL scoring system

Post by Thomas Carey » Mon Aug 13, 2018 7:27 am

viewtopic.php?f=10&t=11281&p=165157#p162047

This was Gev's thing. Quite liking the idea of your first suggestion atm Graeme (would need to run it with past events obvs)
cheers maus

Zarte Siempre
Kiloposter
Posts: 1269
Joined: Sat Jun 30, 2012 8:56 pm
Location: Dadford, Buckinghamshire

Re: FOCAL scoring system

Post by Zarte Siempre » Mon Aug 13, 2018 8:09 am

As Graeme said, several systems were trialled pre-FOCAL.

It was important to us that good players who only turn up occasionally and thus were unlikely to come to Birmingham weren't what ended up in the top 8. That, for lack of better term, tier B players who support the events regularly had a shot at being rewarded for that. That the mechanic was simple to explain to anyone who happened to ask, to the point where they could reasonably work out exactly how they'd done on the day. And equally that it would be easy to follow in terms of looking over the spreadsheet. For this reason, things with a curve (except perhaps a consistent pre-defined one like the trial we did using... think it was some sort of motor racing scoring) were definitely out.

Personally I believe as well that organisers where possible should be incentivised to work harder to promote their event, knowing that some people will base their decisions on where they can take points from. Additionally, I believe that larger events tend to (although there are some exceptions) have much harder fields to get good results out of, and can easily swallow very good players up meaning I don't believe that the difference is as minimal as people seem to think. The issue with this is not only does it over-equalise the winning of some events IMO, but tiering the events also removes much of the incentivisation as the boundaries become so wide. There is a big difference between looking for 5 extra people to tip the points value up by 10, to having to find 10, and if you do, having no need to look any further

Additionally, although much more minorly in the grand scheme, a simple system allows me to extrapolate patterns much more easily, meaning my post-event summaries take me not too much time to do. I'm not committing to continuing them if that's going to make my head hurt on a co-event Sunday when my head often already hurts :)

I get the issues people have with the current system, and I get that it's not perfect. But I felt that it was the right one at the time, and I'm yet to see one that eradicates enough issues to make it worth changing it when it's guaranteed to lose much of the simplicity and self-calculation. Ultimately though it's not my call, it's Jeff's. Appreciate that you've put thought into this though, and apologies I'm not glowing about it.
Possibly the first contestant to accelerate with a mic clipped...

Gavin Chipper
Post-apocalypse
Posts: 8439
Joined: Mon Jan 21, 2008 10:37 pm

Re: FOCAL scoring system

Post by Gavin Chipper » Mon Aug 13, 2018 8:31 am

Reading with interest to see how this develops.

Tim Down
Rookie
Posts: 98
Joined: Fri Feb 12, 2016 9:45 am

Re: FOCAL scoring system

Post by Tim Down » Mon Aug 13, 2018 9:11 am

Zarte Siempre wrote:
Mon Aug 13, 2018 8:09 am
I get the issues people have with the current system, and I get that it's not perfect. But I felt that it was the right one at the time, and I'm yet to see one that eradicates enough issues to make it worth changing it when it's guaranteed to lose much of the simplicity and self-calculation. Ultimately though it's not my call, it's Jeff's. Appreciate that you've put thought into this though, and apologies I'm not glowing about it.
That's fine, no apology needed. I'm not at all precious about my proposal and I'd read the old threads and knew what to expect. Fundamentally, nothing's changed and I'm just bringing it up again now that it actually affects me. I didn't look beyond a linear relationship between event placings and FOCAL points because of the desire for simplicity expressed by you (and others?) but I'd be just as happy with something more complicated.

I think the problem I outlined (too many points for middling finishes at big events) is serious enough to warrant change and I'd prioritise fixing it over simplicity. However, I don't have to deal with people asking about the scoring system. Does it happen a lot? Finally, I enjoy your post-event summaries and I'd be sad if they stopped.

Zarte Siempre
Kiloposter
Posts: 1269
Joined: Sat Jun 30, 2012 8:56 pm
Location: Dadford, Buckinghamshire

Re: FOCAL scoring system

Post by Zarte Siempre » Mon Aug 13, 2018 9:26 am

Tim Down wrote:
Mon Aug 13, 2018 9:11 am
I think the problem I outlined (too many points for middling finishes at big events) is serious enough to warrant change and I'd prioritise fixing it over simplicity. However, I don't have to deal with people asking about the scoring system. Does it happen a lot? Finally, I enjoy your post-event summaries and I'd be sad if they stopped.
A reasonable number of people when they first turn up ask about it, and then again when they first start attending enough for it to be relevant. And then again when they sober up. And in some cases again when they couldn't be bothered to listen properly the first times.
Possibly the first contestant to accelerate with a mic clipped...

Gavin Chipper
Post-apocalypse
Posts: 8439
Joined: Mon Jan 21, 2008 10:37 pm

Re: FOCAL scoring system

Post by Gavin Chipper » Mon Aug 13, 2018 3:54 pm

Zarte Siempre wrote:
Mon Aug 13, 2018 8:09 am
Personally I believe as well that organisers where possible should be incentivised to work harder to promote their event, knowing that some people will base their decisions on where they can take points from. Additionally, I believe that larger events tend to (although there are some exceptions) have much harder fields to get good results out of, and can easily swallow very good players up meaning I don't believe that the difference is as minimal as people seem to think. The issue with this is not only does it over-equalise the winning of some events IMO, but tiering the events also removes much of the incentivisation as the boundaries become so wide. There is a big difference between looking for 5 extra people to tip the points value up by 10, to having to find 10, and if you do, having no need to look any further
The large events (well, I'm thinking London in particular) often have a lot of inexperienced players, so turning up there and finishing in the middle means you can play fairly averagely but outscore a winner somewhere else.

Also, while I agree that organisers should promote their events, you shouldn't be punishing the players who go to those events if the organisers don't promote them well! You're punishing the wring people! Also, if we're looking at incentives, the points system actually encourages players to not really bother with the events that aren't already big. It's likely to keep the big events big and the small events small.

The stuff I've bolded, I'm not sure I really understand to be honest.

User avatar
Mark Deeks
Kiloposter
Posts: 1796
Joined: Tue Jul 12, 2011 3:15 am

Re: FOCAL scoring system

Post by Mark Deeks » Wed Aug 15, 2018 1:31 pm

I feel that with only very rare exceptions, no matter how deep a field is, the winners always go through the same cycle; random draw round one, then two frigging tough follow-up rounds. So to overly weight the points awarded to large fields seems in error, for you aren't playing the vast majority of those people anyway. With tons of respect to Oli Garner, it's not really any more difficult to win a 70+ person event than it is a 30+ person event. Your paths to the final are pretty much the same.
Eoin Monaghan wrote:
He may not be liked on here, but you have to give some credit to Mark

User avatar
Graeme Cole
Series 65 Champion
Posts: 1528
Joined: Tue Jul 06, 2010 9:59 pm

Re: FOCAL scoring system

Post by Graeme Cole » Sun Aug 19, 2018 11:48 am

Zarte Siempre wrote:
Mon Aug 13, 2018 8:09 am
That the mechanic was simple to explain to anyone who happened to ask, to the point where they could reasonably work out exactly how they'd done on the day. And equally that it would be easy to follow in terms of looking over the spreadsheet. For this reason, things with a curve (except perhaps a consistent pre-defined one like the trial we did using... think it was some sort of motor racing scoring) were definitely out.
What if the number of points scored for each position stayed the same as it is now, but there were additionally a fixed number of points awarded to every player for attending an event - say 40? The last-placed player at an event would get 42 points, the second-to-last player 44, and so on, until the first-placed player, who gets 40 + 2 * (number of players).

It wouldn't remove the perhaps disproportionate advantage you get from finishing mid-table at a large event compared to winning a smaller one, but it would go some way towards mitigating the problem where if you miss a large event, it's very difficult to catch up even if you perform very well at multiple smaller events.

For example, suppose person A attends two 20-player events and wins them both, and person B attends one 60-player event and comes 21st. Under the present system they would both score 80, but if each event gave a fixed 40 points to each player, person A would score 160 and person B would score 120.

Obviously 40 is just an example, and this per-event bonus would need to be carefully chosen. If it's too low then it wouldn't have enough effect. If it's too high then the number of events you attended would become more important than how well you played at them.

I don't know if this was one of the systems you trialled, but if so, I'd be interested to hear how it performed.

Gavin Chipper
Post-apocalypse
Posts: 8439
Joined: Mon Jan 21, 2008 10:37 pm

Re: FOCAL scoring system

Post by Gavin Chipper » Sun Aug 19, 2018 1:48 pm

Graeme Cole wrote:
Sun Aug 19, 2018 11:48 am
What if...

It wouldn't remove the perhaps disproportionate advantage you get from finishing mid-table at a large event compared to winning a smaller one
Then why even invent such a weird system? There are systems that sort that problem out with adding weird arbitrary numbers on the end as an afterthought!

User avatar
Graeme Cole
Series 65 Champion
Posts: 1528
Joined: Tue Jul 06, 2010 9:59 pm

Re: FOCAL scoring system

Post by Graeme Cole » Sun Aug 19, 2018 4:12 pm

Gavin Chipper wrote:
Sun Aug 19, 2018 1:48 pm
Graeme Cole wrote:
Sun Aug 19, 2018 11:48 am
What if...

It wouldn't remove the perhaps disproportionate advantage you get from finishing mid-table at a large event compared to winning a smaller one
Then why even invent such a weird system? There are systems that sort that problem out with adding weird arbitrary numbers on the end as an afterthought!
I give an example of what problem it addresses - someone being disproportionately penalised for missing a large event when they did well at a number of smaller ones. What system that sorts the problem out are you thinking of? And does it use a curve?

Gavin Chipper
Post-apocalypse
Posts: 8439
Joined: Mon Jan 21, 2008 10:37 pm

Re: FOCAL scoring system

Post by Gavin Chipper » Mon Aug 20, 2018 9:09 am

Graeme Cole wrote:
Sun Aug 19, 2018 4:12 pm
Gavin Chipper wrote:
Sun Aug 19, 2018 1:48 pm
Graeme Cole wrote:
Sun Aug 19, 2018 11:48 am
What if...

It wouldn't remove the perhaps disproportionate advantage you get from finishing mid-table at a large event compared to winning a smaller one
Then why even invent such a weird system? There are systems that sort that problem out with adding weird arbitrary numbers on the end as an afterthought!
I give an example of what problem it addresses - someone being disproportionately penalised for missing a large event when they did well at a number of smaller ones. What system that sorts the problem out are you thinking of? And does it use a curve?
I've already given a system that I think would work. Basically, in a one-person event, they'd get 50; for two it would be 66.7, 33.3; for three it would be 75, 50, 25 etc. There's a formula out there somewhere but I can't be bothered to find/work it out again right now. I think it's the fairest linear system, and the basic principle is that you get points for the proportion of people that you beat, not the absolute number. I think the only downside is that you get non-integer scores, but I can't see why that actually matters really.

Chris Marshall came up with a similar system where the winner gets a fixed 100 points and last place gets 1 (I think) and the rest are equidistant in between. Obviously the winner of small tournaments is then disproportionately awarded and the loser disproportionately punished, but in practice I think the two systems would give a similar-looking table at the end of the season.

Thomas Cappleman
Acolyte
Posts: 180
Joined: Fri Nov 07, 2008 9:42 pm

Re: FOCAL scoring system

Post by Thomas Cappleman » Mon Aug 20, 2018 9:40 am

Gavin Chipper wrote:
Mon Aug 20, 2018 9:09 am
I've already given a system that I think would work. Basically, in a one-person event, they'd get 50; for two it would be 66.7, 33.3; for three it would be 75, 50, 25 etc. There's a formula out there somewhere but I can't be bothered to find/work it out again right now. I think it's the fairest linear system, and the basic principle is that you get points for the proportion of people that you beat, not the absolute number. I think the only downside is that you get non-integer scores, but I can't see why that actually matters really.

Chris Marshall came up with a similar system where the winner gets a fixed 100 points and last place gets 1 (I think) and the rest are equidistant in between. Obviously the winner of small tournaments is then disproportionately awarded and the loser disproportionately punished, but in practice I think the two systems would give a similar-looking table at the end of the season.
In practice for all but the smallest events these 2 are basically indistinguishable. For example, in a 19 person event the winner under your system gets 95, and Chris' gets 100, and that difference gets smaller as you approach the middle.

I think this is my preferred system (maybe better if you round off the non-integer points). It's still simple enough to describe, and comes closer to rewarding ability fairly. As Mark said, winning a 30 person event is not much easier than winning a 70 person event, but is substantially harder than coming 40th at a 70 person event.

Gavin Chipper
Post-apocalypse
Posts: 8439
Joined: Mon Jan 21, 2008 10:37 pm

Re: FOCAL scoring system

Post by Gavin Chipper » Mon Aug 20, 2018 10:43 pm

The other thing with Chris's system is that the score you get for coming last is arbitrary and a bit like Graeme's idea of awarding points for turning up.

Tim Down
Rookie
Posts: 98
Joined: Fri Feb 12, 2016 9:45 am

Re: FOCAL scoring system

Post by Tim Down » Fri Aug 24, 2018 4:50 pm

Thomas Cappleman wrote:
Mon Aug 20, 2018 9:40 am
Gavin Chipper wrote:
Mon Aug 20, 2018 9:09 am
I've already given a system that I think would work. Basically, in a one-person event, they'd get 50; for two it would be 66.7, 33.3; for three it would be 75, 50, 25 etc. There's a formula out there somewhere but I can't be bothered to find/work it out again right now. I think it's the fairest linear system, and the basic principle is that you get points for the proportion of people that you beat, not the absolute number. I think the only downside is that you get non-integer scores, but I can't see why that actually matters really.

Chris Marshall came up with a similar system where the winner gets a fixed 100 points and last place gets 1 (I think) and the rest are equidistant in between. Obviously the winner of small tournaments is then disproportionately awarded and the loser disproportionately punished, but in practice I think the two systems would give a similar-looking table at the end of the season.
In practice for all but the smallest events these 2 are basically indistinguishable. For example, in a 19 person event the winner under your system gets 95, and Chris' gets 100, and that difference gets smaller as you approach the middle.

I think this is my preferred system (maybe better if you round off the non-integer points). It's still simple enough to describe, and comes closer to rewarding ability fairly. As Mark said, winning a 30 person event is not much easier than winning a 70 person event, but is substantially harder than coming 40th at a 70 person event.
It's my preferred system too and Chris's is the same as mine but with the same value for each event. I only bothered trying to come up with something different because Gevin's had been rejected last time this was discussed. I think we should do rounding for readability, so to reduce the error, I'd suggest increasing the number of points per event to, say, 1000.

Gavin Chipper
Post-apocalypse
Posts: 8439
Joined: Mon Jan 21, 2008 10:37 pm

Re: FOCAL scoring system

Post by Gavin Chipper » Fri Aug 24, 2018 8:49 pm

I agree that it makes sense to do 1000 points if you want to stick to integers.

Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest