Setting Conundrums at Co Events

Discussion and announcements relating to unofficial Countdown competitions, held online or in real life. Observation, discussion, reflection, and other stuff ending in -ion.
Post Reply
JackHurst
Series 63 Champion
Posts: 2011
Joined: Tue Jan 20, 2009 8:40 pm

Setting Conundrums at Co Events

Post by JackHurst »

Setting good conundrums for co-events sounds like quite an undertaking. If you are doing Lincoln style, you have to print hundreds. You ideally want to check for no repeats, unique solutions and a good spread of difficulty. For a final I guess you want to find something special. Hosts seem to do a great job overall and it's really appreciated :)

I'm interested to hear from hosts. What's your method for setting and validating your conundrums? Are there helpful tools? Would you like more tools? Do you have a particular difficulty in mind for the spread and is there a good way to judge difficulty?

And from players, what makes a good set of conundrums, and is there any small difference that could be made to improve things in terms of which conundrums end up being used? I can't think of much.

And for players and hosts, what do you think about an appropriate difficulty level for finals? What do you think about events where top tables get conundrums from "the hard envelope"? In particular I think EOY finals is an interesting one because you have 9 conundrums worth of "Final" difficulty conundrums to supply for the finalists.
Matt Rutherford
Acolyte
Posts: 125
Joined: Sun Dec 22, 2019 10:41 pm
Location: Birmingham's Eastern Fleapit

Re: Setting Conundrums at Co Events

Post by Matt Rutherford »

I'll add my own thoughts below, but I'm going to bung this post from Graeme in the 'How To Host An Event For Dummies' thread.
It is utterly brilliant and where I started when writing cons for Brum 2022
My thoughts on conundrums, and setting them. Some of you may have heard these from me in person, perhaps several times, but this seems a suitable thread in which to write it all down so I can stop going on about it at events.

tl;dr: Don't choose obscure words for conundrums as a lazy way of making them difficult!

When setting a pile of conundrums for a co-event, there is a temptation to pick obscure words for the "difficult" conundrums, the ones intended to be given out on crucials between high-rated players, or in the final. Choosing conundrums whose difficulty arises from the low probability of either player even knowing the word is seen as an easy way of manufacturing lists of difficult conundrums.

There are two fundamental problems with this approach. First, it misunderstands the point of puzzle setting. Second, it doesn't even work.

Puzzle setter and former Only Connect question editor Alan Connor describes the setter's role in The Joy of Quiz: "Novice setters sometimes fancy that their job is to beat the contestant; it is not. Their job is to lose following a struggle."

Alan Connor was talking about quiz questions, but the same applies to conundrums or any other kind of puzzle. No conundrum setter should be patting themselves on the back because their super-hard final conundrum "defeated" the finalists.

Of course, it's not at all easy to judge whether the players will get it, and sometimes they won't. But if when you reveal the answer, it's a virtually unknown word, the players and most of the spectators will utter a collective "...?", with the sense that it was a bit of a wasted 30 seconds. On the other hand, if they recognise the word, the collectively-uttered punctuation will be more like "!", a much better ending to the tournament.

Consider the spectacle for the audience, as well, especially for the final conundrum. "I'd have got that" or "I could have got that" is always better than "I'd never have got that". Setters are - rightly - concerned about making it too easy in order to avoid the crapshoot of an instant buzzer race, but so often they tend to overcompensate the other way and it goes unsolved. Conundrum solves around the 15-25 second region are rarest, but when they happen, it's because the setter pulled off their job perfectly - to lose, after a struggle.

In the COLON 2018 final, the conundrum, WETHARDON, lasted over 20 seconds being stared at by the two finalists before one of them got it. And even if they hadn't got it, when the answer was revealed there would have been no anticlimactic "...?" Great conundrum.

It's worth noting that for all the conundrums at COLON - not just the final - Jen specifically set out to make a range of conundrum difficulties without resorting to obscure words for difficulty. There were probably more conundrums written for that event (well over 200) than for any other co-event in history, so if you're thinking of hosting an event there's no reason you can't do that too.

Another approach, if you don't mind deviating from Countdown tradition, is to do what Milton Keynes did this year, and make the conundrums ten letters long. In the past, MK's grand final conundrums were known for being so hard that the answers didn't even look like words (ACHOLURIC, anyone?), but with ten-letter conundrums it was much easier to find words that the finalists hadn't practised to death on apterous but which were still normal, everyday words.

Let's have a look at the second fundamental problem - the misconception that ramping up the obscurity provides more of a challenge to the higher-ranked players. It doesn't. The best players have played all the conundrums on apterous, and what makes a conundrum difficult to them doesn't have a great deal to do with how often the word is used. As an example, consider the following two conundrum scrambles.

ISAWKHAZI

INERTSTAR

The answer to the top one is an obscure word which most people outside of word games don't know. The answer to the bottom one is a very common word which you'll all know. However, I guarantee you that most expert players will spot the top one instantly (maybe you have already?), and take some time over the bottom one (still haven't got it?). The difficulty is nothing to do with the obscurity of the words, it's about the distribution of the letters.

For the top conundrum, a seasoned player will instantly spot the W, the Z, the K, the As and Is, and buzz in immediately with the answer. Most of the top players have seen this word before, and it's so easy to spot that they'll get it straight away. Furthermore, in terms of catering for a range of skill levels, this one is the worst of all worlds; a top player will get it instantly with no effort, but a novice player has almost no chance of getting it no matter how much they try.

Now let's look at the bottom conundrum. It's got lots of common letters. It's also got lots of building blocks of words, which conundrum solvers often use as short-cuts, and which in this case are nearly all red herrings. There's -IEST, there's -ATE, there's INTER-, -ISE, -ISER, -ITE... none of them are correct. Even if you didn't get the answer* in 30 seconds, the reaction upon seeing it is "ah!" rather than "what's that?" Even better, although this is a difficult conundrum, there's no reason a novice player couldn't solve it, and it certainly wouldn't be unfair for this conundrum to crop up in their game. They might even get it before a significantly higher-rated opponent, which would have been an impossibility with the other conundrum.

* This conundrum was used for a game at Co:Leam in 2013, and is what I usually use as an example of a "good" conundrum. While it was a valid conundrum at the time, nowadays it has an alternative, more obscure, solution.

Here's all of the above, distilled down into a graph. Imagine every conundrum is somewhere on this graph. The x-axis represents word obscurity, and the y-axis represents the difficulty of unscrambling the word assuming it's in your vocabulary. I'm not saying don't ever set obscure words (players shouldn't be entitled to assume the word won't be obscure), but I'd recommend using them only rarely. Most co-event conundrums ought to be on the left-hand half of the graph. To control conundrum difficulty, vary the position on the y-axis rather than the x-axis!

Code: Select all

Difficulty in unscrambling
   ^
   |  The "rewarded effort" corner,    The "WTF is that" corner,
   |  e.g. RESTRAINT.                  e.g. ACHOLURIC,
   |       HANDWROTE                        MRIDANGAM
   |
   |
   |  The "buzzer race" corner,        The "worst of all worlds" corner,
   |  e.g. EXPLODING                   e.g. WAKIZASHI
   |
   +---------------------------------------------------------->   Obscurity of word
Last edited by Matt Rutherford on Sun Oct 01, 2023 4:13 pm, edited 1 time in total.
The Vicar of Dudley*

*(Not ordained, nor do I live Dudley. Godspeed!)
Matt Rutherford
Acolyte
Posts: 125
Joined: Sun Dec 22, 2019 10:41 pm
Location: Birmingham's Eastern Fleapit

Re: Setting Conundrums at Co Events

Post by Matt Rutherford »

Further to that...
Initial Writing
-Make them like the show (i.e, some semblance of words in the scramble rather than just random). Makes things better.
-When looking for words, I check with Lexplorer that they are good. Some escape, but do check all as much as possible. Often go through conundrum attacks for some inspiration.
-Never bung one in that is so hard no-one is going to get it. Buzzer races are OK sometimes. Best used occasionally.

At the Event
-Previously, I have separated cons by Easy (1-3 hardness), Medium (4-7 hardness), Hard (8+), with some variation depending on scramble or word. However, at Rugby 2023, two strong players (around 1900 and 1700 respectively) missed ESUBTONIC -> BOUNCIEST because they were bracing for something more difficult than it was.
-Therefore, in future I will separate them when writing them so there are equal numbers of each. But for distribution, they will be in one envelope and given out (random-ish).
-Slight tangent, but if you are hosting, and it is crucial, collar someone to video it . There have been some disagreements previously. For Lincoln-Table style, collar either the main host or someone who has finished playing. For Bristol-Classroom style, have two-three conundrums for the room so some players can video crucials if need be. VAR on the cons has made sure that any close buzzes or disagreements have been solved as fairly as possible.
The Vicar of Dudley*

*(Not ordained, nor do I live Dudley. Godspeed!)
Thomas Cappleman
Series 72 Champion
Posts: 332
Joined: Fri Nov 07, 2008 9:42 pm

Re: Setting Conundrums at Co Events

Post by Thomas Cappleman »

Ruth's approach at London was initially to play a bunch of cons on your app and save off some interesting ones, but they tended too easy. As she's not a current member, my next suggestions was to look at past games on Apterous from a random date ages ago, and use those. She then made her own scrambles for them and checked them all in the app, which caught a few mistakes before they went out. Obviously this meant they were all ones on the Apterous con list, but finding ones that aren't is really hard.

They were then sorted into envelopes based on the Apterous difficulty (something like 1-3, 1-7 and 8-10). We discussed after, and next time they'd probably all have a wider range but weighted - so a top table's likely but not guaranteed to have a tough one.

She'd picked one she expected to be hard for the final, with a tough-ish but definitely gettable one if needed as a second. Once I was out of the running I checked them and agreed. I expected the first to still go very quickly, and it did, but there's not much you can do when it's Ahmed vs Ronan.
JackHurst
Series 63 Champion
Posts: 2011
Joined: Tue Jan 20, 2009 8:40 pm

Re: Setting Conundrums at Co Events

Post by JackHurst »

Graeme wrote: My thoughts on conundrums, and setting them...
Lots of good stuff in here from Graeme as per usual. Two points opinions I hold as counterpoints:
- The stuff about the struggle is really interesting and I broadly agree. I worry that people might read the post and interpret it to mean that as a host, you want every conundrum to have been solved at the end of the day. In reality, for each conundrum you don't know what will happen on the day. You might have found a really appropriate conundrum that gets insta-solved because somebody happened to have seen it in the morning when they were practising on the train ride up. You might also set something that should have been solved, but both players bottled it under the pressure of the crucial. It's fine (and expected) if quite a few go unsolved at the end of the day, as that's naturally going to happen out of randomness
- Although the overall advice in the post is great, if you rigorously apply it to all of the conundrums that you set, you reduce the pool of available conundrums to use, and increase the predictability. IMO A good balance is to still throw in a few things that are a bit left field to keep people on their toes.
JackHurst
Series 63 Champion
Posts: 2011
Joined: Tue Jan 20, 2009 8:40 pm

Re: Setting Conundrums at Co Events

Post by JackHurst »

Thomas Cappleman wrote: Sun Oct 01, 2023 4:30 pm We discussed after, and next time they'd probably all have a wider range but weighted - so a top table's likely but not guaranteed to have a tough one.
I think this is a great way to go. You need a bit of randomness to make it less predictable.
JackHurst
Series 63 Champion
Posts: 2011
Joined: Tue Jan 20, 2009 8:40 pm

Re: Setting Conundrums at Co Events

Post by JackHurst »

Matt Rutherford wrote: Sun Oct 01, 2023 4:12 pm At the Event
-Previously, I have separated cons by Easy (1-3 hardness), Medium (4-7 hardness), Hard (8+), with some variation depending on scramble or word. However, at Rugby 2023, two strong players (around 1900 and 1700 respectively) missed ESUBTONIC -> BOUNCIEST because they were bracing for something more difficult than it was.
-Therefore, in future I will separate them when writing them so there are equal numbers of each. But for distribution, they will be in one envelope and given out (random-ish).
I think you might be overcompensating a bit here. IMO it's fine that it went missed, and it perhaps more an indicator of the pressure of the crucial adding extra difficulty for the player :)

See above posts about the idea of weighted envelopes. For example, you could have:
- Envelope 1: 70% easy, 30% Medium
- Envelope 2: 20% Easy, 70% Medium, 10% Hard
- Envelope 3: 20% Medium, 70% Hard, 10% Feindish
(or any other blend that you think is suitable :) )
JackHurst
Series 63 Champion
Posts: 2011
Joined: Tue Jan 20, 2009 8:40 pm

Re: Setting Conundrums at Co Events

Post by JackHurst »

Thomas Cappleman wrote: Sun Oct 01, 2023 4:30 pm Ruth's approach at London was initially to play a bunch of cons on your app and save off some interesting ones, but they tended too easy.
Yeah this is by design, mainly for 2 reasons:
- The game has way more beginner players than any other Countdown platform and it keeps it more accessible to have easier conundrums
- If the hard conundrums came out too often they'd no longer be hard.
Thomas Cappleman wrote: Sun Oct 01, 2023 4:30 pm Obviously this meant they were all ones on the Apterous con list, but finding ones that aren't is really hard.
I think this is a common theme for hosts, wanting to find conundrums not on the ascension list from apterous. I've heard 2 good suggestions if you wanted to use apterous as the data source:
- Look through nasty conundrum attacks (remembering to discount the ones which are +S endings)
- Look through Rex's Goatdown games

If there's enough demand I can add a tool in the word lookup to check how I've classified the difficulty, but it might be a bit hit and miss.
Gavin Chipper
Post-apocalypse
Posts: 13275
Joined: Mon Jan 21, 2008 10:37 pm

Re: Setting Conundrums at Co Events

Post by Gavin Chipper »

I've always found the conundrum a slightly awkward and clunky part of Countdown, as it's the only part that's subject to the whims of the organiser and open to bias. It's basically impossible to make it completely "right" in that respect, so you just have to do the best you can. One thing I might question:
JackHurst wrote: Sun Oct 01, 2023 3:25 pmWhat do you think about events where top tables get conundrums from "the hard envelope"?
Thomas Cappleman wrote: Sun Oct 01, 2023 4:30 pmso a top table's likely but not guaranteed to have a tough one.
During the preliminary phase, everyone is competing against each other for points as well as wins. Because of that, it seems unfair to give the top table a harder conundrum. In most cases it won't matter because someone will solve it, but when they don't, 10 points have gone missing relative to a game on another table where an easier conundrum got solved. The Swiss pairs seeding system is intrinsically biased against players who have done well so far anyway, and this is adding another source of bias. Obviously you want to avoid buzzer races, but this is at least something to consider.
Fiona T
Kiloposter
Posts: 1482
Joined: Mon Mar 18, 2019 12:54 pm

Re: Setting Conundrums at Co Events

Post by Fiona T »

Gavin Chipper wrote: Sun Oct 01, 2023 7:10 pm I've always found the conundrum a slightly awkward and clunky part of Countdown, as it's the only part that's subject to the whims of the organiser and open to bias. It's basically impossible to make it completely "right" in that respect, so you just have to do the best you can. One thing I might question:
JackHurst wrote: Sun Oct 01, 2023 3:25 pmWhat do you think about events where top tables get conundrums from "the hard envelope"?
Thomas Cappleman wrote: Sun Oct 01, 2023 4:30 pmso a top table's likely but not guaranteed to have a tough one.
During the preliminary phase, everyone is competing against each other for points as well as wins. Because of that, it seems unfair to give the top table a harder conundrum. In most cases it won't matter because someone will solve it, but when they don't, 10 points have gone missing relative to a game on another table where an easier conundrum got solved. The Swiss pairs seeding system is intrinsically biased against players who have done well so far anyway, and this is adding another source of bias. Obviously you want to avoid buzzer races, but this is at least something to consider.
I think it's more about giving the bottom tables easier conundrums - and those players are unlikely to be competing for the top places.

Bristol is more interesting - far fewer conundrums so more care/thought can be put into them, but everyone in the room gets the same one.
JackHurst
Series 63 Champion
Posts: 2011
Joined: Tue Jan 20, 2009 8:40 pm

Re: Setting Conundrums at Co Events

Post by JackHurst »

Gavin Chipper wrote: Sun Oct 01, 2023 7:10 pm it seems unfair to give the top table a harder conundrum.
Your point is 100% valid, but....

Gavin Chipper wrote: Sun Oct 01, 2023 7:10 pm The Swiss pairs seeding system is intrinsically biased against players who have done well so far anyway, and this is adding another source of bias.
The hard conundrum penalty is absolutely tiny compare to the Swiss pairs penalty.
User avatar
Thomas Carey
Kiloposter
Posts: 1480
Joined: Sun Jan 09, 2011 4:17 pm
Location: North-West of Bradford
Contact:

Re: Setting Conundrums at Co Events

Post by Thomas Carey »

I love Graeme's post and generally think jack's 'blend' post is the way to go - obviously there's no objective measure of difficulty so something you think is pretty tough one player might think is a piece of piss or vice versa but that's part of the fun.

Another food for thought since someone mentioned Bristol style - what do we think of the thing where once you've played a free games and have a relatively good idea of how well the players in each game are doing, having a few conundrums for different groups of people? So you might give ELLENGRAY to some of the lower ranked games and ALLENGRAY to the games at the highest end etc. I quite like this (but not enough to bother defending it) but I know a lot of people don't - thoughts?
cheers maus
Adam S Latchford
Acolyte
Posts: 119
Joined: Mon Jun 13, 2022 8:47 am

Re: Setting Conundrums at Co Events

Post by Adam S Latchford »

One additional check I did for added ones was (for super hard cons) to look up recent dictionary additions, to hunt through apterous for 9s that have never ever been a conundrum, and also to look at my list of every single available 9 that I procured (valid as of about 6 months ago), remove the plurals and then look for absolute fiendish ones.

For the easier ones, I test them on non countdown people

For the medium ones, i test them on the immediate countdown people who surround me who aren't likely going to events.

Seemed to work well at rugby though msr set about 80% of them.
User avatar
Ben Wilson
Legend
Posts: 4546
Joined: Fri Jan 11, 2008 5:05 pm
Location: North Hykeham

Re: Setting Conundrums at Co Events

Post by Ben Wilson »

I kinda feel compelled to contribute here, having probably set more co-event conundrums than anyone else lol.

In the past, conundrums at COLIN were 'graded', in that lower tables got easier ones and higher tables got harder ones. The main reason for this was to stop conundrums being reaction tests, which are especially bad as it puts pressure on the game host to adjudicate correctly, and also runs the risk of being anticlimactic (the final of 'I can't believe it's not COLIN in 2021 arguably suffered from this- https://www.apterous.org/viewgame.php?game=2797439 ). The latter is probably not a huge concern for most event attendees, admittedly.

The current method at COLIN (which will be implemented at January's event) is to pick a mixture of difficulty 4-8 conundrums from apterous and everybody gets assigned them at random, right up until the final round, when everyone in with a shot at the final (basically anyone on 3 or more wins) gets non-ascension conundrums for reasons described above (the same applies to the final). These conundrums are carefully curated to be guessable, though- typically they'll be obscure compound words like PHOTOEMIT and REDTHROAT from previous events. Both those words beat the finalists, incidentally, but not the crowd (okay, only Rob Foster got the second one but I'm still counting it). I've not had any complaints about this method yet, but I'm always open to constructive criticism.
JackHurst
Series 63 Champion
Posts: 2011
Joined: Tue Jan 20, 2009 8:40 pm

Re: Setting Conundrums at Co Events

Post by JackHurst »

Ben Wilson wrote: Mon Oct 02, 2023 5:17 pm I've not had any complaints about this method yet, but I'm always open to constructive criticism.
It's a good system!

Since a few hosts have posted here, I would note that I like that each host has a different system, I think it works well overall if you go to many events that you never end up with the same things happening all of the time.
JackHurst
Series 63 Champion
Posts: 2011
Joined: Tue Jan 20, 2009 8:40 pm

Re: Setting Conundrums at Co Events

Post by JackHurst »

Does anybody have opinions for FOCAL finals? This event is a bit different from the rest because it's the top players from the year, so higher average standard, and more crucials are likely. There's also the open going on at the same time which I think typically ends up being as high a standard as an average event.

Last year there was a system of Conundrums for finalists, and separate conundrums for the open. It worked quite well and also helped with logistics of verifying crucial conundrums, so it was a big success. It also meant that in theory the conundrums for finalists could have been harder if desired, but I don't think that was actually implemented.
Post Reply