"Stop trying to think so much in complicated math and fall back on common sense.  I spin one fair coin, 50% chance of Queen, 50% of Text.  If the first coin comes up Queen, I take out a new coin that's biased to have a 60% chance of coming up Queen.  If the first coin comes up Text, I take out a different new coin that's biased to have a 40% chance of coming up Queen."

"Suppose I was actually going to carry out that procedure.  You don't get to observe the prior coinspin.  What's your betting chance that, when I spin whichever new coin I take out, it shows up Queen?"

"It has to be 50%, which is the same as 0.5*0.6 + 0.5*0.4. But if the first coin is in this case the before-chance, we don't know that it's 50/50, do we?"

She has ever gone through cycles like this getting tortured a few times, but being tortured is what happens to good Asmodeans when they're wrong, right?

"I mean, in this case, you know because I told you to assume that setup, and answer me conditional on the assumption."

"To answer the question," drawls Alexandre, looking up from his scribbled notes, "if we assume that 0.1, 0.2, and 0.4 all start with equal prior likelihoods, and similarly for 0.6, 0.8, and 0.9, it's (729/100000 +  2048/100000 + 3456/100000), divided by three because there are three equal possibilities, compared to (2304/100000 + 512/100000 + 81/100000) divided by three for the same reason, gives you -" (he's been doing calculations while Willa talks) "- roughly two thousand seventy over one hundred thousand, as opposed to a little under seven hundred over one hundred thousand, giving you a ratio of a little under three to one in favor of the first bucket of theories against the second bucket."

Wait no that's obviously wrong oh damn it -

"Does that work?" asks Korva, sounding genuinely unsure. "That gives you a lower probability for both of the buckets that had three hypotheses in them than for the bucket that had only one hypothesis, even though the results seem more consistent with a true answer of .4 than with .5."

"That's correct!  If you spin a fair three-sided coin and then, depending on the results, start flipping a new biased coin with 0.1 or 0.2 or 0.4 propensity to Queen, your chances of getting Text Queen Queen Text Text are around 2%.  Whereas if you just start spinning a fair two-sided coin, your chance of getting Text Queen Queen Text Text is around 3%."

"The 0.4 sub-hypothesis is most likely to generate Text Queen Queen Text Text.  But that hypothesis starts out with only one-third of the prior probability mass in the 'less than 50%' bucket of hypotheses.  The bucket as a whole is less likely to generate the sequence than the sub-hypothesis inside it."

"Incidentally, one would not in Baseline say that the sub-hypothesis of 0.4 propensity started out with a 1/3 'likelihood' of being true, but that it started out with a 1/3 'prior-probability' of being true.  Some of the reason why I've been using those Baseline words is because they have precise meanings, whereas Taldane has a bunch of words like chance and probability and likelihood that all apparently mean the same thing why does this crazy language even have that don't actually answer me unless it's important somehow."

- Wait, Keltham thinks he's right??? But he's wrong!

...Alexandre thinks he's just going to stick with 'the odds of him failing the most important class in his life have gone down', and bedamned to the truth.

She's so confused???

...she's going to raise her hand. Because she feels like she's being confused in a smart way and not a dumb way, for once, but if they've belabored this point for too long then he can ignore her. That's how raising hands works, right?

Points finger at Korva.

"I think I must be confused about how the buckets work. Because, like - say we have one hypothesis that the true value is exactly 50%, and one hypothesis that the true value is less than 50%, and one hypothesis that the true value is more than 50%. It seems like the likelihood - or probability, sorry, I didn't catch the distinction - it seems like it must be more likely that the true value is below 50% than that it's exactly 50%, even though the below-50% space also includes values like 0%, which has obviously already been outright disproven. So - I guess I feel like if .4 is more likely to be right than .5, I don't see why the hypothesis that covers a bigger space, that also includes the value that the true value is most likely to be closest to, becomes less likely just because that hypothesis also includes values that are much less likely to be right than some other narrower hypothesis. - I'm not sure I said all that right."

"Okay, wow, people's intuitions about probability work really differently when they haven't been raised as dath ilani children.  I think probably you just need to invent a whole lot of problems and play around with them, the way anyone - well, the way any dath ilani kids would do as kids?"

"Let's say I spin a fair twosided meta-meta-coin... let all coins be assumed two-sided and assumed fair unless explicitly stated otherwise."

"Anyways, new procedure.  First, I spin a meta-meta-coin.  If the meta-meta-coin comes up Text, I spin an 'objectlevel'-coin five times.  If the meta-meta-coin comes up Queen, I spin a three-sided meta-coin.  Then depending on the result of that coin, I fivetimes spin a biased 'objectlevel'-coin with 0.1 or 0.2 or 0.4 propensity to produce Queens."

Keltham will attempt to whiteboard this:

meta-meta-coin / \ 1/2 1/2 / \ meta-coin (0.5)-propensity / | \ objectlevel-coin 1/3 1/3 1/3 / | \ (0.1) (0.2) (0.4)

"It's actually just true that if you don't know any of the meta-spins, and just see my unknown objectlevel-coin producing Text Queen Queen Text Text, there's roughly two chances in five that my meta-meta-coin came up Queen and picked a biased coin, and three chances in five that my meta-meta-coin came up Text and picked a fair coin."

"Why?  Because when you end up with a biased coin, it's sometimes biased 0.4, but two-thirds of the time biased 0.1 or 0.2.  Mostly you'll see fewer Queens, when the meta-meta-coin comes up Queen.  When you see Text Queen Queen Text Text, that could be because the meta-coin was flipped and selected the 0.4 coin, but more likely, the meta-meta-coin selected a fair objectlevel-coin and that fair objectlevel-coin happened to produce two Queens and three Texts."

"A dath ilani kid would now be 'programming' a 'computer' to run a million simulations of this procedure and show them how many cases of Text Queen Queen Text Text were generated by the meta-meta-coin coming up Queens versus Text and verifying that's how it actually played out.  Here... we'd need to find a three-sided coin and a ten-sided coin, or maybe a cube and an 'icosahedron', regular 20-sided solid, to spin.  And then we'd probably have to do a few hundred spins to collect enough 'data' for the 'statistics', if the math didn't feel intuitive."

"But I'd hope that - the first time the meta-meta-coin came up Queen, and you spun a meta-coin and it selected 0.1, and the resulting objectlevel-coin produced Text Text Text Text Text, it might become more intuitive why, when the meta-meta-coin comes up Queen, what you mostly expect to see is mostly Text.  So when you don't see that, it's not likely the meta-meta-coin came up Queen."

"That... makes sense, but how do we know that there isn't instead a four-sided meta-coin that picks among all the possible coins, including the fair one?"

"In terms of this particular problem?  Because I told you so."

"Why did I tell you so?  Because I was trying to pump the intuition that - assuming 0.1, 0.2, 0.4 equally prior-probable within the 'less than 0.5' bucket - that bucket was then less 'likely' to yield Text Queen Queen Text Text than the 0.5 bucket, even though 0.4 was in that bucket.  I was trying to pump that intuition by showing that, if the whole biased bucket and the whole unbiased bucket started out equally probable, then, after seeing Text Queen Queen Text Text, we'd think the unbiased bucket had become more probable and the biased bucket less probable.  So what we saw must have a lower 'likelihood' in the biased bucket that starts out with 0.1, 0.2, and 0.4 having equal prior-probabilities."

"I mean, we could argue about how that would go in real life, instead of a thought experiment.  You could say that all four hypotheses are equally simple to describe out loud and should therefore be around equally probable.  I could then counterargue that if we're talking about an actual coin, then in real life, most coins are probably pretty close to being fair random-number generators when spun - though I ought to actually verify that here, before I bet anything important on it.  So it should actually take hundreds of observations before we start believing the coin is 40% biased towards Queen, I would argue; five coinflips is nowhere near enough.  Therefore, I'd conclude, 'the coin is biased 40% Queen' is a lot less likely than 'the coin is an unbiased random generator'."

"But it would be better if arguments like that didn't have to appear in our 'published-experimental-reports'.  Which is one angle towards 'grokking' an underlying central reason why 'published-experimental-reports' ought to summarize likelihoods for hypotheses that are more like 'observational likelihood if this coin has 0.4 Queen propensity', and less like 'observational likelihood if this coin has a less than 50% Queen propensity'."

"If you just summarize for the reader 'What is the likelihood of my data, in the world where the coin comes up 40% Queen?  The world of 50% Queen?  The world of 10% Queen?' then you don't have to confront the question of whether 40% Queen was 1/3 as prior-probable or equally prior-probable with 50% Queen."

"Oh, and, uh, to make it explicit:"

Examples of Baseline terms for 'prior', 'likelihood', 'posterior':

'Prior' coin is 40%-Queens:

P( Q=0.4 )    = 1/6

'Prior' coin is 50%-Queens:

P( Q=0.5 )    = 1/2

'Likelihood' of TQQTT, if coin is 40%-Queens:

P( TQQTT ◁ Q=0.4 )     = 0.03456

'Likelihood' of TQQTT, if coin is 50%-Queen (fair):

P( TQQTT ◁ Q=0.5 )     = 0.03125

'Posterior' coin is 40%-Queens, after seeing TQQTT:

P( Q=0.4 ◁ TQQTT )     = 3456 / (729 + 2048 + 3456 + 3125*3)    >(1/5), <(1/4)

'Prior' coin is <50%-Queens:

P( Q<0.5 )    = P( Q=0.1 ) + P( Q=0.2 ) + P( Q=0.4 )   = 1/6 + 1/6 + 1/6   = 1/2

'Likelihood' of TQQTT if <50% Q:

P( TQQTT ◁ Q<0.5 )     = 1/3 * .00729  +  1/3 * .02048  +  1/3 * .03456   ~ 0.02

"How am I doing, Korva Tallandria?"

"I - think I get that, as far as it goes, with the coins? But I don't immediately see how it applies to the people. - if it applies to the people the same way, I'm assuming it does but I'm not sure I should be."

"I mean, the sense in which it also applies to the people, is that your report should summarize the 'likelihood' that your results were generated by a 10% propensity for all-10s to guess within 30 minutes, not the 'likelihood' that your results were generated by a less than 50% propensity for all-10s to guess within 30 minutes.  Because to do the latter thing you have to make a bunch of weird assumptions, and your math is just going to get more and more needlessly complicated as you dig yourself in further."

"And by way of showing how much further into complicated trouble you'd end up digging yourself:"

"Again, let's say we were going by bucketed hypotheses.  One hypothesis, the meta-coin hypothesis, says that there's a 1/3 chance we live in a world where all-10s have a 10% propensity to solve 2-4-6 in 30 minutes, 1/3 chance it's 20% propensity, 1/3 40%.  The other hypothesis, the fair-coin hypothesis, says we live in a world where all-10s have a 50% propensity to solve in 30."

"We don't actually need to consider the probability of these two hypotheses relative to each other.  Let's say we test five all-10 subjects and get NO YES YES NO NO, meaning two subjects guessed within the time limit, three didn't.  Our experimental report is just going to summarize the 0.2 'likelihood' of the data assuming the meta-coin hypothesis bucket, and the 0.3 'likelihood' of the data assuming the fair-coin hypothesis.  That's true regardless of the 'relative prior-odds' of the two hypotheses relative to each other."

"So we publish our report.  0.2 likelihood for the less-than-50% bucket, 0.3 likelihood for the 50%-propensity hypothesis.  There's a questionable assumption that 10%, 20%, and 40% were all 1/3 likely assuming the propensity was under 50%, but fine, whatever, we've got to assume some 'prior distribution' to report a combined likelihood on that whole bucket all at once, yo."

"Along come some replicators.  They test 5 more people.  They get YES NO NO NO YES, so also two subjects who guessed and three who didn't."

"Now what?  What does the combined evidence say?  Anybody want to give the obvious wrong answer?"

"Obvious wrong answer:  This new data also has 0.2 likelihood on the less-than-50% bucket, and 0.3 likelihood on the 50% bucket, so the combined likelihood across the two experiments is 0.2 times 0.2 equals 0.4... equals 0.04 assuming less-than-50%, and 0.3 * 0.3 = 0.09 assuming 50%."

"And why isn't that just totally right?  Or maybe I'm trolling you by calling it the wrong answer, and it is right?  Candidates only, you can message Ione if you're worried your reply is stupid."

"Are we still not considering any other - hypotheses, and just interested in the relative ratios?"

"Yup!"

"We've become more confident, which sounds right... does this get the same result as if we did one experiment with ten people in the first place? I'm not sure if it does..." She's going to start checking the math.

"Well, everyone else do feel free to follow along and try the math on that part.  Raise your hand when done, practice accuracy before you practice speed."

"After all, a Lawful way of looking at the world shouldn't care whether you call your collected 'data' by the name of one experiment or two experiments.  You should always get exactly the same answer either way.  Though, to be sure, you might have occasional occasion to notice that different 'data-subsets' might have been collected under possibly different conditions."

Pilar raises her hand before anyone else.  She's faster than even Asmodia once she knows exactly which rules to follow and which procedure to execute, and in this case she knows exactly what to do.

The first of the new researchers to finish speaks up immediately - "About Fifty-five hundred and sixty-seven to ninety-seven hundred sixty-six," she says, "which really isn't the same thing as four to nine."

"Um, sorry for not being explicit - if I say for people to raise they're hand when they're done, that's so I know everyone is done, and meanwhile, everybody gets a chance to try on their own before hearing anybody else's result."

"At least you didn't say how you did the calculation, so others can, perhaps, do their own calculations and see if they think yours is correct."

Keltham comes over to look at the scratch paper, if any; what calculation seems to have been done?

Squared both sides of 729/100000, 2048/100000, and 3456/100000, divided it by three, then compared it to 3125/100000 with both sides squared, then canceled out the denominator since it was the same on both sides of the equation, with some approximations for doing faster arithmetic.