Korva doesn't think that she could do that stuff on the board unassisted right now, so she's feeling a little bitter about Asmodia (who is, after all, the only other person in the room qualified to teach classes about alien math concepts right now) declaring it obvious.

It doesn't actually look that complicated, so hopefully Korva will be able to confirm its obviousness and apply it to future things later on, when she's gotten a transcript of this class and her brain isn't dribbling out of her ears. Right now she's busy panicking about all of the rapid-fire baseline words in this lecture that haven't been explained and don't seem to have direct Taldane equivalents, and also trying to figure out what exactly an experiment is. It seems like that might be kind of important to figure out, given that she's apparently not allowed to do any of them without publicly announcing it and giving her predictions in advance.

She's not going to ask this, because a bunch of other people seem to know it already, and if she's in the slower half of the class they're going to kick her out and ensure that her soul is permanently worthless.

"Some additional points, when applying this Law inside of Science!, that might not be obvious... let me actually 'whiteboard' the three points, so I don't forget the later two while talking about the first one."

#1 - 'Published-experimental-reports' usually don't assign 'priors' or calculate 'posteriors', they just report all cheap details of the raw data, and maybe calculate some 'likelihoods' from obvious hypotheses

#2 - Separate experiments are usually supposed to avert 'conditional-dependencies', watch out for when that isn't true

#3 - If every obvious hypothesis has unexpectedly low 'likelihood' over all the combined data, it means the true theory wasn't in your starting set, often that different experiments had different hidden conditions

"Point #1, a 'published-experimental-report' wouldn't make up 'priors' or calculate a 'posterior' the way Asmodia did in her example.   That would be a weird kind of thing to claim as the knowledge of Civilization!  Somebody looking at your report might've seen other evidence.  They could be working on a secret Governance project, and know about evidence you can't access at all.  They might be considering hypotheses you haven't thought of at all.  Or, just, if you don't have formal ways of assigning 'priors', they might have more life experience in guessing those."

"Making claims about what Civilization should believe in the light of an experiment is a prediction market's job, not the job of one experimenter who publishes a report."

"If there's very standard or very obvious particular hypotheses to consider, you might make people's lives easier by precomputing the likelihood of the data on each of those hypotheses.  But one of the big cautions is to consider that other people might have hypotheses you don't, which means you should try to report more details of the data than you already know to be necessary."

"Taking the 2-4-6 example.  Suppose that people just hadn't thought of the possibility that there was 'temporal-dependence' in the results, that they hadn't broken out of the mindset where they were expecting the Keltham-environment to be a timeless 'function' that always produced identical output measurements for identical inputs.  They did run across one duplicate input, but didn't notice it was a duplicate input when they recorded the 'NO' - maybe because two different researchers were querying the environment separately."

"Now think of how inconvenienced you'd be, if that data was reported with input-ordering information missing.  Somebody reading through later, couldn't see which inputs had happened before each other, because you'd implicitly believed that the Keltham-environment was 'time-invariant' and the inputs were 'exchangeable' in the ordering."

"And think of how helpful it would be, if the person had instead happened to include 'timestamps' on each input, even if they didn't know it was important, so you could notice the Keltham-environment taking longer to answer on some inputs, even if they'd missed that themselves."

"If the experiment was on people, the full report includes pseudonyms you can use to contact any of the experimental subjects, just in case somebody comes along later and wants to test subjects to see if they've got a particular 'gene' in their heredity, say."

"Obviously, newspapers can't report all the data of an experiment in the middle of the text.  Even most people who read the 'published-experimental-report' directly won't review all the raw data themselves.  Summaries matter!  So yeah, the report will say, briefly up front and in more detail at the end, 'Here's the theoretical likelihoods of the data for the major hypotheses under consideration.'  The newspaper story will say that, plus, 'Here's how prediction markets on related observables shifted immediately after the results came out.'"

"But the real report, above all, is the data itself - what Reality answered back to you when you asked it a question."

"Or, how did the 'Watcher-over-children' put this... some more of this is coming back to me now..."

"It's not the job of the 'experimentalist' who writes the 'published-experimental-report' to say what Civilization believes, to tell everybody else what to believe."

"It's not the experimentalist's role in society to say what's true."

"That's the job of prediction markets to say."

"Only prediction-market traders get to tell everybody what they should believe.  Because they're the ones who'll lose money if they're wrong.  And if you think you know better, step up and bet yourself, because anybody can."

"So I'd guess there was a failure mode they were worried about, where 'experimentalists' start to take on a social role of saying what's true, and don't lose a lot of money when they're wrong?  Or something like that, anyways.  Though that failure mode itself sounds a bit ill-defined to me, what happens when two experimentalists point in different directions?"

"But yeah, the whole Science! system is put together as carefully as Governance.  Unfortunately I did not pay as much attention to exactly why everything had to be exactly that way, because I wasn't skeptical about the Science! system the same way I was skeptical of Governance as a kid."

Most people, in most parts of the world, probably don't have enough money that it would matter that much, even to them, if they lost it. Not that she's going to point this out.

Keltham's view of a world dominated by prediction-market-traders telling people what to believe is honestly adorable. Would he reconsider, after Asmodeus just was smarter than he is, and won all the bets, and took all his money? Which, after all, would just happen.

"Did this start with - a government doing it, and then doing so well they conquered everywhere else? I don't see how you'd know it was the best way if there aren't other ways it beat out."

"Would you accept 'the prediction markets say the results won't be as good if we don't use prediction markets' as an answer?"

“No.”

"Me neither."

"One answer is that there's always cities inside Civilization trying something different, because, if there is anything dath ilan lacks, it is not people who think they might have Better Ideas.  So far, none of the cities trying something other than prediction markets and traditional Science! have produced vast quantities of new knowledge and technology enabling them to impress the rest of dath ilan into stunned agreement."

"If you mean historically... we don't know a lot of things about our own history that you might otherwise expect us to know, for reasons I'm not getting into at this point.  I'm guessing that dath ilan - had a different trajectory of Intelligence, that something catastrophic happened in Golarion to drive the average Intelligence level down to 10 after writing had been invented at average Intelligence 12 and spellcraft had been invented at average Intelligence 14, or whenever the leading geniuses would be smart enough to do that.  Or writing got carried over from a world that is cousin or common ancestor to dath ilan.  And then something happened to lower the average Intelligence here..."

"My point is, I'm guessing that, by the time prediction markets were around, people generally were smart enough and had enough Law that they wouldn't go to war and conquest, that they'd do something else which was not that, and just, you know, form Civilization, because why not."

"I'm also guessing none of you are going to believe that, and there are obviously other options."

"Maybe prediction markets came along, and Science! came along, and there were factions that refused to adopt that, that didn't know Law, didn't want to be taught Law, didn't want their children to be taught Law.  And then the way of Civilization or what became Civilization, if they thought anything like Civilization thinks now, would be to say - you can refuse to learn the Law yourself, and that's fine, but you cannot choose for your children that they'll have no chance to learn it.  And everybody in your faction is going to have their head 'cryopreserved' upon their death, because to refuse that, is the one mistake that people cannot learn from in time; and you do not pass the competence test to credibly claim to Civilization that you know all the reasons not to do that, and you are choosing to do it anyways."

"And the people in those factions would tell pre-Civilization to go die in a fire.  After which pre-Civilization would say sorry, and come in with superior technology that shrugged off whatever they had in the way of primitive pre-scientific explosives, and teach their children, and save everyone's heads when they died, and plan to apologize about that a thousand years later."

"I - am not sure a world of Kelthams would do exactly that.  I'm not sure we wouldn't, either.  Children are not their parents' stuff, children don't have imaginary ownership-tags pointing to their parents.  I think in the Kelthamverse we'd probably - be less inclined to storm in and do things anyways - if it was about people telling us they didn't want their heads cryopreserved.  Because their heads, their souls, are their own stuff and not ours."

"The children?  Are not anybody's stuff.  If the parents in that dissident faction were, like, not letting their kids own stuff?  The kids there aren't allowed to buy books with Law in them?  I think the Kelthamians probably invade them over that.  All the adults in the Kelthamverse used to be children themselves."

Some parts of those stories are plausible. Some parts are not at all plausible. There are a lot of things she'd like to object to, in that, now that she's thinking about what things are true, but you never object to claims about history, and you super extra double don't object to claims about history with the threat of The Wall hanging over everyone.

...even so, Korva makes a note to ask the project leadership whether they are permitted access to the known information about ancient Azlant, and about the more recent trajectory of standardized wizarding education over the course of the last hundred years, and maybe also about the origins of wizardry itself, if anyone remembers them, and how they relate to other kinds of arcane magic. If that seems like a safe thing to ask for. Obviously.

Asmodia's tempted to just kidnap Korva Tallandria off the Project and put her to work on the Wall, frankly.

This might be a slightly dangerous question, but - "Why is average intelligence important for inventions? For societies, yes -" he cannot imagine the average person in his village functioning in dath ilan "- but how does it contribute to inventing writing, except as it produces inventors of genius?" He has the idea - you could have a broader distribution with more failures and more geniuses or a narrower with more nobodies and what matters for this is the geniuses not the average - he's just not sure how to express it -

"You need both?  There's a lot of work to be done in Science! that doesn't need to be done by geniuses but does take Intelligence 18.  That'd be one of my primary guesses for why Golarion is missing so many dath ilani ideas despite having +6 headbands and artifact headbands.  My other primary guess is that even the artifact headbands are only enhancing some of the key aspects of mentation that go into Science!.  At some later point, we may experiment to see whether Fox's Cunning, Owl's Wisdom, and Eagle's Splendour all together actually help naive subjects on the 2-4-6 challenge, at all.  Though that would be only one of many tests."

Willa's been musing over something in her head, but then she gets a mental approval from Asmodia to just ask about it out loud!

"If your pre-Civilization mostly paid attention to prediction markets when deciding what to do and what to believe, couldn't the factions around it just convince it not to attack them by buying into and distorting their markets? Or to do lots of other things that might not be in its interest? It sounds like a huge vulnerability."

"Do you know what we call it in Civilization when somebody comes in and spends a lot of money to try to shift a prediction market away from the Lawful value?"

"In Baseline:  'Free money!'"

"Prediction markets don't generate money out of nothing.  You can only make as much money off a prediction market by being right, as there is wrong money to bet against you, for you to take.  A ton of wrong money coming into a prediction market is a subsidy that can pay smart people the cost of their time to come in, stare at that market, figure out what the correct price ought to be, and buy up shares trading significantly away from that price."

"From the perspectives of smart traders, it doesn't matter whether the person is foolishly trying to distort the market, or just foolish.  It's a subsidy to them either way.  The more stupid money goes in to be taken, the more smart people crowd around competing to take it at exactly the right price.  People who try to distort prediction markets just end up making them more accurate.  You'd need to be able to literally outbid the rest of Civilization combined, to still be standing there buying shares at a bad price after everyone else in Civilization who wanted free money had run out of money they could use to take your money."

"That's why Civilization trusts prediction markets so much.  They're a form of speech where lying costs you, and then if you try to lie anyways no matter how much that costs you, it just makes the truth stronger."

"To be clear, there is a fundamental vulnerability of this sort, but you can't exploit it just by being rich.  The Hypothetical Corrupted Keepers could prevent Civilization from doing anything they didn't want Civilization to do, by bidding in the prediction markets, openly as the Keepers, to say that dreadful bad results will happen if Civilization does that.  Relatively few people would want to bet against the Keepers - not because it's illegal, to be clear, but because you'd probably lose all your money unless you're Literally Nemamel."

"Then Civilization doesn't do that thing, and the market on that policy never resolves, and the Keepers don't lose any money for lying that way."

"This is a central problem in all of decision theory - that we never get to observe the results of the actions we don't take.  We never get to see our expected utility estimates tested for everything we thought had less than optimal utility.  There's a lot of places where Civilization solves that problem by randomizing slightly, if doing slightly the wrong thing won't be catastrophic.  But the really important predictions - you don't do that there.  And that does open you up to assault by the Hypothetical Corrupted Keepers, or other actors inside Civilization who can afford to put up a lot of money and come up with persuasive arguments and have enough of a reputation to make other traders nervous."

"But it requires acting openly, to make the other traders nervous to bet against your reputation.  There's a limit to how much you can corrupt Civilization's decisions and markets that way, because if you do it a lot, people will be suspicious.  And not just anyone can do it, either."

"...that said, yes, the Keepers have bid against things in inscrutable ways, now and then.  It wouldn't be an especially helpful act to compile a public list of all the times they've done that, but they've done that even in markets I've been tracking.  To this day I have absolutely no idea why the Keepers fear long-term consequences specified to the rest of us only as 'people will later vote that was a bad idea', if Civilization makes a harder push on teaching average kids more 'computer-science' once my generation's kids are slightly smarter.  I mean, it's very credible that 'computer-science' reshapes some people's thoughts in some internally-damaging direction, which the Keepers would rather not point out explicitly for obvious reasons.  It doesn't obviously fit into any plan of corrupt world domination.  But... yeah, what the Keepers bid against, largely doesn't get done, and if they were Hypothetically Corrupted, they could in fact be steering Civilization that way."

Alexandre is now curious what computer-science is. Anything the Keepers don't want him to know has to be powerful.

Asmodia is now curious what computer-science is and if it's going to DESTROY ALL OF GOLARION unless SOMEBODY STOPS IT where SOMEBODY is probably going to be ASMODIA because the rest of Project Lawful is composed of SUICIDAL DISASTER MONKEYS.

The mortal is thinking CORRECTLY again.

It's UNFORTUNATE that this mortal already has an ownership-tag pointing somewhere in Hell, preventing Her from clericing it.  Otolmens has never really understood before what some gods seem to see in their clerics.

Now, however, Otolmens sees.  This mortal would not just be a USEFUL TOOL.

This mortal would be a useful tool requiring MINIMAL ONGOING MAINTENANCE.

"Anyways, I'm going to move onto #2 now, in hopes of eventually traversing this conversational 'data-structure'."

"Point #2, since I was dealing with the same Asmodia over time, I had to consider the 'conditional-dependencies' in my 'evidence' and couldn't treat them as 'approximately-independent'.  I had to ask, assuming that Asmodia was writing down a prediction that I was trolling her, having updated my model of Asmodia off that fact, what chance would she then have of putting a probability on that prediction?  That's not my baseline probability for Asmodia putting probabilities on things, it's the Asmodia who was already predicting well enough to know I was trolling her, which is an Asmodia more tuned in on the rhythm-that-is-Keltham than the Asmodia who didn't do that."

"Then I had to consider that question separately for each 'possible-world' of Ordinary, Conspiracy, Timetravel.  The facts inside a single world can interact among themselves, the facts between worlds don't."

"Uh, not in this example, anyways.  Different possible worlds can start interacting if you get into multi-agent logical decision theory.  Even in this example, Conspiracy Asmodia would be trying to guess what Keltham would think Ordinary Asmodia would do.  But those complications shouldn't come up every day, and in most of Science! we can ignore them."

"In Science!, usually, when the 'replicators' set up a new experiment at a new location, the results there shouldn't be 'causally-entangled' 'to-any-significant-degree' with the results of the previous experiment.  If you find yourself thinking that what you expect given a particular hypothesis, on the new experiment, is being influenced by what happened in the previous experiment, it means you're not narrowing down your hypotheses enough.  Which will hugely complicate any efforts to combine your data, if you've got to consider how all your pieces of data are interacting with each other."

"If I was really making a serious run at updating on Ordinary versus Conspiracy, I'd need to consider multiple possible Conspiracies, drawn finely enough that what I saw in one piece of evidence didn't much affect my likelihoods on other pieces of evidence..."

"Actually, this would probably be a lot clearer with some math."

"So let's do that, considering the hypothetical case of us experimenting to see whether enhancing all three Golarion-known mental stats by +4 would help 'previously-unexposed-subjects' on the 2-4-6 challenge."

"Simplifying things down way too far, for the purposes of keeping the math simple on its very first presentation, we'll say that the only variable we measure is whether or not somebody solves the challenge during their first five minutes."

"Now, how would you do Science! to that, to see whether +4 to all mental stats helped?  I'm going to let researchers and candidates from any tier answer, because I expect this one to be pretty hard just starting out.  But if I'm wrong about that and Asmodia or Carissa are seeing it immediately, do call Prediction rather than just tossing out the complete answer."

She still doesn't know what Science is or what it means to do it to something!!! But - there's an obvious thing to look at, isn't there -

"Well - test whether people who read 14 in every mental stat get it within five minutes more often than people who test 10 in each?"

"With what resources? Strategies will vary depending on what we can afford."

"Yeah, you're going to have some trouble finding people with exactly 14 in all mental stats, I predict, given Cheliax's current state of gathering statistics on its population.  Even if you can find them by checking with all the wizard academies where they measure that, paying to Teleport people in sounds quite expensive."  (The back of Keltham's mind also notes that this proposal is something a not-too-competent Conspiracy Korva might float, if their world secretly had much better stat-gathering and/or cheaper Teleports, and they hadn't fully updated on the fake world not having that fact true; and Conspiracy Alexandre might be trying for a hasty backpedal there.)

"But let's leave that aside for now.  Let's also leave aside that people with all 14s may have differences that aren't just about the mental stats, like, maybe they came from richer families and ate better as children and got better 'nutrition' and that's part of why they're smarter now.  Maybe they have smarter parents and their parents owned more books, or better books."

"Leave all that aside.  We test 5 people with all 10s, and 5 people with all 14s.  At those intelligence levels, five minutes won't cut it, given our previous experience here.  Let's give them thirty.  2 out of 5 people with all 10s get it in thirty minutes, 3 out of 5 people with all 14s get it in thirty minutes."

"What have you learned?  What do you now believe?"