Why did people tell her to ask it if it was a dumb question!!

"Well, uh, I guess I was thinking that if different countries are doing their Science secretly, then that's sort of like what you were contrasting it with, the experimentation of a small group or a faction, and I was wondering whether a worldwide effort was a necessary part of the concept. I guess you might not need political unification for a worldwide effort like that, at least on some things, but I don't think it happens automatically - the markets would have to know about each other, both in terms of not being secret and in terms of having the ability to reliably trade information even about things that aren't secret."

"Oh, sure, there's all kinds of secret research in Civilization.  Startups working on something proprietary, probably some stuff Governance does that would be socially-infohazardous like scalable-weapons research, and, one assumes, almost everything to do with the Keepers."

"That's just, like, mini-Science! where you pretend to be your own little Civilization.  Obviously if you could have one big Civilization under the Law of Coordination you could also have several smaller ones being governed by the same rules.  The only difference I can see is that any smaller mini-Science! like that would be 'shoulder-standing' the bigger Science! project of the rest of Civilization of which it's part."

Relayed to Korva:  We needed a question for Keltham that wasn't about 'newspapers' while we tried to figure out how broadsheets worked in alterCheliax.  Your sacrifice and obedience is acknowledged.

Oh that makes sense. She is glad to have been of service to the project, and surprised and grateful for the context. She's going to nod and shut up now.

"Okay, what else is part of Science!... replication and reproducibility, right, you'd think that would be obvious and emergent-from the rest of the setup, but it got taught to me as a distinct principle so somebody thought it needed separate focus and emphasis."

"If in Civilization you report that some procedure works for refining ten pounds of spellsilver ore into twenty pounds of spellsilver, the prediction markets on the result of a general or generic experiment like that one, don't all immediately go to 100 and pay out.  In this case, even if you took 'video recordings' of your experiment so people knew you weren't just lying - that you had, at least, put in a lot of effort to fake the video - market traders would still be pretty skeptical.  The relevant prediction market would be on what would happen if known, competent, careful, previously validated, third-party professional 'replicators' tried the same experiment."

"And I doubt that prediction market would go very far above 0%, if the claim was that you could refine ten pounds of ore into twenty pounds of spellsilver, assuming spellsilver was otherwise known to be mostly pure metal and not have components that could be drawn from the air or other refining materials."

"Now, if you're the original discoverer, you probably find that pretty annoying, right?  You published your experimental results, and those stinky prediction markets still don't believe you?  So after double-checking your own work - at least, you double-check it first if you're at all sane or smart -"

"Well, mostly, you discover that enormous blatant error you made, where the scales you were using to measure the ore weight were broken."

"But let's say that doesn't happen.  Then you buy up all those prediction-market shares that are trading at under 1% that your experiment replicates.  The market price starts to go up, obviously, but as it starts to get above 2%, more traders start to come in, checking to see if they think they can make a quick 2% profit off you.  Your prediction sounds completely nuts, they figure this is just a prank or somebody making a weird point, so they start to sell at 2%."

"You buy more.  Some of the less confident counter-traders drop out, when they see you're willing to spend that much."

"Sometimes the way the story goes from here is that the probability gets as high as 20%.  If the proposition at stake is an important one, 20% is high enough that an outside 'venture philanthropist' will come in and fund a 'replication' of the research done by professional 'replicators', because the 'impact' of 'replicating' the original maniacal-experiment, if the results 'reproduce', will sell for more than five times the cost of doing the 'replication'."

"Let's say that doesn't happen, because the idea you can refine ten pounds of ore to twenty pounds of spellsilver is such a ridiculous one that your counterparties are still confident.  They keep buying the prediction's price down even as you keep trying to buy it up.  It never gets above 4%."

"Once you've spent a large portion of your savings on amassing a huge position like that, you spend a pretty large chunk of whatever's left, on the actually quite expensive project of paying respected 'replicators' with the 'certs' to actually resolve the prediction market, to test your method.  Which they will do very very carefully, taking videos of everything, that you inspect to say beforehand rather than afterwards if you think they're doing anything wrong along the way."

"Usually the way this story ends is that somebody loses a chunk of their savings and gains a valuable life lesson about overconfidence."

"Sometimes, very rarely, it ends with a lot of shocked prediction-market traders who lost a much bigger collective chunk of their own wealth, and a story that makes all the newspapers.  Your name becomes one of the glorious Science!-heroic stories that inspires the next generation to not just believe what everybody else believes.  World-class professional sex workers will compete to seduce you just to be able to advertise to their other clients that they won that competition."

"Also you can sell your prediction market shares at a 25-fold profit.  But that's not why you did it, really.  The money is just how - how other people can know that it was all real, because real money changed hands."

... "Where does the typical researcher get the funds to pay for all these 'prediction-market shares' and 'replications'?" It's not like researchers are wizard with spells to sell, right; dath ilan doesn't have wizards.

"Well, if you are ten years old, and your parents aren't very wealthy and indulgent, and your theory isn't so persuasive that you could convince other 'venture scientists' to go in with you on it, then you are not in fact going to pull this off."

"That's considered a 'feature' of Civilization, not a 'bug'.  It prevents Civilization from being snowed under with futile replication attempts for silly ideas.  You can be poor and persuasive and still get an expensive experiment performed, or rich and unpersuasive, but you've got to be at least one of rich and persuasive."

"That said, let's say that the prestigious replicators earn four times what you do per workday, and it'll take five of them working for ten days to replicate your experiment, plus some relatively lower costs to rent equipment.  Then their cost is on the order of two hundred days of your salary.  If you save a third of what you earn for five years, you can pay for one experiment like that and still have two-thirds of your savings left over."

"Also to be clear, if an experimental report is being put forward by anyone who's not a ten-year-old, there will always be some prediction market about whether a 'replication' would 'reproduce' those results.  You're expected to state your initial price and subsidize that market at least a little, which is how you say to the world that you actually believe your own results at all.  But Civilization knowing more stuff, is to some irreducible degree a 'non-rival non-excludable public good', so the 'philanthropists' come in and subsidize a process where some experiments will be randomly selected for 'replication'.  That is, the selection is random from a probability distribution that goes according to complicated formulas that include the market odds, the 'market-volume', and any advance-obvious 'impact'."

"The point being, almost any experiment has some probability of being actually 'replicated', in which case the conditional prediction market on the outcome of the 'replication', will pay out.  That incentivizes accurate prediction markets and careful trading, even on relatively minor experiments that wouldn't usually pay to get 'replicated' by one of the big expensive high-trust professional 'replication' firms."

"Or as my teachers emphasized to me a lot - if an experiment didn't have any prediction market price on 'reproducing', how would you have any idea what Civilization thought of it?  Would there really be a sense in which Civilization collectively knew anything about the result?  There being a prediction market, with some chance of the replication actually being performed in order to incentivize careful trading, is a key condition for an experiment really being part of Science! at all.  Without that, maybe somebody knows something about that result, but Civilization doesn't know anything about it."

"Probably the people who actually think about Science! in tremendous detail are betting that, if we stop making prediction markets for everything, Science! will start to fall apart.  So that part gets emphasized to children, to prevent that scenario?  I don't know the details, unfortunately, I'm just guessing based on how much different stuff got emphasized to me."

"Are there - other known Science failure modes, ways all of Civilization stays wrong about something?"

"So another precaution they emphasized was 'preregistration', you describe the experiment before you perform it and open a prediction market on the results before they're in.  That forces Civilization to make its own prediction in advance.  It lets anybody object beforehand instead of afterwards, if they have an issue with your experimental procedure.  It makes them look less persuasive if they wait until you get a result they don't like, to claim something was wrong with your methodology.  On a Civilizational level, we were told that 'preregistration' guards against a scenario where people think that boring or expected results are 'failures' and don't bother to report on those, until somebody by random luck gets a misleading exciting result and reports on that."

"There's a whole dramatic debate about whether or not, in principle, we ought to pay any attention to results that aren't 'preregistered', if somebody reports on them anyways.  Because of how, if you say we should throw the results like that away, it means we're virtuously refusing to update on what is in fact evidence.  Conversely, if we pay attention to results like that, we're creating bad systemic incentives."

"Obviously, that scenario basically never comes up in real life.  To the extent it ever did come up in real life, I figure people would just toss the original result and pay a 'replicator' to do it over again, if it was anything important."

"But, uh, I'm realizing as I say it, that none of this stuff is what we need to think about on a mini-Science! project the size of Project Lawful.  We're small enough that we can just know about all the experiments everyone is doing, and say all our individual probabilities on them out loud in advance.  We don't have professional replicators to appeal to, all we can do basically is run any important experiments twice with a different researcher in charge."

"I'm sort of flailing here, going through things in random order, as you can probably tell.  In dath ilan we all sort of grow up knowing how Science! is supposed to work, if only vaguely to start with, so at no point does anybody try to teach you the whole thing all at once.  There's just little pieces here and there where you learn in more detail how some bit of Science! works."

"What else is there to doing Science!... I suppose there's integrating the results from multiple experiments, reporting them in a unified way so that multiple lines of evidence can be collected?  Somebody mentioned that part already?  That part is sort of obvious if you know any Law of Probability, there's only one obviously correct way to do that...  Well, no, there was that one kid in class who invented his own wacky way of doing it, so it's not that obvious.  Probably I need to get out of the mindset of believing that saying obvious things will bore everyone here.  Golarion isn't doing it yet, so it's not that obvious."

"Though collecting results across multiple experiments is also something you'd do just as an individual investigating something on your own, meaning it's not really Science! as such... but you know, whatever, it'll be actually useful to the Project.  I'm just going to talk about accumulating data across different people's experiments, whether or not you call that by the term 'Science!'"

"Or no, let's go ahead and maniacally experiment to determine whether it actually is obvious or I just grew up that way.  Asmodia, how would you collect together the results from three experiments meant to distinguish among three hypotheses?"

"Keltham, you literally ran that exact problem on me earlier, when you were 'trolling' me, remember?  Luckily for your real question, I'd worked it out on my own before then, while wearing the artifact headband, and I can confirm that it's obvious."

"But yeah, say you've got three hypotheses, like, 'Ordinary Asmodia', 'Conspiracy Asmodia', and 'Time-Traveling Asmodia'.  They all start out with some probability-weights attached to them - Keltham didn't actually say what his prior weights were, but let's say they were 0.90 Ordinary, 0.06 Conspiracy, 0.03 Time-Travel, and 0.01 for Something Else but we're not going to consider that part -"

"Asmodia, I was supposed to be dead forever and found myself in another dimension with 'alternatephysics' that is both 'economicmagic' and 'conceptualmagic' like the story isn't even trying to be respectable about considering their consequences separately, was immediately allocated a research harem containing masochists, then the god of mind-altering substances cursed one of my researchers to give out cookies and Governance forgot to tell me about that for two days.  I am WELL above 1% on the None-Of-The-Above Hypothesis."

"People in dath ilan are above 1% on the What If Everything About Reality Is Actually Completely Different From How It Looks Hypothesis.  It's just that, usually, there's not much practical you can do about that."

"Uh, 80% Ordinary Asmodia, 10% Conspiracy Asmodia, 3% Time-Traveling Asmodia?  Actually I'm just going to run with that.  Speaking as either Ordinary Asmodia or somebody pretending to be her in great detail, Ordinary Asmodia doesn't want to help Conspiracy Asmodia by asking Keltham to reveal what his probabilities on Conspiracy were at the time."

"Now you're getting it, or realistically pretending to have just now gotten it."

Security, tell some of these idiots to look incredulous or ask 'What?' or something.  AlterCheliax did not have me using this exact case as an Important Lesson in Probability-Law and Also Conspiracy Maintenance and Also Keltham; their alter-personas should be confused about now.

"Um, what?"

"That was probably a little high-context, wasn't it."

"Really?  D'you think?"

"All right, different example.  Suppose there's a murder case with three possible suspects -"

"Asmodia, you can't just not explain now and leave them wondering, that would be mean."

"Keltham is tracking on an ongoing basis the possibility that his new world of Golarion is constructing an elaborate lie around him, including, for example, there not really being any such country as Cheliax.  Do not worry about this or try to do anything proactive about it, Keltham is a dath ilani and will not automatically conclude that everything is a giant lie even if he is looking for possible evidence of that.  We just need to be ourselves, and have faith in his ability to discern reality as reality."

"Keltham was also, I think briefly but I haven't actually asked, tracking the possibility that my overnight personality change and sudden mastery of the Law of Probability was due to my mind traveling a few months backward in time, rather than Manohar dropping an artifact headband on me for two hours."

"Keltham then observed me do two things and fail to do a third.  I forget what his exact probabilities were, at this point," she'll probably still remember them with awful clarity a hundred years later, "but the point is, he - guessed? calculated? I don't understand this part very well - some probabilities for how likely I was to do thing #1, if I was Ordinary or Conspiracy or Time-Traveler.  Then probabilities for my doing thing #2, and then probabilities for my doing #3.  Except I didn't do thing #3, so negate those probabilities - I mean, subtract them from 1..."

"Specifically, I'd just offered Asmodia a lower salary than all of my other researchers, she'd just written something down on a piece of paper, and what I was predicting was, one, the chance that what she'd written was a Prediction about me trolling her, two, if she'd made that Prediction, the chance that she'd assign a probability to her statement, three, if the first two things happened, the chance that she would write down that I would predict her Prediction."

Asmodia goes up to the board and writes the example:

                       Ordinary      Conspiracy    Time-Traveler
                       --------      ----------    --------------
Prior:                   .80            .10            .03

'Writes' (Y)             .50            .20            .80
'Quantifies' (Y)         .60            .70            .90
'Prediction' (N)        1 - .40        1 - .20        1 - .90

"And then - Keltham didn't actually say this part, because it was obvious given what we already learned - he multiplied his starting probabilities by all the likelihoods for each of the events, and ended up with... give me a second..."

Product:                 .14            .011           .002

"Not that it was actually that, because those weren't actually his probabilities, but that's the obvious way to combine multiple pieces of evidence.  Uh, though I'm leaving out the part where what really changes, is the ratio between the chances, not really the chances themselves.  At the end it's like... 13 times as much Ordinary as Conspiracy, and 5 times as much Conspiracy as Time-Traveler, where previously it was 8 times as much Ordinary as Conspiracy and 3 times as much Conspiracy as Time-Traveler."

"If we didn't already have the Law and were looking for Law-fragments, I'd observe that it doesn't matter what order you consider the data in, because it doesn't matter what order you multiply numbers.  Or that you could take the first results and multiply by those and call the result your new 'prior', and then multiply by the second and third results to get the new 'posterior', and that'd also be the same.  Those seem like the kind of 'coherence-constraints' the Law would need to obey, if we didn't know the Law already, or if it wasn't so easy to find a simple Law that fits together like that."

"That's basically correct, except that I didn't actually calculate any prior odds or multiply them?  When you're dealing with weird hypotheses on the order of 'my new world is actually a Conspiracy at a low level of sophistication distinguishable to me from reality', where your own mind is doing a lot of thrashing, it can make more sense for a non-Keeper like myself to make up the 'likelihoods', calculate the combined likelihood, and then just... sort of let my intuitive mind keep track of the intuitive update that it feels after I stare at those likelihoods a bit?  If I was keeping formal track of the chance of Conspiracy, it would probably swing all over the place, because of the degree to which I'm making all of the likelihoods up.  The thing to notice is if it starts to feel to me like there's a trend, after I keep on making up likelihoods."

"But, yes, that's the Law for combining the results from multiple experiments."