This post has the following content warnings:
Maybe the real unethical experimentation on nonconsenting subjects was the friends we made along the way
« Previous Post
+ Show First Post
Total: 155
Posts Per Page:
Permalink

"I'm not going to be your secretary but I wouldn't be having this chat if I weren't trying to convince you to get someone else to—put a pin on that."

Permalink

Agreeable nod. "Pinned."

Permalink

"Anyway! Number two. You're ignoring second-order effects, and don't try to say you aren't 'cause of course you are, it's not computable in polynomial time or if it is the exponent is huge. Obviously I can't compute them either but that's why you get, like—that thing I said earlier. About how if I act to stab people who help me in the back just because they didn't help me literally as much as possible then I don't get any help at all and I don't get to keep my brain where it is when evil scientists want to experiment on me? 

"That thing is like, like being reliably some kind of person, some person people can cooperate with because you're, like... you don't just do whatever you want? If you just do whatever you want based on your red ink or whatever then the only way people can predict you is if they know you super well to actually be able to predict you entirely

"...I'm not explaining this well. It's like why we have laws—kinda—no I mean it totally is, if it were better to not have laws and just live completely independently of everyone then that's what people would do but it's better to live in society because we can use society and the kinds of rules we get because those rules are made out of the ways ignoring second-order effects came back to bite us in the ass. Collectively."

Her speech gets progressively more animated and she gesticulates wildly, barely looking at him or paying attention to him. Once she gets going she gets going. "And obviously I'm not saying you shouldn't ever break rules, or that all rules are good or whatever, the process isn't perfect. But it's a compounding thing, the more rules you break—and not just laws, I mean rules of living with other people and common decency and not being a villain—the more rules you break the more reason you need to have to break them! It's a probability thing, like a compound, you know, it doesn't, it's not the same as, breaking one really bad rule doesn't make breaking all of the other rules not cost anything. And when you break as many rules as you're doing, as Rekenber is doing, then the other people who are working to kill Surt look at you and don't see a potential ally, they see a villain that they have to stop or would have if they didn't have a ton of other stuff on their plates, and that circles back to the first point."

Permalink

Thoma stares in delight as she gets going. When she reaches the stopping point, he claps lightly.

"– may I lay out my own thoughts on your point?" he asks. "I don't know if you want to finish out your argument before counterpoints."

Permalink

"No, go ahead." She needs to catch her breath anyway.

Permalink

"Excellent. My counterpoint is that I am evil. I am dispositionally incapable of following any societal rule, especially the ones that matter. And you have left out an important point, namely that I receive asymmetrical advantages from being evil just as you do from being... whatever you are, nice I suppose. If I were not evil, I wouldn't have nearly the number of human corpses or live subjects to experiment on. I wouldn't have Rekenber's resources. I might be obligated not to use some particular solution to the Surt problem if it offended the sensibilities - I'd be surprised if killing him involved trapping his spirit in a human child who then suffered agonies for the rest of time, but it's not the sort of thing I like to rule out, you know?"

He scoops up the knifier homunculus in his excitement, petting its needle-hair in exactly and only the correct direction. "I have further thoughts. But I imagine you also have further thoughts. Please continue, this is immensely helpful."

Permalink

She kinda doubts it but whatever she's not really expecting this to work even if he's indulging her she's just buying time. ...well not just but at least in large part. 

"I call bullshit. Sure, maybe you have a disposition or whatever, but you're still human and can still control your own actions or create ways to do it. And that's, you know, exactly the thing I mean by having people check you? And either your goal is killing Surt or out isn't, saying that it's killing Surt but also you like hurting babies is you just deluding yourself because you want to do whatever you feel like doing. And I guess you can, if you want, but that's the sort of thing that'll obviously get you killed because even if life's not a novel it still has people who can sucker punch Surt and survive and those people often—I already said this.

"And like—governments make decisions that kill people too? And not just them, hospitals do too. And sure, Schwarzwald's government is also evil but I hear there are ones that aren't. And if there aren't there should be. And hospitals aren't evil, mostly, I'm pretty sure, and they do red ink maths all the time and that's what it looks like when you're trying to save people and you have to make choices that'll definitely kill some of them."

Permalink

"Mm - you misunderstand me, I think. I don't 'like hurting babies'. I just... don't have the kind of moral instinct that lets people operate in society. Red ink is the closest I get, and I had to pull it out of a dream. You're right, that it doesn't look like what good people do."

Permalink

"You don't need instinct you just need rules. Or—not social ones. Personal ones. If you pulled the red ink out of a dream then you can take the next step, you can make yourself into a tool to achieve your goals." Otherwise what are you for? What is anyone for? "Figure out a sort of—there's two kinds of things to do, right? You can try to decide, on the fly, what you'll do, every time, always be calculating the 'best' way to get what you want, or you can be the type of person who gets what they want, and I don't mean play a role that's aesthetically inspired by people's idea of successful people, I mean figuring out what kinds of stable—algorithms—procedures—you need to have that'll make you reliably get there. And obviously I think this is better than the other thing or I wouldn't be talking about it but one of the reasons is that it works even under pressure, even when you don't have time to think because a huge disgusting insect with too many legs got attached to your face. You can fall back into someone who will get what they want even—

"You're good at maths. It's being able to look for global optima, not just local ones, in your optimisation function."

Permalink

He pets the sharp homunculus some more. The winged one flaps over from circling Lucky, to perch on his shoulder and pat his head. Ari is still shivering happily in her arms, but he starts shivering a little less happily.

Thoma himself slowly loses his facial expression. It's odd to watch. He frowns, and then it leaches away, until the tension in his jaw is the only muscle working in his face.

"...she didn't know any better than I did," he says. His voice is just as blank. "The one with the red ink. She'd been told some rules, and... she cared about someone. Enough to follow the rules. But I don't have weights for an algorithm. She had someone doing all of that for her."

Permalink

...that feels important. Like, beyond the way he's getting all serious, that feels important. "She?" The one with the red ink. "The one you dreamt about?"

Permalink

"Yes. ...I dream about her, but she wasn't a dream. The dream doesn't know names, but I saw a particular battle and looked her up. Yrsa. One of the first true alchemists. Sister to the lord of Heorot."

Permalink

...he dreams someone real? That sounds like something out of a novel.

But entertaining the idea for a moment... "There's a part of me that's like... Look I'm not stupid. I know you're just humouring me because you find me interesting. But I meant everything I said. And if you went, fine, I want to try this being good thing, then I—don't want to punish you for making that decision? And you've got enough resources that you'd be really useful if you were properly—cooperating. And I'd want to enable that. If I could."

Permalink

That gets an expression out of him. "I am not humoring you. Do you know how godsdamned hard it is to find someone intelligent, who knows how to explain ethics, and who already has too much confidential information to be allowed to live such that I can actually tell them anything? Obviously I kidnapped you to evaluate whether to experiment on you, but this is much more important than that."

Permalink

"Well I guess I won't be able to help very long then if you're gonna kill me regardless but fine, ask me anything about ethics." She is so not qualified for this but she's been doing things she's not qualified for for years. Someone has to and not enough qualified people are around.

Permalink

"I may or may not actually kill you," he clarifies. "You have too much information to live by Rekenber's standards, but no one except me knows you were there and if I can satisfy myself that you can keep an oath not to use this intelligence against me I might just let you go. And depending on how well you convince me, I may find myself needing to sabotage Rekenber to make up for my own sins, in which case it wouldn't matter in the slightest."

Permalink

"...of course you can trust me to keep an oath, weren't you listening—" It's reasonable for people to doubt words your say under threat of death, Lucky. "Okay, maybe you can't trust my words that far I guess but if I make a promise I keep it. I claim. It's—so the, like, failure of my philosophy is that a lot of it isn't possible to verify, right? The next time an evil scientist kidnaps me they won't know that I cooperated with this one, but—it still matters and still translates to other parts of my life and if in general I act like this then anyone who pays attention will notice, you know? Which I guess you wouldn't since it's not like I'm famous or anything. But still, it's important."

Permalink

"Lucky, I have absolute faith that you're smart enough to say that. And it certainly helps, in that if you hadn't already made the suggestion for your legs back I probably would be experimenting on you. But, as you say: I need to pay attention. And that may take a little while."

Permalink

"Yeah, yeah," she sighs. "So, what, am I to be kept as a pet until you're sufficiently sure of my honesty or whatever? And in case you aren't I get experimented upon and/or killed, so I'll have a lot of incentive to keep acting honest even if I don't mean it?"

Man, she really should learn how to shut up sometime.

Permalink

"If you don't mean it," Thoma says, "please do lie to my face. Nothing could end this test more quickly."

Permalink

"If you're such a good lie detector then what's the point of the test?"

Shut uuuuuuuup Luckyyyyyyyyy.

Permalink

"There is, actually, a difference between believing in a philosophy of oathkeeping, and actually being someone who will keep an oath under all foreseeable conditions. The former, I've already determined to be true. The latter will be difficult to confirm without personally torturing you. Which would be –" he mimes checking a small cheat-card "– wrong. So I need to determine the kind of person you are by other means. So you will be my honored guest, for a time."

Permalink

"Fine," she sighs. "Do I at least get to tinker with my legs? I had ideas I wanted to try implement and I'll die of boredom if I'm locked in this room."

Permalink

"Swear you won't use it against me? I suppose I could count that under the earlier oath, but it seems good to be on the same page. Also, you have the run of the house, caveat ask me about any locks."

Permalink

"I swear I won't use it against you. Or to try to escape, that'd be dickish too."

Total: 155
Posts Per Page: