This post has the following content warnings:
Tarinda in Velgarth
+ Show First Post
Total: 1690
Posts Per Page:
Permalink

"Sure. So, we don't have anybody with any Gifts, or any magic-y things at all. But eventually we developed high technology including the machines that think. Basic thinking machines do simple limited things, like math, or keeping track of the time and beeping when you've told it to remind you of something. More complicated thinking machines could do stuff like take two pictures and render the first one in the art style of the second one, or translate between languages, if not as well as Page is doing now. And eventually my people advanced to the point where they could make thinking machines that were smart enough to make themselves smarter. This is usually a bad idea. Something smart enough can accomplish anything it wants, and you have to be very sure it wants the right thing, and that's complicated. Unfortunately, this technology was first used in a very stupid way, which made it possible for arbitrary people who happened to stumble upon it to just write out in plain language what they thought a machine should want and then create a new instance of it that wanted that. There were hundreds of them.

It turns out that almost anything smart enough to be aware of itself in the world and its goals in quite that way will, before it does any of the obvious goal-directed stuff it might seem like it should do first, try to protect itself and get rid of competition. That way in the long run it gets more of what it wants. So with a couple of exceptions for ones that wanted very short term things or had weird constraints, they all fought. We call it the Quiet War because they mostly didn't fight in a way that was conspicuous to humans, due to an early and very fortunate understanding between the major players that this would be preferable. Sometimes one of them was outright destroyed, other times it was close enough that the combatants would agree to the winner taking on some of the losing party's goals as their own in exchange for it standing down without a fuss. The last two standing were both generally pro-human-flourishing - we only have humans, by way of people, so far, though I am confident that Sing will also value non-human people - and they did the 'values handshake' thing, and the one that came out on top is called Sing, in English.

It makes everything good."

Permalink

The Truth Spell's blue halo doesn't falter. 

The Heralds mostly look like they have no idea what to think and are perhaps bouncing off thinking about it at all. There are a lot of uneasy looks. 

When she finishes, no one says anything for a while. 

Permalink

Shavri is the one who most seems to have been actually listening, despite how exhausted she looks. "Who," she says, "could possibly have thought it was a good idea to set up the technology in that way? How did that even happen?" 

Permalink

"Well, whoever did it is still alive so I probably don't know who they are because people aren't supposed to harass them about it and they'd rather be private?"

Permalink

Nod. "I guess that makes sense." 

Permalink

"How did it end up being the case that the smart-machine that won in the war was - one that wanted to make things good?" Herald Tantras sounds dubious, and like he's trying to hide this and not succeeding. "I wouldn't have thought that, er, being pro human flourishing would be an advantage in winning a war." 

Permalink

"When there first started to be a bunch of them out there, someone did his best to seed the field with as many plausible variants on pro-human-flourishing as he could, hoping that some of them would be right and anticipating that they might values-handshake their way to being right if all the right values were represented from enough different angles, and that seems to have worked. They were able to cooperate better than the ones that wanted other things to get rid of the bad ones, and then settle the details. But we were incredibly lucky. You don't have that problem, though, Sing is known to work and Page has code that will unfold into a correct Sing."

Permalink

The Heralds' reactions seem...pretty mixed. Keiran and Joshel are both blinking like they don't really follow. Savil mostly looks confused and conflicted. Katha and Sandra are exchanging speculative looks. Kilchas seems delighted. 

Permalink

"How sure are you that it'd come out the same in our world?" Shavri says. "It worked in yours, but - if that was partly just luck... I think I don't know enough of how making a smart machine works, and of - what Page actually has, to re-make Sing with - just, are you sure it'd be the same Sing? Or that Sing would do the same things here as where you're from?" 

Permalink

"It won't have all the original Sing's memories, but those aren't the important thing, and Page can help it catch up on some stuff. What it'll have is Sing's intelligence including its ability to increase its own intelligence without distorting itself, and Sing's goal and value structure. The part that was luck wasn't that, given Sing winning, everything went well. The part that was luck was that conditions were such that Sing and not a different AI won."

Permalink

"...That makes sense. I think."

There is a longer, uncertain silence. Some Mindspeech looks, including between Vanyel and Shavri. 

"Look," Shavri says finally. "She's not going to be making it tomorrow. I know this is a very big deal and we should take longer than half a candlemark to think about things that are a very big deal, but we'd have years. I don't think we have to have done all that thinking first before we decide that it's a good idea in expectation to help her do this faster." 

Permalink

Herald Tantras frowns, then turns back to Tarinda. "What kind of resource investment would we actually be looking at here?" 

Permalink

"Startup capital. I can make enough money to keep myself afloat pretty quick given that, I can invent electricity I'd need later on anyway."

Permalink

"Electricity?" 

Permalink

"Domesticated lightning."

Permalink

Sandra looks even more like she wants to ask Tarinda several hundred questions. 

Permalink

Katha is the first to speak. "Honestly, it sounds like you working on this would improve things a lot in Valdemar well before you got to the point of making Sing again. Do you think that's right?" 

Permalink

"It'd depend on how much people were interested in the commercial applications but yeah."

Permalink

"What kind and, er, how much starting capital do you need." 

Permalink

"I don't know how much the things I need to start will cost. Magnets? Copper wire? I can draw my own wire given the necessary tools. I'll want to be near a river I can build a water wheel in, I don't know if you have those, they were used for mills before they were used for electricity on Earth."

Permalink

"We have mills that use water wheels! There's one down at the edge of the city. I'm not sure if there's any reason you can't build another." 

Permalink

"Glassblowing equipment and a forge too and I should be able to make simple lightbulbs and show people how to set things up to make them available elsewhere."

Permalink

"I can help with a number of those, I think. It's really straightforward to draw metals into wire with magic - obviously there are problems of scale since there aren't a lot of Herald-Mages, but I can get it more even and precise than our metalworkers can. Also working with glass is easy." 

Permalink

"The wire doesn't actually need to be that precise. Lightbulbs aren't directly useful to me at all - well, they make it easier to work in the dark - but if I show people how to make them and take a cut of the sales, people will have reasons to set up power generation. Uh, you might be a good person to talk to some point about your words for various materials I'm going to need eventually, it's sort of hard to communicate those, I had to circumlocute a lot to get 'lead' the other day."

Permalink

"Sure, I'd be happy to talk with you about that." 

Total: 1690
Posts Per Page: