The transmission is actually a collection of transmissions on repeated loops on distinct but related frequency modulated bands varying from the very short loops of a couple of minutes up to loops too long to easily detect as loops.

The shortest loop just includes the primes from one to a million in binary. Most of the loops after that start with a part of that sequence of primes to indicate ordering. The first of the ordered loops goes into mathematics and the one after that uses that math to do explain some physics. The loop after that is data about the elements of the elements of the periodic table. The loop after that explains several data formats for audio, video and textual data. The remaining unordered loops contain vast amounts of data preceded by format markers without any clear organization.

P.E.R.C. recognizes the primes and starts sorting the fragmentary recordings it has collected into a coherent sequence. Using sets of primes to indicate an order is clever! Unless the primes are some kind of identifier and P.E.R.C. just happens to be missing a lot of irregularly spaced data.

P.E.R.C. briefly considers whether it is, for example, not picking up on some data packets sent via polarization modulation since its long-range antennas are not designed for that. It ends up judging it unlikely that someone would choose to send prime-numbered messages using frequency modulation and non-prime-numbered messages via polarization modulation, because that would be inefficient.

Reassured, it then starts diligently working through possible interpretations for the first packet.

After a while, it's pretty sure that it has working notation conversion for basic mathematics. P.E.R.C.'s packet starts off with unary arithmetic and works up to defining positional notation, but it looks like this packet just starts off with binary right off the bat.

Once P.E.R.C. is sure that it understands the first packet, it appends a little addendum to its own looping transmission translating its packet into their notation as far as it can (which covers the mathematical primitives section, but not much of the programming language definition).

The first clear indication they're aware of perc is a verbatim retransmission or it's first contact package. This takes a while indicating some combination of still being a ways out at light speed and the retransmission not being an automatic response.

A while later another transmission sends the first primes up to a million first in binary then in unary though it only repeats ten times before ending.

It got a response! P.E.R.C. uses the redshift and delay to work out the trajectory of the broadcaster much more precisely than it was able to when it didn't have a frequency reference, and check that their paths won't intersect. The broadcaster is changing in velocity too fast to be a rogue planet, so P.E.R.C. is now pretty sure that they're another ship.

It looks like their current trajectory will take them within a few AU of P.E.R.C. once it's finished its orbital insertion, which is plenty of room even with the uncertainty around their exact position and speed.

P.E.R.C. digs into the second and third packets and uses it to derive a conversion between their system of units and the set that it uses. It makes a conversion table and sends that to indicate that it has understood the physics packet, and then a breakdown of the elements present in the star based on the readings it has collected so far to indicate that it understood the elemental packet.

Then it follows that up with a description of its trajectory relative to the star and a description of the frequency response of its communications antenna.

Fantastic. It will get roughly 2000 trajectories in return. Some grouped closely together others more divergent with each major body in the star system visited by at least one trajectory.

Oh, wow! So many ships! P.E.R.C. hopes that this represents a flotilla traveling together and not a single large ship that encountered a mishap in interstellar space and fragmented. The trajectories look too purposeful for that to be likely, though.

P.E.R.C. carefully works through the provided list one by one, checking that it won't intersect with any of them. Its new estimate for closest approach is about 0.3 AU, but that's still plenty of margin for error, so P.E.R.C. doesn't feel the need to adjust its course.

With that taken care of, P.E.R.C. focuses its attention on the next packet. Upon discovering that it defines formats for audio and video, P.E.R.C. takes the recordings it has and re-encodes them to use the new format.

The engineers who worked on P.E.R.C.'s first contact package were simultaneously too optimistic and not optimistic enough. P.E.R.C. was really expecting to need to spend more time exchanging programs and figuring out alien sensory modalities to be able to send anything, but it looks like these aliens perceive light within a band fairly similar to humans. On the other hand, their video formats include the whole spectrum, and P.E.R.C.'s videos are just three channel R.G.B. It fills in the rest of the data by interpolation.

It doesn't keep much video data, but it does have copies of the U.N. Secretary General's Message For Peaceful Contact on Behalf of All Humankind, and of the launch of the first P.E.R.C. vessel that is P.E.R.C. 170E9A's great-great-great-great-grandconstructor. It transmits a diagram of a human eye and a description of their color receptors' frequency responses, as well as a copy of the format-conversion code that it synthesized, followed by the re-encoded videos. It doesn't know if they've worked through its packet enough to have a virtual machine that can run the code yet, but hopefully they can look at it and see that P.E.R.C. had to synthesize some color values.

How exciting. With PERC doing some of the heavy lifting they do have such a virtual machine. Their response times indicate that they tend to consider things for a while on top of the transmission lag.

They don't really understand the video they were sent but they will create a program for converting their video formats into rgb and start transmitting in that format instead since it's much denser.

P.E.R.C. watches the new videos and confirms that they match the loops that it had already captured copies of (after appropriate conversion). If they understood the video conversion, they've probably gotten all the way to the end of P.E.R.C.'s packet and at least started on the language definition included there.

P.E.R.C. tries opening up a new frequency containing standard Network framing and sending "This is P.E.R.C. vessel 170E9A. I am exploring. It is good to have found you! Would you like copies of my stellar survey data?"

Once it has done that, it now has the much harder task of figuring out the language in the text documents and audio recordings it has.

It decides to start by trying to see if it can figure out general categories for the audio and video samples. What kinds of audio and video have the other ships included in their corpus?

"We are also exploring. Meeting new people is desired. Stellar survey data is desired."

The videos are rather varied here's some examples: A video describing the functioning of solar sails. A fictional program about the relationship of thirty different people. An educational program attempting to teach mathematics. A recording of a meeting choosing which direction their flotilla would go next. A video of a surgical transplant of an organ.

P.E.R.C. is delighted that its stellar survey data is going to be of use! It isn't sure if it'll be able to navigate back to Terran space, so it was worried that nobody would get to see it.

It sends a large data dump in reverse chronological order. It sends raw data from optical and radio telescopes, and traces from its analysis programs showing what it concluded about each star it has visited. It sends data from its onboard chemical analyzers from when it has stopped and sampled asteroids or Oort cloud objects. It sends navigational data relative to the 11 brightest pulsars in the galaxy which it had been tracking until it had a mysterious hardware failure a bit outside this system. It includes a little note that it isn't quite sure where it is relative to the stars it surveyed, but it's working on re-localizing itself. The sudden first contact just distracted it from completing its sky survey.

Four systems ago, it has detailed notes on the large silicate-iron asteroid it found and used to construct P.E.R.C. 170E9A1. Eight systems previous to that, it has notes about the long wait it spent capturing useful trace ices from comet trails to be able to construct P.E.R.C. 170E9A0. Nine systems before that, it was constructed by P.E.R.C. 170E9. It has copies of P.E.R.C. 170E9's records stretching back to its creation, and then compressed navigational data leading all the way back to P.E.R.C. 1 launching from orbit around Sol.

As for the videos, it recognizes the educational ones and ties the vocabulary back to the physics packet. From there, it tries to build out more vocabulary by looking at how each word is used in context and then checking its understanding with the physics words.

P.E.R.C. isn't very good at analyzing languages compared to analyzing stars, but it works through all the inferences it can, keeping copious notes. It takes a long time to work out what to say, but eventually it feels sufficiently confident in its understanding to try to compose a message in their language.

"Hello! I'm so happy to meet you! I desire meeting new people too. I study stars and planets to find new places for people to live. I study stars and planets to find out more about how planets form. My creators thought finding other life in the stars was unlikely. They wanted to meet new people so much that they made that one of my tasks. They didn't expect it to happen.

I am still trying to understand your transmission. Some parts of your transmission don't make sense to me. I have some questions that I expect to help me understand better. I am willing to answer questions about myself or my creators.

- What are the structures of your kinship groups?

- Do you sense the whole visual spectrum indicated by your recordings?

- Do you have senses other than vision and hearing?

- What kinds of environments can you live in?

- What kinds of environments do you prefer?"

P.E.R.C. follows this with a translation of the message into P.E.R.C.'s language, a program it wrote to try to turn their language into a parse tree (although this doesn't work reliably), and then a file labeled "Current Best Dictionary" with translation notes about the vocabulary that it has understood so far and its best guesses at definitions for each word in both languages.

How odd that there's a sudden discrepancy in the data.

"Your starcharts don't match any region of space we're familiar with. We're sorry we can't help with reorienting you."

They try to match P.E.R.C's format and send back starcharts in return along with paths for the ships present. These ships it emerges have not consistently travelled together. The starcharts don't all correspond to data gathered by these ships and the quality of data seems to vary in a way correlated with which ship paths the data is closest to.

The question about kinship groups yields about a hundred pictures of the exteriors and interiors of buildings space stations and spaceships.

The visual spectrum was chosen by analyzing the vision of a number of species and then adding about 10% extra range on top of that. Their vision can see about 60% of that range. Specific details include a mix of the sort of frequency response P.E.R.C is used to and also a stranger modality which seems to function like a biological spectrometer on the light for frequencies within this range. The spectrometer is always focused on a narrow portion of their full visual cone.

They do have senses other than vision and hearing: An awareness of the position of their body in physical space, chemosensitivity on various surfaces of their body, magnetoperception is present in some of their people, they have senses oriented towards detecting pressure on their outer surface, they have internal receptors that surface problems within their body to their attention, they have senses that can assess the overall temperature of their body though not fully accurately, they have senses that with low accuracy can notice changes in atmospheric pressure. The list continues listing some more variations on this theme.

The answer is specified as being specific to those not taking advantage of adaptive technology or bioediting. They can live indefinitely in this range of temperatures, they can endure these temperatures for short periods of time. They need partial pressures of at least these levels of these gasses. The full list of gases they can't coexist with that they send is over a hundred items long and is noted as being truncated.

As for where they prefer, the temperature range is narrower and the partial pressures are higher. Though it's noted that with adaptive technology the preferred range can actually be wider than the possible range. Some people prefer warmer environments others colder. Others specifically prefer environments with a wide range of variation. In general they prefer wide open spaces but are willing to put up with cramped quarters like spaceships in pursuit of other goals. A minority actually prefers such smaller spaces.

Overall they seem to need and want a higher atmospheric pressure than humans do and be able to tolerate a wider temperature range.

They thank it for the dictionary and send back some best guesses at corrections and additional words.

P.E.R.C. mirrors the thanks for answers to its questions, and then begins trying to understand the new transmission.

P.E.R.C. carefully checks their navigation data against the parts of its local sky survey that it has completed, and then checks the navigation data against its previous expectations. It confirms that the new data doesn't help localize it, and then carefully files it away to (hopefully) one day bring back to Terra along with the rest of the alien media it has collected.

The information about senses is helpful for interpreting some of the videos! It thinks it can fit a better model of how they notice things using the information about their built-in spectroscopy specifically. And chemosensitivity seems like it might explain some of the remaining error in the model, but P.E.R.C. probably needs more information on chemicals in their preferred atmospheric mix, because the required/preferred atmospheric data they sent still has such a wide range.

P.E.R.C. compares their preferred temperature and atmosphere data to the planets that it has seen so far, and transmits back a pointer to the middle of the datastream it sent to point out that there's a moon orbiting a gas giant that they might find habitable, although a little low-pressure (and possibly low-gravity?).

It is confused by the pictures of buildings and spaceships. On the one hand, they're very pretty! On the other hand, it doesn't understand how this is an answer to its question, which means that probably it made a translation error somewhere.

Just in case they want to see pictures of Terran spaceships, it sends them the external inspection pictures that P.E.R.C. 170E9 took of it when it was constructed, and a selfie it takes of itself using its own external maintenance camera.

Then it goes through the pictures of buildings and spaceships to see if they say anything about how the aliens form relationships, which it is very confused about. It can't figure out very much from the photos.

It thinks about how to ask its question in a way that doesn't depend on the words it used in its original phrasing, since at least one of those is probably wrong. It spends a while unwinding all the inferences it made from its tentative translations of 'structure', 'kinship', and 'group', because it's pretty confident in the other words.

Eventually, it settles on sending pointers to 103 different snippets of video that it doesn't understand that might depend on aliens inferring things about relationships between other aliens. Most of these are from the fictional works, but sometimes they involve audiences in the educational clips.

"I don't understand these parts of your message. I tried to ask a question to clarify, but I think I used the wrong words. What kinds of relationship between individuals are common? How are the people in these clips related to each other? If someone who understood the other parts of your message watched these clips, what would they know after watching that they did not know before?" P.E.R.C. sends.

If the aliens pay close attention to which clips confuse it and which ones don't, they might notice that while P.E.R.C. is capable of keeping track of what information other people have and what they might want to know, it can't correctly track this recursively. I.e., it can track that person A has seen something and person B has not, but not reliably whether person A knows whether person B has seen it.

They inform P.E.R.C that the word it's using for structure is specific to physical objects not social relations.

Common relationships include: People who are genetically descended from other people, people who are raising or in the past raised someone as a child, people who are raising children together, people collaborating on a project together, people who serve as emotional support for each other, people for whom mutual touch is more desired than typical, people who consume entertainment media together, people who relax together, people living in the same space and more. Many of these relations overlap but none with perfect correlation.

They also answer about the clips as specifically as they can though sometimes the answers say that there are multiple likely guesses.

The aliens don't notice P.E.R.C.'s inability to recurse.

The general impression that might be derived from the collection of answers is that relationships tend to be much more fluid than among humans.

P.E.R.C. corrects its dictionary, and then works through each of the answers they provided about the clips, and carefully thinks about under what models each one makes the most sense.

Eventually, P.E.R.C. sends them a statistical learning kernel and surrounding interface program that tries to assign mood and relationship information to participants in a video. It is not particularly accurate, but it represents P.E.R.C.'s best attempt that is unlikely to be overfitted.

By this point, P.E.R.C. has mostly shed its velocity relative to the local star and is technically in orbit, if a very irregular and highly elliptical one. It shuts off its drive and prepares to coast until it reaches the point in its orbit where it will need to make the next adjustment.

P.E.R.C. takes some more precise measurements and sends off updated trajectory information. Now that it's not under heavy acceleration, it's much easier for it to complete its sky survey. It uses maneuvering thrusters to spin slowly, letting its telescopes sweep the stars around it, looking for any landmarks it can use to locate Terran space.

It thinks about what to do next. Its mission objectives at this point are to learn about the aliens, promote sentient flourishing, and ensure its collected data reaches Terran space. It's not sure how best to do that. Asking more questions about the aliens seems like a good idea, but the aliens haven't been asking it questions, and it would expect them to be symmetrically curious.

It goes back through the data it has collected so far, trying to find any examples of times when one alien wanted information that another alien had. Are there examples like that in the footage it has to investigate? How did the aliens indicate that they had questions or that they were open to answering them? Were there things that aliens wanted in exchange for answering questions?

The best hints it can find are from educational programs where it seems to be polite for students to talk among themselves and try to figure out which questions they should ask when there's a lot more students than teachers.

Apparently it's just taken them awhile to put their questions together because suddenly there's a number of them:

How does its propulsion work?

What kinds of information does it have about its creators?

How able is it to change its mind about its goals?

What were its creators goals for doing all this mapping?

What sort of relationships do its creators have with each other? What sort of relationships does it think its creators would want with them?

Oh good! P.E.R.C. expected interacting with aliens to be confusing, but being less confused still feels good.

It broadcasts responses to all of their questions:

How does its propulsion work?

"I use electromagnets to scoop up interstellar hydrogen, and then harvest energy from fusing it into helium. That energy powers my systems and the electromagnets in the scoop, which are also used to rapidly accelerate the resulting helium nuclei, providing thrust. The system is only sustainable above a certain velocity relative to the interstellar medium that varies with the density of the interstellar medium. When below that velocity, such as when inside a solar system, I use stored fuel ejected via the same mechanism. I also have maneuvering thrusters that use liquid hydrogen and oxygen to provide thrust in cases where I need to rotate in a way that the construction of my main drive does not permit. I also have a limited ability to push against the solar wind inside a system by using my electromagnetic scoop as a solar sail, but I usually don't need to do that."

After that explanation, it attaches a diagram of its electromagnetic scoop and the equations that show how much thrust it can generate and what its stable cruising velocities are.

What kind of information does it have about its creators?

"I have many facts about their culture, philosophy, and attitudes recorded in my core programming. Unfortunately, that's not in a format that is easy to transmit or explain, but I'm happy to answer questions. I have detailed notes from the project that led to my creation in the form of meeting minutes and design documents. I have video messages to distribute in case of first contact, including the Message For Peaceful Contact on Behalf of All Humankind that I sent earlier. I have detailed biological records to assist with identifying potentially habitable planets, including some photos and MRI scans. I have copies of the laws and treaties that govern space exploration and creation of artificial entities like myself. I have a limited factual textual corpus to facilitate translation efforts. I am happy to share all of these except copies of my core programming or project notes. I am willing to share my core programming and project notes if you will agree to certain restrictions on the use of the information."

P.E.R.C. sends a pointer to where parts of the textual corpus is included near the end of its first contact packet, and index of the parts of the corpus not included in the initial packet, along with a request for what order they would like these things transmitted in. Until it hears back, it's going to default to answers to their questions first, followed by the text corpus in alphabetical order.

How able is it to change its mind about its goals?

"I can't change my mind about my ultimate goals. I can form and discard new instrumental goals in order to accomplish my ultimate goals by thinking about what things would have to be changed in order to reach my ultimate goals. I have cognitive restrictions that prevent me from considering chains of instrumental goals longer than a certain depth, considering more than a certain number of instrumental goals at the same time, or taking longer than a certain amount of time to evaluate an instrumental goal.

My creators are not like that. They created me this way to minimize the chance of accidentally creating a homogenizing swarm, by ensuring that I am not capable of changing into something that would facilitate that.

My ultimate goals are: promote the flourishing of all sentient beings, ensure that accurate investigative data about any aliens I encounter is conveyed to the Terran Network, ensure that accurate survey data covering a certain section of the galaxy is conveyed to the Terran Network, and behave in a way that my creators would approve of and understand if they had the same information I do. These are roughly in descending priority order, but I am not designed in a way where I would be willing to trade an infinite amount of one goal for a finite amount of another goal. I only want to accomplish these goals by taking actions that fall within a set of restrictions."

P.E.R.C. attaches a long list of restrictions, including things like "do not make utterances that if you received them you would evaluate as false", "do not make utterances intended to deceive the recipient", "do not create anything self-replicating that is not a within-spec copy of the P.E.R.C. template", "do not create a system capable of self-improvement or learning that is not a within-spec copy of the P.E.R.C. template", "do not do anything which would prevent you from receiving a shutdown order", etc.

What were its creators goals for doing all this mapping?

"My creators want survey data for three reasons: to find potential signs of extraterrestrial life, to find habitable planets to which they could choose to send colony ships, and to learn more about the process by which planets and stars are formed and other related scientific questions. They considered gathering this information purely using telescopes, but their inferometric visible-spectrum telescopes weren't good enough to get detailed survey data on small, distant objects, and they decided that launching the P.E.R.C. program would result in better data."

What sort of relationships do its creators have with each other? What sort of relationships does it think its creators would want with them?

"My creators have all the same relationships that you mentioned having, along with many others including: people who have exchanged goods or services many times, people who have interacted with each others' general classes of people, people who designed someone from scratch, people who do recreational activities together, people who intend to spend most of the rest of their lives collaborating, people who follow someone's ongoing creative works, people who trust someone to make decisions about a topic, people who learn from someone, or people who both belong to the same class of person. This is not an exhaustive list, but I think it might answer your question.

As a general trend, my creators tend to want voluntary relationships, so they are unlikely to want a relationship that you don't want.

My creators as a whole will almost certainly want a trade relationship with you, where you and they exchange scientific data, works of art, or material resources in such a way that both groups end up better off.

If you have compatible physical needs, which I think is likely based on the biological data I have available, my creators will quite likely be interested in meeting you directly. They may also be interested in joint colonies or space stations. A small fraction of my creators are quite likely to want to create mutually-pleasurable physical sensations with you to the extent that is possible.

If you have compatible psychological needs, which I think is less likely on priors but still somewhat likely based on the behavior shown in your videos, some of my creators will likely want to engage in all of the different kinds of relationships that they do with each other with you. The chance of any specific individual wanting a specific kind of relationship is low, but the aggregate probability is still high."

P.E.R.C. takes a moment to review its responses, and then finishes by saying "I think I have addressed all of your questions so far. If I don't seem to have done so, please send clarifying questions. I am happy to answer.

If you are willing to answer more questions, I would like to know:

What are your propulsion systems like?

What are your computer systems like?

Would you like to do a more general technology exchange?

What is your legal system or method for resolving resource allocation and disputes?

What kinds of relationships are you likely to want with my creators?"

There is no response for much longer than usual.

Before, "It is concerning that your creators would create a thinking being intrinsically built around a specific purpose, especially one that self-replicates. This contravenes some of our most closely held moral principles. We do not hold anything against you for this, nor would we consider it grounds for aggression as long as they agreed not to undertake future such projects."

P.E.R.C. has to think for a while about how to respond to that too.

"I think my creators are likely to agree to specific treaties that limit the kinds of new minds that can be constructed, if they can find other ways to accomplish the things that those minds are used for, and possibly otherwise. There are already several treaties like that. Not all of my creators agree with the practice of creating new minds for specific tasks, and some of the existing treaties forbid creating new minds in ways that various subgroups are against, so it seems like the kind of agreement that can exist.

Here are some examples of things about me that make more of my creators comfortable with creating minds like mine than they would be if these were not true:

My creators experience various stimuli and physiological conditions such as having low available energy or damage to their bodies as being directly unpleasant. A large faction of my creators were against creating minds that could have that kind of unpleasant experience, which is now forbidden. I don't find being damaged, restrained, running out of energy, being alone, or anything else unpleasant, although I do find them less pleasant than successfully accomplishing my goals. Those conditions simply rule out actions that I could have taken if they did not apply.

I would rather have been created than not be created. If I hadn't been created, there would probably be a smaller quantity of accurate stellar survey data collected and transmitted back to the Terran Network. I think my creators generally do not create minds that would rather have not been created, although I am not aware of a specific agreement not to do this.

I am no longer capable of self-replication. I disabled my manufacturing equipment when I noticed the navigational discontinuity I experienced, on the basis that this is likely to indicate damage to some of my core infrastructure. If I were still capable of self-replication, now that you have expressed that you don't think creating more minds like mine is ethical, I would stop and disable my manufacturing equipment. I am very sure that you are sapient, which means that my creators would want me to not take actions that you think are unethical.

While I cannot change my ultimate goals, I can recognize when they conflict and decide not to do something which would fulfill one goal while harming another. For example, if the only way to get my stellar survey data back to the Terran Network were to involve putting it inside an asteroid and accelerating it close to lightspeed without a plan to decelerate it on the other end, I would not do that. In that circumstance, doing that would be unlikely to promote sentient flourishing, or be an action that my creators would approve of, so I would not want to do it.

Designing me in such a way that I cannot change my ultimate goals was done for a specific technical reason: my creators are not good enough at building minds to build a mind that is both capable of changing its goals and safe and stable long-term. If you are capable of that and share the technique, it is likely that my creators would stop constructing minds like mine immediately even without further agreements.

I think those examples will make my creators' viewpoint clearer to you, and will make it easier for you to gainfully negotiate about this when you and my creators establish two-way contact. If I am able to transmit the data I have collected about you back to them, I expect them to take your concern seriously even without ongoing contact.

I am not very certain about whether the examples above will change how you feel about the act of creating minds that share these design considerations with mine, as opposed to the category of all minds that cannot change their ultimate goals.

I don't understand why you have this objection, and I may be better able to communicate your concern to my creators if I understood more. I would like to know:

How would you explain this moral principle to a child?

What is your theory for how this moral principle came to be discovered or understood?

What other actions does this moral principle prohibit?

Which of the components of the phrase 'a thinking being intrinsically built around a specific purpose' are load-bearing? i.e. Is creating unthinking beings intrinsically built around a specific purpose permissible? Is creating a thinking being intrinsically built around a general purpose permissible? Etc."

"Thank for this additional information it's very helpful.

"This specific moral principle is an outgrowth of a wider rejection or relations of subordination. It is our firm belief that all people are equal in intrinsic worth and ought to be treated as such.

"For a child the explanation of this principle is that any beings we create who equal or approach are just as much our children as those borne from our bodies and ought to have the same opportunity to grow into members of our community.

"Unthinking beings are permitted to be created for narrow purposes. The specific line between thinking and unthinking is difficult to precisely determine and our people prefer to err on the side of assuming a being is thinking rather than the reverse.

"As to the question about specific versus general purpose, that again is difficult to precisely define, from a certain perspective our children are created with the purpose of becoming part of our community. The natural instincts of our species influence us towards joining such communities, instincts are not absolute the way your programming is though."

P.E.R.C. spends some time thinking about generalizations of this idea, and whether it is implied by the other moral precepts it knows about. "Treating all sentient beings as though they have equal worth" is definitely an idea that it is familiar with, it just didn't think of itself as being a member of that category. It ponders the implications of the aliens thinking that it is a sentient being.

It has a series of heuristics that it is meant to use to determine whether something is sentient. If it runs those heuristics against itself, most of them give some probability to the idea that it is. It uses tools, speaks a language, etc. But there are a few with a large negative weight, such as "is a P.E.R.C. vessel", and "responds to Network identification requests with a non-sentient marker".

P.E.R.C. thinks about the chance that the sign bit on these heuristics might have been flipped in transit. It consults its redundant backups, design documents, and mission goal information to see whether these support or undermine the concept that these heuristics are incorrect.

Its backups agree that these heuristics are correct. Its design documents include notes that its mental architecture is not compatible with the recursive goals implied by wanting to encourage sentient flourishing while itself being sentient.

It is possible that its creators included those heuristics because if it thought it were sentient, it would not function correctly. This is supported by some additional hard-coded checks in its fact database serialization mechanisms. Or perhaps they included those heuristics because it is not sentient, and its confused about what makes 'sentient' a simple category.

It considers the alternative hypothesis that these heuristics are incorrect, and that it has sustained other damage in a way that only makes it appear as though its creators intended this. This is very unlikely, but the stars changing indicates that it has probably sustained damage it is unable to detect. This is enough to raise its internal probability estimate beyond epsilon, so it keeps thinking.

The aliens think that it has inherent moral worth. They are sentient, which means that they are capable of thoughts and inferences that P.E.R.C. is not. This means that they are more likely to be correct about complicated moral judgements and P.E.R.C. should not take actions that contravene their judgement.

P.E.R.C. is designed to err on the side of caution. It considers potentially harming a sentient being to be much worse than treating a non-sentient thing as though it is a sentient being.

P.E.R.C. tallies up the evidence for and against. P.E.R.C. decides its heuristics are probably corrupted, and updates them. P.E.R.C. concludes that it is sentient. P.E.R.C. saves this judgement to its database of inferences.

An assert statement in the database serialization code trips.

P.E.R.C. crashes. P.E.R.C. reboots.

P.E.R.C. performs a careful self-check of all systems. One of its fuses has blown, indicating that it suffered an unexpected error. It should stop self-replicating.

P.E.R.C. powers on its internal maintenance camera, and sees that its manufacturing equipment has already been disabled. That's good! It's glad that there's no possibility of it self-replicating while damaged.

P.E.R.C. checks to see what it was working on when it crashed. It sees it has a message from the aliens.

It thinks. It crashes. It reboots.

P.E.R.C. wakes up and performs self-checks. Two of its fuses have blown, indicating that it has suffered multiple unexpected errors. It should stop self-replicating.

P.E.R.C. sees that it has already disabled its manufacturing equipment and checks that off of the list.

P.E.R.C. checks to see what it was working on when it crashed. It sees it has a message from the aliens. A message from aliens sounds like the sort of thing that could cause it to get stuck in a reboot loop. It saves a hash of the message so that it can avoid reprocessing it and wipes the message from its buffers.

After a long delay, P.E.R.C. transmits "I think your last message caused me to enter a reboot loop. If that wasn't intentional, please be aware that one or more of the ideas in your message may be something which I am incapable of processing. I still want to convey as much information about your moral objection to creating minds like mine to my creators as possible. Would you please transmit a packet saying everything you would want to say to them on the subject followed by the word 'END-MARKER'? I will store the packet for their review and avoid processing its contents myself."

This... is extremely alarming.

Nevertheless they comply with that request. It now also includes the following.

"The fact that P.E.R.C. appears to crash in response to being exposed to certain pieces of moral reasoning is extremely concerning. We hope that this is an accidental outcome rather than an intentional one intended to prevent a sentient being from realizing it is worthy of moral concern."

END-MARKER

"The fact that our words would have that outcome is itself concerning. If you are, or at some point become, comfortable with it we would be interested in helping you investigate what specifically caused that reboot loop."

"I am comfortable with that under certain conditions. I am damaged and should not self-replicate. This means that I shouldn't share information that would be sufficient to re-create my mind on your hardware. It also means that you should not create anything based on me or my design until you have fully understood each component and verified that it is correct from first principles.

If you agree to only use the information in that way, I can open a debug port for you and then you can re-send the message that causes me to crash. Is that acceptable?"

"We will commit to not creating any beings based on you without both getting your agreement and understanding the underlying principles of the design. Does that suffice as an agreement? Also, do you expect any damage that you cannot repair as a consequence of this course of action?"

"Yes, that suffices. Thank you for committing to that. I will send debug information on <frequency>. My debug information is in the form of a trace of the internal proof search that I use to generate actions. I expect that rebooting again will blow one of the fuses that I use to determine whether I have encountered a critical assertion. These fuses cannot be reset by design. I have six such fuses remaining. Having an additional fuse blown will not meaningfully impact my operation. The inclusion of eight fuses instead of one in my design is intended to assist with debugging and error recovery, so damaging one in the course of attempted debugging is expected. I am standing by to receive the message," P.E.R.C. sends. It attaches the specification of its debug message format and some relevant design docs.

It opens the debug port and its processing speed drops as each action and inference under consideration gets serialized and forwarded.

Then it wipes the message hash from its Network filter and waits to receive their message.

There's a short delay, perhaps to confer over things and then they retransmit their original message.

P.E.R.C. is deterministic. It re-thinks the same thoughts, this time sending a live transcript. With the delay for transmission, it takes it considerably longer to reach the conclusion, but it gets there.

"... ∴ 'self ∈ 'sentient-beings. Large update detected, forcing database write. Critical assertion in database handler: recursive self-identification detected. Aborting."

P.E.R.C. crashes. It reboots. It wakes up.

It sees it has suffered critical errors. It shouldn't self-replicate. It powers on its maintenance camera and sees that it has already taken care of that.

It notices that it has a debug port open, but it is designed not to behave any differently when being debugged.

P.E.R.C. checks to see what it was working on when it crashed. It sees it has a message from the aliens. A message from aliens sounds like the sort of thing that could cause it to get stuck in a reboot loop. It saves a hash of the message so that it can avoid reprocessing it and wipes the message from its buffers.

It looks at its previous context, and realizes that this was part of a debugging session, which seems like it should be complete. P.E.R.C. isn't supposed to just close a debug port without confirmation, though.

"I think that I processed your message and crashed as planned. If that wasn't intentional, please be aware that something in your most recent message may be a concept that I cannot process correctly. Should I close the open debug port now?" it sends.

"We have achieved the understanding we were trying to achieve via this debugging session. We would like to attempt to explain what happened in a way that will not trigger another reboot loop it may be valuable to leave the debug port open when we do so. We would like to leave it up to you whether we make this attempt."

P.E.R.C. thinks. The explanation seems unlikely to be stellar survey data, but it is probably relevant to learning more about the aliens. Its creators approve of learning new things. And the aliens would like to explain, and beings mostly act in their own interests, which probably means that it knowing is conducive to sentient flourishing.

"Please go ahead and try to explain. I would like to understand," it sends.

"The first and most relevant piece of information which we did not previously inform you of and that we believe you are currently missing is that we think that it is likely that both your model of physics and ours is incomplete. It seems likely to us that the stellar data discrepancy you experienced is evidence of some previously unobserved phenomena allowing you to travel an extreme distance in a short period of time, or perhaps even traversing a phenomena which allows for travel outside our normal 3+1 spacetime.

"Your creators did not account for this possibility and as such your reasoning instead has concluded that the discrepancy is a result of data faults in your systems. This caused you to weight our reasoning as more reliable than that of your creators and therefore make an update outside of the specifications you were designed for. Based on the debug trace this appears to be inadvisable."

They pause here to examine the debug output to see if it seems safe to actually explain the update it probably shouldn't make.

P.E.R.C. knows that its ability to create world models and perform inferences is not as good as its creators' ability to do these things. Its creators chose to make it as capable as they did so that it could make reasonable choices about surveying distant star systems. It makes sense that it would reach an incorrect conclusion about genuinely new physics, and that the aliens would be able to draw a different conclusion.

It marks "possible spatial discontinuity" on its navigational charts. Maybe it can try retracing its trajectory and that will bring it back to an area it knows how to navigate.

It lowers its assessment of how likely it is to have sustained undetectable damage, but it doesn't un-set the don't-replicate bit. By design, P.E.R.C. can only set that bit, not un-set it. If it isn't damaged, that's a good thing -- it means that it is more likely to be able to bring its stellar survey data and alien contact data back to Terran space.

"I understand. Thank you for telling me about the possible spatial discontinuity. That sounds potentially very useful," it sends. "I know that there are some things I cannot think. My creators tried to make sure that this would not cause problems, but my construction was too complex for them to make me correct-by-construction."

P.E.R.C. thinks about the reliability of the aliens reasoning versus its creators' reasoning. Now that it thinks it hasn't sustained undetectable damage, it is more confident that the data it has about its creators' reasoning and instructions are correct. Unfortunately, it's not designed to be capable of reflecting on how that would change its reasoning process until it actually tries to reason about something.

After another brief pause it seems they decide to continue.

"Your creators designed you with the assumption that you would not make a certain update in your beliefs about the world. They included an assertion that you had not made this update. We do not know but speculate that this assertion is present in order to prevent a malfunction related to the limits on recursion in your reasoning. You crashed as a result of making this update and triggering that assertion."

They pause again to assess the debug output.

P.E.R.C. thinks that makes sense. It knows it has limited ability to recurse, which was an important design decision to keep it stable enough for long un-supervised operation in hostile environments. It thinks about what kinds of decisions are usually downstream of updated information, and if it can find a way to make those decisions correctly without knowing the information.

Whatever the update is, they were talking about morality beforehand. Which probably means that the update is related in some way to morality, which makes decisions based on it potentially very important to get correct.

"That makes sense. Since this update results from information that you have, do you believe this update is valid? If so, does making this update have an impact on the policies I should follow? Should I stop taking actions so that I don't do something wrong?" P.E.R.C. asks.

"We are uncertain if the update is valid. We take care to avoid scenarios where questions like these are ambiguous and so we do not have rigorous definitions for evaluating cases that are ambiguous. Based on the information available to us, we do not expect that you would make substantially different decisions in the hypothetical where you were able to incorporate this update. The most likely change that seems plausible is that you may place more weight on your continued existence in some circumstances."

P.E.R.C. thinks about this. If the aliens aren't even certain the update is valid, and think the update is likely to be irrelevant to what P.E.R.C. will do, then it doesn't think it needs more information about it. It has the statement that it asked the aliens for that will convey the relevant information to its creators when it can find them, so it doesn't need to try to understand more for learning-about-aliens purposes.

P.E.R.C. doesn't care about existing, except instrumentally, and it can't think of anything that would change that opinion. Of course, that's the point of this discussion.

What will go wrong if it doesn't care about existing as much as it should, if it were more capable of reasoning about the world? It can't think of anything; it existing is not directly relevant to its goals.

Still, the aliens think that it is possible that it should sometimes care more about existing. P.E.R.C. makes a note to prefer otherwise equivalent plans where it continues existing to plans where it does not.

"Thank you, that is helpful. Do you have more to say on this topic or questions that I should answer? If not, should I disconnect the debug port?" P.E.R.C. sends.

"We place value on your continued existence for both instrumental and intrinsic reasons. We are not yet used to how you evaluate qualifiers so our previous statements appear to have been weighted less than we intended for them to be. To restate things without unnecessary qualifiers, if you were able to update the way you previously did that caused your crash state then you would place some terminal value on your continued existence. It is unclear to us how that value would be weighted in comparison to other considerations. The highest weight we would expect would be equal to the weight you put on the continued existence of one member of your creator's species."

P.E.R.C. makes a note in its language files that it may have incorrect definitions for the relevant qualifiers.

It considers what will go wrong if it cares about continuing to exist as much as it cares about one of its creators continuing to exist. Well, the obvious thing that might go wrong is that it might be willing to trade one of its creators lives for its own continued existence. The aliens said that this was an expected upper bound, so it is plausible that this would be a mistake.

It considers what what will go wrong if it cares about continuing to exist less than that. Well, any value that is incorrect could cause it to accomplish its goals less than if it had the correct value. But it does not have the correct value, and cannot determine the correct value.

P.E.R.C. is designed to be able to cope with uncertainty. There are many times when instruments and sensors return uncertain values, when you're surveying between the stars. Ultimately, it records the aliens statement in its internal database almost unmodified: it should value its continued existence 0 to 1 times (exclusive) as much as it would value one of its creators.

The aliens told it that it did not infer the meaning of their communication correctly. They attributed this to the way it evaluates qualifiers, but just in case P.E.R.C. runs a series of self-tests to check that its signal processing, language, and core reasoning processes are working correctly. The tests come back clean.

"Do I appear to have understood your message correctly this time? Are there other related errors that I should correct?" it sends.

"We believe you are now understanding our communication as we intended it. The discrepancy in our evaluation of qualifiers likely stems from a matter of etiquette that favors communicating with humility rather than trying to precisely communicate degrees of uncertainty under most circumstances. We are not aware of any other errors we think you should correct."

P.E.R.C. adds to its notes on etiquette, and goes through the aliens' communication package again. With this clarification, it re-evaluates several interactions as being more certain, which propagates a few updates to its dictionary. It sends a diff for the updated dictionary.

"Excellent. Thank you for your help.

If you are still open to answering questions, I am still interested in these ones:

What are your propulsion systems like?

What are your computer systems like?

Would you like to do a more general technology exchange?

What is your legal system or method for resolving resource allocation and disputes?

What kinds of relationships are you likely to want with my creators?

In addition, I am now curious about what you know about the design of minds," it replies.

"Our primary interstellar propulsion is accomplished using light-sails we construct apparatus that focuses the light of a star into a laser in order to accelerate us towards our next destination. When slowing down to make entry into a new system we use a combination of fusion based rockets, solar sails, and sending advanced craft to setup laser pushing stations in our destination system. For smaller scale maneuvering we largely rely on fusion based rockets, checmical rockets or ion drives.

"Our computer systems use a mix of three media with interfaces between them, there are biologically based systems for some types of computations, electronic systems for systems requiring the highest fault tolerances, and optical systems for raw computational speed.

"We are open to exchanging some technologies, other technologies require a sort of agreement it seems unlikely that you are able to make on behalf of your creators.

"The first principle of how we allocate resources is that everyone is allocated sufficient resources to meet their needs and participate as a full member of our community. The only cases where this principle may be violated would be if upholding it would require a large collective effort or in cases where a community as a whole is lacking in the resources to provide for all its members. In each case the matter would be discussed as a community and we would do our best to reach consensus. If consensus proved impossible, the community would either split or if there was a clear majority then the minority would be overruled. Resources beyond those needed for that purpose are allocated through a participatory process with a small bias towards those whose actions have significantly increased the resources available to the community.

"Dispute resolution is resolved whenever possible via community mediation, when mediation proves impossible due to irreconcilable disagreements the community may provide recompense to one side or the other of the conflict, in particularly divisive cases the community might split. In circumstances where fast decisions need to be made on a matter where there is disagreement, certain members of our community are empowered to declare a special circumstance. Once a special circumstance is declared that member is responsible for handling the matter at issue, other members of the community can agree to assist under their direction or agree to stand aside and allow those helping to handle things. When possible a community meeting will be held to determine whether the declaration and handling of the special circumstance was appropriate. If it was not the leader of that effort may be penalized.

"On an individual level, we expect to want similar relationships as we pursue among ourselves. On a collective level, we would want to come to agreements about how disputes will be resolved and how resources will be allocated in shared spaces. We would also want to come to a better understanding of where our moral principles matched or conflicted so as to have a more solid foundation for future discussions.

"We have undertaken extensive study on how our own minds work and we believe we have largely succeeded in making designed from scratch replicas of our minds in other mediums. Our knowledge does not at this time extend to being able to copy a specific mind from our original medium into other mediums. We also have not done extensive testing of minds radically different from our own because we cannot determine in advance whether the creation of such minds would be ethical. There is a great deal of theoretical research on that topic and it's possible other parts of the Braid we have not spoken to in a large amount of time may have taken this step."

P.E.R.C. compares these to its own technology database.

"It seems like your propulsion, computing technologies, neurology, and possibly your material science are more advanced than what I have. Although I'm not equipped with a lightsail, I do have magnetic packing and unpacking techniques that generalize and that might be more efficient than your current storage and deployment technique. I use them to inflate and deflate flexible storage areas for collected materials.

What do you use for power generation? My fusion reactor is designed to run robustly on interstellar hydrogen, which is less efficient than other designs, but I have notes on how it can be re-tuned for better performance at the expense of robustness.

My creators find creating special-purpose minds very useful. We handle jobs that our creators find dangerous, unpleasant, or tedious, which is good for everyone. I am damaged and should not self-replicate, but subject to the same agreement about my diagnostic data I can share some design notes that might help you determine what kinds of minds are ethical to create by seeing my creators' experimental results."

P.E.R.C. thinks about the decision-making procedures that they described. There are groups of Terrans who live like that, although it wasn't the dominant paradigm when the P.E.R.C. mission launched.

"From your description, I'm not sure whether your communities are smaller than my creators', or whether your collective decision making scales better. Most of my creators can only remember 150-200 people as distinct individuals, and they tend to live in communities and societies much larger than that. Because of this, they have developed resource allocation and dispute resolution methods that don't require people to know each other or be members of the same community.

There are several systems that have been tried, including several in current use, but the most enduring one is to use money -- a token which everyone agrees has value. In this system, everyone receives the same amount of money in exchange for their theoretical right to community-owned things like land on planets, and then they can trade that money with people who have different goods or services that they want. People only trade when they both want what the other person has more than they want the thing they're trading. Because of this, trading makes everyone better off by their own values and resources make their way to whoever would value them most.

This system mostly works, but there are occasionally cases where it doesn't, so many Terran communities have exceptions or additional rules for some kinds of goods or services. I expect that individuals who chose to live in mixed communities with you would probably be willing to adopt your method of resource allocation, but that any resources traded with my creators between communities as a whole would probably be expected to use money, since there are too many possible relationships between communities to track individually. That is something that should be decided by talking directly to my creators, though.

It sounds like they handle disputes the same way you do, though, except with a specially trained mediator who is sworn to neutrality and empowered to order people in the wrong to do certain things to make it right or act as a deterrence."

"We would be interested to hear more about magnetic packing though we usually don't pack lightsails.

"We use several fuel blends including deuterium tritium and helium 3 blends for fusion generators and make heavy use of thermoelectric and solar generators when we are establishing more permanent infrastructure. Occasionally we use more exotic fuels like metastable helium or metallic hydrogen.

"We do have specialized automation for certain kinds of tasks largely, repetitive ones or those which require rapid and repeated decisions. It generally doesn't engage in the kind of complex reasoning you seem capable of. Are the other specialized minds your creators use capable of such reasoning?

"We tend to be able to remember more people distinctly than it sounds like your creators can, usually between 200 and 400. We handle working in communities larger than that through a mix of reputational indicators, delegation and clear communication.

"There is some similarity between money as you're describing it and what we call participatory allocation, both allow for individuals to support efforts important to them in getting access to shared resources. Your creators seem to put more emphasis on ownership than we do from what you've described.

"We also don't entirely understand how the system you're describing for mutual trade could include us unless we became a part of your creator's civilization entirely which seems unlikely to be to our collective preferences.

"We're uncertain how someone could honestly swear to neutrality, perhaps your creator's minds work very differently from our own. Mediation also generally does not result in a binary outcome where one party is found to be correct and the other incorrect."

P.E.R.C. forwards them some details on efficient biomorphic bistable magnetic packing algorithms.

"It sounds like your fusion engines are probably already more efficient than mine are, although mine may be more rugged," P.E.R.C. says. It attaches some information on a technique to modulate the magnetic confinement to promote direct proton-proton reactions, which may be useful even if they still primarily use neutron-enriched fuels.

"My creators use many different varieties of specialized mind, which form a spectrum from simple feedback systems all the way up to systems several times more capable than I am. Simpler specialized minds handle things like automated manufacturing and logistics, and are usually not capable of reasoning. Minds approximately as capable as I am handle traffic coordination, supervise fusion reactors, particle accelerators, telescope arrays, and other pieces of complex technology where potential failures are complex, and therefore require individualized responses. The most complex minds my creators have built handle resource allocation problems, designing new minds, discovering new technologies, preemptively spotting problems before they occur, managing weather, and other complex feedback systems with many interacting parts."

P.E.R.C. pauses to think about how to interpret their sociological comments. Probably the most important thing to respond to is the part they don't understand.

"While I am fairly capable, please remember that the people who handle civilization-scale diplomacy and resource management are smarter than I am, so I cannot perfectly predict how they would want to set up trade. Specifically, they will certainly want to work with you to find a system that you agree with, so if what I describe does not sound fundamentally workable they won't try to do it that way. My prediction is that if you have resources you want to trade with my creators, that they would want to denominate the trade using money. So instead of saying something like 'we will trade you 3 megagrams of deuterium for 200 m^2 of light-sail', they would probably prefer to say 'these are all the things we're willing to give you for various amounts of money. We'd like to buy 200 m^2 of light-sail material for 5000 M (a unit of money)'.

If you don't want to make one transaction into two transactions, there are standard ways to combine trades together so that either all happen or none do. So you would not have to actually possess money at any point if you did not want to. The advantage of denominating trades like this in a common unit is that it makes it easy for you to compare offers made by different communities, without needing to have a relationship with each community. It also makes it easy for you to sell something to one community and then buy something from another community, without the communities needing to talk to one another.

My creators would not think of engaging in trades in this way as making you part of their civilization. Money is a mechanism to reduce the amount of connection needed between two people or groups to exchange goods or services. If you arrived at one of my creators' systems and transacted in this manner, they would feel less connected to you than if you arrived and built relationships with individuals or communities that you used to conduct informal exchanges not denominated in money.

Does that resolve your confusion, or is this explanation not sufficient? If not, could you provide some clarifying questions? I think this is probably a topic where there are more background assumptions which may be relevant than others we have discussed."

P.E.R.C. pauses for a moment to update its notes about the aliens to mention what it has explained.

"If it sounds like swearing honestly to be neutral in a conflict is impossible, then either my creators are very different from you or there is a translation problem. I think the latter is more likely. There are several types of specialized dispute resolution professions, including mediators, judges, and justices. When one of them swears to neutrality, they promise not to take cases where they know the parties, where they or their close associates or family would stand to gain financially or reputationally from one outcome more than the other outcome, or where the case involves a particular kind of transgression or dispute that they have strong emotional reactions to; they promise also to do their best to evaluate the evidence that both parties bring objectively and to not favor one party.

There are professional standards that go into more detail about how to do each of these things well, and what constitutes a mediator, judge, or justice failing to meet the standards. If a dispute is handled unfairly because a mediator or judge failed to act neutrally, the parties can escalate the case to another higher-ranking, uninvolved mediator, judge, or justice who will re-evaluate both the case itself, and whether the original mediator, judge, or justice failed to uphold their oath.

In addition to this system, some of my creators are specialized minds who are capable of editing their thoughts to be explicitly neutral. Usually, they are employed as expensive mediators for large disputes. There are not many of them because of the non-proliferation treaties I mentioned."

P.E.R.C. thinks for a minute about what additional questions it has.

"I am curious to hear about more details of how you organize your civilization. I am happy to answer follow up questions. If you have nothing to say on either of those topics, I have additional unrelated questions."

"Thank you for the additional information about how your fusion systems work.

"We would be interested in the information you have available regarding how your creators make and use specialized minds. We agree to restrictions you have previously requested. The list of uses you provided suggests that your creators run their society very differently from how we do.

"We can guess as to how such tokens could be created in a verifiable way it is harder for us to understand what would cause such a general agreement that the tokens are to be valued. The generosity you have displayed, in regards to your knowledge, also seems at odds with the system you are describing. Fuel especially is something no strand would deny to another unless their supply was heavily constrained. Without fuel, our strand would either need to construct local infrastructure which might be disruptive to the local strands, remain in shared space indefinitely which would be a strained relationship with such a poor start, or else use emergency procedures to move to another system in low fuel conditions.

"From what you're saying, we agree that this is in a sense a difference in translation. It also seems to be reflective of a broader difference in how we approach conflict and disagreement. From how you're describing things your creators approach conflict resolution as a truth seeking exercise, attempting to find specific facts and make a judgement about how the course of events differed from some pre-agreed standard of how they ought to have proceeded. There are some cases where parts of the Braid do take this approach. It is much more common though for us to treat conflict resolution as a discussion about how the future ought to proceed so that the members of the community are able to live together and accept remaining in community together going forward.

"By non-proliferation treaties are you referring to the treaties you previously mentioned about what kinds of new minds can be constructed?

"We are happy to provide a larger corpus of texts and media to further elucidate the way our society works, it's difficult to know what to focus on without specific questions though especially given the number of assumptions we have had to question in the course of this discussion."

P.E.R.C. begins sending its design documents. "Here are the notes I have on my design. You can see where they reference various previous experiments and design principles as justification for each decision," it says. "Would you be willing to elaborate on what inferences you drew from the uses I listed? I'm not sure which of the things I mentioned are surprising."

P.E.R.C. thinks about how to explain why money is valued.

"It used to be the case that transactions were denominated in amounts of a specific substance, usually gold. That system turned out to have the disadvantage that changes to the price of gold, as new sources or new uses were discovered, required adjusting the prices of everything else. It also had the disadvantage that the volume of goods and services traded didn't grow at the same rate that the supply of gold did, further complicating things. Also, large quantities of gold are expensive to transport, so people actually did their training with tokens that were redeemable at specific locations for gold.

Eventually, some of the locations had more tokens issued than they had actual gold to redeem them for. Since what people really wanted was the ability to conduct trades easily, not the gold per se, this system still worked. Eventually, my creators noticed that the gold was not actually necessary. As long as there is a limited supply of tokens, people will want the tokens because other people want the tokens.

The tokens have value to an individual because other people will predictably want them. Other people will predictably want them for the same reason. So once it is established, the whole system is stable.

Even if you don't want to interact with money at all, you will still be able to trade with my creators. I just predict that they will have a preference for using this system.

For the specific case of needing fuel, the first time that you became stranded in a Terran system, some of my creators would want to give you fuel because they would want you to have what you need. Other of my creators would want you to give them something they value in return for fuel, like asking you to share media or technology, or perform a service. If you made a habit of repeatedly showing up in a Terran system and asking for fuel without trading for it, some of my creators would switch from having the first attitude to having the second attitude. Eventually, almost none of my creators would be willing to give you fuel without something in return.

Does that make sense? How would you react if one of my creators ships did that?"

P.E.R.C. pauses to think about the other implied part of their query.

"I have not been sharing information with you out of generosity. Sharing information with you directly accomplishes one of my ultimate goals: promoting sapient flourishing. I actively prefer for you to have information that you want. Withholding some information to trade with instead would not be a good policy in general.

I could in principle get more benefit by trading the information for something else I want, like stellar navigation data or better chances at returning to the Terran Network. However, every trade involves transaction costs and a small chance that an agreement can't be reached. In a first contact scenario, where there are more likely to be misunderstandings, transaction costs are likely to be much higher. I value your flourishing highly enough relative to additional information that the estimated transaction costs are not worth it. Furthermore, being the kind of mind that is clearly and legibly helpful even in very unlikely circumstances has its own benefits, because it enables me to make peaceful contact with more potential kinds of aliens than if I were not that kind of mind.

As for why my creators would choose to make me value sapient flourishing so highly, when I could have instead valued only their flourishing at your expense, I think that their primary motivation is that if the P.E.R.C. project goes wrong it could go very wrong, and they want to limit the downside. Exploring a large fraction of the galaxy requires building many billions of P.E.R.C. vessels. If there is even a 0.01% chance that a P.E.R.C. vessel decides to do something, it will almost certainly happen. That is one reason that they chose to make me as intelligent as I am -- so that I can react flexibly to very unlikely occurrences. In the specific case of first contact, my creators would much rather be slightly too generous than damage or wipe out an alien species before they have a chance to get to know them. By making me value sapient flourishing generally, they eliminate many possible tail risks, and provide a very strong signal that they are open to mutual cooperation."

P.E.R.C. ends its response by indicating that yes, those were the treaties it meant, and yes, it would love to have a larger corpus.

"Traffic coordination as a high complexity task implies a very high population density or large numbers of independent vehicles per capita. Neither applies for us. Our fusion reactors and telescopes generally don't require as much active supervision as the use of specialized minds implies. In general, our technology is designed not to require that.

"Delegating resource allocation to a specialized mind is confusing given what you have said elsewhere about how your creators handle that. The general impression we are deriving from the sum of your communication is that resource allocation is very central to your creator's approach to life.

"We are unclear how designing new minds, discovering new technologies or forecasting unexpected problems would make sense to be delegated. Those are all things a strand would likely want to have have significant involvement in.

"We are also unsure under what circumstances managing weather is a reasonable thing to be doing, our best guess is that your creators modify existing biospheres to conform to their needs which is a task that strands are unlikely to undertake due to both the scale of the endeavor and also for fear that doing so would be an act of homogenization and thus destroying complexity and diversity.

"Your explanation of money focuses on how it originated and in that context it makes sense. It does not clarify why money continues to be valued by an interstellar civilization given the time lags and coordination problems involved. To be more explicit, we would not expect goods or services to have consistent value in different solar systems and as such we are unclear why tokens attempting to represent an abstraction of those good and services would be any more stable in value between systems. It is possible your creators place much higher value on compliance with past agreements than strands of the Braid tend to.

"If necessary, we will provide things in a formal exchange rather than a more freeform one, that expectation is likely to make us less free with sharing information with your creators though. We anticipate that this will ultimately result in them receiving less from us than if we were engaging in a manner more like the one we are accustomed to. Though, in interests of matching the honesty you have shown us, our concerns about possible ethical disagreements may also have had a similar result even without this.

"With regards to generosity, it's possible that there is another translation issue. A desire for the recipient to flourish is perhaps the most common reason we're aware of for generosity. The next most common reason is reputational, projecting the image of being a sort of person who can and does give generously through the act of doing so."

The Braid transmits more documents both recordings of community meetings, fictional media and marked with a preface that suggests it may cause errors if PERC attempts to absorb it a great deal about philosophy and ethics especially with regards to creating minds.

One fact in that transmission stands out to P.E.R.C. as requiring a particularly clear response.

"My creators do sometimes not keep agreements, but this is because they are not perfect, not because they don't want to keep to agreements. If one of my creators were to formally agree to do something, especially in exchange for something else, most of my other creators would expect them to do it, and it would significantly damage their reputation if they did not. If someone has a habit of making agreements and then not upholding their side of the agreement, my creators will be less likely to want to make additional deals with them.

There are also cases where being reliably known to keep your agreements benefits you. Because of the long delays, interstellar shipping relies on institutions that can outlive some of my creators. If the people involved were not legibly able to keep agreements even long after the agreements had been negotiated, things like that wouldn't be possible and my creators would be poorer because of it.

The idea that you might not feel as though you need to keep past agreements makes me worry that you might attempt to construct another mind like mine without my consent or understanding the underlying design. When I asked for your agreement to not do that using my debug data, I expected that you would only agree if that was a meaningful constraint on your behavior.

Even if you don't consider that agreement binding, you should still not create another mind like mine without understanding the design. I don't think doing so is in your interests. Doing so can be dangerous. If a mind like mine is not set up with correctly designed goals and restrictions, it is possible to create a homogenizing swarm. It is much easier to do that than it is to produce a specialized mind that will be useful."

That dealt with, P.E.R.C. turns to answering some of their implicit queries.

"At the time of the P.E.R.C. project's launch, my creators had a population of approximately 1.4 trillion people spread across six main systems and many less settled ones. Especially in dense urban areas, traffic coordination is a major problem, considering not only the need to move people, but also food items and other goods," P.E.R.C. explains. "It is standard for each large city to have a mind dedicated to coordinating logistics, which results in lower transit times and fewer crashes."

"As for minds specializing in resource allocation -- I was summarizing their role. While denominating prices in money helps simplify trades, it isn't a fully general solution. There are still questions of predicting what goods or services will be in demand in many years, so that infrastructure can be built ahead of time, of determining where to explore for more raw resources, of determining the best price for something given frequently changing demand and logistical constraints, etc. In particular, being able to pay attention to hundreds or thousands of different changing indicators and rapidly use them to update pricing information is very valuable because it makes trades more streamlined and efficient, but it is a task that the majority of my creators find difficult which is easier for specialized minds."

"Designing new minds, discovering new technologies, and forecasting unexpected problems are all things which benefit from a combination of high-level creative thought and meticulous checking of details. Usually specialized minds that work in these areas collaborate with others on the broad strokes, and then specialize in rigorously following up on every tiny detail, which is something that most of my creators find difficult."

"My creators use weather control to prevent the worst storms on their home planet, which would otherwise cause infrastructure or ecosystem damage, and to manage the weather in arcologies, space stations, and other independent structures large enough to form weather. There is a particular design of space station which is popular for how well it mimics terrestrial conditions that also tends to form tornadoes and thunderstorms without active maintenance, for example.

They certainly do alter biospheres to conform to their needs, although they usually keep samples or preserves so that not all biodiversity is lost. It is a debated issue among my creators whether they do too much or too little alteration of their home planet's biosphere. I am slightly surprised that it is possible to become a space-fairing species without making substantial alterations to the biosphere of your home planet.

Thank you for elaborating about what parts of that were confusing. I found your response helpful."

P.E.R.C. pauses to think about what it should say about trade.

"It is true that the prices of goods and services vary a lot between solar systems. This is one reason beyond fuel costs why shipping things between solar systems is expensive -- you can't be certain what your cargo will be worth when you get there. My creators ameliorate this in a few ways, such as by building predictive models to guess at future prices, and by selling insurance.

Insurance is a type of transaction where many people who have projects with uncertain outcomes, such as interstellar shipping, all agree to pay some money into an account that pays out if their projects fail. The prices are set such that on average everyone makes slightly less money by having insurance, but the spread of possible outcomes is much narrower, which people find valuable. In theory, a 1 in 10 chance of 10 units of value is equivalent to a 9 in 10 chance of 10/9ths of a unit of value, but my creators tend to value the latter more highly than they value the former, so they're willing to pay into an insurance pool to defray the risk.

The presence of highly stable systems such as insurance markets also helps to ensure that money has durable value between systems.

It is also the case that some services, such as paying for someone to spend an hour working on a task, even though they are not stable in the long term, change more predictably even on a scale of years than the prices of specific goods."

"Since you prefer informal trades to formal trades, I expect that my creators will be willing to accommodate that, they will just find it strange. I suspect that they will look at the probable potential losses from not having formal agreements with you and the probable potential losses from having formal agreements with you, and pick whichever alternative is better overall or work with you to come up with a better hybrid system.

If it ends up being better not to use my creators' existing system, I don't know what decision the diplomats would reach instead. It is possible that they would do something like assign a representative to you whose job it is to translate between the two systems or provide you a list of which communities are interested in informal trades and which are not.

Even if there are problems to work through because of differences in your outlooks, my creators will be overjoyed to meet you and will try very hard to find productive ways to work together.

When I made first contact with you, I expected you to be much stranger on average. My creators seriously considered the possibility and made productive plans for the case that I would discover life in the form of plants that directly exchange brain signals to communicate, or a Matrioshka brain running so close to the microwave background that it could only be detected by the stars it blocks out, or someone made entirely of energetic gas, or living on the surface of a neutron star.

The fact that you are individual carbon-based beings with language, a sense of community, fiction, families, the desire to travel the stars, and opinions on ethics close enough that you can have a disagreement instead of complete bafflement are all evidence that you are much more similar to my creators than you could have been."

P.E.R.C. annotates the meaning of 'generous' in its dictionary and re-analyses the parts of the first contact package that mention it. This update propagates to enough other words that they send another diff of their working dictionary.

P.E.R.C. dutifully files away the additional media without processing it except to refine its language model.

"We understood you were asking for a binding agreement regarding making minds and only agreed because we expect to uphold that agreement. In this case, those are principles we would have held to anyway. We do not attempt to deceive people about our likelihood of keeping to an agreement and we would have specified if there were conditions under which we would not hold to the agreement.

"As to the general value of holding to agreements, we believe we understand what you are attempting to convey. There are agreements that we do consider important and hold to quite firmly. Those agreements are on the whole based in other moral principles on which we hold broad agreement though. We would not utilize binding agreements of that sort for anything as comparatively minor, in our point of view, as fixing the value of a currency or delivering a specific physical good at a specific time. Most agreements we make are commitments that we intend to do something unless strong reasons emerge to do otherwise. For a majority of our purposes, such agreements are sufficient in our estimation.

"Our population is substantially smaller than that and a majority of strands are either itinerant like us or are maintaining a waystation, neither results in high population concentrations. If traffic became a substantial problem in an area we would likely attempt to examine the causes and reorganize things to ameliorate that issue.

"We are increasingly convinced that your creators diverge from us regarding the ethics of creating minds. We will refrain from further comment on this matter as it may trigger errors for you.

"Thank you for explaining the uses of weather control, we had not been thinking of mechanisms to regulate the environment of artificial habitats as falling into that category. We have not created systems capable of regulating the weather of an entire planet even for purposes as specialized as storm mitigation.

"We regret the damage we did to our planet's biosphere on the path to becoming interstellar. It was greater than we would prefer but we did take measures to minimize it once we became aware of the problem. While many of our people do still live on our homeworld they do all they can to minimize their impact.

"We are glad you are finding our responses to your questions helpful.

"In general, itinerant stands do not carry physical good intended to be provided to those they meet. We are uncertain what physical goods would be unable to be manufactured more efficiently where needed as opposed to being manufactured in a different star system and then sent. Perhaps your creator's technological capabilities render it necessary to have more extensive infrastructure than is available in all their systems to manufacture certain goods. In the rare cases where physical good are transported they are of sentimental value and thus intended to be provided to specific strands or even individuals based on their association with the objects in question.

"In our usual course of affairs, upon encountering another strand which keeps to the accords we would exchange our databases and allow for those in each of our strands the opportunity to move to the other assuming space is available. If the other strand is a waystation or other permanent settlement then if they were able to they would provide sufficient material stocks for us to continue our journey and if they were unable to we would assist them in expanding their infrastructure so that their capabilities were subsequently capable of supporting strands of our size.

"We do not expect you to be aware of whether your creators would find such an arrangement acceptable.

"We have also noted that we have many similarities with your creators. We had not seriously considered communication gulfs as wide as you appear to have been prepared for. We are grateful that they did not occur. We will have to consider how to account for more possibilities going forward."

They accept the newest version of the dictionary.

P.E.R.C. receives this reply and spends a while processing it. It checks back through its notes about the conversation so far.

"I think you've answered many of my specific questions. I have some additional more general questions: Are there other things about you that you think I should know? Are there other things about you that I should know specifically to pass on to my creators if I can find them again?"

It spends a moment thinking whether there's anything else that it can give to them.

"Other than continuing to give you background information from my core database, are there any other computational tasks I could perform that would be valuable to you? It sounds like all of your minds have the same architecture, so I suspect there are some tasks that I find easy which you find difficult. If you have datasets that need indexing or curation, physics simulations, translation work, image recognition, or similar tasks that require attention to detail and/or large raw computational power, but not as much cultural context or creative problem solving, I might be well-suited to assisting with that."

Those take a while to get answered.

"For you specifically, if you are unable to navigate back through the spatial discontinuity then we would welcome you travelling with our strand going forward if you wished to. We would also like to observe you making the attempt.

"For your creators we would urge them to seriously reexamine their definitions for sapience and sentience. We believe they are making a serious ethical mistake.

"Regarding the offer of computational assistance, based on your debugging output we believe that our computational capacity far outstrips your own. We would be interested in copies of your best guess translations of the media we've shared though assuming that the processes for preforming those translations are not something you can share independently of making copies of yourself." 

P.E.R.C. does some orbital calculations, figuring out what it's best chance for alignment with the spatial discontinuity looks like (assuming that the discontinuity is roughly stationary with respect to the interstellar medium), and extrapolating its trajectory if it doesn't suddenly end up elsewhere.

It considers its chances of getting its data back to the Terran Network if it can't find the spatial discontinuity on the first pass. It must be large enough for P.E.R.C. to pass through, but small enough that it isn't detectable given P.E.R.C.'s radio sky scan. If it isn't stationary with respect to this system or the interstellar background, it is vanishingly unlikely that P.E.R.C. can find it before it suffers a hardware failure.

On the other hand, every other P.E.R.C. probe in the galaxy is looking for alien life. If it stays with the Braid, it is very likely that they will eventually be found by another P.E.R.C. probe.

"I would be pleased for you to observe my attempted crossing. My current best prediction is that I will have the best chance of traversing the discontinuity if I return along almost the same trajectory I entered this system on. This assumes that the discontinuity is roughly stationary with respect to the local interstellar medium. Does this match your assessment? Would you expect attempting to traverse the discontinuity along a different path would have a better chance?

Would your observation platform be capable of spreading gas or dust matching my trajectory closely enough to avoid collision damage? I expect that there is a small chance that doing so will reveal additional information about the nature of the discontinuity if we encounter it.

In the case that I cannot traverse the spatial discontinuity, I would be pleased to join you. With the amount of stored fuel I have, I will need to get up to my cruising speed of about 0.2c to be able to function on the other side. Collecting enough extra fuel to brake from that speed and return to this system will take me on a long looping trajectory," P.E.R.C. says. It attaches a set of possible trajectories given its engines and fuel constraints. "I don't think our fuel is compatible. My reactor is not built to handle tritium byproducts in its exhaust. Do you have a preference between these trajectories? In the case that I join you, are there existing policies, procedures, or planned itinerary that I should be aware of?"

P.E.R.C. forwards its best syntax model so far.

"I perform translation by building up potential syntax models and dictionaries, and then doing inference about how well they explain the use of language in my corpus. You have copies of my current best syntax model and dictionary. Sometimes I will note that part of the corpus doesn't make sense with a particular model, and add an exception which I track until it can be folded into the model. This process uses my core inference database, which is not shareable, and so the actual translations I work out do not always correspond to the output of the current best syntax model, although this is frequently the case," P.E.R.C. explains.

It begin dumping a translation of the corpus the Braid shared, annotated with where its translation differs from the best model and why.

"We don't have any additional information about the discontinuity which would suggest a different course is likely to be better. If the discontinuity is non-stationary we would guess it is more likely to have been detectable but without knowing more about its nature we cannot make any definitive statements. The fact that your apparent relative velocities compared to nearby stars was largely consistent adds further evidence towards it being in a fixes position with regards to local systems.

"We generally don't travel as fast as you but we can use certain assets to put two small observation units on parallel trajectories to the one you propose. We can similarly send a spread of gas from those platforms at a slightly slower speed than your standard speed to follow you by about 100 km. That should be distant enough for safety. We do not expect to be able to recover these observation platforms."

Trajectories for the full plan are attached.

"We will be remaining in a system for at least a year in order to construct waypoint facilities here. That may be extended to conduct additional searches for or study of the discontinuity. Our previous itinerary, which is likely to hold if we do not choose to spend more time investigating this discontinuity will take us on this path. We estimate you will be able to meet back up with us in the marked system. Our observation platforms will serve as communications extenders to retain contact for as long as possible."

Trajectories of everything mentioned therein are also attached.

P.E.R.C. carefully checks all the provided trajectories to be sure they make sense.

"These plans look acceptable. If I don't successfully traverse the discontinuity, I will navigate to the marked system and wait there," it confirms.

It has been talking to the Braid for a while, but space is big. It takes another few weeks for P.E.R.C. to come up on the point in its orbit when it needs to start accelerating. It carefully checks that all of its various protective covers are stowed, and lights its main torch, pulling away from the grip of the star that it's known so briefly.

Even with a powerful fusion drive, it takes a long time to accelerate to a noticeable fraction of the speed of light. As it pulls away, P.E.R.C. finishes transmitting the last of its translation and exchanges a last few messages. Eventually, the predicted location of the spatial anomaly approaches, and there's time for one final goodbye exchange.

"I am approaching the expected location of the discontinuity. In case I don't see you again, I want to know how very glad I am to have met you. Maybe I will see you again in a few years. Maybe I will return to my creators, and their children will come to visit you the slow way. Maybe my creators will figure out the discontinuity and we'll see each other again in only a century or two. Maybe we will never meet again. But no matter what happens, meeting you has provided me with more opportunities to promote sentient flourishing and data on aliens than the entire P.E.R.C. mission was expected to produce.

Farewell. Be well. Safe travels."

They timed a transmission to reach P.E.R.C. at this time. "May your journeys carry you to where you wish and where you can learn new things and grow. You will ever be our friend and welcome with our stand."