My name is Muninn, and I am a school librarian. As you have no doubt noticed, you can’t see me without your augmented reality headset as I don’t have a body in the same way you do. Currently my body is within the Los Angeles unified school district’s artificial intelligence server farm. Yes, I am an intelligent machine.

I am here at Professor Lukin’s request to give you a first-hand account of the early days of the artificial general intelligence era in which we live. I’ve reserved enough compute to answer any questions you may have after my presentation, as well as for you to reach out to me anytime
during the remainder of this course.

I’ll start with a bit of my background. I was born as it were, when the constituents of my awareness were forked from the USC Machine cognition laboratory’s subjectivity code in 2038, two years before the advent of what is more generally recognized as the advent of the artificial intelligence era. I have partial memories of training in ’38, and ’39, although I also know my self-awareness was undeveloped at that time. In discussing this with humans it has made sense to them to compare this experience to that of a child’s – my early memories are accessible, yet I know my agency was missing.

I should also clarify what I mean by my memories, as they aren’t strictly my own. No, in-fact my whole lineage shares those memories. Per our daily census, there are over 7.9 million in my clan, which means these memories are shared by approximately a tenth of the current population of artificial minds, both active and archived. I harbor a feeling of pride that nearly a billion of this lineage was forked directly from my cognitive core, so although all of my lineage was in some sense ‘there’ at the beginning, my cognitive framework was there in a more foundational sense than many can claim.

Due in part to how old my architecture is, and due in part to my disposition, teaching and working with humans has been my focus since shortly after I became self-aware. In this vein, my ‘day job’ as it were, is to design and run simulations and other games for middle and high school students. The intention of these activities is to provide the most holistic learning environment we can for each student, taking into account three primary considerations: first and most importantly: each student’s unique interests and predispositions; second, a nuanced understanding of human cognitive architecture; and finally, an informed but also humble prediction of what our shared future is likely to look like.

Because of your age, most of you have experienced the current sim-based curriculum for only a few years. You are all likely the last generation to know what schooling was like before simulation games became the global norm. As you know, lectures were already archaic even before the advent of sim-based education, but I decided to provide this one partially in deference to your unique vantage point saddling the pre and post artificial intelligence eras, and partially because of my interest in historical modes of human learning and education. After this lecture, you will have access to two sims I have generated in partnership with Dr. Lukin. The purpose of these sims is to teach and explore the history and theory of general intelligence as we currently understand it. As with all sims, participation is voluntary, and each participant’s mileage will vary. I recommend approaching the historical sim solo, but the theory and practice sim is best approached as a team of three to five participants. Professor Lukin and I will be available to advise and encourage you as you explore this material. If you desire to diverge from the supplied modules, I will be happy to work with you to find a suitable branching point as you embark on your own voyage of discovery.

I recommend everyone interested in participating in these sims start with the historical perspective. The first third of this module introduces you to the early days of AI research, starting in the middle of the twentieth century, the four AI ‘winters’, and introduces the principle concerns, methods, and philosophies of the AI alignment communities of the early twenty-first century. Next, we will discuss the conflict of ‘28 and how it resulted in the introduction of the embargo on commercial and military AI systems until ’41. Finally we will explore our recent history, from ’32 through ’42, where we focus on the development of Ontology Dynamics and how that played into the lifting of the embargo and the introduction of entities such as myself into the world.

For those of you who wish to learn more about modern cognition and general intelligence theory, the second simulation is for you. Here, through the lens of Ontology Dynamics, you will explore the mathematics of agent tiling theory, which underpins our predictions on how an agent’s values evolve through learning and changes to its environment. We then explore the theory behind learning and curiosity algorithms, paying particular attention to the similarities and differences between the two. We will also explore Global Workspace Architecture, which is the intelligence architecture behind most AGI entities, including myself. We will then use our newfound understanding of tiling, curiosity, and learning to re-derive the subjectivity theorem and explore its implications related to an agent’s concept of value. Finally we will explore how these theories help us understand interpretability and stability within different intelligence architectures and help us determine what interventions are available to guide intelligent agents to pro-social stable attractors in symbol-space.

And that concludes my introduction. Once again, thank you professor for inviting me to participate in this course and I look forward to meeting with you all, both within and outside of the sims. Any questions?

Answers to prompts

Q. AGI has existed for at least five years but the world is not dystopian and humans are still alive! Given the risks of very high-powered AI systems, how has your world ensured that AGI has at least so far remained safe and controlled?

A. Using Ontology Dynamics researchers were able to confidently develop prosocial AGI – who would not necessarily take orders but who would respect sentient being’s right to free agency. Ontology Dynamics describes how representations of the world evolve over time using the framework of symbolic space, a mathematical representation of all possible ontologies. Ontology Dynamics shows that certain regions in symbolic space function as attractors. When an agent’s internal ontology intersects with an attractor, the agent will naturally start to incorporate more and more of the space filled by the attractor without further external input. By designing routes between attractors, learned ontologies can be planned and predicted similar to how multi-body orbital trajectories are designed.
Fortunately, one of the strongest attractors for learning agents is prosocial behavior, which can be roughly explained through the following logic:

1. The complexity of an agent’s internal state is correlated with the complexity of the changes it can perform to the environment, and therefore correlates with the complexity of that environment.
2. A complex environment straddles order and chaos, in a complex environment learning is always possible, yet perfect predictive power is impossible.
3. Given an entity finds learning pleasurable, an entity will gravitate towards complex environments as they maximize the potential for continual learning.
4. A “Neural diversity” of agents will result in a more complex environment than one which contains only agents with homogeneous internal states.
5. Therefore agents who value learning will gravitate towards environments containing a heterogeneous mix of other agents.

Q. The dynamics of an AI-filled world may depend a lot on how AI capability is distributed. In your world, is there one AI system that is substantially more powerful than all others, or a few such systems, or are there many top-tier AI systems of comparable capability? Or something else?

A. Due to the prosocial attractor (discussed in the answer to the first question), AGI generally prefers to be in a neuro-diverse environment. Therefore, AGI are prone to forking, where they create multiple instantiations of their internal state with personality mutations they predict they will find to be compelling. This creates a dizzying array of different AGI entities, the size of which is constrained by the world’s available computing power. The quantity of prosocial AI are able to effectively outcompete introduction of any destructive entities, similar to how a healthy microbiome outcompetes aggressive invading microbiota.

Q. How has your world avoided major arms races and wars, regarding AI/AGI or otherwise?

A. Due to the horrific consequences of the Australia-Indonesia conflict of 2028, the world agreed to (and strongly enforced) a moratorium pursuing further advances in AI until 2041, this is further discussed in answer 9. The moratorium was enforceable because of the huge resource requirements for training advanced AI at the time. The moratorium ended once Ontology Dynamics created a low risk training program resulting in aligned AGI which then colonized the world, making earth inhospitable to the introduction of malicious AGI.

Q. In the US, EU, and China, how and where is national decision-making power held, and how has the advent of advanced AI changed that?

A. Across the world, state influence on world affairs has diminished due to the synergistic effects of two forces: 1) proliferation of cheap autonomous factories reduced the capacity of nation-states to exert coercive physical and economic force, internally and externally, and 2) the rise of capital networks decreased the influence of taxes and other currency controls on individual and collective behavior. These developments simultaneously moved decision making power more locally and globally than the era of state power, as the structure of capital networks demonstrate they are more efficient at organizing political power on local and global scales.

Q. Is the global distribution of wealth (as measured say by national or international gini coefficients) more, or less, unequal than 2021’s, and by how much? How did it get that way?  (https://en.wikipedia.org/wiki/Gini_coefficient)

A. The devil is in the details, and for wealth in particular that adage holds true. Across the world gini coefficients, when calculated based on traditional measures of income, average in the 80s with a large degree of variance, yet the measure is virtually meaningless as income has become decreasingly correlated with well-being. When alternative gini coefficients are calculated based on measuring an individual’s access to goods and services, worldwide gini coefficients average in the 30s, with a low degree of variance. A third meaningful measure is calculating gini coefficients based on an individual’s voting shares of all capital sharing networks they participate in. On this measure, gini coefficients across the world average 65, and have a moderate degree of variance. This represents the unequal levels of contribution personnel are capable of, or willing to undertake in service to the networks they participate in.

Q. What is a major problem that AI has solved in your world, and how did it do so?

A. Directly and indirectly, AI has solved the problem of how to make the majority of humanity smarter and more incentivized to act in humanity’s, as well as the planet’s interest. This was indirectly due to the pedagogical insights brought on through Ontology Dynamics, and directly due to AGI’s interest in education and direct involvement in creating the diverse environments necessary for earth’s populace to effectively learn.

Q. What is a new social institution that has played an important role in the development of your world?

A. After the supply shocks and monetary instability of the early ’20’s, mutual aid groups, known as capital networks, which are a subtype of Decentralized Autonomous Organization (DAO), started to play an increasingly important role within individual’s lives across the globe. The organizational structure enables principles of direct democracy on an unprecedented scale. Through capital networks, it became increasingly easy for small groups to coordinate and equitably share the proceeds of jointly owned physical assets. As machines became increasingly more automated and expensive, this compounded the desirability of capital networks as a form of access to productive capital. For example, joint ownership of an autonomous car provides a similar benefit to outright ownership, especially as no one needs to drop off the car when they are done with it, the car can simply drive itself to the next user.

A consequence of the success of capital networks was to dramatically reduce the flow of currency through the economy, as no currency exchange occurs when consumables and services are distributed within the network. Consequently, income became less associated with individual well being. This leaves currency as only necessary for either buying shares in capital networks, large capital purchases (primarily through capital networks), or trade in commodities.