Our mission is to ensure human flourishing in the age of AGI.

We define human flourishing in terms of whether people are able to live meaningful lives.

Our understanding of meaning, values, wisdom and flourishing has been shaped by a decade of research into relevant work in philosophy, sociology and economics, and real-world experimentation.

We are not in the business of prescribing what it means to live a meaningful life. Furthermore, making decisions about what’s meaningful for others prevents them from practicing their agency and developing their values – essential ingredients for a meaningful life. Instead, we want to help build AI systems that empower that agency and self-definition.

We believe there is a connection between meaning, values and wisdom, which we elaborate on below. Our understanding of this draws from philosophers like Charles Taylor, Ruth Chang and David Velleman, and economists like Amartya Sen.

The word “values” can mean many things, but it is often used to mean a consideration as we live our lives and make choices that demarcates the meaningful choices from the merely logistical or incidental choices. A meaningful choice is one that expresses our values. Another way to say this is that some of our choices are movements towards a way of living that is intrinsically meaningful to us. We call this kind of value a “source of meaning”. We use this terminology to mark out this kind of “value” from other things commonly referred to as values (such as norms and ideological commitments), defined as such:

• A norm is an idea about how to live that’s dictated by a social environment. For example, there are norms around professionalism, politeness and gender.

• An ideological commitment is an idea about how to live you want to convince others of, like for example being an environmentalist or into polyamory.

• An internalized norm is an idea about how to live that was dictated to you by a previous social environment, and which you have since internalized, such as being a “responsible older sister” or a “tough street kid”.

Norms are not bad – they keep social groups together. Ideally, however, norms around us should be in service of our “sources of meaning”. When this is not the case, we feel oppressed by our surroundings, forced to comply with a way of life that doesn’t lead to our own flourishing.

Our conception of “Sources of Meaning” is also different from goals and preferences.

A goal is something you want to accomplish, something you would rather be done with already if you could, like for example passing a test, or becoming a homeowner. Sources of meaning, on the other hand, are intrinsic motivations we want to be present for, like learning about a field you’re interested in, or creating a stable environment for you and your family to grow in.

Preferences are evaluations with no normative value. Your preference for a certain color in jeans, for example, says little about who you aspire to be. Your sources of meaning are instead a “preference” for a way of living with normative value. For example, aspiring to a certain type of authenticity.

A preference can be underpinned by a source of meaning – for example, your preference towards blue jeans might be an expression of living with a certain attention to aesthetics. The preference could also be underpinned by an addiction, a desire to comply with a norm, or another motivation.

We have spent years developing robust methods for extracting sources of meaning from people and separating them from norms, ideological commitments, preferences and goals. For more on this methodology, see the work done by the school for social design.

Things commonly thought of as “bad values” – nazism, sadism, consumerism – are not sources of meaning but ideological commitments. In our processes, these data objects are filtered out in favor of underlying sources of meaning, if they exist.

In the case where these containers do contain some meaningful way of living, the source of meaning tends to be quite different from the container. For instance, people may experience a sense of meaningful solidarity, pushing a cause shunned by the masses, in the case of nazism. This sense of solidarity could be equally or better served by more prosocial containers.

For more on the difference between sources of meaning and containers, see here.

In our research, we capture sources of meaning by finding paths of attention that feel meaningful to attend to. This results in a data object we call a “values card”.

These ”values cards” have been used to guide design at major companies, and were used to find wise values voted on by both Republicans and Democrats on controversial topics like abortion, in a project we did with OpenAI.

You can try out a prototype that elicits a values card from a meaningful experience you have had here.

For more on how we think about attention and values, and for our theory of meaning and social choice, see here.

We think of wisdom as knowing which value (or, source of meaning) to live by given a particular context. For example, when advising someone considering getting an abortion, is it better to listen attentively, or help them identify trusted mentors that could guide them?

We are building on top of a theory of moral learning that has been fleshed out by philosophers like Charles Taylor and Ruth Chang. In short, we think there is a developmental structure to values. We operate by a set of values, and upon realizing our set is incomplete for a novel situation, we “grapple” with the situation, leading to a new value that we can understand as a gain in wisdom, because it fixes a kind of problem with our previous set of values. Living at the edge of our wisdom, by the values we think best constitute a good life, tends to feel meaningful.

For more on this, see our thoughts on moral learning.

We believe wisdom is dispersed amongst us, rather than concentrated in a small elite. People who have lived through unique life experiences, forcing them to grapple with unique moral situations, are likely to have the wisest values for those contexts. And there are many incommensurable wise ways of living, each depending on the subject and context.

Rather than averaging a set of values representative of humanity, we are trying to detect the “frontier of wisdom”. We do this by democratically building a moral graph of values and contexts, linked by wisdom gained in transitioning between values, to collectively determine the wisest values for each situation.

You can read the report on our work on moral graphs and democratic fine-tuning here.

We believe it is possible to build ASW (Artificial Superwisdom) – systems that are wiser than any one person alive, that can find win-win solutions we didn’t know existed. Perhaps it is even necessary to face the challenges of the 21st century.

Our work on moral graphs could theoretically be scaled to superwisdom – systems that develop new values, like we do, through “moral self-play”.

How to do this in practice is an open research question.

We don’t know yet, but it is possible. However, we still need to have a good conception of superwisdom to verify this. It is also likely that, just as current LLMs have a good map of human wisdom from reading massive amounts of text but still need RLHF to act upon it, superintelligence might need direction to act upon its latent wisdom.

We think of alignment more broadly than this. Traditionally, alignment has been defined as aligning AI with instructions (operator intent). We believe that if that succeeds, we are still likely to have a bad future due to values-driven humans being replaced by AI in social systems like markets, democracies and the military. There are many instances of a human inside of such a larger system using their sense of what’s right to refuse orders and incentives, or do something against the self-interest of the system. Replacing these actors with AI would mean replacing them with artificial sociopaths carrying out any directive with precision, entirely neglecting human norms and values.

For more on our thinking about this, see here.

We also think AI offers the opportunity to create a flourishing future in which people live vastly more meaningful lives than now.

You cannot create a good future by only focusing on how to prevent bad ones.

We don’t have a great answer to this question yet. Market dynamics will likely favor systems that optimize for individual preferences.

This is part of the motivation behind our second line of work, helping society adapt to AI. We think there is an alternative to current market structures, where LLMs act as coordinators. Thanks to being able to understand everyone’s desires and wishes on a deep level, such market/LLM hybrids could create a future that rivals preference-based LLMs and markets, due to life inside of this structure being preferable.

Wise AI

Sourcing values, fine-tuning models on our notion of wisdom and values, researching systems that extrapolate and build their own moral graphs.

Helping society adapt to AI

Researching LLM/market hybrids, building a tool that elicits values from people and coordinates based on what they find meaningful, and running events where this tool is used.

Together with OpenAI, we created our first moral graph from a representative sample of 500 Americans through a new democratic process we built.

We are currently fine-tuning a model on this moral graph, comparing it to alternatives like preference-based RLHF and Constitutional AI. For more on this project, see our introduction post and results report.

We have built a community tool that elicits people’s values from meaningful experiences they have had and matches them together with resources and activities based on this.

Joe has researched the question of “what is worth maximizing” for a decade. He co-founded CHT and the school for social design to research and develop the values-elicitation methodology we build upon.

For our notion of values and moral learning, we are building on a lineage of late-20th century philosophers and sociologists, that includes Charles Taylor, David Velleman, Ruth Chang, Martha Nussbaum, and Amartya Sen.

We’re currently looking for brilliant AI researchers, community builders and funders. We’re also looking to build connections within all major LLM labs.

You can reach us at hello@meaningalignment.org.

You can also donate directly to us here.