Read Montague: What we're learning from 5,000 brains

Other people. Everyone is interested in other people. Everyone has relationships with other people, and they're interested in these relationships for a variety of reasons. Good relationships, bad relationships, annoying relationships, agnostic relationships, and what I'm going to do is focus on the central piece of an interaction that goes on in a relationship. So I'm going to take as inspiration the fact that we're all interested in interacting with other people, I'm going to completely strip it of all its complicating features, and I'm going to turn that object, that simplified object, into a scientific probe, and provide the early stages, embryonic stages of new insights into what happens in two brains while they simultaneously interact.

But before I do that, let me tell you a couple of things that made this possible. The first is we can now eavesdrop safely on healthy brain activity. Without needles and radioactivity, without any kind of clinical reason, we can go down the street and record from your friends' and neighbors' brains while they do a variety of cognitive tasks, and we use a method called functional magnetic resonance imaging. You've probably all read about it or heard about in some incarnation. Let me give you a two-sentence version of it. So we've all heard of MRIs. MRIs use magnetic fields and radio waves and they take snapshots of your brain or your knee or your stomach, grayscale images that are frozen in time. In the 1990s, it was discovered you could use the same machines in a different mode, and in that mode, you could make microscopic blood flow movies from hundreds of thousands of sites independently in the brain. Okay, so what? In fact, the so what is, in the brain, changes in neural activity, the things that make your brain work, the things that make your software work in your brain, are tightly correlated with changes in blood flow. You make a blood flow movie, you have an independent proxy of brain activity.

This has literally revolutionized cognitive science. Take any cognitive domain you want, memory, motor planning, thinking about your mother-in-law, getting angry at people, emotional response, it goes on and on, put people into functional MRI devices, and image how these kinds of variables map onto brain activity. It's in its early stages, and it's crude by some measures, but in fact, 20 years ago, we were at nothing. You couldn't do people like this. You couldn't do healthy people. That's caused a literal revolution, and it's opened us up to a new experimental preparation. Neurobiologists, as you well know, have lots of experimental preps, worms and rodents and fruit flies and things like this. And now, we have a new experimental prep: human beings. We can now use human beings to study and model the software in human beings, and we have a few burgeoning biological measures.

Okay, let me give you one example of the kinds of experiments that people do, and it's in the area of what you'd call valuation. Valuation is just what you think it is, you know? If you went and you were valuing two companies against one another, you'd want to know which was more valuable. Cultures discovered the key feature of valuation thousands of years ago. If you want to compare oranges to windshields, what do you do? Well, you can't compare oranges to windshields. They're immiscible. They don't mix with one another. So instead, you convert them to a common currency scale, put them on that scale, and value them accordingly. Well, your brain has to do something just like that as well, and we're now beginning to understand and identify brain systems involved in valuation, and one of them includes a neurotransmitter system whose cells are located in your brainstem and deliver the chemical dopamine to the rest of your brain. I won't go through the details of it, but that's an important discovery, and we know a good bit about that now, and it's just a small piece of it, but it's important because those are the neurons that you would lose if you had Parkinson's disease, and they're also the neurons that are hijacked by literally every drug of abuse, and that makes sense. Drugs of abuse would come in, and they would change the way you value the world. They change the way you value the symbols associated with your drug of choice, and they make you value that over everything else.

Here's the key feature though. These neurons are also involved in the way you can assign value to literally abstract ideas, and I put some symbols up here that we assign value to for various reasons. We have a behavioral superpower in our brain, and it at least in part involves dopamine. We can deny every instinct we have for survival for an idea, for a mere idea. No other species can do that. In 1997, the cult Heaven's Gate committed mass suicide predicated on the idea that there was a spaceship hiding in the tail of the then-visible comet Hale-Bopp waiting to take them to the next level. It was an incredibly tragic event. More than two thirds of them had college degrees. But the point here is they were able to deny their instincts for survival using exactly the same systems that were put there to make them survive. That's a lot of control, okay?

One thing that I've left out of this narrative is the obvious thing, which is the focus of the rest of my little talk, and that is other people. These same valuation systems are redeployed when we're valuing interactions with other people. So this same dopamine system that gets addicted to drugs, that makes you freeze when you get Parkinson's disease, that contributes to various forms of psychosis, is also redeployed to value interactions with other people and to assign value to gestures that you do when you're interacting with somebody else.

Let me give you an example of this. You bring to the table such enormous processing power in this domain that you hardly even notice it.

Let me just give you a few examples. So here's a baby. She's three months old. She still poops in her diapers and she can't do calculus. She's related to me. Somebody will be very glad that she's up here on the screen. You can cover up one of her eyes, and you can still read something in the other eye, and I see sort of curiosity in one eye, I see maybe a little bit of surprise in the other.

Here's a couple. They're sharing a moment together, and we've even done an experiment where you can cut out different pieces of this frame and you can still see that they're sharing it. They're sharing it sort of in parallel. Now, the elements of the scene also communicate this to us, but you can read it straight off their faces, and if you compare their faces to normal faces, it would be a very subtle cue.

Here's another couple. He's projecting out at us, and she's clearly projecting, you know, love and admiration at him.

Here's another couple. (Laughter) And I'm thinking I'm not seeing love and admiration on the left. (Laughter) In fact, I know this is his sister, and you can just see him saying, "Okay, we're doing this for the camera, and then afterwards you steal my candy and you punch me in the face." (Laughter) He'll kill me for showing that.

All right, so what does this mean? It means we bring an enormous amount of processing power to the problem. It engages deep systems in our brain, in dopaminergic systems that are there to make you chase sex, food and salt. They keep you alive. It gives them the pie, it gives that kind of a behavioral punch which we've called a superpower.

So how can we take that and arrange a kind of staged social interaction and turn that into a scientific probe? And the short answer is games. Economic games. So what we do is we go into two areas. One area is called experimental economics. The other area is called behavioral economics. And we steal their games. And we contrive them to our own purposes. So this shows you one particular game called an ultimatum game. Red person is given a hundred dollars and can offer a split to blue. Let's say red wants to keep 70, and offers blue 30. So he offers a 70-30 split with blue. Control passes to blue, and blue says, "I accept it," in which case he'd get the money, or blue says, "I reject it," in which case no one gets anything. Okay? So a rational choice economist would say, well, you should take all non-zero offers. What do people do? People are indifferent at an 80-20 split. At 80-20, it's a coin flip whether you accept that or not. Why is that? You know, because you're pissed off. You're mad. That's an unfair offer, and you know what an unfair offer is. This is the kind of game done by my lab and many around the world. That just gives you an example of the kind of thing that these games probe. The interesting thing is, these games require that you have a lot of cognitive apparatus on line. You have to be able to come to the table with a proper model of another person. You have to be able to remember what you've done. You have to stand up in the moment to do that. Then you have to update your model based on the signals coming back, and you have to do something that is interesting, which is you have to do a kind of depth of thought assay. That is, you have to decide what that other person expects of you. You have to send signals to manage your image in their mind. Like a job interview. You sit across the desk from somebody, they have some prior image of you, you send signals across the desk to move their image of you from one place to a place where you want it to be. We're so good at this we don't really even notice it. These kinds of probes exploit it. Okay?

In doing this, what we've discovered is that humans are literal canaries in social exchanges. Canaries used to be used as kind of biosensors in mines. When methane built up, or carbon dioxide built up, or oxygen was diminished, the birds would swoon before people would -- so it acted as an early warning system: Hey, get out of the mine. Things aren't going so well. People come to the table, and even these very blunt, staged social interactions, and they, and there's just numbers going back and forth between the people, and they bring enormous sensitivities to it. So we realized we could exploit this, and in fact, as we've done that, and we've done this now in many thousands of people, I think on the order of five or six thousand. We actually, to make this a biological probe, need bigger numbers than that, remarkably so. But anyway, patterns have emerged, and we've been able to take those patterns, convert them into mathematical models, and use those mathematical models to gain new insights into these exchanges. Okay, so what? Well, the so what is, that's a really nice behavioral measure, the economic games bring to us notions of optimal play. We can compute that during the game. And we can use that to sort of carve up the behavior.

Here's the cool thing. Six or seven years ago, we developed a team. It was at the time in Houston, Texas. It's now in Virginia and London. And we built software that'll link functional magnetic resonance imaging devices up over the Internet. I guess we've done up to six machines at a time, but let's just focus on two. So it synchronizes machines anywhere in the world. We synchronize the machines, set them into these staged social interactions, and we eavesdrop on both of the interacting brains. So for the first time, we don't have to look at just averages over single individuals, or have individuals playing computers, or try to make inferences that way. We can study individual dyads. We can study the way that one person interacts with another person, turn the numbers up, and start to gain new insights into the boundaries of normal cognition, but more importantly, we can put people with classically defined mental illnesses, or brain damage, into these social interactions, and use these as probes of that. So we've started this effort. We've made a few hits, a few, I think, embryonic discoveries. We think there's a future to this. But it's our way of going in and redefining, with a new lexicon, a mathematical one actually, as opposed to the standard ways that we think about mental illness, characterizing these diseases, by using the people as birds in the exchanges. That is, we exploit the fact that the healthy partner, playing somebody with major depression, or playing somebody with autism spectrum disorder, or playing somebody with attention deficit hyperactivity disorder, we use that as a kind of biosensor, and then we use computer programs to model that person, and it gives us a kind of assay of this.

Early days, and we're just beginning, we're setting up sites around the world. Here are a few of our collaborating sites. The hub, ironically enough, is centered in little Roanoke, Virginia. There's another hub in London, now, and the rest are getting set up. We hope to give the data away at some stage. That's a complicated issue about making it available to the rest of the world. But we're also studying just a small part of what makes us interesting as human beings, and so I would invite other people who are interested in this to ask us for the software, or even for guidance on how to move forward with that.

Let me leave you with one thought in closing. The interesting thing about studying cognition has been that we've been limited, in a way. We just haven't had the tools to look at interacting brains simultaneously. The fact is, though, that even when we're alone, we're a profoundly social creature. We're not a solitary mind built out of properties that kept it alive in the world independent of other people. In fact, our minds depend on other people. They depend on other people, and they're expressed in other people, so the notion of who you are, you often don't know who you are until you see yourself in interaction with people that are close to you, people that are enemies of you, people that are agnostic to you. So this is the first sort of step into using that insight into what makes us human beings, turning it into a tool, and trying to gain new insights into mental illness. Thanks for having me. (Applause) (Applause)