## Evolving useful delusions to promote cooperation

This joint work with Marcel Montrey and Thomas Shultz combines — to be consistent with the interdisciplinary theme of this symposium — ideas from biology, economics, a little bit of cognitive science, and the approach is through applied mathematics. This post is a transcript of a presentation I gave on March 27th and covers part of my presentation today at Swarmfest.

## Ethnocentrism, religion, and austerity: a science poster for the humanities

Artem Kaznatcheev and I presented a poster on May 4th at the University of British Columbia to a highly interdisciplinary conference on religion. The conference acronym is CERC, which translates as Cultural Evolution of Religion Research Consortium. Most of the 60-some attendees are religion scholars and social scientists from North American and European universities. Many are also participants in a large partnership grant from the Social Sciences and Humanities Research Council of Canada (SSHRC), spearheaded by Ted Slingerland, an East Asian scholar at UBC. Some preliminary conversations with attendees indicated considerable apprehension about how researchers from the humanities and sciences would get on. Many of us are familiar with collaborative difficulties even in our own narrow domains. Skepticism was fairly common.

As far as I know, our poster was the only computer simulation presented at the meeting. We titled it Agent-based modeling of the evolution of “religion”, with scare quotes around religion because of the superficial and off-hand way we treated it. Because we know from experience that simulations can be a tough sell even at a scientific psychology conference, we were curious about whether and how this poster would fly in this broader meeting.

## PSYC 532 (2012): Evolutionary Game Theory and Cognition

Past Thursday was my fourth time guest lecturing for Tom’s Cognitive Science course. I owe Tom and the students a big thank you. I had a great time presenting, and hope I was able to share some of my enthusiasm for evolutionary games.

I modified the presentation (pdf slides) by combining lecture and discussion. Before the lecture the students read my evolving cooperation post and watched Robert Wright’s “Evolution of compassion”. Based on this, they prepared discussion points and answers to:

1. What is kin selection? What is green-beard effect or ethnocentrism? How do you think kin selection could be related to the green-beard effect or ethnocentrism?
2. What does Wright say compassion is from a biological point of view? Do you think this is a reasonable definition?
3. Can a rational agent be compassionate? Is understanding the indirect benefits (to yourself or your genes) that your actions produce essential for compassion?
4. What simplifying assumptions does evolutionary game theory make when modeling agents? Are these assumptions reasonable?
5. Can compassion or cooperation evolve in an inviscid environment? What about a spatially structured one?
6. What is reciprocal altruism, direct reciprocity and indirect reciprocity?
7. What is a zero-sum game? Does a non-zero-sum relationship guarantee that compassion will emerge?
8. Is the Prisoner’s dilemma a zero-sum game? Can you have a competitive environment that is non-zero sum?

During the lecture, we would pause to discuss these questions. As always, the class was enthusiastic and shared many unique viewpoints on the topics. Unfortunately, I did not sufficiently reduce the material from last year and with the discussion we ran out of time. This means that we did not get to the ethnocentrism section of the slides. For students that want to understand that section, I recommend: Evolution of ethnocentrism in the Hammond and Axelrod model.

To the students: thank you for being a great audience and I encourage you to continue discussing the questions above in the comments of this post.

## Introduction to evolving cooperation

Since 2009, I’ve had a yearly routine of guest lecturing for Tom’s Cognitive Science course. The way I’ve structured the class was by assigning videos to watch before the lecture so that I could build on them. Last year, I started posting the video ahead of time on the blog: my 2009 TEDxMcGill talk, Robert Wright’s evolution of compassion, and Howard Rheingold’s new power of collaboration. However, instead of just presenting a link with very little commenatry, this time I decided to write a transcript with my talk that I seeded with references and links for the curious. The text is not an exact recreation of the words, but a pretty close fit that is meant to serve as a gentle introduction to the evolution of cooperation.

Earlier today, we heard about the social evolution of language and to a certain extent we heard about the emergence and evolution of zero. We even heard about our current economic affairs and such. I am going to talk about all of these things and, in particular, continue the evolutionary theme and talk about the evolution of cooperation in society and elsewhere.

We’ve all come across ideas of the greater good, altruism, cooperation or the sacrifice of an individual for the good of others. In biology, we have an analogous concept where we look at the willingness of certain individuals to give up some of their reproductive potential to increase the reproductive potential of others. This paradoxical concept in the social sciences is grappled with by philosophers, sociologists, and political scientists. In the biological context, it is obviously an important question to biologists.

Now, the question really becomes as to how and why does this cooperation emerge? First, we are going to look at this from the biological point of view, connect it to the social sciences, and then to everything else.

Currently, biology is really shaped by Darwin, Wallace and their theory of evolution by natural selection. It is a unifying theme and tie of modern biology. The interesting feature of biology is that it is an explicitly competitive framework: organisms compete against other organisms for their reproduction. Our question becomes: how does cooperation emerge in such a competitive environment?

We know this cooperation does emerge because it is essential for all the complexity we see. It is essential for single cells to come together into multi-cellular organisms, for the emergence of ant colonies, and even human society. We want to study this and try to answer these questions. But how do you create a competitive environment in a mathematical framework? We borrow from game theory the idea of Prisoner’s dilemma, or in my case I prefer the Knitter’s dilemma. This is one of many possible models of a competitive environment, and the most used in the literature.

In the Knitter’s dilemma there are two players. One of them is Alice. Alice produces yarn, but she doesn’t have any needles, and she wants to sew a sweater. In the society that she lives, knitting sweaters is frowned upon, so she can’t go ask for needles publicly. Bob, on the other hand, produces needles but not yarn. He also wants to sew a sweater. So they decide: “okay, lets go out into the woods late at night, bring briefcases with our respected goods and trade”.

Alice has a dilemma: should she include yarn in her briefcase (indicated by the green briefcase in the figure below)? Or should she not (signified by the red)? If Bob includes needles (first column), and Alice includes yarn then she gets the benefit $b$ of going home and knitting a sweater, but she does pay a small cost $c$ for giving away some of her yarn. Alternatively, if Bob brings needles, but she’s tricky and doesn’t bring her yarn then she gets all the benefit of going home and making a sweater without paying even the marginal cost of giving away some of her yarn. If Bob brings an empty briefcase (second column), and Alice brings yarn as she said she would then Alice pays a small cost in giving some of her yarn away without benefit of being able to make a sweater. Alternatively, if she also brings an empty briefcase then they just met in the middle of the night, traded empty briefcases, and everybody goes back with the no payoff.

It seems that no matter what Bob does, it is better for Alice to bring an empty briefcase, what we call defection, than to cooperate by bringing a full briefcase. This sets up the basic idea of a competitive environment. The rational strategy, or the Nash equilibrium, for this game is for both individuals to defect and bring empty briefcases. However, from outside the game we can see that if they both do what they said they would and cooperate then they are both better of. That is captured by the Pareto optimum in green.

Of course, as mentioned earlier by Andy, we cannot always expect people to be rational and make all these decisions based on reasoning. Evolutionary game theory comes from the perspective of modeling Alice and Bob as simple agents that have a trait that is passed down to their offspring. This is shown below by green circles for players that cooperate and red circles for ones that don’t. In the standard model, we will pair them off randomly and they will play the game. So a green and a green is two cooperators; they both went home and made a sweater. Two reds both went empty handed. After interaction we disseminate them through the population and let them reproduce according to how the game affected their potential. Higher for people that received a large benefit, and lower chance to reproduce to people who only paid costs. We cycle this for a while, and what we observe is more and more red emerging. All the green cooperation starts to go away. This captures the basic intuition that a competitive environment breeds defection.

Of course, you and I can think of some ways to overcome this dilemma. Evolutionary game theorists have also been there and thought of it (Nowak, 2006). They thought of three models of how to avoid it. The first is Hamilton’s (1964) kin selection: Bob’s actually your uncle, and you’re willing to work with him. You’ll bring the yarn as you said you would. Alternatively, you’ve encountered Bob many times before and he has always included needles in his briefcase. You are much more willing to work with him. This is Trivers’ (1971) direct reciprocity, and you’ll include your yarn. Finally, indirect reciprocity (Nowak & Sigmund, 1998): you’ve heard that Bob is an honest man that always brings needles as he says he will. So you are much more likely to cooperate with him.

All these things seem pretty simple to us, but if we’re an amoeba floating around in some soup (and microbes do play games; Lenski & Velicer 2001) then it’s not quiet as obvious that we can do any of these things. Recognizing kin, remembering past interactions, or social constructs like reputation become very difficult. Hence, I look at the more primitive methods such as spatial/network reciprocity or viscosity.

Earlier, Paul mentioned that if we have a turbulent environment it becomes very hard for us to live. Hence the idea that we introduce some structure into our environment. We populate all our agents inside a small grid where they can interact with their neighbors and reproduce into neighboring squares.

Alternatively, we can borrow an idea from the selfish gene approach to evolution called the green-beard effect. This was introduced by Hamilton (1964) & Dawkins’ Selfish Gene. This is a gene that produces three phenotypical effects: (1) it produces an arbitrary marker which we call the beard (or in our case circles and squares), (2) it allows you to recognize this trait in others, not their strategy just the trait/beard, and (3) it allows you to change your strategy depending on what trait/beard you observe. As before, you can cooperate or defect with other circles, or if you meet a square then you can also chose to cooperate or defect. You have four possible strategies that are drawn in the figure below. In human culture, cooperating with those that are like you (i.e. other circles) and defecting against those that are squares is the idea of ethnocentrism. Here we bring back the social context a little bit by looking at this as a simple model of human evolution, too.

We can combine the two models, by looking at little circles and squares of different colors inside a grid, and seeing how the population will evolve with time. The results we observe are that we do see cooperation emerge, but sadly it is an ethnocentric sort of cooperation. We can see it from the below graph where the y-axis is proportion of cooperative interactions: the higher up you are in the graph, the more cooperation is happening, so the better it is. In the blue model we have agents that can distinguish between circles and squares living inside a spatial lattice. In the green we see a model with spatial structure, but no cognitive ability to adjust based on tags. In the red and the yellow you can see models where there is no spatial structure, or there is no ability to recognize people based on if they are a circle or a square. In these restricted models cooperation does not consistently emerge. Although in the tags with no space model in yellow there is occasional bifurcation of cooperation highlighted by the black circle and arrow.

Proportion of cooperation versus evolutionary cycle for four different conditions. In blue is the standard H&A model; green preserves local child placement but eliminates tags; yellow has tags but no local child placement; red is both inviscid and tag-less. The lines are from averaging 30 simulations for each condition, and thickness represents standard error. Figure appeared in Kaznatcheev & Shultz (2011).

This gives us a suggestion of how evolution could have shaped the way we are today, and how evolution could have shaped the common trend of ethnocentrism in humans. The model doesn’t propose ways to overcome ethnocentrism, but one thing it does is at least create cooperation among scientists who use it. In particular, the number of different fields (represented in one of my favorite xkcd comics, below) that use these sort of models.

Sociologists and political scientists use these models for peace building and conflict resolution (eg. Hammond & Axelrod, 2006). In this case cooperation would be working towards peace, and defection could be sending a mortar round into the neighboring village. Psychologists look at games like the Prisoner’s dilemma (or the Knitter’s dilemma in my case) and say “well, humans tend to cooperate in certain settings. Why is that? Can we find an evolutionary backing for that?” In our running example by looking at ethnocentrism (eg. Shultz, Hartshorn, & Kaznathceev, 2009). Biologists look at how the first molecules came together to form life, or how single cells started to form multi-cellular organisms. Even in cancer research (eg. Axelrod, Axelrod, & Pienta, 2006) and the spread of infectious disease such as the swine flu (eg. Read & Keeling, 2003). Even chemists and physicists use this as a model of self-organizing behavior and a toy model of non-linear dynamics (eg. Szabo & Fath, 2007). Of course, it comes back to computer scientists and mathematicians, who use this for studying network structure and distributive computing. The reason all these fields can be unified by the mathematical idea underlying evolution seems kind of strange. The reason this can happen is because of the simple nature of evolution. Evolution can occur in any system where information is copied in a noisy environment. Thus, all these fields can cooperate together in working on finding answers to the emergence and evolution of cooperation. Hopefully, starting with the scientists working together on these questions, we can get people around the world to also cooperate.

### References

Axelrod, R., Axelrod, D. E., & Pienta, K. J. (2006). Evolution of cooperation among tumor cells. Proceedings of the National Academy of Sciences, 103(36), 13474-13479.

Hamilton, W. D. (1964). The Genetical Evolution of Social Behavior. Journal of Theoretical Biology 7 (1): 1–16.

Hammond, R. A., & Axelrod, R. (2006). The evolution of ethnocentrism. Journal of Conflict Resolution, 50(6), 926-936.

Kaznatcheev, A., & Shultz, T.R. (2011). Ethnocentrism Maintains Cooperation, but Keeping One’s Children Close Fuels It. Proceedings of the 33rd Annual Conference of the Cognitive Science Society, 3174-3179

Lenski, R. E., & Velicer, G. J. (2001). Games microbes play. Selection, 1(1), 89-96.

Nowak MA (2006). Five rules for the evolution of cooperation. Science (New York, N.Y.), 314 (5805), 1560-3 PMID: 17158317

Nowak, M. A., & Sigmund, K. (1998). Evolution of indirect reciprocity by image scoring. Nature, 393(6685), 573-577.

Read, J. M., & Keeling, M. J. (2003). Disease evolution on networks: the role of contact structure. Proceedings of the Royal Society of London. Series B: Biological Sciences, 270(1516), 699-708.

Shultz, T. R., Hartshorn, M., & Kaznatcheev, A. (2009). Why is ethnocentrism more common than humanitarianism. In Proceedings of the 31st annual conference of the cognitive science society (pp. 2100-2105).

Szabo, G., & Fath, G. (2007). Evolutionary games on graphs. Physics Reports, 446 (4-6), 97-216

Trivers, R. L. (1971). The evolution of reciprocal altruism. Quarterly review of biology, 35-57.

## Presentation on evolutionary game theory and cognition

Last week Julian sent me an encouraging email:

did you know that you come up first in google for a search “evolutionary theory mcgill university”? You beat all the profs!

The specific link he was talking about was to my slides from the first time I gave a guest lecture for Tom’s Cognitive Science course in 2009. Today, I gave a similar lecture again; my 3rd year in a row giving a guest lecture for PSYC532. The slides are available here.

I am very happy Tom invited me. It is always fun to share my passion for EGT with students, and I like motivating the connections to cognition. As if often the case, some of the questions during the presentation got me thinking. A particular question I enjoyed, was along the lines of:

If humanitarians cooperate with everyone, and ethnocentrics only cooperate with in-group, then how can we have lower levels of cooperation when the world is dominated by humanitarians?

This was in reference to a result I presented in [Kaz10] about the decrease in cooperative interactions as the cognitive costs of ethnocentrism increases. In particular, even though ethnocentrics are replaced by humanitarians in the population, we don’t see an increase in the proportion of cooperative interactions. In fact, it triggers a decrease in the proportion of cooperative interactions.

I started with my usual answer of the humanitarians allowing more selfish agents to survive, but then realized a second important factor. When the ethnocentric agents are a minority, they no longer form giant same-tag clusters, and are thus much more likely to be defecting (since they are meeting agents of other tags) than cooperating. Thus, the sizable minority of ethnocentrics tend to defect and decrease the proportion of cooperation when living among a majority of humanitarians. On the other hand, when the ethnocentrics are in the majority they are in same-tag clumps and thus tend to cooperate. Of course, I should more closely analyze the simulation data to test this story.

Another attentive student caught a mistake on slide 14 (page 19 of pdf). I have left it for consistency, but hopefully other attentive readers will also notice it. Thank you for being sharp and catching an error I’ve had in my slides for 2 or 3 years now!

To all the students that listened and asked great questions: thank you! If you have any more queries please ask them in the comments. To everyone: how often do you get new insights from the questions you receive during your presentations?

### References

[Kaz10] Kaznatcheev, A. (2010) “The cognitive cost of ethnocentrism.” Proceedings of the 32nd annual conference of the cognitive science society. [pdf]