Diversity and persistence of group tags under replicator dynamics

Everyday I walk to the Stabile Research Building to drink espresso and sit in my cozy — although oversaturated with screens — office. Oh, and to chat about research with great people like Arturo Araujo, David Basanta, Jill Gallaher, Jacob Scott, Robert Vander Velde and other Moffitters. This walk to the office takes about 30 minutes each way, so I spend it listening to podcasts. For the past few weeks, upon recommendation from a friend, I’ve started listing to the archive of the Very Bad Wizards. This is a casual — although oversaturated with rude jokes — conversation between David Pizarro and Tamler Sommers on various aspects of the psychology and philosophy of morality. They aim at an atmosphere of two researchers chatting at the bar; although their conversation is over Skype and drinks. It is similar to the atmosphere that I want to promote here at TheEGG. Except they are funny.

While walking this Wednesday, I listed to episode 39 of Very Bad Wizards. Here the duo opens with a Wilson & Haidt’s TIME quiz meant to quantify to what extent you are liberal or conservative.^[1] They are 63% liberal.^[2]

To do the quiz, you are asked to rate 12 statements (well, 11 and one question about browsers) on a six point Likert scale from strongly disagree to strongly agree. Here are the three that caught my attention:

6. If I heard that a new restaurant in my neighborhood blended the cuisines of two very different cultures, that would make me want to try it.
7. My government should treat lives of its citizens as being much more valuable than lives in other countries.^[3]
10. I wish the world did not have nations or borders and we were all part of one big group.^[4]

Do you strongly agree? Strongly disagree? What was your overall place on the liberal-conservative scale?

Regardless of your answers, the statements probably remind you of an important aspect of your daily experience. The world is divided into a diversity of groups, and they coexist in a tension between their arbitrary, often artificial, nature and the important meaning that they hold to both their own members and others. Often this division is accompanied by ethnocentrism — a favoring of the in-group at the expensive of, or sometimes with direct hostility toward, the out-group — that seems difficult to circumvent through simply expanding our moral in-group. These statements also confront you with the image of what a world without group lines might look like; would it be more cooperative or would it succumb to the egalitarian dilemma?^[5]

As you know, dear reader, here at TheEGG we’ve grappled with some of these questions. Mostly by playing with the Hammond & Axelrod model of ethnocentrism (2006; also see: Hartshorn, Kaznatcheev & Shultz, 2012). Recently, Jansson’s (2015) extension of my early work on the robustness of ethnocentrism (Kaznatcheev, 2010) has motivated me to continue this thread. A couple of weeks ago I sketched how to reduce the dimensionality of the replicator equations governing tag-based games. Today, I will use this representation to look at how properties of the game affect the persistence and diversity of tags.

Short history of iterated prisoner’s dilemma tournaments

Nineteen Eighty — if I had to pick the year that computational modeling invaded evolutionary game theory then that would be it. In March, 1980 — exactly thirty-five years ago — was when Robert Axelrod, a professor of political science at University of Michigan, published the results of his first tournament for iterated prisoner’s dilemma in the Journal of Conflict Resolution. Game theory experts, especially those specializing in Prisoner’s dilemma, from the disciplines of psychology, political science, economics, sociology, and mathematics submitted 14 FORTRAN programs to compete in a round-robin tournament coded by Axelrod and his research assistant Jeff Pynnonen. If you want to relive these early days of evolutionary game theory but have forgotten FORTRAN and only speak Python then I recommend submitting a strategy to an analogous tournament by Vincent Knight on GitHub. But before I tell you more about submitting, dear reader, I want to celebrate the anniversary of Axelrod’s paper by sharing more about the original tournament.

Maybe it will give you some ideas for strategies.

Hunger Games themed semi-iterated prisoner’s dilemma tournament

With all the talk surrounding it, crowdsourcing science might seem like a new concept and it might be true for citizen science efforts, but it is definitely an old trick to source your research to other researchers. In fact, evolutionary game theory was born (or at least popularized) by one such crowdsourcing exercise; in 1980, Robert Axelrod wanted to find out the best strategy for iterated prisoner’s dilemma and reached out to prominent researchers for strategy submissions to a round-robin tournmanet. Tit-for-tat was the winning strategy, but the real victor was Axelrod. His 1981 paper with Hamilton analyzing the result went on to become a standard reference in applications of game theory to social questions (at least outside of economics), agent-based modeling, and — of course — evolutionary game theory. Of Axelrod’s sizeable 47,222 (at time of writing) citations, almost half (23,370) come from this single paper. The tradition of tournaments continues among researchers, I’ve even discussed an imitation tournament on imitation previously.

The cynical moral of the tale: if you want to be noticed then run a game theory tournament. The folks at Brilliant.org — a website offering weekly olympiad-style challange problems in math and physics — took this message to heart, coupled it to the tried-and-true marketing technique of linking to a popular movie/book franchise, and decided to run a Hunger Games themed semi-iterated Prisoner’s dillema tournament. Submit a quick explanation of your strategy and Python script to play the game, and you could be one of the 5 winners of the $1,000 grand prize. Hooray! The submission deadline is August 18th, 2013 and all you need is a Brilliant account and it seems that these are free. If you are a reader of TheEGG blog then I recommend submitting a strategy, and discussing it in the comments (either before or after the deadline); I am interested to see what you come up with.

Evolving useful delusions to promote cooperation

This joint work with Marcel Montrey and Thomas Shultz combines — to be consistent with the interdisciplinary theme of this symposium — ideas from biology, economics, a little bit of cognitive science, and the approach is through applied mathematics. This post is a transcript of a presentation I gave on March 27th and covers part of my presentation today at Swarmfest.

Evolve ethnocentrism in your spare time

Running an agent based simulation really isn’t that complex. While there’s no shortage of ready-made software packages for ABM (like Repast and NetLogo), all you really need is a good, high-level programming language and a code editor.

As you may have noticed from other blog posts, we have spent quite a bit of time studying agent based models of ethnocentric evolution. To coincide with the publication of our paper (Hartshorn, Kaznatcheev & Shultz, 2013) on the evolution of ethnocentrism in the Journal of Artificial Societies and Social Simulation (JASSS), we thought it would be fun to provide a hands-on tutorial so you can replicate the model yourself. There’s a lot to cover here, so we won’t get into the scientific description of the model itself, but you can read a good synopsis in my executive summary, or Artem’s general overview.

This post assumes no programming background, just a computer, patience, and some curiosity. That being said, you will be compiling a small Java program and modifying its source code, so if words like “compile,” “source code,” and “Java” strike terror in your heart, consider yourself forewarned. It’s actually not that scary. In Estonia they’re teaching kids to program in first grade, and you’re smarter than a first grader…right?!

How ethnocentrism evolves: a simulation of evolutionary dynamics

Cooperation is a paradox—it just doesn’t make sense. Why should I help you when there’s no direct benefit for me? Artem, Professor Tom Shultz, and I have been working for quite some time on a paper about cooperation, and we’re psyched to announce that it’s finally been published in The Journal of Artificial Societies and Social Simulation (JASSS). JASSS is an open web journal, so you can view the full text of our article for free on their website. Or you could skip the 8000 or so words and check out this summary post.

Natural algorithms and the sciences

Today, I am passing through New York City on my way to Princeton’s Center for Computational Intractability for a workshop on Natural Algorithms and the Sciences (NA&S). The two day meeting will cover everything from molecular algorithms for learning and experiments on artificial cells to bounded rationality in decision-making and the effects of network topology on the evolution of collective migration. In other words, right at home for TheEGG blog. The full mission statement:

The workshop will bring together researchers from computer science, mathematics, physics, biology, and engineering to explore interactions among algorithms, dynamical systems, statistical physics, and complexity theory (in all senses of the term).

Learning and evolution are different dynamics

A couple of weeks ago, if you randomly woke me in the middle of the night and demanded to know the fundamental difference between evolution and learning as adaptive processes, I would probably respond: “how did you get into my house? and umm… I guess they are mostly the same, it is just a matter of time-scales and domain.” This answer stems from my urge to generalize and find the overarching similarities between ideas, and evolution and learning share a lot in common. Both are more likely to propagate effective behaviors than ineffective and both generate novelty in randomized and often unguided process: mutation and innovation. In fact, in evolutionary game theory social imitation and reproduction are used almost interchangeably in mathematical models. Most computational models can be interpreted either as biological or cultural evolution without changing any code, just the words used to describe the agents.

In the Hammond & Axelrod (2006) model of ethnocentrism, for example, we can stretch the whole range of biological to cultural evolution depending on our interpertation:

• If we interpret the agents as single bacteria and the tags are quorum markers, then we are obviously in the standard green-beard effect regime and our evolution can only be interpreted as biological.
• If we interpret the agents as humans (or other animals) and tags as skin color (or other physical trait) then our strategy transmission might be biological or cultural, but the tag transmission is clearly biological.
• If we interpret the agents as humans and tags as language accents, then both transmissions are cultural with only a little room to argue for biology.
• Finally, if we interpret agents as villages and tags as their religion then it is almost impossible to argue for biology and the dynamics must clearly be of cultural evolution.

But, we never changed any specifications of the model, just the language we used to describe it so the dynamics were invariant. I usually view this generality as an advantage of the model; we can reason about either dynamic: cultural or biological. However, it can also be a weakness, the dynamics are underspecified and inaccurate representations of both!

From a practical point of view, if I want to combine evolution and learning in one model then it doesn’t make sense to do so (and expect anything interesting) if they follow the same exact dynamics. Since I am becoming more interested in social learning and its potential analogies to evolutionary game theory, it is important to figure out what fundamental differences the two adaptive process might have. Thankfully, evolutionary economists have already thought about this.

For an evolutionary economist: the agents are corporations and the heritable material is business-practices. In domain they are squarely working with learning and cultural evolution, but they view the resulting dynamics as analogous to the biology from which they borrow name. Since agent-based modeling is an important methodology for these economists, they have thought about the similarities and differences of evolutionary and learning models carefully.

Brenner (1998) explicitly compares models of evolutionary and learning. For evolution, he takes the EGT model of replicator-mutator dynamics, and for learning he looks at his earlier Variation-Imitation-Decision (VID) model (Brenner 1996). Since the VID model doesn’t seem to be a standard approach, I won’t go into the details of the technical comparison. I will instead highlight the distinction Brenner draws that I think generalize to most models of evolution and learning: objectivity of fitness.

In a biological settings, we have a clear objective measure of fitness: number of offspring. As such, it is relatively uncontroversial to associate a given behavior with a fitness value. In a lot of social learning settings, the same approach is also followed, but it is not as obvious. The fitness of a meme is subjective and varies between potential adopters. Some agents might be more susceptible to a given idea given the ideas they already hold, their past history with various behaviors (invidiual historicity), or just general outlook; other agents might be less so. Two agents might observe the same behavior, and the first might think the behavior is good (and thus maybe worth imitating) and another will conclude that it is not a helpful behavior (and thus probably not worth imitating). In a general social settings, we cannot view a heritable trait as having an inherent fitness, it depends on the agents that will consider it for copying.

If we wanted to incorporate a lack of objective fitness into an EGT model, we could do this in the objective versus subjective rationality model. In this model, each agent has a different subjective conception of what game the objective game of the environment is. As such, if Alice and Bob views the behavior of Eve then they will judge its effectiveness not by Eve’s conception of the game (that they doesn’t know) but by their own, as such Alice might calculate one utility for the behavior she saw Eve display, and Bob could calculate a completely different utility. From the point of view of imitation, Eve’s behavior would not have an inherent fitness. At the same time, the obj-vs-subj model also has elements of standard evolution (in the vertical transmission of conceptions of the game) and can be a good groundwork for building models that capture the different dynamics of evolution and learning.

Now, if you break into my house in the middle of the night to question me about evolution and learning then — while I wait for the cops to come remove you — I might explain the importance of objective versus subjective measures of fitness.

References

Brenner, T. (1996) Learning in a repeated decision process: A mutation-imitation-decision model. Papers on Economics and Evolution #9603, Max-Planck-Institut, Jena.

Brenner, T. (1998). Can evolutionary algorithms describe learning processes? Journal of Evolutionary Economics, 8 (3), 271-283 DOI: 10.1007/s001910050064

Hammond, R., & Axelrod, R. (2006). The Evolution of Ethnocentrism. Journal of Conflict Resolution, 50(6): 926-936

Social learning dilemma

Last week, my father sent me a link to the 100 top-ranked specialties in the sciences and social sciences. The Web of Knowledge report considered 10 broad areas^[1] of natural and social science, and for each one listed 10 research fronts that they consider as the key fields to watch in 2013 and are “hot areas that may not otherwise be readily identified”. A subtle hint from my dad that I should refocus my research efforts? Strange advice to get from a parent, especially since you would usually expect classic words of wisdom like: “if all your friends jumped off a bridge, would you jump too?”

So, which advice should I follow? Should I innovate and focus on my own fields of interest, or should I imitate and follow the trends? Conveniently, the field best equipped to answer this question, i.e. “social learning strategies and decision making”, was sixth of the top ten research fronts for “Economics, Psychology, and Other Social Sciences”^[2].

For the individual, there are two sides to social learning. On the one hand, social learning is tempting because it allows agents to avoids the effort and risk of innovation. On the other hand, social learning can be error-prone and lead individuals to acquire inappropriate and outdated information if the the environment is constantly changing. For the group, social learning is great for preserving and spreading effective behavior. However, if a group has only social learners then in a changing environment it will not be able to innovate new behavior and average fitness will decrease as the fixed set of available behaviors in the population becomes outdated. Since I always want to hit every nail with the evolutionary game theory hammer, this seems like a public goods game. The public good is effective behaviors, defection is frequent imitation, and cooperation is frequent innovation.

Although we can trace the study of evolution of cooperation to Peter Kropotkin, the modern treatment — especially via agent-based modeling — was driven by the innovative thoughts of Robert Axelrod. Axelrod & Hamilton (1981) ran a computer tournament where other researchers submitted strategies for playing the iterated prisoners’ dilemma. The clarity of their presentation, and the surprising effectiveness of an extremely simple tit-for-tat strategy motivated much of the current work on cooperation. True to their subject matter, Rendell et al. (2010) imitated Axelrod and ran their own computer tournament of social learning strategies, offering 10,000 euros for the best submission. By cosmic coincidence, the prize went to students of cooperation: Daniel Cownden and Tim Lillicrap, two graduate students at Queen’s University, the former a student of mathematician and notable inclusive-fitness theorist Peter Taylor.

A restless multi-armed bandit served as the learning environment. The agent could select which of 100 arms to pull in order to receive a payoff drawn independently (for each arm) from an exponential distribution. It was made “restless” by changing the payoff after each pull with probability . A dynamic environment was chosen because copying outdated information is believed to be a central weakness of social learning, and because Papadimitriou & Tsitsiklis (1999) showed that solving this bandit (finding an optimal policy) is PSPACE-complete^[3], or in laymen terms: very intractable.

Participants submitted specifications for learning strategies that could perform one of three actions at each time step:

• Innovate — the basic form of asocial learning, the move returns accurate information about the payoff of a randomly selected behavior that is not already known by the agent.
• Observe — the basic form of social learning, the observe move returns noisy information about the behavior and payoff being demonstrated by a randomly selected individual. This could return nothing if no other agent played an exploit move this round, or if the behavior was identical to one the focal agent already knows. If some agent is selected for observation then unlike the perfect information of innovate, noise is added: with probability a randomly chosen behavior is reported instead of the one performed by the selected agent, and the payoff received is reported with Gaussian noise with variance .
• Exploit — the only way to acquire payoffs by using one of the behaviors that the agent has previously added to its repertoire with innovate and observe moves. Since no payoff is given during innovate and observe, they carry an inherent opportunity cost of not exploiting existing behavior.

The payoffs were used to drive replicator dynamics via a death-birth process. The fitness of an agent was given by their total accumulated payoff divided by the number of rounds they have been alive for. At each round, every agent in the population had a 1/50 probability of expiring. The resulting empty spots were filled by offspring of the remaining agents, with probability of being selected for reproduction proportional to agent fitness. Offspring inherited their parents’ learning strategy, unless a mutation occurred, in which case the offspring would have the strategy of a randomly selected learning strategy from those considered in the simulation.

A total of 104 learning strategies were received for the tournament. Most were from academics, but three were from high school students (with one placing in the top 10). A pairwise tournament was held to test the probability of a strategy invading any other strategy (i.e, if a single individual with a new strategy is introduced into a homogeneous population of another strategy).This round-robin tournament was used to select the 10 best strategies for advancement to the melee stage. During the round-robin , , were kept fixed, only during the melee stage with all of the top-10 strategies present did the experimenters vary these parameters.

Mean score depending the proportion of learning actions (both INNOVATE and OBSERVE) in the left figure, and the proportion of OBSERVE actions in the right figure. These are figures 2C and 2A from Rendell et al. (2010).

Unsurprisingly using lots of EXPLOIT moves is essential to good performance, since this is the only way to earn payoff. In other words: less learning and more doing. However, a certain minimal amount of learning is needed to get your doing off the ground, of this learning there is a clear positive correlation between the amount of social learning and success in invading other strategies. The best strategies used the limited information given to them to estimate and used that to better predict and quickly react to changes in the environment. However, they also relied completely on social learning, waiting for other agents to innovate new strategies or for to accidently give a new behavior for their repertoire. Since evolution (unlike the classical assumptions of rationality) cares about relative and not absolute payoffs, it didn’t matter to these agents that they were not doing as well as they could be, as long as they were doing as well as (or better than) their opponents^[4]. OBSERVE moves and a good estimate of environmental change allowed the agents to minimize their number of non-EXPLOIT moves and since their exploits paid as well as their opponents (who they were copying) they ended up having equal or better payoff (due to less learning and more exploiting).

Average individual fitness of the top 10 strategies when in a homogenous environment. The best strategy from the multi-strategy competitions is on the left and the tenth best is on the right. Note that the best strategies for when all 10 strategies are present are the worst for when they are alone. This is figure 1D from Rendell et al. (2010).

My view of social learning as an antisocial strategy is strengthened by the strategy’s low fitness when in isolation. The figure to the left shows this result, with the data-points more to the left corresponding to strategies that did better in the melee. Strategies 1, 2, and 4 are the pure social learners. The height of the data points shows how well a strategy performed when faced only against itself. The strategies that did best in the heterogeneous setting of the 10 strategy melee performed the worst when they were in a homogeneous populations with only agents of the same type. This is in line with Rendell, Fogarty, & Laland (2010) observation that social learning can decrease the overall fitness of the population. Social learners fare even worse when they can’t make occasional random mistakes in copying behavior, without these errors all innovation disappears from the population and average fitness plummets. Social learners are free-riding on the innovation of asocial agents.

I would be interested in pursuing this heuristic connection between learning and social dilemmas further. The interactions of learners with each other and the environment can be seen as an evolutionary game: can we calculate the explicit payoff matrix of this game in terms of environmental and strategy parameters? Does this game belong to the Prisoners’ dilemma or Hawk-Dove (or other) region of cooperate-defect games? The heuristic view of innovation as a public good and the lack of stable co-existence of imitators and innovators suggests that the dynamics are PD. However, Rendell, Fogarty, & Laland (2010) show social learning can sometimes spread better on a grid structure, this is contrary to the effects of PD on grids, but consistent with observations for HD (Hauert & Doebeli, 2004). Since the two studies use very different social learning strategies, it could be the case that depending on parameters, we can achieve either PD or HD dynamics.

Regardless of which social dilemma is in play, we know that slight spatial structure enhances cooperation. This means that I expect that if — instead of inviscid interactions — I repeated Rendell et al. (2010) on a regular random graph then we would see more innovation. Similarly, if we introduced selection on the level of groups then groups with more innovators would fare better and spread the innovative strategy throughout the population.

So what does this mean for how I should take my father’s implicit advice? First: stop learning and start doing; I need to spend more time writing up results into papers instead of learning new things. Unfortunately for you, my dear reader, this could mean fewer blog posts on fun papers and more on my boring work! In terms of following research trends, or innovating new themes, I think a more thorough analysis is needed. It would be interesting to extend my preliminary ramblings on citation network dynamics to incorporate this work on social learning. For now, I am happy to know that at least some of things I’m interested are — in Twitter speak — trending.

Notes and References

1. Way too broad for my taste, one category was “Mathematics, Computer Science, and Engineering”; talk about a tease-and-trick. After reading the first two items I was excited to see a whole section dedicated to results like theoretical computer science, only to have my dreams dashed by ‘Engineering’. Turns out that Thomson Reuters and I have very different ideas on what ‘Mathematics’ means and how it should be grouped.
2. Note that my interest weren’t absent from the list, with “financial crisis, liquidity, and corporate governance” appearing tenth for “Economics, Psychology, and Other Social Sciences” and even selected for a special more in-depth highlight. Evolutionary thinking also appeared in tenth place for the poorly titled “Mathematics, Computer Science and Engineering” area as “Differential evolution algorithm and memetic computation”. It is nice to know that these topics are popular, although I am usually not a fan of the engineering approach to computational models of evolution since their goal is to solve problems using evolution, not answer questions about evolution.
3. High-impact general science publications like Nature, Science, and their more recent offshoots (like the open-access Scientific Reports) are awful at presenting theoretical computer science. It is no different in this case, Papadimitriou and Tsitsiklis (1999) is a worst-case result that requires more freedom in the problem instances to encode the necessary structure for a reduction to known hard problems. Although their theorem is about restless bandits, the reduction needs a more general formulation in terms of arbitrary deterministic finite-dimensional Markov chains instead of the specific distributions used by Rendell et al. (2010). I am pretty sure that the optimal policy for the obvious generalization (i.e. arms instead of 100, but generated in the same way) of the stochastic environment can be learned efficiently; there is just not enough structure there to encode a hard problem. Since I want to understand multi-armed bandits better, anyways, I might find the optimal algorithm and write about it in a future post.
4. This sort of “I just want to beat you” behavior, reminds me of the irrational defection towards the out-group that I observed in the harmony game for tag-based models (Kaznatcheev, 2010).

Axelrod, R., & Hamilton, W. D. (1981). The evolution of cooperation. Science, 211(4489), 1390-1396.

Hauert, C., & Doebeli, M. (2004). Spatial structure often inhibits the evolution of cooperation in the snowdrift game. Nature, 428(6983), 643-646.

Kaznatcheev, A. (2010). Robustness of ethnocentrism to changes in inter-personal interactions. Complex Adaptive Systems – AAAI Fall Symposium. (pdf)

Papadimitriou, C. H., & Tsitsiklis, J. N. (1999). The complexity of optimal queuing network control. Mathematics of Operations Research, 24(2): 293-305.

Rendell L, Boyd R, Cownden D, Enquist M, Eriksson K, Feldman MW, Fogarty L, Ghirlanda S, Lillicrap T, & Laland KN (2010). Why copy others? Insights from the social learning strategies tournament. Science, 328 (5975), 208-213 PMID: 20378813

Rendell, L., Fogarty, L., & Laland, K. N. (2010). Rogers’ paradox recast and and resolved: population structure and the evolution of social learning strategies” Evolution 64(2): 534-548.

Environmental austerity and the anarchist Prince of Mutual Aid

Prince Pyotr Alexeyevich Kropotkin

Any good story starts with a colourful character, a complicated character, and — to be complacent with modern leftist literature — an anarchist intellectual well-versed in (but critical and questioning of) Marxism; enter Pyotr Alexeyevich Kropotkin. Today he is best known as one of the founders and leading theorist of anarcho-communism, but in his time he was better known as an anti-Tsarist revolutionary, zoologist, geographer and explorer. Kropotkin was born to the Prince of Smolensk, a descendant of the Rurik dynasty that ruled and eventually unified many of the Principalities and Duchies of Rus into the Tsardom of Russia. By the 9 December 1842 birth of our protagonist, Russia had been under Romanov rule for over 200 years, but the house of Rurik still held great importance. Even though the young boy renounced his Princely title at age 12, he was well-off and educated in the prestigious Corps of Pages. There he rose to the highest ranks and became the personal page of Tsar Alexander II. Upon graduation this entitled Kropotkin to his choice of post, and our first plot twist.

Analogous to the irresistible pull of critical theory on modern liberal-arts students, the young Kropotkin was seduced by the leftist thought of his day: French encyclopédistes, the rise of Russian liberal-revolutionary literature, and his personal disenfranchisement with and doubt of the Tsar’s “liberal” reputation. Instead of choosing a comfortable position in European Russia, the recent graduate requested to be sent to the newly annexed Siberian provinces and in 1862 was off to Chita. This city has a personal significance to me, it is where my grandfather was stationed over 100 years later and most of my mother’s childhood was spent there. Chita has become a minor place of pilgrimage for modern anarchists, but it (and the other Siberian administrative centre at Irkutsk) did not hold Kropotkin’s attention for long.

Unable to enact substantial change as an administrator, he followed his passion as a naturalist. In 1864, he took command of a geographic survey expedition into Manchuria. Having read Darwin’s On the Origin of Species when it was published 5 years earlier, Kropotkin embarked on a distinctly Siberian variant of the HMS Beagle — sleigh dogs instead of wind to power his way. His hope was to observe the same ‘tooth and claw’ competition as Darwin, but instead he saw primarily cooperation. In the harsh environment of Siberia, it wasn’t a struggle of beast versus beast, but animal against environment.

From 1890 to 1896, Kropotkin published his Siberian observations as a series of essays in the British monthly literary magazine, Nineteenth Century. Motivated as a response to Huxley’s “The Struggle for Existence”, the essays highlighted cooperation among nonhuman animals, in primitive societies and medieval cities, and in contemporary times. He concluded that not competition, but cooperation, were the most important factors in survival and the evolution of species. Kropotkin assembled the essays into book form, and in 1902 published Mutual Aid: A Factor of Evolution. A magnum opus on cooperation, much like E.O. Wilson Sociobiology of nearly 75 years later, Kropotkin started from the social insects and traced a common thread to the human society around him; he was the first student of cooperation.

Unfortunately, his mechanism for cooperation did not extend beyond group selection. Kropotkin left it to modern researchers to find more basic engines of altruism. Only now are we starting to build mathematical, computational, and living models. To study cooperation in the laboratory, especially when looking at the effect of environmental austerity, Strassmann & Queller (2011) have proposed the social microbe Dictyostelium discoideum or slime mold as the perfect model. These single-cell soil-dwelling amoeba are capable of working together under austere conditions, and even display rudimentary swarm intelligence. A long time expert on slime molds, John Bonner of Princeton University, made a video of them during his undergraduate years at Harvard:

Under plentiful conditions, D. discoideum are solitary predators of bacteria, which they consume by engulfment. If the environment deteriorates and the amoebae begin to starve, then they enter a social stage. Using their quorum-sensing mechanism they check if enough other amoebae are present in the area and then aggregate into a mound. They coat themselves with a slime (that gives them their name) and move together as a unit, until they find a good location to fruit. The slime mold then extends a stalk up from the soil with most cells forming a spore at the top. At a certain height, the spore is released, allowing the amoebae at the top to disperse to greener pastures; the cells in the stalk die. Since all the cells are free-living independent organisms during the non-social stage, this shows the clearest form of altruism: fellow D. discoideum sacrificing their own lives in order to give their brethren a chance at a future.

Sadly, most evolutionary game theory models assume constant population size and no resource variability. In these models, it is difficult to introduce a parameter analogous to environmental austerity. To allow for resource limitations, we need to introduce variable population sizes and thus create an ecological game. I explored this modification for a Hammond & Axelrod-like model back in the summer of 2009 and thought I would share some results here.

The agents inhabit a toroidal lattice, and each round the agents interacts with their 4 adjacent neighbours via the Prisoner’s dilemma. The payoffs are added to their default birth rate, reproduction is asexual and into adjacent empty sites. At each time step, each agent has a fixed (0.25) probability of expiring and vacating its site. The worlds start empty and are gradually filled with agents.

This figure has three graphs; in each figure the line thickness represents standard error from averaging 30 independent runs. The leftmost graph is the proportion of cooperation versus cycle, with two conditions for default birth rate: 0.24 (high austerity; top line) and 0.28 (low austerity; bottom line). The two figures on the right show the total number of cooperators (blue) and defectors (red). The rightmost graph has time flowing from right to left. The left panel is high austerity (def ptr = 0.24) and the right panel is low austerity (def ptr = 0.28).

Above are the results for a Prisoner’s dilemma interaction with — a rather competitive environment. Matching Shultz, Hartshorn, & Kaznatcheev (2009) and consistent with Milbiner, Cremer, & Frey (2010), we can see an early spike in the number of cooperators as the world reaches its carrying capacity. After this transient period, the dynamics shift and defection becomes more competitive. The dynamics settle to a stable distribution of cooperators and defectors. The proportion of cooperation depends heavily on the environmental austerity. In a harsh environment with a low default birth rate of 0.24, the agents band together and cooperate and in a plentiful environment with high default birth rate of 0.28, defection dominates. As Kropotkin observed: cooperation is essential to surviving environmental austerity.

Analogous to the results from the Hauert, Homles, & Doebeli (2006) ecological public-goods game the proportion of cooperation tends to bifurcate around default birth rate equal to to the death rate (0.25), although I don’t present the visuals here. The increase in default birth rate results in a slight increase in the world population at saturation, but even by raw number there are more cooperators in the high austerity than the low austerity setting. Thus, it is not simply defectors benefiting more from the decrease in austerity (since defectors go from a regime where clusters are not-self sustaining (def ptr = 0.24) to one where it is (def ptr = 0.28)), but also an effect of defectors out-competing and disproportionately exploiting and crowding out the cooperators.

Each graph is evolutionary cycles versus proportion of cooperation, line thickness is standard error from averaging 30 independent runs. Environmental austerity decreases from the left graph (where default birth rate is equal to death rate) to the right (where their ratio is 1.1). The blue line is the model where agents can discriminate based on arbitrary non-strategy related tag (the green-beard effect/ethnocentrism are possible) and the green line is simulations where no conditional strategy is possible.

If agents are allowed to condition their behavior on an arbitrary tag then the ethnocentric population is better able to maintain higher levels of cooperation as environmental austerity decreases. In the tag-based model, it would be interesting to know if there is a parameter range where varying environmental austerity can take us from a regime of humanitarian (unconditional cooperator) dominance, to ethnocentric dominance (cooperate with in-group, defect from out-group), to a selfish (unconditional defection) world. I am also curious to know how the irrational hostility I observed in the tag-based harmony game (Kaznatcheev, 2010) would fare as the environment turns hostile. Will groups overcome their biases against each other, or will they compete even more for the more limited resource? Nearly 150 years after Peter Kropotkin’s Siberian expedition, the curtain is still up and basic questions on mutual aid in austere environments remain!

References

Hauert, C., Holmes, M., & Doebeli, M. (2006). Evolutionary games and population dynamics: maintenance of cooperation in public goods games. Proceedings of the Royal Society B: Biological Sciences, 273(1600): 2565-2571

Kaznatcheev, A. (2010). Robustness of ethnocentrism to changes in inter-personal interactions. Complex Adaptive Systems – AAAI Fall Symposium. [pdf]

Melbinger, A., Cremer, J., & Frey, E. (2010). Evolutionary game theory in growing populations. Physical Review Letters, 105(17): 178101. [arXiv pdf]

Shultz, T. R., Hartshorn, M., & Kaznatcheev, A. (2009). Why is ethnocentrism more common than humanitarianism? In N. A. Taatgen & H. van Rijn (Eds.), Proceedings of the 31st Annual Conference of the Cognitive Science Society (pp. 2100-2105). Austin, TX: Cognitive Science Society. [pdf]

Strassmann, J., & Queller, D. (2011). Evolution of cooperation and control of cheating in a social microbe Proceedings of the National Academy of Sciences, 108 (2), 10855-10862 DOI: 10.1073/pnas.1102451108