Learning and evolution are different dynamics

A couple of weeks ago, if you randomly woke me in the middle of the night and demanded to know the fundamental difference between evolution and learning as adaptive processes, I would probably respond: “how did you get into my house? and umm… I guess they are mostly the same, it is just a matter of time-scales and domain.” This answer stems from my urge to generalize and find the overarching similarities between ideas, and evolution and learning share a lot in common. Both are more likely to propagate effective behaviors than ineffective and both generate novelty in randomized and often unguided process: mutation and innovation. In fact, in evolutionary game theory social imitation and reproduction are used almost interchangeably in mathematical models. Most computational models can be interpreted either as biological or cultural evolution without changing any code, just the words used to describe the agents.

In the Hammond & Axelrod (2006) model of ethnocentrism, for example, we can stretch the whole range of biological to cultural evolution depending on our interpertation:

  • If we interpret the agents as single bacteria and the tags are quorum markers, then we are obviously in the standard green-beard effect regime and our evolution can only be interpreted as biological.
  • If we interpret the agents as humans (or other animals) and tags as skin color (or other physical trait) then our strategy transmission might be biological or cultural, but the tag transmission is clearly biological.
  • If we interpret the agents as humans and tags as language accents, then both transmissions are cultural with only a little room to argue for biology.
  • Finally, if we interpret agents as villages and tags as their religion then it is almost impossible to argue for biology and the dynamics must clearly be of cultural evolution.
  • But, we never changed any specifications of the model, just the language we used to describe it so the dynamics were invariant. I usually view this generality as an advantage of the model; we can reason about either dynamic: cultural or biological. However, it can also be a weakness, the dynamics are underspecified and inaccurate representations of both!

    From a practical point of view, if I want to combine evolution and learning in one model then it doesn’t make sense to do so (and expect anything interesting) if they follow the same exact dynamics. Since I am becoming more interested in social learning and its potential analogies to evolutionary game theory, it is important to figure out what fundamental differences the two adaptive process might have. Thankfully, evolutionary economists have already thought about this.

    For an evolutionary economist: the agents are corporations and the heritable material is business-practices. In domain they are squarely working with learning and cultural evolution, but they view the resulting dynamics as analogous to the biology from which they borrow name. Since agent-based modeling is an important methodology for these economists, they have thought about the similarities and differences of evolutionary and learning models carefully.

    Brenner (1998) explicitly compares models of evolutionary and learning. For evolution, he takes the EGT model of replicator-mutator dynamics, and for learning he looks at his earlier Variation-Imitation-Decision (VID) model (Brenner 1996). Since the VID model doesn’t seem to be a standard approach, I won’t go into the details of the technical comparison. I will instead highlight the distinction Brenner draws that I think generalize to most models of evolution and learning: objectivity of fitness.

    In a biological settings, we have a clear objective measure of fitness: number of offspring. As such, it is relatively uncontroversial to associate a given behavior with a fitness value. In a lot of social learning settings, the same approach is also followed, but it is not as obvious. The fitness of a meme is subjective and varies between potential adopters. Some agents might be more susceptible to a given idea given the ideas they already hold, their past history with various behaviors (invidiual historicity), or just general outlook; other agents might be less so. Two agents might observe the same behavior, and the first might think the behavior is good (and thus maybe worth imitating) and another will conclude that it is not a helpful behavior (and thus probably not worth imitating). In a general social settings, we cannot view a heritable trait as having an inherent fitness, it depends on the agents that will consider it for copying.

    If we wanted to incorporate a lack of objective fitness into an EGT model, we could do this in the objective versus subjective rationality model. In this model, each agent has a different subjective conception of what game the objective game of the environment is. As such, if Alice and Bob views the behavior of Eve then they will judge its effectiveness not by Eve’s conception of the game (that they doesn’t know) but by their own, as such Alice might calculate one utility for the behavior she saw Eve display, and Bob could calculate a completely different utility. From the point of view of imitation, Eve’s behavior would not have an inherent fitness. At the same time, the obj-vs-subj model also has elements of standard evolution (in the vertical transmission of conceptions of the game) and can be a good groundwork for building models that capture the different dynamics of evolution and learning.

    Now, if you break into my house in the middle of the night to question me about evolution and learning then — while I wait for the cops to come remove you — I might explain the importance of objective versus subjective measures of fitness.

    References

    Brenner, T. (1996) Learning in a repeated decision process: A mutation-imitation-decision model. Papers on Economics and Evolution #9603, Max-Planck-Institut, Jena.

    Brenner, T. (1998). Can evolutionary algorithms describe learning processes? Journal of Evolutionary Economics, 8 (3), 271-283 DOI: 10.1007/s001910050064

    Hammond, R., & Axelrod, R. (2006). The Evolution of Ethnocentrism. Journal of Conflict Resolution, 50(6): 926-936

Advertisements

About Artem Kaznatcheev
From the Department of Computer Science at Oxford University and Department of Translational Hematology & Oncology Research at Cleveland Clinic, I marvel at the world through algorithmic lenses. My mind is drawn to evolutionary dynamics, theoretical computer science, mathematical oncology, computational learning theory, and philosophy of science. Previously I was at the Department of Integrated Mathematical Oncology at Moffitt Cancer Center, and the School of Computer Science and Department of Psychology at McGill University. In a past life, I worried about quantum queries at the Institute for Quantum Computing and Department of Combinatorics & Optimization at University of Waterloo and as a visitor to the Centre for Quantum Technologies at National University of Singapore. Meander with me on Google+ and Twitter.

12 Responses to Learning and evolution are different dynamics

  1. Thomas Shultz says:

    Interesting and helpful post on what can be a confusing issue about culture and biology! If we want to study the interactions of culture and biology, we should at least understand their similarities and differences. At some abstract level, the algorithms can look very similar across biology and culture and even be identical, as Artem notes. At a more detailed level, a number of important differences can be noted, and they go beyond issues of interpretation of simulations. This comment focuses on two such differences: time scale and coding. Biological evolution operates on a relatively slow time scale. With long developmental periods and small mutation rates, it can take a long time for changes in species to emerge. Nested within evolutionary cycles are faster developmental cycles. And nested within those are even faster learning cycles. In terms of coding, evolutionary changes are coded in genotypes, while learning is coded in synaptic modifications. Some social learning methods, such as theory passing in humans, are quite different than mere imitation, and are perhaps less likely to be confused with genetic change. Once such differences are clarified, interactions can be examined. The evolution of learning methods can be studied, as can the impact of learning on biological evolution. Notwithstanding the immense practical difficulties in separating biological evolution from direct experiential effects, including epigenetics of gene expression, failure to understand fundamental differences will surely obscure clear examination of interactions. Tom Shultz

    • Thanks for the thorough comment, Tom! I agree that it is important to catalogue all the differences we know between evolution, development, and learning. However, I think it is important to always formalize these differences. Otherwise it isn’t clear if it is just a different of words (important to not mistake this for real difference) or an actual difference in dynamics.

      For example, on the level of timescales, the issue is very domain specific; in some domains, evolution can be extremely rapid, looking like a “sudden leap”. The example I give in my answer in the link is of affinity maturation which is an evolutionary processes in the human immune response that happens on the timescales of a few days and just a few generations of B cells. The mutation level differences between biological and cultural evolution are also a bit of a stretch when looking at the evolutionary dynamics of cancer or HIV; in fact, one of the few places where EGT has proven to be effective is in showing how we can cause cancer or strains of HIV to mutate themselves into extinction.

      Even the nesting of learning inside development inside of evolution is an oversimplification and does not distinguish cultural from biological evolution. Although some gene-culture co-evolutionary approaches are controversial, most notably Joseph Henrich’s Cultural Brain Hypothesis, others are well established. The one I am most familiar with is the cultural development of herding of milk-producing animals, and the evolution of enzymes to help us process milk after infancy; this cultural innovation had appeared at two distinct times in two distinct regions, and along side of it, two distinct ways of digesting milk evolved and the genes spread alongside the cultural transmission of how to raise and milk animals. In a slightly less clear cut, but more drastic, example: Adam Benton recently blogged about how the cultural skills of cultivating grains helped to distinguish us from our human-cow cousins and survive longer. Clearly, in these cases learning (at least the social/cultural kind) is not nested within evolution but driving it and adapting to it.

      Even theory passing, is not unique to learning. Although I don’t know if this is observed in nature together, but the mechanisms exist in evolution, so it definitely could be a possibility. If we look at single-cell learning then Lamarkian or epigenetic transfer would be equivalent to theory-passing.

      The difference in coding between genes and neural-nets is also not clear cut for me. As mentioned in the previous paragraph, if we look at single-cell learners then there are no neurons to adjust for learning. The information is encoded in the levels of specific macromolecules like proteins, which isn’t that fundamentally different from genes encoded in the specific macromolecules called DNA. But even if we restrict ourselves to brain-learning, the difference is a verbal one, since neither how genes map to phenotype nor how neuronal connections map to behavior are understood in full detail (although the biologists are closer). At this point, we have to treat both as black boxes, and sure we can assume they are different, but the question becomes how? Can we show that the difference in substrate produces a dynamically relevant difference? The analogue would be in computers, it doesn’t matter if my memory is vacuum tubes, RAM, or Turing Tape, the dynamics are still the same.

      Even the distinction of objectivity of fitness that I raise in this post, might not be as fundamental as I thought. Jacob Scott suggests on G+ that similar subjectivity of fitness/utility might exist when we are dealing with evolutionary dynamics of cancer. However, I am not convinced that this is the same sort of subjectivity as the one I mention, but I will need to think about it carefully and probably build a model to convince myself of their equivalence or difference.

      I think all the distinctions you mention are very promising, but we have to pursue them in a detail that extends beyond verbal theories. We cannot confine ourselves to popular high-level conceptions of evolution and learning but touches directly the microbiology and neuroscience.

  2. Torbjörn Larsson, OM says:

    I (interested in astrobiology) thought the differences between learning and evolution was various mechanisms such as imitation in learning and near neutral drift in evolution, but the similarities or rather analogies was in the adaptive mechanisms of variation and selection. (Where positive selection is analogous to trail-and-reward, and negative to trial-and-punishment.)

    As for a clear objective measure of fitness, it seems to me measurements of relative fitness is the norm while absolute fitness are rarer because it demands absolute population numbers. [ http://en.wikipedia.org/wiki/Fitness_(biology) ] Neither fitness is number of offspring, but averages over populations of ratios of increase/decrease of alleles, and the two concepts are related. Interestingly, absolute fitness is differential reproduction: “probability of survival multiplied by the average fecundity.”

    So variation is inherent, and averaging a must. I don’t really see an inherent difference between your description of individual outcomes of behavior that can modify (or not) other’s behavior by learning and a biological description of traits that can survive (or not) by selection. (“That rat was accidentally caught by a rat.”)

    • I agree that the distinction between relative fitness and absolute fitness is extremely important, and one of the distinguishing factors between biology and economics. However, the sort of subjectivity inherent in relative fitness is not as complicated as the subjective nature of utility for memes, since it varies much more smoothly (or more often is just a single normalizing term for the whole population).

      If I wanted to really introduce a subjective component to biological fitness then I would probably turn to sexual populations, in this case the mechanisms of sexual selection might produce subjective effects similar to what is possible in a social setting. With asexual populations though, fitness still has a common currency, and the introduction of relative fitness is a choice of the scientists to simplify the description of their system. In a social setting, there just isn’t a common currency for measuring fitness of memes, since each measuring agent can have a different utility function.

      I don’t think I understand your last paragraph. Could you elaborate on it?

  3. Pingback: Abiogenesis and Evolution | Party Dress Shops Articles

  4. Pingback: Cooperation through useful delusions: quasi-magical thinking and subjective utility | Theory, Evolution, and Games Group

  5. Pingback: Cataloging a year of blogging: from behavior to society and mind | Theory, Evolution, and Games Group

  6. Pingback: Phenotypic plasticity, learning, and evolution | Theory, Evolution, and Games Group

  7. Pingback: Evolution is a special kind of (machine) learning | Theory, Evolution, and Games Group

  8. Pingback: Useful delusions, interface theory of perception, and religion. | Theory, Evolution, and Games Group

  9. Pingback: Misbeliefs, evolution and games: a positive case | Theory, Evolution, and Games Group

  10. Pingback: Memes, compound strategies, and factoring the replicator equation | Theory, Evolution, and Games Group

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s