It seems like my initial experiments have been a bit awry, but this is how you learn! I was tweaking the IPD code to use encoded weightings rather than classic logic to start creating the genetic algorithm code to evolve this behaviour.

I recreated all the strategies as weights using a simple encoding denoting their probability to cooperate. They are encoded as {I, R, S, T, P} where I = initial, R = reward (c,c), S = sucker (c,d), T = temptation (d,c), P = punishment (d,d). As an example, all-cooperate and TFT strategies would be:

AllCooperate = 1.0, 1.0, 1.0, 1.0, 1.0
TFT = 1.0, 1.0, 0.0, 1.0, 0.0

So All-C will always cooperate, from the outset and continue to. TFT will cooperate initially, then cooperate after reward or temptation and defect after sucker or punishment.

After doing this, running the code brought out some different results to my previous code. Even without any mistakes or noise, Pavlov was absolutely annihilating all the competition, including TFT, with the exception of course of an All-Defect which would persist in very small, stable clusters.

After checking it wasn’t a bug in my latest code, I noticed there were a few small bugs in my previous code. I also tweaked a bit of the spatial logic (allowing the propagation of strategies that are equal in points). The Pavlov strategy was still unbelievably quick to propagate and take over the world, in fact, noise helped the TFT strategy to gain a bit of traction. With a 20% mistake rate, TFT can stay for some generations, exploiting the All-D cells that divide up the pockets of Pavlov.

I introduced a generous-TFT strategy that ignores a third of all defections and still cooperates. As expected (GTFT is well-documented as a strong strategy), this faired much better overall, co-existing nicely with Pavlov even after several thousand iterations.

It is good to see the strategies now stacking up by cooperative behaviour: TFT rules when pitted against random, All-C and All-D strategies. Introduce Pavlov and it takes over. Introduce a GTFT and it exists where TFT couldn’t.