Evolving Cooperative IPD Strategies

As I continue to play with the IPD, I created code to genetically evolve IPD strategies to see if cooperation could be borne out of random behaviours. I used the standard GA I’ve created in Wintermute, with each agent represented by the five weights detailed in my last post and with fitness calculated as the average points earned in each bout (this was subtracted from 5 in order to allow the GA to search for a minimum). I then created a population of 500 agents with a mutation and elitism rate of 0.5% per generation.

It took me a while to tweak the GA to start working, but I finally got it with fascinating results. Here is a chart of the distribution of strategies along with the best fitness for each iteration:

Continue reading “Evolving Cooperative IPD Strategies”

More Experimenting with the Spatial IPD

It seems like my initial experiments have been a bit awry, but this is how you learn! I was tweaking the IPD code to use encoded weightings rather than classic logic to start creating the genetic algorithm code to evolve this behaviour.

I recreated all the strategies as weights using a simple encoding denoting their probability to cooperate. They are encoded as {I, R, S, T, P} where I = initial, R = reward (c,c), S = sucker (c,d), T = temptation (d,c), P = punishment (d,d). As an example, all-cooperate and TFT strategies would be:

AllCooperate = 1.0, 1.0, 1.0, 1.0, 1.0
TFT = 1.0, 1.0, 0.0, 1.0, 0.0

So All-C will always cooperate, from the outset and continue to. TFT will cooperate initially, then cooperate after reward or temptation and defect after sucker or punishment. Continue reading “More Experimenting with the Spatial IPD”

Understanding Cooperation with the Spatial Iterated Prisoner’s Dilemma

Not exactly a snappy way to start my new blog, but…

I just finished reading Critical Mass by Phillip Ball and one of the topics discussed was the iterated prisoner’s dilemma. I had implemented a Spatial IPD as part of the Generation5 JDK and more recently as part of the C# port, ‘Wintermute’.

Prior to reading Critical Mass though, I saw the SIPD as an interesting cellular automata implementation of a famous game theory exercise but never fully experimented with the model’s insights. In his book, Ball detailed how IPD can be used to show how cooperation is evolved as being more beneficial than selfish behaviour. Even more fascinating was the IPD’s ability to show how more cooperative strategies are more beneficial, until you get to a completely cooperative world — at which point, a very selfish strategy can quickly take advantage of the “naivety” of the world.

I decided to expand my code to explore this further to see if I could replicate this.

Continue reading “Understanding Cooperation with the Spatial Iterated Prisoner’s Dilemma”