Kohonen Neural Network Experimentations

A few weeks ago I updated my Kohonen neural network code to support circular rows and columns as well as some simple additional visualizations which allowed for some interesting experimentation.

Circular rows and columns follow a simple premise — the neighbourhood influence effect of training can now wrap around both row and column, allowing for circular, cylindrical and torodial geometries.

Think of it like this, a network with a single row wrapped around allows for a network to learn a circular topology. A nice, simple example of this is the Travelling Salesman Problem as you need one continuous, circular route to be determined. Below there is an example of a Kohonen network run over 350 iterations. The initial phase starts to spread the nodes out over the map, with the second phase making more localized adjustments:

Kohonen-based TSP Solver

It is worth noting that the Kohonen-based solver isn’t perfect, sometimes choosing interim junctions where there is no city, but it is an interesting, reasonably practical use of a different kind of Kohonen topology. Continue reading “Kohonen Neural Network Experimentations”

Evolving Cooperative IPD Strategies

As I continue to play with the IPD, I created code to genetically evolve IPD strategies to see if cooperation could be borne out of random behaviours. I used the standard GA I’ve created in Wintermute, with each agent represented by the five weights detailed in my last post and with fitness calculated as the average points earned in each bout (this was subtracted from 5 in order to allow the GA to search for a minimum). I then created a population of 500 agents with a mutation and elitism rate of 0.5% per generation.

It took me a while to tweak the GA to start working, but I finally got it with fascinating results. Here is a chart of the distribution of strategies along with the best fitness for each iteration:

Continue reading “Evolving Cooperative IPD Strategies”

More Experimenting with the Spatial IPD

It seems like my initial experiments have been a bit awry, but this is how you learn! I was tweaking the IPD code to use encoded weightings rather than classic logic to start creating the genetic algorithm code to evolve this behaviour.

I recreated all the strategies as weights using a simple encoding denoting their probability to cooperate. They are encoded as {I, R, S, T, P} where I = initial, R = reward (c,c), S = sucker (c,d), T = temptation (d,c), P = punishment (d,d). As an example, all-cooperate and TFT strategies would be:

AllCooperate = 1.0, 1.0, 1.0, 1.0, 1.0
TFT = 1.0, 1.0, 0.0, 1.0, 0.0

So All-C will always cooperate, from the outset and continue to. TFT will cooperate initially, then cooperate after reward or temptation and defect after sucker or punishment. Continue reading “More Experimenting with the Spatial IPD”

Understanding Cooperation with the Spatial Iterated Prisoner’s Dilemma

Not exactly a snappy way to start my new blog, but…

I just finished reading Critical Mass by Phillip Ball and one of the topics discussed was the iterated prisoner’s dilemma. I had implemented a Spatial IPD as part of the Generation5 JDK and more recently as part of the C# port, ‘Wintermute’.

Prior to reading Critical Mass though, I saw the SIPD as an interesting cellular automata implementation of a famous game theory exercise but never fully experimented with the model’s insights. In his book, Ball detailed how IPD can be used to show how cooperation is evolved as being more beneficial than selfish behaviour. Even more fascinating was the IPD’s ability to show how more cooperative strategies are more beneficial, until you get to a completely cooperative world — at which point, a very selfish strategy can quickly take advantage of the “naivety” of the world.

I decided to expand my code to explore this further to see if I could replicate this.

Continue reading “Understanding Cooperation with the Spatial Iterated Prisoner’s Dilemma”