A few weeks ago I updated my Kohonen neural network code to support circular rows and columns as well as some simple additional visualizations which allowed for some interesting experimentation.
Circular rows and columns follow a simple premise — the neighbourhood influence effect of training can now wrap around both row and column, allowing for circular, cylindrical and torodial geometries.
Think of it like this, a network with a single row wrapped around allows for a network to learn a circular topology. A nice, simple example of this is the Travelling Salesman Problem as you need one continuous, circular route to be determined. Below there is an example of a Kohonen network run over 350 iterations. The initial phase starts to spread the nodes out over the map, with the second phase making more localized adjustments:
Kohonen-based TSP Solver
It is worth noting that the Kohonen-based solver isn’t perfect, sometimes choosing interim junctions where there is no city, but it is an interesting, reasonably practical use of a different kind of Kohonen topology.
Next up was updating the visualizations to include a unified distance matrix (or U-Matrix). This is effectively a fancy way of saying to plot the Eucledian distances between the weights at each network node. As Kohonen networks are great at compressing multi-dimensional inputs, the U-Matrix is a great way to visualize how the network is spread out.
Below are two screenshots of the time-honoured Kohonen network learning random RGB values, along with the same network but visualized using a U-Matrix:
Kohonen RGB (top), U-Matrix (bottom)
The U-Matrix is best seen in motion, as you can see the influence of the circular neighbourhoods gradually reducing as well as the difference the different phases affect the learning weights. It also helps visualize the peaks that occur in weights, highlighting specialized nodes (or over-training).
I’ve still not got around to rewriting my image recognition and processing code that allowed me to do some great stuff (like this and this!). C# image processing seems painfully slow outside of unmanaged code.
One thought on “Kohonen Neural Network Experimentations”
Hi James,
Just wanna say thank you for the code and also ask if you have anything on using KOHONEN for climate modelling?
Hi James,
Just wanna say thank you for the code and also ask if you have anything on using KOHONEN for climate modelling?