Week 2

I. Goals

Final program will consist of, at minimum:
- The neural network engine, responsible for simulating the networks. It may be possible to use public domain class libraries to reduce coding effort; one such package is available here.
- The genetic algorithm, responsible for breeding networks, determining which ones survive into the next generation, etc. There is a fairly impressive C++ genetic algorithms library (freeware) available here.
- A suite of functions to visualize individual neural networks.
- Routines to visualize the evolution of the networks.
- GUI code to hold everything together.
This tool could be used (a) by professionals to help evolve neural networks to solve specific problems, and (b) by students to get an intuitive understanding of how neural networks and the genetic algorithm work.

Ability to load networks created by other programs (f.e. EasyNN, QwikNet, etc.)
Ability to "play God" by selecting which neural networks breed together, modifying the edge weights or topology of existing networks, etc.
Ability to view the family tree of a neural network, and possibly to animate the transitions between generations.

In Neural Networks, Volume 8, Number 6, the article "Radial Basis Function Network Configuration Using Genetic Algorithms" (pp. 877-890) described a genetic algorithm used to solve a fluid dynamics problem. The number of nodes was allowed to vary, and they found that successful networks tended to have between 15 and 60 nodes.
The screen shots below illustrate the appearance of my prototype interface with 15, 30, and 60 nodes.

15 nodes:

30 nodes:

60 nodes:

Clearly, the presentation of the edge weights becomes questionable at 30 nodes, and absolutely incomprehensible at 60.

The number of edges is (n-1)² for a graph with n nodes. This quadratic expansion will make it difficult for any visualization system to represent the edge weights.
The clutter in the pictures above could be reduced by using line-drawing routines that add to, rather than overwrite, the pixels underneath. It will also help when opacity is used instead of brightness.
Presentation of edge weights could be reserved for detail view of each node. In the normal view, you could just have some indication of the total weight of the edges going into and out of each node, to make it easy to hone in on the important nodes.

List of public domain neural networks tools available here.
I have found a very good paper entitled "Evolutionary Design of Neural Architectures - A Preliminary Taxonomy and Guide to Literature" which provides an overview of most of the issues involved. A PostScript version is available here. Among the points I noted are:
- Evolution of network topology seems to be taken as a given; the edge weights are sometimes evolved, sometimes determined by back propagation.
- The article states that "Despite much research activity in this area . . . the design of artificial neural networks (ANNs) for specific applications . . . is, to a large extent, a process of trial and error, relying mostly on past experience with similar applications." This seems to support the need for a visualization tool.
- Neural networks are not always fully connected, as I had thought; in fact a connectivity matrix is one of the encoding schemes for topology. This may alleviate, to some extent, the scaling problems mentioned above.
The article has a 300+ entry guide to relevant papers, which should greatly ease further research in this area.