|
|
|
|
|
|
 |
Unsupervised Learning in Neural NetworksSelf-organising mapsSensory systems in biology are often organised in such a way that stimuli that are close to each other in their properties often activate cells that are themselves close to each other. A good example of this is the visual system. At a very basic level objects that are spatially adjacent in the visual world activate photoreceptors that are also adjacent to each other, and these in turn activate neurones with the cortical visual system. Hubel and Weisel (1962) demonstrated electro physiologically that this is also true in a more abstract sense. They found that in visual area 1 of the mammalian brain neurones were tunes to the "orientation" of a stimulus. That is, if a grating of alternate dark and light lines is presented, the cells respond most strongly when they are orientated in a particular way. The researchers found that cells in the cortex were organised topographically with each adjacent cell having a slightly different preferred orientation. Although many of the systems described so far would be able to learn to recognise certain orientations they are not necessarily organised topographically. It is possible to train networks based on competition that are able to create such maps automatically. This was initially shown by C. von der Malsburg in 1973 but was popularised by Kohonen in 1982 and later in his book in 1984. He proposed networks consisting of a layer of nodes each of which is connected to all inputs and connected to some of their surrounding nodes. Input vectors or patterns are presented and the node k whose weight vector is closest to the input is selected. Although the network could use competitive dynamics to perform this task, Kohonen postulates that this is done with a supervisory engine (the justification being that the network could have done it, but would have taken longer and been "messier"). In fact, the rule Kohonen uses in his book is simply to look for the node that has the smallest value for the length of the vector difference. After a node is selected its weights are trained along with other nodes in its vicinity to closer resemble the input. This process is repeated over several cycles with the size of the neighbourhood falling over time allowing for the net to distinguish between progressively finer details. As the size of the neighbourhood falls over time, the learning rate also falls ensuring "stability" so that the network doesn't forget previous work. Eventually, after learning, weights will be organised in such a way that topographically close nodes are sensitive to inputs that are physically similar. In addition to the above model, many other researchers have
proposed competitive nets. One of the most important is Stephen Grossberg
who created the Adaptive Resonance Theory (ART). ASRT is concerned with
the development of networks in which the number of neurones organise data
is not assigned but is determined by the nets sensitivity to detail within
the data set (the "vigilance" parameter). Another significant
contribution is by Fukushima (1975) who developed a multilayer net called
the "neocognitron" which recognises characters and is based
on early processing in the visual system. The structure is complex and
is a good example of a large network model that is based on its biological
counterpart. Previous | Next | Page 1 2 3 4 Index | Unsupervised Learning | Self-organising maps References: Hubel, D. and Wiesel, T. Receptive fields, binocular interaction and functional architecture in the cat's visual cortex. Journal of Physiology, 160:106 - 154, 1962. von der Malsburg, C. Self-organization of orientation sensitive cells in the striate cortex. Kybernetik, 14:85 - 100, 1973. Kohonen, T. Self-organized formation of topologically correct feature maps. Biological Cybernetics, 43:59 - 69, 1982. Kohonen, T. Self-organization and associative memory. Springer Verlag. 1984 Grossberg, S. Competitive learning: from interactive activation to adaptive resonance. Cognitive Science, 11:23 - 63, 1987. Fukushima, K. Cognitron: a self-organizing multilayered neural network. Biological Cybernetics, 20:121 - 136, 1975. © 2008 Marcus bros
|
|
|
|
