|
|
|
|
|
|
 |
Neural Networks ComponentsArtificial NeuronesArtificial neural networks consist of a number of processing units termed neurones that are connected together. In order to understand the function of these artificial neurones it is important to first consider the characteristics of biological neurones. A biological neurone receives input from its neighbours via electrochemical synapses. The nature of these synapses determines whether inputs are excitatory or inhibitory, and the strength of their effects. Signals are then transmitted along the hair-like dendrites of the neurone until they reach the base of the axon (the axon-hillock). At this point the neurone integrates the effects of its many inputs over time. If a critical threshold is surpassed, the cell fires an action potential that travels along the axon and provides the neurones output. Although clearly an oversimplification the above account provides a working representation that is often used in artificial neural networks. One model proposed by McCulloch and Pitts in 1943 to reproduce neuronal function is known as the Threshold Logic Unit (TLU): signals (action potentials) appear at the neurones synapses. Each synapse has a weight (strength) with excitatory and inhibitory actions modelled using positive and negative values respectively. The product of the signal (x) and weight (w) of each synapse approximate the effect of each signal. The weighted signals are then summed to produce an overall activation (a). For example, consider a three-input unit with weights (1, -0.5, 1), that is w1 = 1, w2 = -0.5 and w3 = 1, and suppose it is presented with signals (1,1,0) so that x1 = 1, x2 = 1 and x3 = 0. Using the above equation the activation is given by:
If this activation exceeds a threshold theta the unit produces a signal y. The presence of an action potential output is denoted by "1" and absence by "0".
Now, it is generally accepted that in real neurones information is encoded in terms of frequency of firing rather than merely the presence or absence of an action potential. There are two approaches that are often used in dealing with this issue. First, artificial neurones may themselves vary the frequency of the occurrence of "1" in a pulse stream. Here, activation is used to determine the probability of firing at a given moment. Alternatively, neurones may output a range of values between 0 and 1 rather than simple binary. These values can be thought of as representing different firing frequencies. In both these methods the way that the activation is converted to the output (either probability of firing or an output value) is termed the "transfer function". Transfer functions that are commonly supported include sine, hyperbolic tangents, or sigmoid. A modification of the TLU was introduced by Grossberg to more explicitly introduce the concept of time. Unlike the TLU, weighted inputs are not simply added to produce an overall activation. Instead inputs are also summated over time to contribute to a rate of change of activation. To avoid confusion with previous notation it is necessary to introduce a further symbol for the weighted sum of inputs:
The rate of change of the activation, da/dt, is then defined by:
The first term gives rise to activation decay and the second
represents input from other neurones. A unit like this is sometimes known
as a leaky integrator. The "water tank" analogy is often used
to illustrate the function of the integrator. Water enters the tank through
a hose at a rate of "s" litres a minute. The depth of the water
is "a" metres. If water were only allowed to enter the tank
the rate of change of depth would be proportional to s, or da/dt = beta
x s where beta is a constant. On the other hand, if water were only allowed
to leave the tank through a drain, the rate of change of water level would
be proportional to the depth or da/dt = -a x alpha where alpha is a constant.
To return to our artificial neurone, the result of a leaky integrator
is that immediate withdrawal of an input causes a gradual reduction in
the activation. This gradual rather than immediate fall in the activation
allows multiple sequential inputs to build on each other until the threshold
is surpassed. Index | Components | Artificial Neurones
References: W. McCulloch and W. Pitts. A logical calculus of the ideas immanent in nervous activity. Bulletin of Mathematical Biophysics, 7:115 - 133, 1943 Grossberg, S. Adaptive pattern classification and universal recoding: 1. parallel development and coding of neural feature detectors. Biological Cybernetics, 23:121 - 134, 1976 © 2008 Marcus bros
|
|
|
|
