Introduction to Artificial Neural Nets 2

Finishing off the Perceptron

In the last post we laid down the basics of an artificial neuron, but we're not quite finished on our road to the generally used artificial neuron yet. I left you with a single neuron that had two dendrites, each of which could be stimulated to activity or not.

However, if you know about neurons, you will know a few facts that cause problems with this simple model.

1) Any number of pre-synaptic neurons can connect to a dendrite, and our model supposes a one input - one dendrite relationship.

2) Pre-synaptic neurons can also connect to the cell body, and our model supposes they only connect to dendrites.

3) The synapse (connection between the pre-synaptic and post-synaptic neuron) of a biological neuron is capable of changing its strength in order to amplify or attenuate (weaken) the the message from the pre-synaptic neuron.

These facts are all easy to incorporate into our model. Firstly, we need to make a distinction between the inputs to the cell and the synapses that amplify or attenuate the input. To do this, all we need to do is slide the decimals that we used as inputs in our previous model down onto the dentrites. The two pictures below show this:

Now we have a number on each of the dendrites - in this case 0.5 and 0.7, though the number could be anything. We also have an x and a y left as inputs. These variables can take on any value, and are capable of changing from one presentation to the next. So, for example, if x is blue and y is yellow, an input of [1,0] will mean pure blue, [0,1] pure yellow, [1,1] green, and [0,0] no colour at all. These inputs can be presented to the network at any time, just like various sights can be presented to our eye at any time, and the network (or our eye) will recognise them.

The modification above accounts for point 3) in the list I gave. After doing this modification, we can take care of the other two points by a simple semantic change. Instead of thinking of the numbered lines as dendrites, we think of them as synapses, and for every single connection between any pre-synaptic neuron and this post-synaptic neuron, we add a weighted line (a line with a number on saying how strong the connection is). In fact, the only reason that I talked about dendrites at all on the previous post was that the two weighted lines have always looked like dendrites to me, and I find it a good stepping stone between the well-known physical shape of a biological neuron and the abstraction of an artificial neuron.