Introduction to Artificial Neural Nets 1

The simplest model of a neuron – the perceptron

In this picture we can see a neuron with the dendrites, cell body and axon labelled. The dendrites receive information from other neurons via chemical messengers (neurotransmitters), this information is converted to an electrical charge which accumulates in the cell body and which, if it exceeds a certain threshold charge, will send a pulse (actually, a series of pulses) down the axon. This pulse is the output message from this neuron, and when it reaches the end of the axon, it is converted to chemical messengers to be sent to the dendrites of other neurons, so the process repeats.

This whole process is easy to model on a computer. The following two diagrams show how this is done.

This picture is the same as the first we saw, but I've stripped away most of the dendrites to leave just two. This is not a necessity – neurons with any number of dendrites (inputs) can be modelled, but just to keep it simple let's start with two. The two dendrites are labelled x and y. The cell body is still present, and still sums the inputs of the two dendrites, and the axon is still there to send the information on if the threshold is exceeded.

Drawing this in a more formal manner, we get the diagram on the right. All the information in the above picture is included in this one too.

The Σ sign represents the summation of x + y in the cell body.

Ok, so we have a simple model of a neuron (called a perceptron now that it's on our computer). So what is it capable of doing? Well, we can answer this question just by thinking about what comes out of the axon; the output. We only have one neuron, so one axon, and therefore one output. In the case of a biological neuron, it's either firing, or it isn't. We can represent this as either 1 (when it fires) or 0 (when it doesn't). So there's our answer; the perceptron is capable of taking it's inputs, summing them together and categorising them into one of two groups depending on whether the sum is greater than the threshold or not.

Our model does not actually do this yet though. At the moment it just sums the inputs together and tells us what that sum is. So if the x input is 0.5 and the y input is 0.7, the axon will give an output of 1.2. We need to add the threshold function so that it outputs only 1 or 0 (which could mean 'yes' or 'no', 'cat' or 'dog', or any pair of categories). The following diagram includes a threshold function:

The dashed line is the threshold. We haven't yet determined what the value of the threshold is. In a normal biological neuron the threshold is about 15mV (it rests at about -70mV and fires at about -55mV, so the threshold is the difference between these). For our model, let's keep things simple and set the threshold as 1. Therefore if the sum of x + y equals or exceeds 1, the output from the axon will be 1. If the sum is less than 1, the output will be 0.

Right, now we have everything we need to give a little demonstration of the perceptron.

It's going to be a simple demonstration. We want to know when both of the inputs into the neuron are active, so if both x and y receive a message from other neurons, then we want our neuron also to send a message by outputting a 1. On the other hand, if only one of our inputs is receiving a message, or if neither is, then we don't want our neuron to fire, and so a 0 should be outputted. We can use the input values that we set before: the input to x = 0.5 and the input to y = 0.7.

The above diagram shows the case when the inputs to both dendrite x and y are active. The sum exceeds the threshold value, and thus the perceptron outputs a 1, as we wanted.

The three other possible cases are shown in the three diagrams below; when there is an input to x, but not to y, when there's an input to y, but not to x, and when there is no input at all. In all these cases the sum in the cell body does not equal or exceed the threshold value of 1, so the output of the perceptron is 0, equivalent to a neuron not firing. The perceptron does what we have asked it to – it tells us when both of its inputs are active by outputting a 1 as opposed to a 0 in any other case.

Well, if you're anything like me, at this point you'll be thoroughly unimpressed by what a perceptron does! It's really no more than adding a few numbers together and then saying if they are more or less than a given threshold value. This can all be written a lot more concisely in mathematical form:

If sum(inputs) >= 1, then output = 1

else output = 0

So what exactly is the use of the perceptron?

I'll provide three answers to this question:

1. The perceptron above doesn't actually learn anything. Things don't get interesting with neural networks until you get them learning, so hold on in there!
   
2. A neural network is able to generalise from the things they know to things they've never seen before. They can make educated guesses!
3. Nervous systems, including brains, are stunningly good at processing information. They are not only responsible for (almost) everything we do, but actually who we are. They give rise to consciousness. Yet all of these amazing functions are based upon little processing units (more or less) like the perceptron we described above and how they work together.