Some advantages of ANNs

In cognitive science artificial neural networks often come under the banner of 'connectionism' or 'connectionist systems'. This distinguishes them from the symbol processing systems that have traditionally been the foundation of computational work in cog science. It also distinguishes them from ANNs used for purposes other than cognitive science, since ANNs can be applied to all sorts of problems in industry, finance, etc.

Connectionist systems have several advantages. The ones we'll focus on here are graceful degradation and generalisation to data which was not seen in training. Both of these arise because of the distributed nature of the processing in a neural network. To demonstrate these two phenomena, we'll build a simple little network that is able to distinguish fruits based upon their features.

The six green circles are the six feature sensors. The first will detect a value of 1 if the fruit has yellowness, and 0 if it doesn't, the second a value of 1 if the fruit has greenness, and 0 if it doesn't, and so on for the six features listed. All of the six feature sensors are connected to both of the two neurons in the associative layer. The table in the bottom right-hand corner shows the weights for the network. the rows (labelled i) are the two neurons and the columns (labelled j) are the six feature sensors. So, the orange square is the link from feature sensor 1 to neuron 1, and the value of the weight is 0.5 (though you can't see that on the diagram to avoid it looking cluttered), and the green square is the weight between feature sensor 4 and neuron 2. Now, let's put this into Excel...

There's no need to have this network learning, so we can use a similar basic set-up to the Darwinian Grak exercise. However, there are going to be six inputs, so we need to have six weights going into the neuron instead of just two. We also don't need a threshold function here because it's simply going to select which fruit based upon which neuron is more strongly activated. Set up your neuron as such:

The input spots are empty at the moment because the network isn't currently looking at anything. The six weights are set to the values in row one of the table we saw above. Just as in the Darwinian Grak case, the activation is the sum of all inputs*weights, so you will need to add the extra four sensors and weights to the formula. After you've done this, when you double click cell C4 it should bring up the following:

That's neuron 1 set up. Now do the same for neuron 2, bearing in mind that the weights should be the values in row 2 of the table from the first picture.

When it comes to creating the formula for the activation, remember that the inputs to both neurons are the same, so double clicking the cell that contains the activation for neuron 2 should lead to the following display:

And that is our two neuron network finished!

The next step is to create the inputs. I'm going to create a little table below the network to hold these. When we present a fruit to the network, it will be a case of telling the feature sensors to refer to one of the lines of this table. Mine looks like this:

From this table we can see that line 18 is the banana because it stimulates the feature sensors for yellowness, longness, white flesh and thick peelable skin. The line below, line 19, is the apple because it stimulates the feature sensors for greenness, roundness and white flesh.

We can also state at this point that we would like neuron 1 to fire when the network thinks it's a banana that it's looking at, and neuron 2 to fire when it thinks it's an apple it's looking at.

I don't want to have to change all six inputs every time I present a different fruit to the system, so I'm going to automate the presentation process.

• Click on cell E14, and enter a number 1. This will refer to line 1 of the input table we've just created. When we've finished the following few steps, we'll be able to change all the inputs just by changing this to a 2.
• Click on the first of our feature sensors - cell A3.
• Go to the Formulas tabbed menu and click the Insert Function button on the left.
• In the floating window that pops up, select the CHOOSE function (if it's not there, you may need to select 'All' in the drop down category menu).
• In the new floating window that appears for the CHOOSE function, type E14. This means that the feature sensor checks cell E14 to find out which row of the inputs table to look at.
• In the box for value1, type A18. This is the yellowness feature of the banana. In the box for value2, type A19. This is the yellowness feature of the apple. Press OK.
• Open a new CHOOSE box for our second feature sensor - cell A4. This should again reference cell E14, but this time the two values should be B18 and B19, the greenness features of the banana and apple.
• Repeat this process for the other four feature sensors, cells A5, A6, A7 and A8.

Let's check if it works as we intended. Go to cell E14 and change the 1 to a two. Do all the values in our feature sensor cells (A3:A8) change from the banana to the apple? Change E14 back to 1. Do they change back to banana?

That's the network finished. If you present different fruits to the network, it should tell you whether it's an apple or banana by outputting a greater value from either the banana cell (neuron 1 - cell C4) or the apple cell (neuron 2 - cell C11). Just to make it pretty, I like to add a final little bit of code. Click on cell D7 and input the following:

=IF(C4>C11,"banana","apple")

You'll see what this does if you play with the system!

Just in case you have got stuck anywhere along the way, here's a video that shows what's in each of the cells:

Ok, that's the network. Now we can go ahead and demonstrate generalisation and graceful degradation. Generalisation first...

We've seen that the network recognises normal bananas and apples with no problems, but what about an abnormal one. Bananas are green before they become ripe. Let's add an input for a green banana. Modify your input table as such:

You'll also need to update the CHOOSE functions in the six feature sensor cells (A3:A8). Do this by simply adding a comma and the appropriate feature of the green banana to the function. So the CHOOSE function for feature sensor 1 (cell A3) will look like this:

=CHOOSE(E14,A18,A19,A20)

where the final input, A20, is the part I've just added. Do this for all six feature sensors.

Now test the network by changing the value in cell E14 to a 3. What is the output; apple or banana? Check all three inputs again. Is the difference in output values the same between an banana and an apple as between a green banana and an apple? Why or why not?

What features would these things have: a golden delicious apple, a peeled banana, apple sauce?

Design your own inputs for the above and test the outputs. Does the network perform as expected on all of them? Pay attention to the difference between the output values in all cases.

My apple sauce was zero for all of the features except for white flesh and the output was 0.5 from both neurons. This would mean that the network has no idea whether it's an apple or a banana. If you'd never experienced apple sauce before and were unable to smell or taste it, do you think you would be able to tell the difference?

From the above tests, you should gather that this simple little network is able to generalise from the things that it's had experience of before to things it's never experienced by using the knowledge it has about normal apples and bananas. Next onto graceful degradation...

If a system exhibits graceful degradation, it doesn't completely break suddenly because a little bit of it is damaged or because the input to the system is noisy (not perfect). Instead performance decreases a little bit for every little bit of damage or extra noise.

Firstly, we'll simulate a noisy input. In the picture below you can see that I've changed the value of the 'white flesh' feature from a 1 to a 0 (in the orange cell). You could think of this as being caused by a speck on the camera lens, or a cataract on the animal's eye.

Try playing with noisy inputs by changing the values of different features of the banana and apple. You can also try changing the values to values other than one or zero - how about 2? 5? -1? -3? How many features can be noisy before the system becomes unable to distinguish apples and bananas?

Now onto what happens when we damage the system a little bit. In this case, damage refers to tampering with the weights since this is where the information about how to perform is stored. In the picture below you can see that I've changed the weight from the first feature sensor to neuron 1 from a 0.5 to a 0.

How does the network perform if you do this? How much can you change the weight value before the network is unable to perform correctly? How many weights can you change just a little before the network can't perform correctly?

You should find that performance decreases proportionately to the damage you incur on the weights. This is strongly equivalent with the way animals brains are capable of incurring damage without breaking completely: Parkinson's, Alzheimer's, lesions, etc.