Denis Cousineau home page

Me

The PDF documents on this site were made with PrimoPDF.
Get a reader free.

The Notebook documents on this site were made with Mathematica.
Get the reader free.

Pages © Denis Cousineau;
last modified: December 17th, 2009;
page consulted on

The parallel race model

The parallel race model (PRM) is a different kind of neural network.

It is based on a feed-forward architecture similar to the one used in Perceptron. However, the output units have a different kind of integration rule: it fires when its "accumulator is full". The following Figure illustrate a simple PRN. In addition, it is possible to add time-out units, units that are activated after some time has passed.

Although very simple, this network is capable of learning to solve a XOR, thus being a good alternative to standard feed-forward neural network learning with the delta-rule. In fact, it does learn a XOR very rapidly (about a hundred presentations), and does so in the presence of noise and in the presence of redundancy.

Summary of this network

In this web site, we describe the Parallel Race Network, a race model with the ability to simulate cognition autonomously using a formal framework that is identical to the one used by the traditional connectionist networks. The Parallel Race Network assumes that the connections represent abstract units of time rather than strengths of association. Consequently, the connections in the network indicate how quickly the information should be sent to an output unit. The decision is based on a race between the outputs. To make learning functional and autonomous, the Delta rule was modified to fit the time-based assumption of the Parallel Race Network.

Position of the Parallel Race Network

The Parallel Race Network (PRN) is a descendant of the sampling theories often used in psychology. Its main assumption is that the external world is sampled through the senses, but that this process does not occur at predictable moments when a stimulus is presented due to noise, i.e. uncertainty in the reception of information. A decision is made as soon as enough evidence in favor of a response is received. All the possible responses are continuously in competition so that the fastest wins the race. This is often called a Winner-Takes-All process.

When generalizing this class of models, we considered the case where the race is the end product of a full matrix of connections between the senses and the responses. The connections have an important role in deciding what is important and what is not, so that irrelevant information will never reach an accumulator. Further, we explored the possibility that the connections learn the diagnostic value of the inputs by stimulus-response exposure. Thus, PRN determines autonomously what important information is and what is not, resulting in a network that can reproduce many aspect of human cognition.

A "cousin" that shares many features of the PRN is the Perceptron. It is also based on a network of connections between the inputs and the response, and it also has a learning mechanism. Here end the similarities. The PRN adjusts delays so that high-priority information is transmitted immediately. As such, PRN is the first of the time-based network family of models.

Architecture

The architecture of the PRN is a feed-forward network where all the inputs are connected to all the outputs. The inputs code the moment at which an input became available (a time between 0: immediately and infinity: never). Likewise, the connections code the moment at which an input should send its activation to a response (a delay between 0: right now and infinity: never). The decision is done by looking at the fastest inputs that filled the accumulator (thus, it is also a race to fill each accumulator). The learning rule is supervised so that the true solution to each stimulus must be available for feedback.

The following documents explains the mechanisms behind PRM

The original report

Merging Race Models and Adaptive Networks: A Race Network

Subsequent publications

Redefining the rules: Providing race models with a connectionist learning rule

Learning of an XOR problem in the presence of noise and redundancy

A race model of perceptual forced choice reaction time

The source files

In addition, I performed many simulations using a program developed with Mathematica. The source files are provided below:

Actual simulations

The following notebooks explains some of the key elements of the network, and how to use it:

References

Cousineau, D. (2004) Merging Race Models and Adaptive Networks: A Race Network. Psychonomics Bulletin & Review.

Cousineau, D., Lacroix, G., & Hélie, S. (2003) Redefining the rules: Providing race models with a connectionist learning rule. Connection Science, 15, 27-43.

Cousineau, D. (2005) Learning of an XOR problem in the presence of noise and redundancy, Proceedings of the International Joint Conference on Neural Networks.

Huber, D. E., & Cousineau, D. (2003) A race model of perceptual forced choice reaction time. Proceedings of the Cognitive Science Society.

(c) 1998, 2003, Denis Cousineau. (c) 2003 Sébastien Hélie for the Java code