Sunday 6 March 2011

Developer Journal 66 - On Setting Neural Network Parameters

I've been searching and searching for biologically plausible ways to set neuron activation threshold levels. I hate how things seem easy until you start doing them.

I could use a random number, but what should the range be? Surely a neuron that has 1000 inputs should have a different threshold to a neuron that has 10 inputs?

I'm reading, http://nba.uth.tmc.edu/neuroscience/s1/index.htm. It's rather good.

Dendrites collect inputs and pass them to the soma. The soma generates an output if the inputs exceed a threshold. The axon transmits the output to other neurons.

Sending neurons are called pre-synaptic cells and receiving neurons are called post-synaptic cells. Neurons often connect to 10,000 post-synaptic cells.

The other part of the challenge is setting the output of a neuron. How do you make sure the range is between 0 and 1 if you don't know what the range of the input is?

In biological models, the output is 0 or a fixed number, no matter the input.

Neurons have fixed thresholds. If an activation level is high, the output does not increase in size, rather, it increases in frequency.

RELATED

No comments:

Post a Comment