Sunday, 6 March 2011

Developer Journal 66 - On Setting Neural Network Parameters

I've been searching and searching for biologically plausible ways to set neuron activation threshold levels. I hate how things seem easy until you start doing them.

I could use a random number, but what should the range be? Surely a neuron that has 1000 inputs should have a different threshold to a neuron that has 10 inputs?

I'm reading, It's rather good.

Dendrites collect inputs and pass them to the soma. The soma generates an output if the inputs exceed a threshold. The axon transmits the output to other neurons.

Sending neurons are called pre-synaptic cells and receiving neurons are called post-synaptic cells. Neurons often connect to 10,000 post-synaptic cells.

The other part of the challenge is setting the output of a neuron. How do you make sure the range is between 0 and 1 if you don't know what the range of the input is?

In biological models, the output is 0 or a fixed number, no matter the input.

Neurons have fixed thresholds. If an activation level is high, the output does not increase in size, rather, it increases in frequency.


No comments:

Post a Comment