on 11/3/01 9:02 AM, Joshua Bell at [EMAIL PROTECTED] wrote:
> "Dan Minette" <[EMAIL PROTECTED]> wrote:
>>
>> How are neural networks not algorithmic? Each node has a simple set of
>> rules, a simple algorithm. The timing of them passing information can also
>> be described by an algorithm. Perhaps timing difficulties exist, so the
>> same answer will not be obtained twice with the same input, but that can
>> also be described algorithmically, through the use of a pseudo-random
>> number
>> generator.
>
> True - but only in the most pedantic way. You typically can't write an
> algorithm for what a neural network does that doesn't involve "insert a
> neural network here" as a step. For example, image, voice, or optical
> character recognition. You can train a network to do things that we're thus
> far unable to manually construct algorithms for. It may even be that the
> simplest algorithm for the task (recognize X with some probability of
> success in some limited number of steps) does involve a neural network.
>
> Consider a computer performing addition with a binary adder vs. recognizing
> a square with a neural network. We can describe the algorithm of the former
> rather simply. What algorithm is the latter using? Can you describe it in
> any simpler terms other than as a box containing the entire process itself?
I think there is some confusion about the usage of the word 'algorithm'. It
can be used to mean 'effective process' as in the Church-Turing Thesis
(every effective process is implementable by running a suitable program on a
UTM), or it can be used more loosely to mean any set of instructions.
An effective process is essentially a proof sequence and is logically
constrained by Godel's Theorem (with implications that can be argued about.)
But a heuristic 'algorithm' is a good-guesser, not an effective process. It
doesn't produce proof sequences it gives answers with some (high)
probability of being correct.
The program that simulates the neural network is an effective process for
simulating a neural network *but the neural network itself is not an
effective process - it is a heuristic guessing system*
There is nothing to stop a computer program having intuition (good guessing)
and generating interesting mathematical ideas like Goldbach's Conjecture,
but any computer program faces exactly the same problem as any human
mathematician in *proving* something (producing a proof sequence.)
It depends whether you want logical certainty or just very high
probabilities. Very high probabilities are often good enough.
For example, when finding very large prime numbers for use in public key
crypto the software doesn't prove that the numbers are prime, it just tests
them enough (using something like Miller-Rabin) that the possibility of them
not being prime is too small to worry about.
--
William T Goodall
[EMAIL PROTECTED]
http://www.wtgab.demon.co.uk
