Re: [agi] Diminished impact of Moore's Law on AGI due to other bottlenecks

[Shane Legg]

James Rogers wrote:

We just don't need that much raw CPU to get the job done.  The entirety of
our number crunching is integer domain, and just addition/subtraction at
that.  Aside: Since the engine operates on relative values and the dynamic
range of integers on your average machine is quite high (higher than
anything going on in the human brain), there is no reason to NOT use integer
formats, particularly since integer math and manipulation is very fast on
most common hardware.

I had similar thoughts, but when I did some tests on the webmind code
a few years back I was a little surprised to find that floating point
was about as fast as integer math for our application. This seemed to
happen because where you could do some calculation quite directly with
a few floating point operations, you would need more to achieve the same
result with integer math due to extra normalisation operations etc.

I was also surprised to discover that the CPU did double precision
floating point math at about the same speed as single precision floating
point math. I guess it's because a lot of floating point operations are
internally highly parallel and so extra precision don't make much speed
difference?

Anyway, the thing that really did affect performance was the data size
of the numbers being used (whether short, int, long, float, double etc.)
Because we had quite a few RAM cache misses, using a smaller data type
effectively meant that we could have twice as many values in cache at
the same time and each cache miss would bring twice as many new values
into the cache. So it was really the memory bandwidth required by the
size of the data types we were using that slowed things down, not the
time the CPU took to do a calculation on a double precision floating
point number compared to say an a simple int.

I'd always had a bias against using floating point numbers ever
since I used to write code 15 years ago when the CPU's I used
weren't designed for it and it really slowed things down badly.
It's a bit different now however with really fact floating point
cores in CPUs.

Cheers
Shane

-------
To unsubscribe, change your address, or temporarily deactivate your subscription, please go to http://v2.listbox.com/member/?[EMAIL PROTECTED]