On Tue, Aug 25, 2009 at 11:59 AM, Ted Dunning<[email protected]> wrote:
> On Tue, Aug 25, 2009 at 2:57 AM, Steve Loughran <[email protected]> wrote:
>
>> On petascale-level computers, the application codes' CPU instructions are
>>> about 10% floating point (that is, in scientific applications, there are
>>> less floating point instructions than in most floating point benchmarks).
>>> Of the remaining instructions, about 1/3 are memory-related and 2/3 are
>>> integer. Of the integer instructions, 40% are computing memory locations.
>>>
>>
>> cool. I wonder what percentage is vtable/function lookup in OO code versus
>> data retrieval? After all, every time you read or write an instance field in
>> a class instance, there is the this+offset maths before the actual
>> retrieval, though there is a fair amount of CPU support for such offset
>> operations
>
>
> Since the codes running on these machines tend to be matrix oriented
> Fortran, I would expect almost all of this is array index computation.
>
>
>>
>>
>>> So, on the biggest DOE computers, about 50% of the CPU time is spent on
>>> memory-related computations. I found this pretty mind-boggling when I
>>> learned this. It seems to me that the "central" part of the computer is
>>> becoming the bus, not the CPU.
>>>
>>
>> welcome to the new bottlenecks.
>>
>
> Comm ACM had a recent article in which the author measured (informally) the
> throughput available for Disk, SSD and main memory with sequential and
> random access patterns. Sequential disk was slightly faster than random
> access to main memory.
>
> --
> Ted Dunning, CTO
> DeepDyve
>
Warning!
If you decide to switch to 1.6.0u15 with 18.3 you are going to get a
task tracker crash. An odd silent error that appears in logs but does
not manifest itself in a failed status with hadoop. Logs will show:
Exception in thread "main" java.lang.NoClassDefFoundError:
Could_not_reserve_enough_space_for_object_heap
Caused by: java.lang.ClassNotFoundException:
Could_not_reserve_enough_space_for_object_heap
at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320)
Could not find the main class:
Could_not_reserve_enough_space_for_object_heap. Program will exit.
Issue is : HADOOP-5564
Fix is here:
http://www.koopman.me/2009/04/hadoop-0183-could-not-create-the-java-virtual-machine/
