I've wrote a Java app that loaded millions of records, ~2.5GB on disk, into memory, and ready to be analyzed in about 3-4 seconds on my laptop on a warm run. Cold runs took a little longer, I forget the exact time, but it was in the 5-12 second range. The main trick was storing my data by columns rather than by rows. Using the right NIO API you can read massive arrays of primitives (int[], long[], float[]) directly from disk into memory without any Java loops. I imagine this was completely bound by OS level I/O operations. Conceptually, I don't see how you'd squeeze more performance out of this with C-style casting and memory copies. My point is that sometimes you can find high performance tricks even in Java to do the types of tasks you are talking about.
Memory use is a big issue on consumer devices. The JVM may have problems in this area; obviously server applications can generally afford to spare much more RAM, and maybe the JVM has been heavily tuned for that. But nevertheless, I don't think there are any technical barriers that would stop the JVM from having a competitive memory footprint. It seems like there is a decided lack of enthusiasm about Java and the JVM ecosystem. People are just searching for reasons why it isn't good. Does anyone want to see it improve and broaden it's usefulness? On Jan 18, 2:54 pm, Josh Berry <[email protected]> wrote: > One thing you need to be careful with on these benchmarks is memory > usage. Especially targeting the ps3, memory is not exactly free. > Same for throughput of data if you require anything off of the disc. > So, how long does it take to read a meg off the disk and process it as > program data? > > In other words, these benchmarks are likely completely irrelevant. > > Also, you keep looking at stack allocation and unsigned types. I > think the more important "unsafe" trick is the ability to memcpy into > a struct. The unsigned part is one thing, but actually being able to > say the next "5k bytes from the disc is an array of this struct" is > huge. Well.... I say that. It sounds huge. I could definitely be > wrong. :) -- You received this message because you are subscribed to the Google Groups "The Java Posse" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/javaposse?hl=en.
