That Telegraph dataflow paper is pretty long in the tooth. Certainly several of their claims have little force any more (lack of non-blocking I/O, poor thread performance, no unmap, very expensive synchronization for uncontested locks). It is worth that they did all of their tests on the 1.3 JVM and things have come an enormous way since then.
Certainly, it is worth having opaque contains based on byte arrays, but isn't that pretty much what the NIO byte buffers are there to provide? Wouldn't a virtual tuple type that was nothing more than a byte buffer, type and an offset do almost all of what is proposed here? On Thu, May 14, 2009 at 5:33 PM, Alan Gates <ga...@yahoo-inc.com> wrote: > http://wiki.apache.org/pig/PigMemory > > Alan. >