Hi Alan, It looks like Apache HBASE also uses FileChannel map and MappedByteBuffer mechanism. The feature proposed by Andrew Dinn and Jonathan Halliday will be very useful for Big Data frameworks as well and help them to use NVM without a need to go to JNI. Copying HBASE experts Anoop and Ram to this thread.
Apache has API layers to overcome the 2GB limitation through MultiByteBuff class: https://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/nio/MultiByteBuff.html https://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/util/ByteBufferArray.html Some example uses of ByteBuffer in HBASE today: https://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/io/hfile/bucket/FileMmapEngine.html https://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/io/ByteBuffInputStream.html https://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/ipc/ServerRpcConnection.ByteBuffByteInput.html Best Regards, Sandhya -----Original Message----- From: Alan Bateman [mailto:alan.bate...@oracle.com] Sent: Monday, September 24, 2018 1:15 AM To: Andrew Dinn <ad...@redhat.com>; core-libs-dev@openjdk.java.net; hotspot compiler <hotspot-compiler-...@openjdk.java.net>; Aundhe, Shirish <shirish.aun...@intel.com>; Dohrmann, Steve <steve.dohrm...@intel.com>; Viswanathan, Sandhya <sandhya.viswanat...@intel.com>; Deshpande, Vivek R <vivek.r.deshpa...@intel.com>; Jonathan Halliday <jonathan.halli...@redhat.com> Subject: Re: RFR: 8207851 JEP Draft: Support ByteBuffer mapped over non-volatile memory On 21/09/2018 16:44, Andrew Dinn wrote: > Hi Alan, > > Thanks for the response and apologies for failing to notice you had > posted it some days ago (doh!). > > Jonathan Halliday has already explained how Red Hat might want to use > this API. Well, what he said, essentially! In particular, this model > provides a way of ensuring that raw byte data is able to be persisted > coherently from Java with the minimal possible overhead. It would be up > to client code above this layer to implement structuring mechanisms for > how those raw bytes get populated with data and to manage any associated > issues regarding atomicity, consistency and isolation (i.e. to provide > the A, C and I of ACID to this API's D). > > The main point of the JEP is to ensure that this such a performant base > capability is available for known important cases where that is needed > such as, for example, a transaction manager or a distributed cache. If > equivalent middleware written in C can use persistent memory to bring > the persistent storage tier nearer to the CPU and, hence, lower data > durability overheads then we really need an equivalently performant > option in Java or risk Java dropping out as a player in those middleware > markets. > > I am glad to hear that other alternatives might be available and would > be happy to consider them. However, I'm not sure that this means this > option is not still desirable, especially if it is orthogonal to those > other alternatives. Most importantly, this one has the advantage that we > know it is ready to use and will provide benefits (we have already > implemented a journaled transaction log over it with promising results > and someone from our messaging team has already been looking into using > it to persist log messages). Indeed, we also know we can use it to > provide a base for supporting all the use cases addressed by Intel's > libpmem and available to C programmers, e.g. a block store, simply by > implementing Java client libraries that provide managed access to the > persistent buffer along the same lines as the Intel C libraries. > > I'm afraid I am not familiar with Panama 'scopes' and 'pointers' so I > can't really compare options here. Can you point me at any info that > explains what those terms mean and how it might be possible to use them > to access off-heap, persistent data. > I'm not questioning the need to support NVM, instead I'm trying to see whether MappedByteBuffer is the right way to expose this in the standard API. Buffers were designed in JSR-51 with specific use-cases in mind but they are problematic for many off-heap cases as they aren't thread safe, are limited to 2GB, lack confinement, only support homogeneous data (no layout support). At the same time, Project Panama (foreign branch in panama/dev) has the potential to provide the right API to work with memory. I see Jonathan's mail where he seems to be using object serialization so the solution on the table works for his use-case but it may not be the right solution for more general multi-threaded access to NVM. There is some interest in seeing whether this part of Project Panama could be advanced to address many of the cases where developers are resorting to using Unsafe today. There would of course need to be some integration with buffers too. There's no concrete proposal/JEP at this time, I'm just pointing out that many of the complaints about buffers that are really cases where it's the wrong API and the real need is something more fundamental. So where does this leave us? If support for persistent memory is added to FileChannel.map as we've been discussing then it may not be too bad as the API surface is small. The API surface is just new map modes and a MappedByteBuffer::isPersistent method. The force method that specify a range is something useful to add to MBB anyway. If (and I hope when) there is support for memory regions or pointers then I could imagine re-visiting this so that there are alternative ways to get a memory region or pointer that is backed by NVM. If the timing were different then I think we'd skip the new map modes and we would be having a different discussion here. An alternative is course to create the mapped buffer via a JDK-specific API as that would be easier to deprecate and remove in the future if needed. I'm interested to see if there is other input on this topic before it gets locked into extending the standard API. -Alan.
