On 23/07/17 11:28, Andy Seaborne wrote:
On 22/07/17 21:23, Dave Reynolds wrote:
On 22/07/17 20:51, Andy Seaborne wrote:
On 22/07/17 18:03, Andy Seaborne wrote:
On 22/07/17 12:10, Dave Reynolds wrote:
# Summary
We've started seeing an unusual JVM error message in some fuseki
deployments:
java.lang.InternalError: a fault occurred in a recent unsafe memory
access operation in compiled Java code
I don't think Jena itself does any Java unsafe operations.
"The web" seems to think that mmap files, and running out of disk
space can cause this. Or running out of shared mapped space.
Thanks Andy. Yes, that was my conclusion too but as I say there's
plenty of disk and occurring on two machines simultaneously seems to
rule out bad blocks. Not sure what I can do about management of
available mapped space, will consult the web further.
Dave
What was in the stack trace?
It's random, which is one reason I'm sure this isn't a Jena bug.
Examples are:
java.lang.InternalError: a fault occurred in a recent unsafe memory
access operation in compiled Java code
at java.nio.MappedByteBuffer.<init>(MappedByteBuffer.java:89)
at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:195)
at java.nio.DirectByteBuffer.slice(DirectByteBuffer.java:213)
at
org.apache.jena.tdb.base.recordbuffer.RecordBufferPageBase.reset(RecordBufferPageBase.java:82)
at
org.apache.jena.tdb.base.recordbuffer.RecordBufferPageBase.<init>(RecordBufferPageBase.java:74)
at
org.apache.jena.tdb.base.recordbuffer.RecordBufferPage.<init>(RecordBufferPage.java:87)
at
org.apache.jena.tdb.base.recordbuffer.RecordBufferPage.format(RecordBufferPage.java:81)
at
org.apache.jena.tdb.base.recordbuffer.RecordBufferPageMgr$Block2RecordBufferPage.fromBlock(RecordBufferPageMgr.java:82)
at
org.apache.jena.tdb.index.bplustree.BPTreeRecordsMgr$Block2BPTreeRecords.fromBlock(BPTreeRecordsMgr.java:59)
at
org.apache.jena.tdb.index.bplustree.BPTreeRecordsMgr$Block2BPTreeRecords.fromBlock(BPTreeRecordsMgr.java:45)
at
org.apache.jena.tdb.base.page.PageBlockMgr.getRead(PageBlockMgr.java:68)
at
org.apache.jena.tdb.index.bplustree.BPTreeNode.getMgrRead(BPTreeNode.java:163)
...
java.lang.InternalError: a fault occurred in a recent unsafe memory
access operation in compiled Java code
at java.nio.Buffer.checkIndex(Buffer.java:539)
at java.nio.DirectByteBuffer.get(DirectByteBuffer.java:253)
at
org.apache.jena.tdb.base.buffer.RecordBuffer.compare(RecordBuffer.java:196)
at
org.apache.jena.tdb.base.buffer.RecordBuffer.find(RecordBuffer.java:172)
at
org.apache.jena.tdb.base.buffer.RecordBuffer.find(RecordBuffer.java:133)
at
org.apache.jena.tdb.index.bplustree.BPTreeRecords.internalSearch(BPTreeRecords.java:93)
at
org.apache.jena.tdb.index.bplustree.BPTreeNode.internalSearch(BPTreeNode.java:407)
...
java.lang.InternalError: a fault occurred in a recent unsafe memory
access operation in compiled Java code
at java.lang.StringCoding$StringEncoder.encode(StringCoding.java:304)
at java.lang.StringCoding.encode(StringCoding.java:344)
at java.lang.String.getBytes(String.java:918)
at org.apache.jena.atlas.lib.Bytes.string2bytes(Bytes.java:238)
at org.apache.jena.tdb.lib.NodeLib.hash(NodeLib.java:181)
at org.apache.jena.tdb.lib.NodeLib.setHash(NodeLib.java:122)
at
org.apache.jena.tdb.store.nodetable.NodeTableNative.accessIndex(NodeTableNative.java:132)
...
The last is not even in NIO code. Though it does seem to be the heavy
use of NIO that triggers it, most other reports of this error I've found
are in database systems that also make heavy use of NIO.
And yes, the fact I've a variety of stacktraces to choose from is
because problem is now recurrent. Though at the moment I don't have a
more reliable recipe to recreate it other than "one day of normal
workload followed but an overnight SPARQL Update followed by a large
query". I'm working on finding some workable substitute for the "one day
of normal workload" bit before I try experimenting with some possible
workarounds.
Dave
