On 20/06/18 16:04, Dan Pritts wrote:
Andy Seaborne wrote on 6/20/18 6:43 AM:
For a database that does fit in RAM, the actual RAM size is a bit
bigger than the disk size (e.g. nodes are UTF-8 on disk and Java
strings in RAM so x2 bytes + string overheads and this is in-heap).
Yeah, in-heap memory isn't the primary issue here, except that NIO is
apparently using out-of-heap memory to cache everything it puts in the
heap. Crazy.
BufferAllocatorDirect is not used by default - it is used only is
journal spilling to disk for large transactions is enabled which is
not the default. (aside to all: if you are in this situation, please
consider TDB2)
Is TDB2 considered production ready?
Yes - although it is less used than TDB1 due to being younger.
(TDB1 remains the better choice for many small updates)
on a related note, is fuseki compatible with java 9? I looked through
the release notes in Jira and didn't see anything one way or another,
just the bit about being able to build it with java 9.
Should be.
What does not work is building with Java9+ and running on Java8 even
with a target of Java8. The JDK runtime library for ByteBuffers has
changed some method return signatures.
It is not specific to Jena:
https://stackoverflow.com/questions/48693695/java-nio-buffer-not-loading-clear-method-on-runtime
https://jira.mongodb.org/browse/JAVA-2559
...and a separate BufferAllocatorMapped class. I'm not a java
programmer, so it's not simple for me to track what's getting used
where, but the TODO makes me wonder.
Good find - the TODO is misleading though.
Direct ByteBuffers aren't used normally.
My interpretation of the article I posted is that the direct bytebuffers
are preferable to the in-heap ones.
pros and cons.
They are faster but the optimizer gets better and better at avoiding the
overheads.
They require more management - they need to be freed etc because they
are not in the heap (they are malloc space).
The memory mapped files are like "direct" byte buffers : same lower
overheads and also no copy from file system cache to into the JVM.
Of course, that's just one
article, and I don't know what disadvantages they have. One obvious
one is that there aren't knobs to control the size of the buffers like
there are for the heap.
If you get a chance to dump threads and heap as per Rob's suggestion,
that would be great. I can't see a reason for what you are seeing at
the moment.
I'll send you and Rob some stuff directly, it's pretty big. I was
holding off sending until i had a repeat occurrence.
