Andy Seaborne wrote on 6/20/18 6:43 AM:
For a database that does fit in RAM, the actual RAM size is a bit
bigger than the disk size (e.g. nodes are UTF-8 on disk and Java
strings in RAM so x2 bytes + string overheads and this is in-heap).
Yeah, in-heap memory isn't the primary issue here, except that NIO is
apparently using out-of-heap memory to cache everything it puts in the
heap. Crazy.
BufferAllocatorDirect is not used by default - it is used only is
journal spilling to disk for large transactions is enabled which is
not the default. (aside to all: if you are in this situation, please
consider TDB2)
Is TDB2 considered production ready?
on a related note, is fuseki compatible with java 9? I looked through
the release notes in Jira and didn't see anything one way or another,
just the bit about being able to build it with java 9.
...and a separate BufferAllocatorMapped class. I'm not a java
programmer, so it's not simple for me to track what's getting used
where, but the TODO makes me wonder.
Good find - the TODO is misleading though.
Direct ByteBuffers aren't used normally.
My interpretation of the article I posted is that the direct bytebuffers
are preferable to the in-heap ones. Of course, that's just one
article, and I don't know what disadvantages they have. One obvious
one is that there aren't knobs to control the size of the buffers like
there are for the heap.
If you get a chance to dump threads and heap as per Rob's suggestion,
that would be great. I can't see a reason for what you are seeing at
the moment.
I'll send you and Rob some stuff directly, it's pretty big. I was
holding off sending until i had a repeat occurrence.
--
Dan Pritts
ICPSR Computing & Network Services
University of Michigan
<https://www.postbox-inc.com>
