On 18/03/16 21:27, Stephen Allen wrote:
You can use the off-JVM memory that Andy mentions by changing the "mapped"
to "direct" in your config file. That is similar to using a memory mapped
file, except that you are limited by the amount of memory that you have
(but if you have enough virtual memory, then there should be no problem).
That first setting is only for TDB's storage of unwritten blocks. But when
you do large updates, Jena needs to temporarily store all of the tuples
generated by the WHERE clause in memory before applying them in the update.
This is where the spillToDisk comes in, it serializes those temporary
tuples on disk in a regular file instead of holding them in an in-memory
array. That file is not memory mapped, so there should be no problem with
removing it after the update is complete.
So basically, if "direct" works for you, then go with that (or use a
different OS like Linux for the memory mapped approach).
-Stephen
Stephen,
(I got confused by the multiple use of "direct")
Would it make sense to implement the spill space using a BlockMgr? We
have several implementations of BlockMgr including one that uses
traditional file I/O as opposed to memory mapped files.
Andy
