On 16/11/17 11:30, Osma Suominen wrote:
Rob Vesse kirjoitti 16.11.2017 klo 13:13:
This is by design. As has been discussed in the past tdbloader2
produces maximally packed B+Trees by preprocessing data which will
minimise disk space usage.
[...]
As Andy mentioned on an earlier thread tdb2.tdbloader essentially
has the same behaviour as tdbloader, because of the different data
structures low performance should be much better anyway and he did not
think there would be much benefit to having a tdb2.tdbloader2 variant.
Also given the different Data structures I’m not sure if this would be
as practical.
Right. I was just surprised at how big the difference is.
tdbloader2 is both fast and space-efficient,
but does not run on MS Windows.
that makes it a lot more
appealing than tdb2.tdbloader which in my (very limited) experience is
slow and space-hungry (but similar to tdbloader for TDB1).
But the real surprise was the space overhead of named graphs. More than
twice the space just because I decide to put the data in a named graph
instead of the default graph? And that seems to be the case both for
TDB1 (both tdbloader and tdbloader2) and TDB2.
The tdbloader in TDB2 is a simple. I found that this current simple one
was faster than expected (maybe because it is append-only so disk
friendly or maybe because I was using an SSD for testing).
Rather than wait for that work area to be done, I thought it would be
good to contribute and now release TDB2. It's experimental.
The TDB1 loaders could be ported.
TDB2 goals are to address the scal elimiations on transactions, the
write-back queue overload problems, a better architecture e.g. fully
integrate in jena-text transactions, and no quirks about models across
transactions.
The style of tdbloader2 could be used to compact databases down. Teh
current compaction is a simple one, good for getting a trusted one to work.
Andy
