On 23/03/13 15:49, Marco Neumann wrote:
this is not important at the moment but I have tested the tdbloader2
on several different configurations from small virtual machines with
1gb ram to dedicated machines with 8 cores and 128gb ram.
Part of tdblaoder2 are calls to unix sort(1). Only some version have
the --parallel flags so the default setup is single threaded :-(
export SORT_ARGS="--buffer-size=50% --parallel=3"
may do better on the 8 core machine.
Is it faster than tdbloader? I find it makes a difference only for
datasets (> 100e6 triples).
there does not much seem to be much of a difference between the tuples
per seconds in load time. is tdbloader2 throttled to a certain number?
No.
I can see that it takes 50k triples at a time is that configurable?
Not sure what that is - the monitor may report every 50K but it's just a
regular tick point in the stream being loaded.
Andy
