tdbloader2 is around 20% faster than tdbloader. there was a marginal improvement on the 8 core machine, but as far as I can tell negligent.
i am currently looking at a small data set <100M . need to test again with larger sets. Marco On Mon, Mar 25, 2013 at 5:30 PM, Andy Seaborne <[email protected]> wrote: > On 23/03/13 15:49, Marco Neumann wrote: >> >> this is not important at the moment but I have tested the tdbloader2 >> on several different configurations from small virtual machines with >> 1gb ram to dedicated machines with 8 cores and 128gb ram. > > > Part of tdblaoder2 are calls to unix sort(1). Only some version have the > --parallel flags so the default setup is single threaded :-( > > export SORT_ARGS="--buffer-size=50% --parallel=3" > > may do better on the 8 core machine. > > Is it faster than tdbloader? I find it makes a difference only for datasets > (> 100e6 triples). > > >> there does not much seem to be much of a difference between the tuples >> per seconds in load time. is tdbloader2 throttled to a certain number? > > > No. > > >> I can see that it takes 50k triples at a time is that configurable? > > > Not sure what that is - the monitor may report every 50K but it's just a > regular tick point in the stream being loaded. > > Andy > > > -- --- Marco Neumann KONA
