Re: WG: Declining TDB load performance with larger files

2013-11-07 Thread Andy Seaborne
@jena.apache.org Betreff: AW: Declining TDB load performance with larger files I did a comparison of tdbloader vs. tdbloader2. The results are not relieable (machine-dependent, and perhaps even influenced by different background load on the vm cluster), but perhaps even then they may be interesting

AW: AW: Declining TDB load performance with larger files

2013-11-04 Thread Neubert Joachim
Nachricht- Von: Andy Seaborne [mailto:a...@apache.org] Gesendet: Samstag, 2. November 2013 13:40 An: users@jena.apache.org Betreff: Re: AW: Declining TDB load performance with larger files On 01/11/13 12:42, Neubert Joachim wrote: I did a comparison of tdbloader vs. tdbloader2. The results

WG: Declining TDB load performance with larger files

2013-11-04 Thread Neubert Joachim
: Freitag, 1. November 2013 13:43 An: users@jena.apache.org Betreff: AW: Declining TDB load performance with larger files I did a comparison of tdbloader vs. tdbloader2. The results are not relieable (machine-dependent, and perhaps even influenced by different background load on the vm cluster

AW: AW: Declining TDB load performance with larger files

2013-11-03 Thread Neubert Joachim
datasets. Cheers, Joachim -Ursprüngliche Nachricht- Von: Andy Seaborne [mailto:a...@apache.org] Gesendet: Samstag, 2. November 2013 13:40 An: users@jena.apache.org Betreff: Re: AW: Declining TDB load performance with larger files On 01/11/13 12:42, Neubert Joachim wrote: I did

Re: AW: Declining TDB load performance with larger files

2013-11-02 Thread Andy Seaborne
factors are. But for a given setup, more RAM is good especially for the all java tdbloader. Andy Cheers, Joachim -Ursprüngliche Nachricht- Von: Andy Seaborne [mailto:a...@apache.org] Gesendet: Montag, 28. Oktober 2013 16:58 An: users@jena.apache.org Betreff: Re: Declining TDB load

AW: Declining TDB load performance with larger files

2013-11-01 Thread Neubert Joachim
: Re: Declining TDB load performance with larger files Hi Joachim, What is happing is that the system is running out of working space and the disk is being used for real. JAVA_OPTS: -d64 -Xms6g -Xmx10g Don't set -Xmx10g. Try a 2G heap. Don't bother with -Xms. More heap does not help

Declining TDB load performance with larger files

2013-10-28 Thread Neubert Joachim
I'm loading a 111 million triples file (GND German Authority files). For the first roughly 70 million triples, it's really fast (more than 60,000 avg), but then throughput declines continuously to a thousand or just some hundred triples (which brings down the avg to less than 7000). During the

Re: Declining TDB load performance with larger files

2013-10-28 Thread Andy Seaborne
Hi Joachim, What is happing is that the system is running out of working space and the disk is being used for real. JAVA_OPTS: -d64 -Xms6g -Xmx10g Don't set -Xmx10g. Try a 2G heap. Don't bother with -Xms. More heap does not help - in fact, it can make it worse. TDB uses memory mapped

AW: Declining TDB load performance with larger files

2013-10-28 Thread Neubert Joachim
Hi Andy, Thank you so much. I fixed the heap issue, and will check and report my findings. Cheers, Joachim -Ursprüngliche Nachricht- Von: Andy Seaborne [mailto:a...@apache.org] Gesendet: Montag, 28. Oktober 2013 16:58 An: users@jena.apache.org Betreff: Re: Declining TDB load