Re: JENA Loader Benchmarks

2020-06-12 Thread Marco Neumann
dy Seaborne > >> wrote: > >>>>>> > >>>>>>> These messages are logged (to logger > "org.apache.jena.tdb2.loader") - > >>>>> do > >>>>>>> you have log4j.proprties in the cu

Re: JENA Loader Benchmarks

2019-06-23 Thread Andy Seaborne
:13, Marco Neumann wrote: I am collecting jena loader benchmarks. if you have results please post them directly. http://www.lotico.com/index.php/JENA_Loader_Benchmarks tdb2.tdbloader has variations controlled by --loader. --loader= Loader to use: 'basic', 'phased' (default), 'sequential',

Re: JENA Loader Benchmarks

2019-06-23 Thread Marco Neumann
>>>>> INFO Add: 1,000,000 bsbm-5m.nt.gz (Batch: 189,753 / Avg: 157,604) > >>>>> INFO Add: 1,500,000 bsbm-5m.nt.gz (Batch: 205,676 / Avg: 170,920) > >>>>> INFO Add: 2,000,000 bsbm-5m.nt.gz (Batch: 204,248 / Avg: 178,189) > >>>>&g

Re: JENA Loader Benchmarks

2019-06-22 Thread Andy Seaborne
ne wrote: On 14/06/2019 18:13, Marco Neumann wrote: I am collecting jena loader benchmarks. if you have results please post them directly. http://www.lotico.com/index.php/JENA_Loader_Benchmarks tdb2.tdbloader has variations controlled by --loader. --loader= Loader to use: 'basic', 'phased'

Re: JENA Loader Benchmarks

2019-06-20 Thread Marco Neumann
58,881 / Avg: 175,076) >> > > INFOElapsed: 28.56 seconds [2019/06/14 22:51:37 BST] >> > > INFO Finished: /home/afs/Datasets/BSBM/bsbm-5m.nt.gz: 5,000,599 >> tuples >> > > in 28.63s (Avg: 174,644) >> > > INFO Finish - index SPO >>

Re: JENA Loader Benchmarks

2019-06-18 Thread Marco Neumann
> > I currently think RocksDB is possible choice. Initial experiments showed > it works but needs tuning work. The new storage architecture > (jena-dboe-storage) would make it event easier to build. > > Andy > > > > > > > > > > On Fri, Jun 14, 2019 at 8:03 PM An

Re: JENA Loader Benchmarks

2019-06-18 Thread Andy Seaborne
Seaborne wrote: On 14/06/2019 18:13, Marco Neumann wrote: I am collecting jena loader benchmarks. if you have results please post them directly. http://www.lotico.com/index.php/JENA_Loader_Benchmarks tdb2.tdbloader has variations controlled by --loader. --loader= Loader to use: 'basic'

Re: JENA Loader Benchmarks

2019-06-18 Thread Marco Neumann
collecting jena loader benchmarks. if you have results please post > > them directly. > > > > http://www.lotico.com/index.php/JENA_Loader_Benchmarks > > tdb2.tdbloader has variations controlled by --loader. > > --loader= > Loader to use: 'basic', 'phased' (default),

Re: JENA Loader Benchmarks

2019-06-15 Thread Marco Neumann
; INFO Add: 3,000,000 bsbm-5m.nt.gz (Batch: 206,953 / Avg: 186,173) > >>>> INFO Add: 3,500,000 bsbm-5m.nt.gz (Batch: 183,621 / Avg: 185,804) > >>>> INFO Add: 4,000,000 bsbm-5m.nt.gz (Batch: 151,423 / Avg: 180,676) > >>>> INFO Add: 4,500,000 bsbm-5m.nt

Re: JENA Loader Benchmarks

2019-06-15 Thread Lorenz B.
,000,000 bsbm-5m.nt.gz (Batch: 151,423 / Avg: 180,676) >>>> INFO Add: 4,500,000 bsbm-5m.nt.gz (Batch: 152,765 / Avg: 177,081) >>>> INFO Add: 5,000,000 bsbm-5m.nt.gz (Batch: 158,881 / Avg: 175,076) >>>> INFOElapsed: 28.56 seconds [2019/06/14 22:51:37 BST] >>>>

Re: JENA Loader Benchmarks

2019-06-14 Thread Marco Neumann
> INFO Finish - index POS > > > INFO Finish - index OSP > > > INFO Time = 35.572 seconds : Triples = 5,000,599 : Rate = 140,577 /s > > > > > > > > > There is pause after the first "Finished:" - this is finished data in, >

Re: JENA Loader Benchmarks

2019-06-14 Thread Martynas Jusevičius
On 14/06/2019 20:16, Marco Neumann wrote: > > > let me fire up one of the big machines to see what I will get there. > > > currently I have no info display during load with tdb2.tdbloader . if -v > > is > > > specified I get some extra info but no load info

Re: JENA Loader Benchmarks

2019-06-14 Thread Marco Neumann
. if -v > is > > specified I get some extra info but no load info. > > > > On Fri, Jun 14, 2019 at 8:03 PM Andy Seaborne wrote: > > > >> > >> > >> On 14/06/2019 18:13, Marco Neumann wrote: > >>> I am collecting jena loader benchmarks. if you have results p

Re: JENA Loader Benchmarks

2019-06-14 Thread Andy Seaborne
t there. currently I have no info display during load with tdb2.tdbloader . if -v is specified I get some extra info but no load info. On Fri, Jun 14, 2019 at 8:03 PM Andy Seaborne wrote: On 14/06/2019 18:13, Marco Neumann wrote: I am collecting jena loader benchmarks. if you have results pleas

Re: JENA Loader Benchmarks

2019-06-14 Thread Marco Neumann
ann wrote: > > I am collecting jena loader benchmarks. if you have results please post > > them directly. > > > > http://www.lotico.com/index.php/JENA_Loader_Benchmarks > > tdb2.tdbloader has variations controlled by --loader. > > --loader= > Loader to use: 'bas

Re: JENA Loader Benchmarks

2019-06-14 Thread Marco Neumann
collecting jena loader benchmarks. if you have results please post > > them directly. > > > > http://www.lotico.com/index.php/JENA_Loader_Benchmarks > > > > On a linux machine I am using "time" to collect data. > > > > Is there a flag on tdb2.tdblo

Re: JENA Loader Benchmarks

2019-06-14 Thread Andy Seaborne
On 14/06/2019 18:13, Marco Neumann wrote: I am collecting jena loader benchmarks. if you have results please post them directly. http://www.lotico.com/index.php/JENA_Loader_Benchmarks tdb2.tdbloader has variations controlled by --loader. --loader= Loader to use: 'basic', 'phased' (default

Re: JENA Loader Benchmarks

2019-06-14 Thread Andy Seaborne
On 14/06/2019 18:13, Marco Neumann wrote: I am collecting jena loader benchmarks. if you have results please post them directly. http://www.lotico.com/index.php/JENA_Loader_Benchmarks On a linux machine I am using "time" to collect data. Is there a flag on tdb2.tdbloader to r

JENA Loader Benchmarks

2019-06-14 Thread Marco Neumann
I am collecting jena loader benchmarks. if you have results please post them directly. http://www.lotico.com/index.php/JENA_Loader_Benchmarks On a linux machine I am using "time" to collect data. Is there a flag on tdb2.tdbloader to report time and triples per second? I ha