Re: Upload large datasets to fuseki

2021-12-06 Thread Andy Seaborne
On 06/12/2021 19:10, robert.ba...@tiscali.it wrote: Thanks for the reply. The data model is relatively simple, dealing with transactions of goods and user data. I will wait for the next release of Jena. When is it scheduled? "very soon" (the vote is in progress - as we are all

Re: Upload large datasets to fuseki

2021-12-06 Thread robert . barry
Thanks for the reply. The data model is relatively simple, dealing with transactions of goods and user data. I will wait for the next release of Jena. When is it scheduled? Il 06.12.2021 14:49 Andy Seaborne ha scritto: > I guess the end of loading with your setup was very slow. > >

Re: Upload large datasets to fuseki

2021-12-06 Thread Andy Seaborne
I guess the end of loading with your setup was very slow. Marco has mentioned xloader (which is an improved tdbloader2 that works on TDB2). If you can use a machine with more RAM, "tdb2.loader --loader=parallel" for 750m will be fastest but at some point the slow-and-steady xloader

Re: Upload large datasets to fuseki

2021-12-06 Thread Marco Neumann
I am currently experimenting with xloader. It's part of the 4.3 release. It's not as fast as tdb2.tdbloader with the parallel option but it seems to work more gracefully with an extra large datasets. On Mon, Dec 6, 2021 at 1:14 PM wrote: > > > Hello, > > I have to upload 3 billion triples to

Upload large datasets to fuseki

2021-12-06 Thread robert . barry
Hello, I have to upload 3 billion triples to Jena Fuseki. I tried using the following command with a first dataset (0.ttl.gz 1.ttl.gz => 750 million triples): tdb2_tdbloader.bat --loader = parallel --loc datasetX 0.ttl.gz 1.ttl.gz. Loading took about 8 hours to upload 750 milion. The system