Hi Osma, > Am 13.09.2018 um 13:53 schrieb Osma Suominen <[email protected]>: > > Hi Markus, > > A few comments inline. > > Markus Neumann kirjoitti 12.09.2018 klo 18:08: >> 1. Inserting data: >> It takes more than one hour to upload the measurements of a month >> (7.5GB .ttl file ~ 16 Mio triples) (using the data-upload web-interface of >> fuseki) >> Is there a way to do this faster? > > One thing you may want to investigate is using Fuseki with a HDT backend. > Then you could create the HDT files offline and when the loading is done, > replace the files (maybe you need to restart Fuseki). I've not actually done > this myself though. The HDT support is in the hdt-jena project which is not > in the Apache Jena codebase. Its maintenance status is a bit unclear. Marco already pointed that one out and I will look into it. Has anyone an idea of the performance with patches of about 40Mio triples changed?
> >> 2. Updating data: >> We get new model runs 5 times per day. This is data for the next 10 >> days, that needs to be updated every time. >> My idea was to create a named graph "forecast" that holds the latest >> version of this data. >> Every time a new model run arrives, I create a new temporary graph to >> upload the data to. Once this is finished, I move the temporary graph to >> "forecast". >> This seems to do the work twice as it takes 1 hour for the upload an 1 >> hour for the move. > Why do you need to move the temporary graph? The PUT operation is atomic - > the data being loaded will only be visible to queries after the whole > operation is complete. What would the PUT operation look like? As I need to replace data and DELETE/INSERT did not work with a first test, I thought that creating a "temporary" graph and moving it once the upload is complete might be an option... > > -Osma > > -- > Osma Suominen > D.Sc. (Tech), Information Systems Specialist > National Library of Finland > P.O. Box 26 (Kaikukatu 4) > 00014 HELSINGIN YLIOPISTO > Tel. +358 50 3199529 > [email protected] > http://www.nationallibrary.fi Thanks Markus
