As Rob says, details matter here. The amount of data has risen considerable, assuming the version of the code is the same in April and earlier in November, and the size of the machine and the style of queries being asked can be factors.
What queries are you asking? Use of an SSD also makes a big difference, to loading and potentially to query of the dataset is a lot larger then RAM. More RAM is good for query. You can load on a different machine (with SSD) and copy the database about is that helps. On 13 May 2014 10:22, Rob Vesse <[email protected]> wrote: > "Is this significant drop in performance sth expected or maybe I have sth > fundamentally wrong in my set up - which I would need to track and fix." > > We can't tell unless you actually tell us about your setup: OS, RAM, JVM > settings, type of disk the database resides upon, etc - the more details > you can provide the better > > One important thing to be aware of is that TDB uses memory mapped files so > you don't want to set the heap size too high since most of TDB memory > usage is off heap though depending on your queries you'll need the heap to > be reasonably sized as otherwise GC and spill-to-disk will slow down query > evaluation > > In general your dataset is at the upper limit of what TDB can reasonably > handle and if you are trying to build a business on top of a triple store > then you may want to consider commercial options > > Rob > > > On 12/05/2014 15:54, "Ewa Szwed" <[email protected]> wrote: > >>Hello, >>This is me again. :) >>I have the following (very big) problem. >>Last year in November I have loaded freebase dump to Jena TDB and I was >>able to work with it reasonably good and got quite good performance for >>most of my queries. >>Recently I have updated my Jena TDB store with a dump from April. >>Here are some numbers to show the difference between these 2 instances. >> >> >> >>*November 2013* >> >>*April 2014* >> >>Full time of import >> >>262,052 sec /3,03 days >> >>716,121 sec / 8,29 days >> >>Number of triples >> >>1,826,551,456 >> >>2,489,221,915 >> >>Index size (whole dir) >> >>174 GB >> >>333 GB >> >> >>My problem is that my new instance in not performing at all. >>The queries that previously run for a couple of minutes take a couple of >>hours now and it is not acceptable for my business. :( >>So I would like to ask if there is a practical index limit size for Jena >>TDB. Is there anything I can do to improve the performance of it. >>Is this significant drop in performance sth expected or maybe I have sth >>fundamentally wrong in my set up - which I would need to track and fix. >>Please advise. >>Regards, >>Ewa Szwed > > > >
