Re: sparql 1.4 billion triples

2018-12-16 Thread Dick Murray
Be very careful using vmtouch especially if you call -dl as you could very easily and quickly kill a system. I've used this tool on cloud VM's to mitigate cycle times, think DBAN due to public nature of hardware. It's a fast way to an irked OS thrashing around. Dick On Sun, 16 Dec 2018 19:57

Re: sparql 1.4 billion triples

2018-12-16 Thread Siddhesh Rane
I'll be happy to document this. I think FAQ would be a good place. I actually looked further into this and found that the vmtouch functionality is provided in the jdk itself. java.nio.MappedByteBuffer#load method will bring file pages in memory [1]. The way it works is similar to vmtouch, i.e.

Re: sparql 1.4 billion triples

2018-12-16 Thread ajs6f
This seems to be a Linux-only technique that relies on installing and maintaining vmtouch, correct? It doesn't seem that we could support that as a general solution, but would you be interested in writing something that gives the essentials up for someplace in the Jena docs? I'll admit I'm not

Re: sparql 1.4 billion triples

2018-12-16 Thread Jean-Marc Vanel
yphu, you didn't share your query. Maybe the query have questionable features. Did you try a simple but useful query, like getting the first 10 foaf:Person ? Le ven. 7 déc. 2018 à 15:53, y...@zju.edu.cn a écrit : > Dear jena, > I have built a graph with 1.4 billion triples and store it as a

Re: sparql 1.4 billion triples

2018-12-16 Thread Siddhesh Rane
In-memory database has following limitations : 1) Time to create the database. Not a problem if you have a dedicated machine which runs 24/7 where you load data once and the process never exits. But a huge waste of time if you get hardware during certain time slots and you have to load data from

Re: sparql 1.4 billion triples

2018-12-15 Thread ajs6f
What is the advantage to doing that as opposed to using Jena's built-in in-memory dataset? ajs6f > On Dec 15, 2018, at 3:04 AM, Siddhesh Rane wrote: > > Bring the entire database in RAM. > Use "vmtouch " > Get vmtouch from https://hoytech.com/vmtouch/ > > I had used jena for 150M triples and

Re: sparql 1.4 billion triples

2018-12-15 Thread Siddhesh Rane
Bring the entire database in RAM. Use "vmtouch " Get vmtouch from https://hoytech.com/vmtouch/ I had used jena for 150M triples and my performance findings are documented at https://lists.apache.org/thread.html/254968eee3cd04370eafa2f9cc586e238f8a7034cf9ab4cbde3dc8e9@%3Cusers.jena.apache.org%3E

sparql 1.4 billion triples

2018-12-07 Thread 胡云苹
Dear jena, How to ask questions in https://markmail.org/search/? I have built a graph with 1.4 billion triples and store it as a data set in TDB through Fuseki upload system. Now, I try to make some sparql search, the speed is very slow. For example, when I make the sqarql in Fuseki in

sparql 1.4 billion triples

2018-12-07 Thread y...@zju.edu.cn
Dear jena, I have built a graph with 1.4 billion triples and store it as a data set in TDB through Fuseki upload system. Now, I try to make some sparql search, the speed is very slow. For example, when I make the sqarql in Fuseki in the following, it takes 50 seconds. How can I improve the