Hi Bertrand, The approach of putting triples in an ArrayList seems the limiting factor and I don't understand why you do this. TcManager is as scalable as the underlying storage backend (i.e. tdb or sesame).
The clerezza sparql endpoint doesn't need to have the triples in memory. You can have a UnionGraph of multiple graphs even from different backends without any data actually being copied around. The performance of a sparql query may depend on the order of the where clauses. For queries against a set of graphs from the same storage endpoint we will soon implement a transparent fastlane to the sparql-endpoint of the storage provider (if the storage provider supports sparql), but in your case as you are querying against a UnionGraph (and not against named actually stored graphs) this will not make any difference. Cheers, reto On Tue, Jun 15, 2010 at 10:31 AM, Bertrand Delacretaz < [email protected]> wrote: > Hi, > > We are using some Clerezza modules to implement SPARQL queries in IKS FISE. > > I'm wondering about the performance/scalability of SPARQL queries as > done in our ClerezzaSparqlQueryEngine [1], which is based on an > example that Manuel created at the first FISE hackathon. > > That code collects all available triples from the triple store in an > ArrayList, creates an UnionGraph by converting that collection to an > array, and passes that graph to TcManager.executeSparqlQuery(...) > > Unless I'm missing something, it looks like that would use tons of > memory if the TcManager stores lots of triples, is there a way to make > this more scalable? > > Also, is TcManager.executeSparqlQuery(...) efficient and scalable, or > are there usage limitations that we should take into account? > > Thanks for any help, and RTFM links are welcome of course. > > -Bertrand > > [1] > http://iks-project.googlecode.com/svn/sandbox/fise/trunk/clerezza/clerezza-sparql/src/main/java/eu/iksproject/fise/clerezza/sparql/ClerezzaSparqlQueryEngine.java >
