GitHub user rvesse edited a comment on the discussion: Performance issues on small dataset: any recommended configuration or tuning options?
> We updated the JAVA_OPTIONS to -Xms90g -Xmx90g -XX:+UseG1GC > -XX:MaxGCPauseMillis=200 -XX:+ParallelRefProcEnabled -XX:+AlwaysPreTouch > -XX:+UseStringDeduplication Firstly don't set the heap size that high, most of the memory usage for TDB2 (assuming that is your database backend) is off-heap via memory mapped files so you want to leave plenty of memory for the OS to use for that. I wouldn't typically go higher than 8GB heap at your scale (and even that is likely overkill unless your queries are very memory intensive) https://jena.apache.org/documentation/tdb/faqs.html#java-heap > Dataset size: 16,127,232 triples Often performance is data driven so if you can share your dataset that's great, if not (e.g. it's commercially sensitive) can you generate a sample dataset that shows the same performance problems with your queries that you can share? > Query type: SELECT That's insufficient information for us to help you. `SELECT` is the main query type in SPARQL and has many possible clauses, without seeing some sample queries we can't tell whether your queries are the problem > Configuration flags that influence query planning or execution There's some documentation on TDB optimisation at https://jena.apache.org/documentation/tdb/optimizer.html The default query optimiser code is at https://github.com/apache/jena/blob/main/jena-arq/src/main/java/org/apache/jena/sparql/algebra/optimize/OptimizerStd.java and you can see the various ARQ configuration symbols that turn specific optimisations on/off in the code there You may also find it helpful to use the [`qparse`](https://jena.apache.org/documentation/query/cmds.html#arqqparse) command from ARQ to compare the unoptimised and optimised algebras to see if ARQ is actually able to optimise your queries at all GitHub link: https://github.com/apache/jena/discussions/3600#discussioncomment-15011665 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
