On 10/07/2023 12:18, Andy Seaborne wrote:
Laura, Dave,
This doesn't sound like the same issue but let's see.
Sorry for the confusion - these questions are for Laura.
- Is it in a container? If so:
Is it the container being killed OOM or
Java throwing an OOM exception?
Much RAM does the container get? How many threads?
- If not a container, how many CPU Threads are there? How many cores?
- Which form of Fuseki are you using?
what does
java -XX:+PrintFlagsFinal -version \
| grep -i 'M..HeapSize`
say?
How are you sending the queries to the server?
On 09/07/2023 20:33, Laura Morales wrote:
I'm running a job that is submitting a lot of queries to a Fuseki
server, in parallel. My problem is that Fuseki is OOM-killed and I
don't know how to fix this. Some details:
- Fuseki is queried as fast as possible. Queries take around 50-100ms
to complete so I think it's serving 10s of queries each second
Are all the queries about the same amount of work are are some going to
cause significantly more memory use?
It is quite possible to send queries faster than the server can process
them - there is little point sending in parallel more than there are
real CPU threads to service them.
They will interfere and the machine can end up going slower (query of
queries per second).
I don't know exactly the impact on the GC but I think the JVM delays
minor GC's when very busy but that pushes it to do major ones earlier.
A thing to try is use less parallelism.
- Fuseki 4.8. OS is Debian 12 (minimal installation with only OS,
Fuseki, no desktop environments, uses only ~100MB of RAM)
- all the queries are read queries. No updates, inserts, or other
write queries
- all the queries are over HTTP to the Fuseki endpoint
- database is TDB2 (created with tdb2.tdbloader)
- database contains around 2.5M triples
- the machine has 8GB RAM. I've tried on another PC with 16GB and it
completes the job. On 8GB though, it won't
- with -Xmx6G it's killed earlier. With -Xmx2G it's killed later.
Either way it's always killed.
Is it getting OOM at random or do certain queries tend to push it over
he edge?
Is that the machine (container) has 8G RAM and there is no -Xmx setting?
in that case, default setting applies which is 25% of RAM.
A heap dump to know where the memory is going would be useful.
Is there anything that I can tweak to avoid Fuseki getting killed?
Something that isn't "just buy more RAM".
Thank you
