Federico Mariani created CAMEL-24687:
----------------------------------------
Summary: camel-quickjs - every evaluation leaks a QuickJS module;
8 threads exhaust a 1 GB heap in 22 seconds
Key: CAMEL-24687
URL: https://issues.apache.org/jira/browse/CAMEL-24687
Project: Camel
Issue Type: Bug
Affects Versions: 4.23.0
Reporter: Federico Mariani
QuickJS4J executes a module per {{invokeGuestFunction}} call and QuickJS keeps
evaluated modules until its context is freed, so a camel-quickjs engine grows
by about 12 KB of WebAssembly memory per evaluation and never shrinks.
Reproduced twice with the 8-thread load test of the benchmark branch
({{GroovyLoadTest}}, variant {{quickjs}}, {{-Xms1g -Xmx1g}}): heap after young
GC 496 MB at 5 s, 939 MB at 20 s, then {{java.lang.OutOfMemoryError}} in the
producer threads at ~22 s, at about 4,000 msg/s. The GC log shows 919 MB live
after a full GC, i.e. it is retained, not garbage.
Proposed fix (implemented, tested, benchmarked): keep a handle on the engine's
linear memory through the Endive memory factory and recycle a worker thread's
engine (close it, create a fresh one on the next evaluation) once its memory
exceeds {{engineMaxMemory}} (default 64 MB) or it has run
{{engineMaxEvaluations}} (default 50,000) evaluations. With that, the same
5-minute run completes at 8,858 msg/s with no full GC and a stable heap. An
upstream quickjs4j issue (a way to invoke a compiled function without
re-evaluating a module) would remove the need for recycling.
Related: CAMEL-23398 (component), apache/camel#25778.
----
_Found during a benchmark review of camel-groovy and the other scripting
languages (camel-performance-tests branch camel-groovy-perf,
tests/camel-jmh/groovy-results, JMH 1.37, JDK 21, one M1 Pro laptop). A
candidate fix with tests exists on the local apache/camel branch
camel-perf-all-fixes; a PR follows._
_Claude Code on behalf of Croway_
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
