paulk-asert commented on PR #968:
URL: https://github.com/apache/groovy/pull/968#issuecomment-6277341561
Thanks @NecOnIce for the PR and the detailed heap analysis, and apologies
that it sat for so long. I'm going to close this one. As @blackdrag explained,
holding the metaclass weakly makes the runtime keep rebuilding metaclasses for
classes that are still in use, and that costs a lot of performance. A soft
reference doesn't stop a class from being unloaded. It only delays it until
there's memory pressure.
The problem behind the PR has since been tackled in other ways. Here are the
options, starting with the one that usually helps most (some of these are
Groovy 6+ which may or may not help you):
**1. Cache compiled scripts (the main fix for the scenario described)**
About 100 rules producing tens of thousands of `ScriptNNNNN` classes means
every evaluation compiles a new class. Each of those classes brings its own
class metadata and metaclass. Compile each rule once, keyed by its text, and
reuse it:
// once per distinct rule text
Class<? extends Script> cls = shell.parse(ruleText).getClass()
// per evaluation (a fresh instance, so bindings aren't shared between
threads)
Script s = InvokerHelper.createScript(cls, new Binding(vars))
s.run()
When the rule set changes, drop the cache and the `GroovyClassLoader` (or
`GroovyShell`) that compiled it, so the old classes can be unloaded.
**2. `-Dgroovy.use.classvalue=soft` (Groovy 6.0.0+, GROOVY-12281)**
By default Groovy keeps per-class metadata in `java.lang.ClassValue`. An
association on a long-lived class (e.g. `String`) is never released, and it can
hold the Groovy class loader it belongs to (JDK-8136353). In soft mode Groovy
keeps the `ClassValue` fast path but holds each association through a
`SoftReference`. Metadata with state that can't be rebuilt (an installed or
per-instance metaclass) stays strongly held while in use, and anything cleared
is rebuilt on demand. Dispatch performance is the same as the default mode.
**3. `-Dgroovy.use.classvalue=false`**
Stores the associations in a weak-key map instead of `ClassValue`. This also
releases the loader, but each lookup becomes a hash lookup, so very
dispatch-heavy dynamic code can run noticeably slower.
**4. Explicit cleanup**
When a batch of scripts is discarded, call `ClassInfo.remove(cls)` for its
classes (or iterate `ClassInfo.getAllClassInfo()`). The integration guide has
an example in "Class loader release in managed environments".
**5. Compact closures (Groovy 6.0.0+, incubating, GEP-27)**
Each closure literal normally compiles to its own class, so a rule
containing closures adds a class (plus metadata) per closure on top of the
script class. With `@PackedClosures`, or the global flag
`-Dgroovy.target.closure.pack=true`, eligible closures are compiled into a
shared adapter instead of their own classes. That cuts the number of classes,
and metaclasses, per compiled rule. Without the annotation, the flag only packs
closures it can prove are safe: in dynamic code, that means closures with no
free names. Add `-Dgroovy.target.closure.pack.report=true` to see which
closures weren't packed and why. This reduces how much each compilation
creates, but it's not a replacement for caching (option 1).
@blackdrag's idea of starting with a weak reference and switching to soft
after repeated recreation is still open. If anyone wants to take it further,
please raise a JIRA for it based on findings for a more recent Groovy version.
Thanks again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
