[
https://issues.apache.org/jira/browse/GROOVY-10918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17689441#comment-17689441
]
Andriy Rysin commented on GROOVY-10918:
---------------------------------------
Thank you, so my original script had several dozen of String.replaceAll() (it's
for cleaning up texts for NLP, and it grew over several years).
In general it was:
{code:java}
text = text.replaceAll(...)
text = text.replaceAll(...)
{code}
It was all good until somebody said they need to run my script on 300MB text
file and it choke.
So I've split the code into dozen of methods but I still have 2-5 replacements
in each, and if text is 300MB then each replacement still multiply the memory
requirement.
I've also started using different var names, but it complicates the logic:
a) I have to make sure in each next replaceAll I don't forget to use the right
var (which is if you reorder/refactor may be non-trivial)
b) it gets worse if you have conditional replacements - then just introducing
new var does not work, e.g.
{code:java}
def t10 = ...
if( t10.contains('\u2028') ) {
def t11 = t10.replaceAll(/\u2028\n?/, '\n')
t10 = null // shall I force it here?
t10 = t11
t11 = null // shall I force it here?
}
def t12 = t10.replaceAll(...){code}
> Memory leak: local variable values are not discarded
> ----------------------------------------------------
>
> Key: GROOVY-10918
> URL: https://issues.apache.org/jira/browse/GROOVY-10918
> Project: Groovy
> Issue Type: Bug
> Components: groovy-runtime
> Affects Versions: 4.0.9
> Reporter: Andriy Rysin
> Priority: Major
> Labels: MemoryLeak
> Attachments: TestOOM.groovy, TestOOMIcj.java, TestOOMJ.java,
> TestOOM_works.groovy, groovy_oom.png
>
>
> When I run the code below with 3 statements with closures inside the method
> the local var (parameter) values (all 4 of them) are staying in memory.
> I don't see the same problem if I run corresponding Java code with lambdas.
> Run TestOOM.groovy with
> -Xmx600M -XX:+HeapDumpOnOutOfMemoryError
> Notice groovy fails:
> Y
> Z
> java.lang.OutOfMemoryError: Java heap space
> Dumping heap to java_pid147612.hprof ...
> Heap dump file created [497819587 bytes in 0.136 secs]
> Caught: java.lang.OutOfMemoryError: Java heap space
> java.lang.OutOfMemoryError: Java heap space
> at test.TestOOM.test(TestOOM.groovy:31)
> at test.TestOOM.run(TestOOM.groovy:41)
> But Java version does not.
> It looks like all the values of s are still in memory (see screenshot), even
> though previous values should be discarded.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
