[
https://issues.apache.org/jira/browse/LUCENE-9468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17179659#comment-17179659
]
Michael McCandless commented on LUCENE-9468:
--------------------------------------------
Thanks [~dweiss] ... I had ~10K of these gradle buggy tmp files by the time I
noticed it, but it is easy enough for me to workaround.
I do not want to do a {{clean}} on every beasting run, though. That'd just
decrease the already poor test efficiency!
So, I just inserted an explicit {{rm -f
lucene/core/build/tmp/test/jar_extract*}} between beast iterations.
Hmm, OK, I see, it is far more than just these {{jar_extract_*}} files – even
after removing those, I have ~15K other files, created by our tests I think,
totaling ~9.6 GB. OK, I will change my script to {{rm -rf
lucene/core/build/tmp}} each time. They truly are temp files right? I will
not slow down subsequent iterations by removing them?
Admittedly this is an exotic use case! And maybe once we have true beasting
support with gradle (LUCENE-9465) I can switch to that. Most users would
probably remember to insert a {{gradle clean}} now and then. But it is
disturbing that if you fail to do that, you can lose GBs to these supposedly
temporary files!
{quote}There is a related bigger issue with gradle workers leaving their stuff
in user's temp - this is what I really don't like...
This is the issue:
[https://github.com/gradle/gradle/issues/12020]
{quote}
Thanks for finding/sharing that issue. That is indeed annoying, even moreso
than this issue! In fact, I went and ran {{ls /tmp}} on my 128 core beast box.
It took ~8 seconds to come back with (get this!) ~675K supposedly temporary
gradle files! Sheesh. I will also fix my beasting script to remove THOSE ones
too.
I guess proper garbage collection is truly hard.
> "./gradlew -p lucene test" leaks lucene/core/build/tmp/test/jar_extract_XXXX
> files
> ----------------------------------------------------------------------------------
>
> Key: LUCENE-9468
> URL: https://issues.apache.org/jira/browse/LUCENE-9468
> Project: Lucene - Core
> Issue Type: Improvement
> Reporter: Michael McCandless
> Priority: Major
>
> I have been stupid-beasting ({{while(true) "./gradlew -p lucene test"}})
> Lucene core + modules tests, and noticed that I have accumulated many
> (~10.3K) files like these:
> {noformat}
> -rw-r--r-- 1 mike mike 4954 Aug 17 12:49
> lucene/core/build/tmp/test/jar_extract_9943749690454255636_tmp
> -rw-r--r-- 1 mike mike 84929 Aug 17 13:24
> lucene/core/build/tmp/test/jar_extract_9949237584070142535_tmp
> -rw-r--r-- 1 mike mike 14627 Aug 17 14:01
> lucene/core/build/tmp/test/jar_extract_9950285037002822552_tmp
> -rw-r--r-- 1 mike mike 15935 Aug 17 15:10
> lucene/core/build/tmp/test/jar_extract_995066590821695944_tmp
> -rw-r--r-- 1 mike mike 5488 Aug 17 11:23
> lucene/core/build/tmp/test/jar_extract_9952865172929838404_tmp
> -rw-r--r-- 1 mike mike 19078 Aug 17 14:41
> lucene/core/build/tmp/test/jar_extract_9960016969100835830_tmp
> -rw-r--r-- 1 mike mike 5488 Aug 17 10:07
> lucene/core/build/tmp/test/jar_extract_9960479662672452908_tmp
> -rw-r--r-- 1 mike mike 28711 Aug 17 13:07
> lucene/core/build/tmp/test/jar_extract_996420631017213954_tmp
> -rw-r--r-- 1 mike mike 84929 Aug 17 10:30
> lucene/core/build/tmp/test/jar_extract_9964495910786482810_tmp
> -rw-r--r-- 1 mike mike 8034 Aug 17 17:10
> lucene/core/build/tmp/test/jar_extract_9965528236930220207_tmp
> -rw-r--r-- 1 mike mike 43565 Aug 17 11:52
> lucene/core/build/tmp/test/jar_extract_9967892842722777228_tmp
> -rw-r--r-- 1 mike mike 36278 Aug 17 10:36
> lucene/core/build/tmp/test/jar_extract_996836107828729763_tmp
> -rw-r--r-- 1 mike mike 29997 Aug 17 15:48
> lucene/core/build/tmp/test/jar_extract_9968527122717193835_tmp
> -rw-r--r-- 1 mike mike 19078 Aug 17 11:11
> lucene/core/build/tmp/test/jar_extract_9968609693107939092_tmp
> -rw-r--r-- 1 mike mike 9920 Aug 17 14:31
> lucene/core/build/tmp/test/jar_extract_9968809316564216653_tmp
> -rw-r--r-- 1 mike mike 19078 Aug 17 13:22
> lucene/core/build/tmp/test/jar_extract_9969318805542859308_tmp
> -rw-r--r-- 1 mike mike 28711 Aug 17 11:44
> lucene/core/build/tmp/test/jar_extract_9974798403956637924_tmp {noformat}
> It seems to grow by ~64 files (how many cores I configured gradle to use in
> my {{./gradle.properties}} on each iteration of beasting.
> I think {{gradle}} should be cleaning this up maybe?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
