This is actually more complicated. KnnVectorDict uses a mapped byte buffer - such buffers lock the source file on Windows and closing the original channel doesn't really affect their state:
private final MappedByteBuffer vbuffer; It'd need to use the unmapper hack currently present in mmapdirectory to release this buffer. I CC Uwe - perhaps he has an idea how to solve it cleanly. At the moment tests on Windows will fail because the temporary file/ directory can't be unmapped. I think it's a legitimate bug since we do want those resources released eagerly rather than rely on the JVM garbage collector to unmap them. Dawid On Wed, Aug 18, 2021 at 9:25 PM Michael Sokolov <[email protected]> wrote: > > Ah I'll check, thanks for the pointer > > On Wed, Aug 18, 2021 at 12:19 PM Dawid Weiss <[email protected]> wrote: > > > > Looks like something (file handle) is not properly closed - this > > causes temp. file removal to fail on Windows. > > > > D. > > > > On Wed, Aug 18, 2021 at 5:52 PM Policeman Jenkins Server > > <[email protected]> wrote: > > > > > > Build: https://jenkins.thetaphi.de/job/Lucene-main-Windows/10089/ > > > Java: 64bit/jdk-15 -XX:+UseCompressedOops -XX:+UseG1GC > > > > > > 3 tests failed. > > > FAILED: org.apache.lucene.demo.TestDemo.classMethod > > > > > > Error Message: > > > java.io.IOException: Could not remove the following files (in the order > > > of attempts): > > > > > > C:\Users\jenkins\workspace\Lucene-main-Windows\lucene\demo\build\tmp\tests-tmp\lucene.demo.TestDemo_94AFFE7CDE43C29F-001\ContribDemoTest-001\knn-dict.bin: > > > java.nio.file.AccessDeniedException: > > > C:\Users\jenkins\workspace\Lucene-main-Windows\lucene\demo\build\tmp\tests-tmp\lucene.demo.TestDemo_94AFFE7CDE43C29F-001\ContribDemoTest-001\knn-dict.bin > > > > > > C:\Users\jenkins\workspace\Lucene-main-Windows\lucene\demo\build\tmp\tests-tmp\lucene.demo.TestDemo_94AFFE7CDE43C29F-001\ContribDemoTest-001: > > > java.nio.file.DirectoryNotEmptyException: > > > C:\Users\jenkins\workspace\Lucene-main-Windows\lucene\demo\build\tmp\tests-tmp\lucene.demo.TestDemo_94AFFE7CDE43C29F-001\ContribDemoTest-001 > > > > > > C:\Users\jenkins\workspace\Lucene-main-Windows\lucene\demo\build\tmp\tests-tmp\lucene.demo.TestDemo_94AFFE7CDE43C29F-001: > > > java.nio.file.DirectoryNotEmptyException: > > > C:\Users\jenkins\workspace\Lucene-main-Windows\lucene\demo\build\tmp\tests-tmp\lucene.demo.TestDemo_94AFFE7CDE43C29F-001 > > > > > > > > > Stack Trace: > > > java.io.IOException: Could not remove the following files (in the order > > > of attempts): > > > > > > C:\Users\jenkins\workspace\Lucene-main-Windows\lucene\demo\build\tmp\tests-tmp\lucene.demo.TestDemo_94AFFE7CDE43C29F-001\ContribDemoTest-001\knn-dict.bin: > > > java.nio.file.AccessDeniedException: > > > C:\Users\jenkins\workspace\Lucene-main-Windows\lucene\demo\build\tmp\tests-tmp\lucene.demo.TestDemo_94AFFE7CDE43C29F-001\ContribDemoTest-001\knn-dict.bin > > > > > > C:\Users\jenkins\workspace\Lucene-main-Windows\lucene\demo\build\tmp\tests-tmp\lucene.demo.TestDemo_94AFFE7CDE43C29F-001\ContribDemoTest-001: > > > java.nio.file.DirectoryNotEmptyException: > > > C:\Users\jenkins\workspace\Lucene-main-Windows\lucene\demo\build\tmp\tests-tmp\lucene.demo.TestDemo_94AFFE7CDE43C29F-001\ContribDemoTest-001 > > > > > > C:\Users\jenkins\workspace\Lucene-main-Windows\lucene\demo\build\tmp\tests-tmp\lucene.demo.TestDemo_94AFFE7CDE43C29F-001: > > > java.nio.file.DirectoryNotEmptyException: > > > C:\Users\jenkins\workspace\Lucene-main-Windows\lucene\demo\build\tmp\tests-tmp\lucene.demo.TestDemo_94AFFE7CDE43C29F-001 > > > > > > at __randomizedtesting.SeedInfo.seed([94AFFE7CDE43C29F]:0) > > > at org.apache.lucene.util.IOUtils.rm(IOUtils.java:315) > > > at > > > org.apache.lucene.util.TestRuleTemporaryFilesCleanup.afterAlways(TestRuleTemporaryFilesCleanup.java:207) > > > at > > > com.carrotsearch.randomizedtesting.rules.TestRuleAdapter$1.afterAlways(TestRuleAdapter.java:31) > > > at > > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43) > > > at > > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > > > at > > > org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) > > > at > > > org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) > > > at > > > org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44) > > > at > > > org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60) > > > at > > > org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47) > > > at org.junit.rules.RunRules.evaluate(RunRules.java:20) > > > at > > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > > > at > > > com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370) > > > at > > > com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:826) > > > at java.base/java.lang.Thread.run(Thread.java:832) > > > > > > > > > FAILED: org.apache.lucene.demo.knn.TestDemoEmbeddings.classMethod > > > > > > Error Message: > > > java.io.IOException: Could not remove the following files (in the order > > > of attempts): > > > > > > C:\Users\jenkins\workspace\Lucene-main-Windows\lucene\demo\build\tmp\tests-tmp\lucene.demo.knn.TestDemoEmbeddings_94AFFE7CDE43C29F-001\knn-demo-001\dict.bin: > > > java.nio.file.AccessDeniedException: > > > C:\Users\jenkins\workspace\Lucene-main-Windows\lucene\demo\build\tmp\tests-tmp\lucene.demo.knn.TestDemoEmbeddings_94AFFE7CDE43C29F-001\knn-demo-001\dict.bin > > > > > > C:\Users\jenkins\workspace\Lucene-main-Windows\lucene\demo\build\tmp\tests-tmp\lucene.demo.knn.TestDemoEmbeddings_94AFFE7CDE43C29F-001\knn-demo-001: > > > java.nio.file.DirectoryNotEmptyException: > > > C:\Users\jenkins\workspace\Lucene-main-Windows\lucene\demo\build\tmp\tests-tmp\lucene.demo.knn.TestDemoEmbeddings_94AFFE7CDE43C29F-001\knn-demo-001 > > > > > > C:\Users\jenkins\workspace\Lucene-main-Windows\lucene\demo\build\tmp\tests-tmp\lucene.demo.knn.TestDemoEmbeddings_94AFFE7CDE43C29F-001: > > > java.nio.file.DirectoryNotEmptyException: > > > C:\Users\jenkins\workspace\Lucene-main-Windows\lucene\demo\build\tmp\tests-tmp\lucene.demo.knn.TestDemoEmbeddings_94AFFE7CDE43C29F-001 > > > > > > > > > Stack Trace: > > > java.io.IOException: Could not remove the following files (in the order > > > of attempts): > > > > > > C:\Users\jenkins\workspace\Lucene-main-Windows\lucene\demo\build\tmp\tests-tmp\lucene.demo.knn.TestDemoEmbeddings_94AFFE7CDE43C29F-001\knn-demo-001\dict.bin: > > > java.nio.file.AccessDeniedException: > > > C:\Users\jenkins\workspace\Lucene-main-Windows\lucene\demo\build\tmp\tests-tmp\lucene.demo.knn.TestDemoEmbeddings_94AFFE7CDE43C29F-001\knn-demo-001\dict.bin > > > > > > C:\Users\jenkins\workspace\Lucene-main-Windows\lucene\demo\build\tmp\tests-tmp\lucene.demo.knn.TestDemoEmbeddings_94AFFE7CDE43C29F-001\knn-demo-001: > > > java.nio.file.DirectoryNotEmptyException: > > > C:\Users\jenkins\workspace\Lucene-main-Windows\lucene\demo\build\tmp\tests-tmp\lucene.demo.knn.TestDemoEmbeddings_94AFFE7CDE43C29F-001\knn-demo-001 > > > > > > C:\Users\jenkins\workspace\Lucene-main-Windows\lucene\demo\build\tmp\tests-tmp\lucene.demo.knn.TestDemoEmbeddings_94AFFE7CDE43C29F-001: > > > java.nio.file.DirectoryNotEmptyException: > > > C:\Users\jenkins\workspace\Lucene-main-Windows\lucene\demo\build\tmp\tests-tmp\lucene.demo.knn.TestDemoEmbeddings_94AFFE7CDE43C29F-001 > > > > > > at __randomizedtesting.SeedInfo.seed([94AFFE7CDE43C29F]:0) > > > at org.apache.lucene.util.IOUtils.rm(IOUtils.java:315) > > > at > > > org.apache.lucene.util.TestRuleTemporaryFilesCleanup.afterAlways(TestRuleTemporaryFilesCleanup.java:207) > > > at > > > com.carrotsearch.randomizedtesting.rules.TestRuleAdapter$1.afterAlways(TestRuleAdapter.java:31) > > > at > > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43) > > > at > > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > > > at > > > org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) > > > at > > > org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) > > > at > > > org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44) > > > at > > > org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60) > > > at > > > org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47) > > > at org.junit.rules.RunRules.evaluate(RunRules.java:20) > > > at > > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > > > at > > > com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370) > > > at > > > com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:826) > > > at java.base/java.lang.Thread.run(Thread.java:832) > > > > > > > > > FAILED: org.apache.lucene.demo.knn.TestKnnVectorDict.classMethod > > > > > > Error Message: > > > java.io.IOException: Could not remove the following files (in the order > > > of attempts): > > > > > > C:\Users\jenkins\workspace\Lucene-main-Windows\lucene\demo\build\tmp\tests-tmp\lucene.demo.knn.TestKnnVectorDict_94AFFE7CDE43C29F-001\knn-demo-001\dict.bin: > > > java.nio.file.AccessDeniedException: > > > C:\Users\jenkins\workspace\Lucene-main-Windows\lucene\demo\build\tmp\tests-tmp\lucene.demo.knn.TestKnnVectorDict_94AFFE7CDE43C29F-001\knn-demo-001\dict.bin > > > > > > C:\Users\jenkins\workspace\Lucene-main-Windows\lucene\demo\build\tmp\tests-tmp\lucene.demo.knn.TestKnnVectorDict_94AFFE7CDE43C29F-001\knn-demo-001: > > > java.nio.file.DirectoryNotEmptyException: > > > C:\Users\jenkins\workspace\Lucene-main-Windows\lucene\demo\build\tmp\tests-tmp\lucene.demo.knn.TestKnnVectorDict_94AFFE7CDE43C29F-001\knn-demo-001 > > > > > > C:\Users\jenkins\workspace\Lucene-main-Windows\lucene\demo\build\tmp\tests-tmp\lucene.demo.knn.TestKnnVectorDict_94AFFE7CDE43C29F-001: > > > java.nio.file.DirectoryNotEmptyException: > > > C:\Users\jenkins\workspace\Lucene-main-Windows\lucene\demo\build\tmp\tests-tmp\lucene.demo.knn.TestKnnVectorDict_94AFFE7CDE43C29F-001 > > > > > > > > > Stack Trace: > > > java.io.IOException: Could not remove the following files (in the order > > > of attempts): > > > > > > C:\Users\jenkins\workspace\Lucene-main-Windows\lucene\demo\build\tmp\tests-tmp\lucene.demo.knn.TestKnnVectorDict_94AFFE7CDE43C29F-001\knn-demo-001\dict.bin: > > > java.nio.file.AccessDeniedException: > > > C:\Users\jenkins\workspace\Lucene-main-Windows\lucene\demo\build\tmp\tests-tmp\lucene.demo.knn.TestKnnVectorDict_94AFFE7CDE43C29F-001\knn-demo-001\dict.bin > > > > > > C:\Users\jenkins\workspace\Lucene-main-Windows\lucene\demo\build\tmp\tests-tmp\lucene.demo.knn.TestKnnVectorDict_94AFFE7CDE43C29F-001\knn-demo-001: > > > java.nio.file.DirectoryNotEmptyException: > > > C:\Users\jenkins\workspace\Lucene-main-Windows\lucene\demo\build\tmp\tests-tmp\lucene.demo.knn.TestKnnVectorDict_94AFFE7CDE43C29F-001\knn-demo-001 > > > > > > C:\Users\jenkins\workspace\Lucene-main-Windows\lucene\demo\build\tmp\tests-tmp\lucene.demo.knn.TestKnnVectorDict_94AFFE7CDE43C29F-001: > > > java.nio.file.DirectoryNotEmptyException: > > > C:\Users\jenkins\workspace\Lucene-main-Windows\lucene\demo\build\tmp\tests-tmp\lucene.demo.knn.TestKnnVectorDict_94AFFE7CDE43C29F-001 > > > > > > at __randomizedtesting.SeedInfo.seed([94AFFE7CDE43C29F]:0) > > > at org.apache.lucene.util.IOUtils.rm(IOUtils.java:315) > > > at > > > org.apache.lucene.util.TestRuleTemporaryFilesCleanup.afterAlways(TestRuleTemporaryFilesCleanup.java:207) > > > at > > > com.carrotsearch.randomizedtesting.rules.TestRuleAdapter$1.afterAlways(TestRuleAdapter.java:31) > > > at > > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43) > > > at > > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > > > at > > > org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) > > > at > > > org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) > > > at > > > org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44) > > > at > > > org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60) > > > at > > > org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47) > > > at org.junit.rules.RunRules.evaluate(RunRules.java:20) > > > at > > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > > > at > > > com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370) > > > at > > > com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:826) > > > at java.base/java.lang.Thread.run(Thread.java:832) > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: [email protected] > > > For additional commands, e-mail: [email protected] > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [email protected] > > For additional commands, e-mail: [email protected] > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
