Chuck Burgess created STORM-1378:
------------------------------------
Summary: Windows File Handling Issues with Blobstore
Key: STORM-1378
URL: https://issues.apache.org/jira/browse/STORM-1378
Project: Apache Storm
Issue Type: Bug
Environment: Win7 JDK1.7.0_45
Reporter: Chuck Burgess
Priority: Minor
Since the Blobstore merge to master (7029ae), several tests show consistent
issues with file system interaction on Windows.
I first noticed it as several of these in
backtype.storm.multilang-test.test-python:
{code}
30970 [Thread-8] ERROR b.s.event - Error when processing event
java.io.IOException: Unable to delete file:
C:\Users\cburgess\AppData\Local\Temp\6d2e5448-3710-44c0-8e0a-8bfef14d065d\supervisor\tmp\fcc042a9-ddf4-498d-9114-a29626429310\stormconf.ser
at org.apache.commons.io.FileUtils.forceDelete(FileUtils.java:2279)
~[commons-io-2.4.jar:2.4]
at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java:1653)
~[commons-io-2.4.jar:2.4]
at org.apache.commons.io.FileUtils.deleteDirectory(FileUtils.java:1535)
~[commons-io-2.4.jar:2.4]
at org.apache.commons.io.FileUtils.moveDirectory(FileUtils.java:2756)
~[commons-io-2.4.jar:2.4]
at
backtype.storm.daemon.supervisor$fn__2392.invoke(supervisor.clj:1134)
~[classes/:?]
at clojure.lang.MultiFn.invoke(MultiFn.java:243) ~[clojure-1.7.0.jar:?]
at
backtype.storm.daemon.supervisor$mk_synchronize_supervisor$this__2118.invoke(supervisor.clj:565)
~[classes/:?]
at backtype.storm.event$event_manager$fn__5761.invoke(event.clj:40)
[classes/:?]
at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?]
at java.lang.Thread.run(Thread.java:744) [?:1.7.0_45]
{code}
When running a full test build, the LocalizerTest and BlobStoreTest cases stop
the build due to Windows file interaction errors:
{code}
Failed tests:
LocalizerTest.testDirPaths:147 get local user dir doesn't return right value
expected:<...c5-8a25-f0a888dc95e0[/usercache/]user1> but
was:<...c5-8a25-f0a888dc95e0[\usercache\]user1>
Tests in error:
backtype.storm.blobstore.BlobStoreTest.testBasicLocalFs(backtype.storm.blobstore.BlobStoreTest)
Run 1: BlobStoreTest.testBasicLocalFs:164->testBasic:313 ╗ Runtime
java.nio.file.Dire...
Run 2: BlobStoreTest.cleanup:74 ╗ IO Unable to delete file:
\tmp\blob-store-test-0a66...
BlobStoreTest.cleanup:74 ╗ IO Unable to delete file:
\tmp\blob-store-test-9ada...
backtype.storm.blobstore.BlobStoreTest.testLocalFsWithAuth(backtype.storm.blobstore.BlobStoreTest)
Run 1: BlobStoreTest.testLocalFsWithAuth:159->testWithAuthentication:227 ╗
Runtime ja...
Run 2: BlobStoreTest.cleanup:74 ╗ IO Unable to delete file:
\tmp\blob-store-test-594f...
backtype.storm.blobstore.BlobStoreTest.testMultipleLocalFs(backtype.storm.blobstore.BlobStoreTest)
Run 1: BlobStoreTest.testMultipleLocalFs:169->testMultiple:398 ╗ AccessDenied
C:\tmp\...
Run 2: BlobStoreTest.cleanup:74 ╗ IO Unable to delete file:
\tmp\blob-store-test-5356...
LocalizerTest.testArchivesJar:260->testArchives:286 ╗ IO Error getting blobs
LocalizerTest.testArchivesTar:254->testArchives:286 ╗ IO Error getting blobs
backtype.storm.localizer.LocalizerTest.testArchivesTarGz(backtype.storm.localizer.LocalizerTest)
Run 1: LocalizerTest.testArchivesTarGz:248->testArchives:286 ╗ IO Error
getting blobs
Run 2: LocalizerTest.tearDown:125 ╗ IO Unable to delete file:
\tmp\blob-store-localiz...
backtype.storm.localizer.LocalizerTest.testArchivesTgz(backtype.storm.localizer.LocalizerTest)
Run 1: LocalizerTest.testArchivesTgz:236->testArchives:286 ╗ IO Error getting
blobs
Run 2: LocalizerTest.tearDown:125 ╗ IO Unable to delete file:
\tmp\blob-store-localiz...
LocalizerTest.testArchivesZip:242->testArchives:286 ╗ IO Error getting blobs
LocalizerTest.testBasic:363 ╗ IO Error getting blobs
LocalizerTest.testMultipleKeysOneUser:435 ╗ IO Error getting blobs
LocalizerTest.testMultipleUsers:563 ╗ IO Error getting blobs
LocalizerTest.testUpdate:637 ╗ IO Error getting blobs
Tests run: 95, Failures: 1, Errors: 13, Skipped: 0
{code}
Most of these are also due to "Unable to delete file". They also all seem to
be regarding stormconf.ser.
Is it possible that some process/thread that created these stormconf.ser files
has not released/closed their own file interactions before the storm code tries
to delete them?
Because 7029ae looks to be the entire feature squashed into one commit, I
cannot bisect any deeper.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
