ndimiduk commented on issue #1335: HBASE-24000 Simplify CommonFSUtils after upgrading to hadoop 2.10.0 URL: https://github.com/apache/hbase/pull/1335#issuecomment-603569926 `TestWALProcedureStore` passes for me locally. Looks like the test could be written a little more defensively to hand reruns. In this particular case, it seems reruns were blocked because the first invocation didn't delete it's directory, and the same directory path is reused over and over. It's not obvious to me why the first run failed, and there's no additional stack trace depth in the test logs. ``` <testcase name="testLogFileAleadExists" classname="org.apache.hadoop.hbase.procedure2.store.wal.TestWALProcedureStore" time="0.967"> <error message="java.io.IOException: Stream Closed" type="org.apache.hadoop.fs.FSError"><![CDATA[org.apache.hadoop.fs.FSError: java.io.IOException: Stream Closed at org.apache.hadoop.hbase.procedure2.store.wal.TestWALProcedureStore.tearDown(TestWALProcedureStore.java:104) Caused by: java.io.IOException: Stream Closed at org.apache.hadoop.hbase.procedure2.store.wal.TestWALProcedureStore.tearDown(TestWALProcedureStore.java:104) ]]></error> <system-err><![CDATA[2020-03-24 03:59:21,445 INFO [Time-limited test] hbase.ResourceChecker(179): after: procedure2.store.wal.TestWALProcedureStore#testProcIdHoles Thread=8 (was 8), OpenFileDescriptor=89 (was 89), MaxFileDescriptor=1048576 (was 1048576), SystemLoadAverage=1007 (was 1007), ProcessCount=22 (was 22), AvailableMemoryMB=26738 (was 26789) 2020-03-24 03:59:21,453 INFO [Time-limited test] hbase.ResourceChecker(151): before: procedure2.store.wal.TestWALProcedureStore#testLogFileAleadExists Thread=8, OpenFileDescriptor=89, MaxFileDescriptor=1048576, SystemLoadAverage=1007, ProcessCount=22, AvailableMemoryMB=26736 2020-03-24 03:59:21,469 DEBUG [Time-limited test] wal.WALProcedureStore(401): Starting WAL Procedure Store lease recovery 2020-03-24 03:59:21,476 INFO [Time-limited test] wal.WALProcedureStore(1134): Rolled new Procedure Store WAL, id=1 2020-03-24 03:59:21,477 DEBUG [Time-limited test] wal.WALProcedureStore(436): Lease acquired for flushLogId=1 2020-03-24 03:59:21,477 DEBUG [Time-limited test] wal.WALProcedureStore(454): No state logs to replay. 2020-03-24 03:59:21,482 INFO [WALProcedureStoreSyncThread] wal.WALProcedureStore(1134): Rolled new Procedure Store WAL, id=2 2020-03-24 03:59:21,483 INFO [WALProcedureStoreSyncThread] wal.WALProcedureStore(1221): Remove all state logs with ID less than 1, since no active procedures 2020-03-24 03:59:21,483 DEBUG [WALProcedureStoreSyncThread] wal.WALProcedureStore(1243): Removed log=/home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1335/yetus-jdk8-hadoop2-check/src/hbase-procedure/target/test-data/c0fb2ae6-9b4d-e64a-6ae8-b8746f2e897c/proc-logs/pv2-00000000000000000001.log, activeLogs=[/home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1335/yetus-jdk8-hadoop2-check/src/hbase-procedure/target/test-data/c0fb2ae6-9b4d-e64a-6ae8-b8746f2e897c/proc-logs/pv2-00000000000000000002.log] 2020-03-24 03:59:22,393 DEBUG [Time-limited test] wal.WALProcedureStore(401): Starting WAL Procedure Store lease recovery 2020-03-24 03:59:22,395 WARN [Time-limited test] wal.WALProcedureStore(1370): Remove uninitialized log: DeprecatedRawLocalFileStatus{path=file:/home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1335/yetus-jdk8-hadoop2-check/src/hbase-procedure/target/test-data/c0fb2ae6-9b4d-e64a-6ae8-b8746f2e897c/proc-logs/pv2-00000000000000000002.log; isDirectory=false; length=0; replication=1; blocksize=33554432; modification_time=1585022361000; access_time=1585022361474; owner=; group=; permission=rw-rw-rw-; isSymlink=false} 2020-03-24 03:59:22,402 INFO [Time-limited test] wal.WALProcedureStore(1134): Rolled new Procedure Store WAL, id=3 2020-03-24 03:59:22,403 DEBUG [Time-limited test] wal.WALProcedureStore(436): Lease acquired for flushLogId=3 2020-03-24 03:59:22,403 INFO [Time-limited test] wal.WALProcedureStore(330): Stopping the WAL Procedure Store, isAbort=false ]]></system-err> <rerunError message="java.io.IOException: Stream Closed" type="org.apache.hadoop.fs.FSError"> <stackTrace><![CDATA[org.apache.hadoop.fs.FSError: java.io.IOException: Stream Closed at org.apache.hadoop.hbase.procedure2.store.wal.TestWALProcedureStore.testLogFileAleadExists(TestWALProcedureStore.java:687) Caused by: java.io.IOException: Stream Closed at org.apache.hadoop.hbase.procedure2.store.wal.TestWALProcedureStore.testLogFileAleadExists(TestWALProcedureStore.java:687) ]]></stackTrace> <system-err><![CDATA[2020-03-24 03:59:24,459 INFO [Time-limited test] hbase.ResourceChecker(151): before: procedure2.store.wal.TestWALProcedureStore#testLogFileAleadExists Thread=9, OpenFileDescriptor=91, MaxFileDescriptor=1048576, SystemLoadAverage=1054, ProcessCount=22, AvailableMemoryMB=26379 2020-03-24 03:59:24,466 DEBUG [Time-limited test] wal.WALProcedureStore(401): Starting WAL Procedure Store lease recovery 2020-03-24 03:59:24,468 INFO [Time-limited test] wal.WALProcedureStore(1134): Rolled new Procedure Store WAL, id=1 2020-03-24 03:59:24,469 DEBUG [Time-limited test] wal.WALProcedureStore(436): Lease acquired for flushLogId=1 2020-03-24 03:59:24,469 DEBUG [Time-limited test] wal.WALProcedureStore(454): No state logs to replay. 2020-03-24 03:59:24,480 INFO [WALProcedureStoreSyncThread] wal.WALProcedureStore(1134): Rolled new Procedure Store WAL, id=2 2020-03-24 03:59:24,480 INFO [WALProcedureStoreSyncThread] wal.WALProcedureStore(1221): Remove all state logs with ID less than 1, since no active procedures 2020-03-24 03:59:24,481 DEBUG [WALProcedureStoreSyncThread] wal.WALProcedureStore(1243): Removed log=/home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1335/yetus-jdk8-hadoop2-check/src/hbase-procedure/target/test-data/8f7fbc71-2e3e-2dbf-99cd-0c61da4671c4/proc-logs/pv2-00000000000000000001.log, activeLogs=[/home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1335/yetus-jdk8-hadoop2-check/src/hbase-procedure/target/test-data/8f7fbc71-2e3e-2dbf-99cd-0c61da4671c4/proc-logs/pv2-00000000000000000002.log] 2020-03-24 03:59:24,481 DEBUG [Time-limited test] wal.WALProcedureStore(401): Starting WAL Procedure Store lease recovery 2020-03-24 03:59:24,482 WARN [Time-limited test] wal.WALProcedureStore(1370): Remove uninitialized log: DeprecatedRawLocalFileStatus{path=file:/home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1335/yetus-jdk8-hadoop2-check/src/hbase-procedure/target/test-data/8f7fbc71-2e3e-2dbf-99cd-0c61da4671c4/proc-logs/pv2-00000000000000000002.log; isDirectory=false; length=0; replication=1; blocksize=33554432; modification_time=1585022364000; access_time=1585022364466; owner=; group=; permission=rw-rw-rw-; isSymlink=false} 2020-03-24 03:59:24,486 INFO [Time-limited test] wal.WALProcedureStore(330): Stopping the WAL Procedure Store, isAbort=false ]]></system-err> </rerunError> <rerunError message="java.io.IOException: Stream Closed" type="org.apache.hadoop.fs.FSError"> <stackTrace><![CDATA[org.apache.hadoop.fs.FSError: java.io.IOException: Stream Closed at org.apache.hadoop.hbase.procedure2.store.wal.TestWALProcedureStore.testLogFileAleadExists(TestWALProcedureStore.java:687) Caused by: java.io.IOException: Stream Closed at org.apache.hadoop.hbase.procedure2.store.wal.TestWALProcedureStore.testLogFileAleadExists(TestWALProcedureStore.java:687) ]]></stackTrace> <system-err><![CDATA[2020-03-24 03:59:24,545 INFO [Time-limited test] hbase.ResourceChecker(151): before: procedure2.store.wal.TestWALProcedureStore#testLogFileAleadExists Thread=9, OpenFileDescriptor=92, MaxFileDescriptor=1048576, SystemLoadAverage=1054, ProcessCount=21, AvailableMemoryMB=26507 2020-03-24 03:59:24,553 DEBUG [Time-limited test] wal.WALProcedureStore(401): Starting WAL Procedure Store lease recovery 2020-03-24 03:59:24,556 INFO [Time-limited test] wal.WALProcedureStore(1134): Rolled new Procedure Store WAL, id=1 2020-03-24 03:59:24,556 DEBUG [Time-limited test] wal.WALProcedureStore(436): Lease acquired for flushLogId=1 2020-03-24 03:59:24,556 DEBUG [Time-limited test] wal.WALProcedureStore(454): No state logs to replay. 2020-03-24 03:59:24,559 INFO [WALProcedureStoreSyncThread] wal.WALProcedureStore(1134): Rolled new Procedure Store WAL, id=2 2020-03-24 03:59:24,560 INFO [WALProcedureStoreSyncThread] wal.WALProcedureStore(1221): Remove all state logs with ID less than 1, since no active procedures 2020-03-24 03:59:24,561 DEBUG [WALProcedureStoreSyncThread] wal.WALProcedureStore(1243): Removed log=/home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1335/yetus-jdk8-hadoop2-check/src/hbase-procedure/target/test-data/dd76aeb2-fe62-be76-3273-d13f409d446f/proc-logs/pv2-00000000000000000001.log, activeLogs=[/home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1335/yetus-jdk8-hadoop2-check/src/hbase-procedure/target/test-data/dd76aeb2-fe62-be76-3273-d13f409d446f/proc-logs/pv2-00000000000000000002.log] 2020-03-24 03:59:24,561 DEBUG [Time-limited test] wal.WALProcedureStore(401): Starting WAL Procedure Store lease recovery 2020-03-24 03:59:24,561 WARN [Time-limited test] wal.WALProcedureStore(1370): Remove uninitialized log: DeprecatedRawLocalFileStatus{path=file:/home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1335/yetus-jdk8-hadoop2-check/src/hbase-procedure/target/test-data/dd76aeb2-fe62-be76-3273-d13f409d446f/proc-logs/pv2-00000000000000000002.log; isDirectory=false; length=0; replication=1; blocksize=33554432; modification_time=1585022364000; access_time=1585022364554; owner=; group=; permission=rw-rw-rw-; isSymlink=false} 2020-03-24 03:59:24,567 INFO [Time-limited test] wal.WALProcedureStore(330): Stopping the WAL Procedure Store, isAbort=false ]]></system-err> </rerunError> </testcase> ```
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
