[jira] [Updated] (HDFS-15058) TestFsck.testFsckListCorruptFilesBlocks and TestFsck.testFsckListCorruptSnapshotFiles fail some times
[
https://issues.apache.org/jira/browse/HDFS-15058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
liusheng updated HDFS-15058:
Description:
when I am try to run the tests of HDFS, the
*TestFsck.testFsckListCorruptFilesBlocks* and
*TestFsck.testFsckListCorruptSnapshotFiles* tests are easy to fail, see:
{code:java}
06:26:38 [ERROR] Failures:
06:26:38 [ERROR] TestFsck.testFsckListCorruptFilesBlocks:1167
06:26:38 [ERROR] TestFsck.testFsckListCorruptSnapshotFiles:2167
06:26:38 [INFO]
06:26:38 [ERROR] Tests run: 33, Failures: 2, Errors: 0, Skipped: 0
{code}
Both of these two tests failures are mainly because the tests will check the
number of corrupt files after sleep *1000 ms* and the number is not equal to
expected. see:
{noformat}
blk_1073741825 /corruptData/8117051706407353421
blk_1073741825 /corruptData/.snapshot/mySnapShot/8117051706407353421
The filesystem under path '/corruptData' has 2 CORRUPT files2. bad fsck include
snapshot out: The list of corrupt files under path '/corruptData' are:
blk_1073741825 /corruptData/8117051706407353421
blk_1073741825 /corruptData/.snapshot/mySnapShot/8117051706407353421
The filesystem under path '/corruptData' has 2 CORRUPT files2019-12-13
06:26:35,808 [Listener at localhost/44367] INFO hdfs.MiniDFSCluster
(MiniDFSCluster.java:shutdown(2067)) - Shutting down the Mini HDFS Cluster
2019-12-13 06:26:35,808 [Listener at localhost/44367] INFO hdfs.MiniDFSCluster
(MiniDFSCluster.java:shutdownDataNode(2115)) - Shutting down DataNode 0
{noformat}
To fix these two tests, we need to enlarge the sleep time of 1000 ms, according
to my testing, enlaging the time to *5000* ms can make the tests passed every
times.
was:
when I am try to run the tests of HDFS, the
*TestFsck.testFsckListCorruptFilesBlocks* and
*TestFsck.testFsckListCorruptSnapshotFiles* tests are easy to fail, see:
{code:java}
06:26:38 [ERROR] Failures:
06:26:38 [ERROR] TestFsck.testFsckListCorruptFilesBlocks:1167
06:26:38 [ERROR] TestFsck.testFsckListCorruptSnapshotFiles:2167
06:26:38 [INFO]
06:26:38 [ERROR] Tests run: 33, Failures: 2, Errors: 0, Skipped: 0
{code}
Both of these two tests failures are mainly because the tests will check the
number of corrupt files after sleep *1000 ms* and the number is not equal to
expected. see:
{noformat}
blk_1073741825 /corruptData/8117051706407353421
blk_1073741825 /corruptData/.snapshot/mySnapShot/8117051706407353421
The filesystem under path '/corruptData' has 2 CORRUPT files2. bad fsck include
snapshot out: The list of corrupt files under path '/corruptData' are:
blk_1073741825 /corruptData/8117051706407353421
blk_1073741825 /corruptData/.snapshot/mySnapShot/8117051706407353421
The filesystem under path '/corruptData' has 2 CORRUPT files2019-12-13
06:26:35,808 [Listener at localhost/44367] INFO hdfs.MiniDFSCluster
(MiniDFSCluster.java:shutdown(2067)) - Shutting down the Mini HDFS Cluster
2019-12-13 06:26:35,808 [Listener at localhost/44367] INFO hdfs.MiniDFSCluster
(MiniDFSCluster.java:shutdownDataNode(2115)) - Shutting down DataNode 0
{noformat}
To fix these two tests, we need to enlarge the sleep time of 1000 ms, according
to my testing, enlaging the time to *5000* ms can make the tests passed every
times.
> TestFsck.testFsckListCorruptFilesBlocks and
> TestFsck.testFsckListCorruptSnapshotFiles fail some times
> -
>
> Key: HDFS-15058
> URL: https://issues.apache.org/jira/browse/HDFS-15058
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: liusheng
>Priority: Major
>
> when I am try to run the tests of HDFS, the
> *TestFsck.testFsckListCorruptFilesBlocks* and
> *TestFsck.testFsckListCorruptSnapshotFiles* tests are easy to fail, see:
> {code:java}
> 06:26:38 [ERROR] Failures:
> 06:26:38 [ERROR] TestFsck.testFsckListCorruptFilesBlocks:1167
> 06:26:38 [ERROR] TestFsck.testFsckListCorruptSnapshotFiles:2167
> 06:26:38 [INFO]
> 06:26:38 [ERROR] Tests run: 33, Failures: 2, Errors: 0, Skipped: 0
> {code}
> Both of these two tests failures are mainly because the tests will check the
> number of corrupt files after sleep *1000 ms* and the number is not equal to
> expected. see:
> {noformat}
> blk_1073741825 /corruptData/8117051706407353421
> blk_1073741825 /corruptData/.snapshot/mySnapShot/8117051706407353421
> The filesystem under path '/corruptData' has 2 CORRUPT files2. bad fsck
> include snapshot out: The list of corrupt files under path '/corruptData' are:
> blk_1073741825 /corruptData/8117051706407353421
> blk_1073741825 /corruptData/.snapshot/mySnapShot/8117051706407353421
> The filesystem under path '/corruptData' has 2 CORRUPT files2019-12-13
> 06:26:35,808 [Listener at localhost/44367] INFO hdfs.MiniDFSCluster
> (MiniDFSCluster.java:shutdown(2067)) - Shutting
[jira] [Updated] (HDFS-15058) TestFsck.testFsckListCorruptFilesBlocks and TestFsck.testFsckListCorruptSnapshotFiles fail some times
[
https://issues.apache.org/jira/browse/HDFS-15058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
liusheng updated HDFS-15058:
Description:
when I am try to run the tests of HDFS, the
*TestFsck.testFsckListCorruptFilesBlocks* and
*TestFsck.testFsckListCorruptSnapshotFiles* tests are easy to fail, see:
{code:java}
06:26:38 [ERROR] Failures:
06:26:38 [ERROR] TestFsck.testFsckListCorruptFilesBlocks:1167
06:26:38 [ERROR] TestFsck.testFsckListCorruptSnapshotFiles:2167
06:26:38 [INFO]
06:26:38 [ERROR] Tests run: 33, Failures: 2, Errors: 0, Skipped: 0
{code}
Both of these two tests failures are mainly because the tests will check the
number of corrupt files after sleep *1000 ms* and the number is not equal to
expected. see:
{noformat}
blk_1073741825 /corruptData/8117051706407353421
blk_1073741825 /corruptData/.snapshot/mySnapShot/8117051706407353421
The filesystem under path '/corruptData' has 2 CORRUPT files2. bad fsck include
snapshot out: The list of corrupt files under path '/corruptData' are:
blk_1073741825 /corruptData/8117051706407353421
blk_1073741825 /corruptData/.snapshot/mySnapShot/8117051706407353421
The filesystem under path '/corruptData' has 2 CORRUPT files2019-12-13
06:26:35,808 [Listener at localhost/44367] INFO hdfs.MiniDFSCluster
(MiniDFSCluster.java:shutdown(2067)) - Shutting down the Mini HDFS Cluster
2019-12-13 06:26:35,808 [Listener at localhost/44367] INFO hdfs.MiniDFSCluster
(MiniDFSCluster.java:shutdownDataNode(2115)) - Shutting down DataNode 0
{noformat}
To fix these two tests, we need to enlarge the sleep time of 1000 ms, according
to my testing, enlaging the time to *5000* ms can make the tests passed every
times.
was:
when I am try to run the tests of HDFS, the
*TestFsck.testFsckListCorruptFilesBlocks* and
*TestFsck.testFsckListCorruptSnapshotFiles* tests are easy to fail, see:
{code:java}
06:26:38 [ERROR] Failures:
06:26:38 [ERROR] TestFsck.testFsckListCorruptFilesBlocks:1167
06:26:38 [ERROR] TestFsck.testFsckListCorruptSnapshotFiles:2167
06:26:38 [INFO]
06:26:38 [ERROR] Tests run: 33, Failures: 2, Errors: 0, Skipped: 0
{code}
Both of these two tests failures are mainly because the tests will check the
number of corrupt files after sleep *1000 ms* and the number is not equal to
expected. see:
{noformat}
blk_1073741825 /corruptData/8117051706407353421
blk_1073741825 /corruptData/.snapshot/mySnapShot/8117051706407353421
The filesystem under path '/corruptData' has 2 CORRUPT files2. bad fsck include
snapshot out: The list of corrupt files under path '/corruptData' are:
blk_1073741825 /corruptData/8117051706407353421
blk_1073741825 /corruptData/.snapshot/mySnapShot/8117051706407353421
The filesystem under path '/corruptData' has 2 CORRUPT files2019-12-13
06:26:35,808 [Listener at localhost/44367] INFO hdfs.MiniDFSCluster
(MiniDFSCluster.java:shutdown(2067)) - Shutting down the Mini HDFS Cluster
2019-12-13 06:26:35,808 [Listener at localhost/44367] INFO hdfs.MiniDFSCluster
(MiniDFSCluster.java:shutdownDataNode(2115)) - Shutting down DataNode 0
{noformat}
To fix these two tests, we need to enlarge the sleep time of 1000 ms, according
to my testing, enlaging the time to 5000 ms can ensure tests passed every times.
> TestFsck.testFsckListCorruptFilesBlocks and
> TestFsck.testFsckListCorruptSnapshotFiles fail some times
> -
>
> Key: HDFS-15058
> URL: https://issues.apache.org/jira/browse/HDFS-15058
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: liusheng
>Priority: Major
>
> when I am try to run the tests of HDFS, the
> *TestFsck.testFsckListCorruptFilesBlocks* and
> *TestFsck.testFsckListCorruptSnapshotFiles* tests are easy to fail, see:
> {code:java}
> 06:26:38 [ERROR] Failures:
> 06:26:38 [ERROR] TestFsck.testFsckListCorruptFilesBlocks:1167
> 06:26:38 [ERROR] TestFsck.testFsckListCorruptSnapshotFiles:2167
> 06:26:38 [INFO]
> 06:26:38 [ERROR] Tests run: 33, Failures: 2, Errors: 0, Skipped: 0
> {code}
> Both of these two tests failures are mainly because the tests will check the
> number of corrupt files after sleep *1000 ms* and the number is not equal to
> expected. see:
> {noformat}
> blk_1073741825 /corruptData/8117051706407353421
> blk_1073741825 /corruptData/.snapshot/mySnapShot/8117051706407353421
> The filesystem under path '/corruptData' has 2 CORRUPT files2. bad fsck
> include snapshot out: The list of corrupt files under path '/corruptData' are:
> blk_1073741825 /corruptData/8117051706407353421
> blk_1073741825 /corruptData/.snapshot/mySnapShot/8117051706407353421
> The filesystem under path '/corruptData' has 2 CORRUPT files2019-12-13
> 06:26:35,808 [Listener at localhost/44367] INFO hdfs.MiniDFSCluster
> (MiniDFSCluster.java:shutdown(2067)) -
