[
https://issues.apache.org/jira/browse/HDFS-9500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15609303#comment-15609303
]
Hadoop QA commented on HDFS-9500:
---------------------------------
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
40s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
58s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
35s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
8s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
8s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
49s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 42m 26s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 65m 37s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Timed out junit tests | org.apache.hadoop.hdfs.TestLeaseRecovery2 |
| | org.apache.hadoop.hdfs.TestFileChecksum |
| | org.apache.hadoop.hdfs.TestParallelShortCircuitLegacyRead |
| | org.apache.hadoop.hdfs.server.namenode.TestStartup |
| | org.apache.hadoop.hdfs.TestWriteRead |
| | org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure |
| | org.apache.hadoop.fs.viewfs.TestViewFileSystemHdfs |
| | org.apache.hadoop.hdfs.server.blockmanagement.TestDatanodeManager |
| | org.apache.hadoop.hdfs.TestFileCreationClient |
| | org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
|
| | org.apache.hadoop.hdfs.server.datanode.TestNNHandlesBlockReportPerStorage
|
| | org.apache.hadoop.hdfs.server.datanode.TestDataNodeMetrics |
| | org.apache.hadoop.hdfs.server.datanode.TestNNHandlesCombinedBlockReport |
| | org.apache.hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes |
| | org.apache.hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints |
| | org.apache.hadoop.hdfs.server.datanode.TestDataNodeRollingUpgrade |
| | org.apache.hadoop.hdfs.TestMaintenanceState |
| | org.apache.hadoop.hdfs.server.blockmanagement.TestReplicationPolicy |
| | org.apache.hadoop.hdfs.server.namenode.TestFileContextAcl |
| | org.apache.hadoop.fs.TestEnhancedByteBufferAccess |
| | org.apache.hadoop.hdfs.server.namenode.TestLargeDirectoryDelete |
| | org.apache.hadoop.hdfs.TestDFSClientFailover |
| | org.apache.hadoop.hdfs.TestSetrepIncreasing |
| | org.apache.hadoop.hdfs.TestClientProtocolForPipelineRecovery |
| | org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure180 |
| | org.apache.hadoop.hdfs.server.datanode.TestBlockReplacement |
| | org.apache.hadoop.hdfs.server.namenode.TestNameNodeRecovery |
| | org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure160 |
| | org.apache.hadoop.hdfs.TestAclsEndToEnd |
| | org.apache.hadoop.hdfs.server.namenode.TestNamenodeCapacityReport |
| | org.apache.hadoop.hdfs.server.namenode.ha.TestBootstrapStandbyWithQJM |
| | org.apache.hadoop.hdfs.TestFileCreation |
| | org.apache.hadoop.hdfs.server.datanode.TestReadOnlySharedStorage |
| | org.apache.hadoop.hdfs.TestReplication |
| | org.apache.hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot
|
| | org.apache.hadoop.hdfs.server.namenode.TestINodeFile |
| | org.apache.hadoop.hdfs.server.namenode.TestEditLog |
| | org.apache.hadoop.hdfs.server.namenode.TestNameNodeAcl |
| | org.apache.hadoop.hdfs.server.datanode.TestDataNodeLifeline |
| | org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyWriter |
| | org.apache.hadoop.hdfs.server.namenode.ha.TestEditLogTailer |
| | org.apache.hadoop.hdfs.TestSafeModeWithStripedFile |
| | org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure130 |
| | org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl |
| | org.apache.hadoop.hdfs.TestReconstructStripedFile |
| | org.apache.hadoop.hdfs.TestWriteReadStripedFile |
| | org.apache.hadoop.hdfs.server.datanode.TestLargeBlockReport |
| | org.apache.hadoop.hdfs.server.namenode.TestBackupNode |
| | org.apache.hadoop.hdfs.server.namenode.TestFSImage |
| | org.apache.hadoop.hdfs.TestDFSClientRetries |
| | org.apache.hadoop.hdfs.server.namenode.TestQuotaByStorageType |
| | org.apache.hadoop.fs.viewfs.TestViewFileSystemAtHdfsRoot |
| | org.apache.hadoop.hdfs.TestDFSStorageStateRecovery |
| | org.apache.hadoop.hdfs.TestFileAppend3 |
| | org.apache.hadoop.hdfs.TestHDFSFileSystemContract |
| | org.apache.hadoop.hdfs.TestFileAppend2 |
| | org.apache.hadoop.hdfs.server.namenode.ha.TestStandbyIsHot |
| | org.apache.hadoop.hdfs.qjournal.client.TestQJMWithFaults |
| | org.apache.hadoop.hdfs.TestDFSInotifyEventInputStream |
| | org.apache.hadoop.hdfs.server.namenode.TestFSImageWithXAttr |
| | org.apache.hadoop.hdfs.TestDFSPermission |
| | org.apache.hadoop.hdfs.TestDecommission |
| | org.apache.hadoop.hdfs.server.datanode.TestBatchIbr |
| |
org.apache.hadoop.hdfs.server.diskbalancer.command.TestDiskBalancerCommand |
| | org.apache.hadoop.hdfs.server.namenode.TestCacheDirectives |
| | org.apache.hadoop.hdfs.server.namenode.TestAddOverReplicatedStripedBlocks
|
| | org.apache.hadoop.hdfs.server.namenode.TestFSImageWithSnapshot |
| | org.apache.hadoop.hdfs.security.TestDelegationToken |
| | org.apache.hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks |
| | org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotBlocksMap |
| | org.apache.hadoop.hdfs.TestDFSStripedOutputStream |
| | org.apache.hadoop.hdfs.client.impl.TestBlockReaderLocal |
| | org.apache.hadoop.hdfs.server.balancer.TestBalancerWithEncryptedTransfer |
| | org.apache.hadoop.hdfs.server.diskbalancer.TestDiskBalancer |
| |
org.apache.hadoop.hdfs.server.blockmanagement.TestBlocksWithNotEnoughRacks |
| | org.apache.hadoop.hdfs.server.blockmanagement.TestOverReplicatedBlocks |
| |
org.apache.hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped |
| | org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshot |
| | org.apache.hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots |
| | org.apache.hadoop.hdfs.server.namenode.TestFsck |
| |
org.apache.hadoop.hdfs.server.namenode.ha.TestFailoverWithBlockTokensEnabled |
| | org.apache.hadoop.hdfs.server.namenode.ha.TestDNFencing |
| | org.apache.hadoop.hdfs.TestCrcCorruption |
| | org.apache.hadoop.hdfs.server.balancer.TestBalancerWithSaslDataTransfer |
| | org.apache.hadoop.hdfs.server.blockmanagement.TestBlockStatsMXBean |
| | org.apache.hadoop.hdfs.TestLeaseRecovery |
| | org.apache.hadoop.hdfs.server.blockmanagement.TestPendingInvalidateBlock |
| | org.apache.hadoop.hdfs.server.namenode.TestReconstructStripedBlocks |
| | org.apache.hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits |
| | org.apache.hadoop.hdfs.server.namenode.snapshot.TestNestedSnapshots |
| | org.apache.hadoop.hdfs.server.namenode.TestFileTruncate |
| | org.apache.hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes |
| | org.apache.hadoop.hdfs.server.namenode.TestDiskspaceQuotaUpdate |
| |
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestScrLazyPersistFiles |
| | org.apache.hadoop.hdfs.server.namenode.TestListCorruptFileBlocks |
| | org.apache.hadoop.hdfs.server.namenode.TestDeadDatanode |
| | org.apache.hadoop.hdfs.server.mover.TestStorageMover |
| | org.apache.hadoop.hdfs.server.namenode.TestProcessCorruptBlocks |
| | org.apache.hadoop.hdfs.server.balancer.TestBalancer |
| | org.apache.hadoop.hdfs.TestDatanodeReport |
| | org.apache.hadoop.hdfs.TestReadStripedFileWithDecoding |
| |
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistFiles |
| | org.apache.hadoop.hdfs.TestAppendSnapshotTruncate |
| | org.apache.hadoop.hdfs.server.datanode.TestDataNodeMXBean |
| | org.apache.hadoop.hdfs.server.namenode.TestCheckpoint |
| | org.apache.hadoop.hdfs.server.namenode.TestNameNodeMXBean |
| | org.apache.hadoop.hdfs.server.namenode.TestAddStripedBlocks |
| | org.apache.hadoop.hdfs.TestReadStripedFileWithMissingBlocks |
| | org.apache.hadoop.hdfs.TestDataStream |
| | org.apache.hadoop.hdfs.server.namenode.TestNameEditsConfigs |
| | org.apache.hadoop.hdfs.server.datanode.TestDataNodeMultipleRegistrations |
| | org.apache.hadoop.hdfs.server.mover.TestMover |
| | org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure090 |
| |
org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureToleration |
| |
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestSpaceReservation |
| | org.apache.hadoop.hdfs.server.namenode.TestNamenodeRetryCache |
| | org.apache.hadoop.hdfs.server.namenode.TestNestedEncryptionZones |
| | org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure080 |
| | org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure070 |
| | org.apache.hadoop.hdfs.server.diskbalancer.TestDiskBalancerRPC |
| | org.apache.hadoop.hdfs.TestRenameWhileOpen |
| | org.apache.hadoop.hdfs.server.datanode.TestFsDatasetCache |
| |
org.apache.hadoop.hdfs.server.blockmanagement.TestReconstructStripedBlocksWithRackAwareness
|
| | org.apache.hadoop.hdfs.server.namenode.ha.TestDNFencingWithReplication |
| | org.apache.hadoop.hdfs.server.namenode.TestFSEditLogLoader |
| | org.apache.hadoop.metrics2.sink.TestRollingFileSystemSinkWithHdfs |
| | org.apache.hadoop.hdfs.TestFileConcurrentReader |
| | org.apache.hadoop.hdfs.server.datanode.TestBPOfferService |
| | org.apache.hadoop.hdfs.server.blockmanagement.TestPendingReconstruction |
| | org.apache.hadoop.hdfs.server.datanode.TestDirectoryScanner |
| | org.apache.hadoop.hdfs.server.namenode.TestFSImageWithAcl |
| | org.apache.hadoop.hdfs.server.namenode.ha.TestHAStateTransitions |
| | org.apache.hadoop.hdfs.shortcircuit.TestShortCircuitLocalRead |
| |
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaPlacement
|
| | org.apache.hadoop.hdfs.server.namenode.TestHDFSConcat |
| |
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery
|
| | org.apache.hadoop.hdfs.server.namenode.TestDeleteRace |
| | org.apache.hadoop.hdfs.TestDecommissionWithStriped |
| | org.apache.hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA |
| | org.apache.hadoop.hdfs.server.namenode.ha.TestBootstrapStandby |
| | org.apache.hadoop.hdfs.server.namenode.ha.TestStandbyInProgressTail |
| | org.apache.hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA |
| | org.apache.hadoop.hdfs.server.namenode.TestEditLogRace |
| | org.apache.hadoop.hdfs.server.namenode.TestAddBlock |
| | org.apache.hadoop.hdfs.server.datanode.TestBlockScanner |
| | org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotDeletion |
| | org.apache.hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency |
| | org.apache.hadoop.hdfs.TestDistributedFileSystem |
| | org.apache.hadoop.fs.TestSWebHdfsFileContextMainOperations |
| | org.apache.hadoop.hdfs.TestHFlush |
| | org.apache.hadoop.hdfs.server.namenode.ha.TestPipelinesFailover |
| | org.apache.hadoop.fs.TestSymlinkHdfsFileSystem |
| | org.apache.hadoop.hdfs.server.namenode.TestAuditLogs |
| | org.apache.hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFS |
| | org.apache.hadoop.hdfs.server.namenode.TestHostsFiles |
| | org.apache.hadoop.hdfs.TestGetBlocks |
| | org.apache.hadoop.hdfs.server.namenode.ha.TestHASafeMode |
| | org.apache.hadoop.hdfs.server.balancer.TestBalancerWithNodeGroup |
| | org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure |
| | org.apache.hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics |
| | org.apache.hadoop.cli.TestHDFSCLI |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Issue | HDFS-9500 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12835385/HDFS-9500.002.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux f77703842fb8 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28
20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / f209e93 |
| Default Java | 1.8.0_101 |
| findbugs | v3.0.0 |
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/17294/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/17294/testReport/ |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/17294/console |
| Powered by | Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> datanodesSoftwareVersions map may counting wrong when rolling upgrade
> ---------------------------------------------------------------------
>
> Key: HDFS-9500
> URL: https://issues.apache.org/jira/browse/HDFS-9500
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 2.7.1, 2.6.2
> Reporter: Phil Yang
> Assignee: Erik Krogen
> Attachments: 9500-v1.patch, HDFS-9500.000.patch, HDFS-9500.001.patch,
> HDFS-9500.002.patch
>
>
> While rolling upgrading, namenode's website overview will report there are
> two versions datanodes in the cluster, for example, 2.6.0 has x nodes and
> 2.6.2 has y nodes. However, sometimes when I stop a datanode in old version
> and start a new version one, namenode only increases the number of new
> version but not decreases the number of old version. So the total number x+y
> will be larger than the number of datanodes. Even all datanodes are upgraded,
> there will still have the messages that there are several datanode in old
> version. And I must run hdfs dfsadmin -refreshNodes to clear this message.
> I think this issue is caused by DatanodeManager.registerDatanode. If nodeS in
> old version is not alive because of shutting down, it will not pass
> shouldCountVersion, so the number of old version won't be decreased. But this
> method only judges the status of heartbeat and isAlive at that moment, if
> namenode has not removed this node which will decrease the version map and
> this node restarts in the new version, the decrementVersionCount belongs to
> this node will never be executed.
> So the simplest way to fix this is that we always recounting the version map
> in registerDatanode since it is not a heavy operation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
