[jira] [Commented] (HDFS-9191) Typo in Hdfs.java. NoSuchElementException is misspelled
[ https://issues.apache.org/jira/browse/HDFS-9191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14942199#comment-14942199 ] Hudson commented on HDFS-9191: -- FAILURE: Integrated in Hadoop-trunk-Commit #8561 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/8561/]) Move HDFS-9191 to 2.8.0. (aajisaka: rev ed5f51faa8e4b37bbe0fd86de7fef10eab449b6e) * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt > Typo in Hdfs.java. NoSuchElementException is misspelled > - > > Key: HDFS-9191 > URL: https://issues.apache.org/jira/browse/HDFS-9191 > Project: Hadoop HDFS > Issue Type: Bug > Components: HDFS >Reporter: Catherine Palmer >Assignee: Catherine Palmer >Priority: Trivial > Labels: newbie > Fix For: 3.0.0, 2.8.0 > > Attachments: hdfs-9191.patch > > > Line 241 NoSuchElementException has a typo -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9191) Typo in Hdfs.java. NoSuchElementException is misspelled
[ https://issues.apache.org/jira/browse/HDFS-9191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14942221#comment-14942221 ] Hudson commented on HDFS-9191: -- SUCCESS: Integrated in Hadoop-Yarn-trunk #1214 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/1214/]) Move HDFS-9191 to 2.8.0. (aajisaka: rev ed5f51faa8e4b37bbe0fd86de7fef10eab449b6e) * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt > Typo in Hdfs.java. NoSuchElementException is misspelled > - > > Key: HDFS-9191 > URL: https://issues.apache.org/jira/browse/HDFS-9191 > Project: Hadoop HDFS > Issue Type: Bug > Components: HDFS >Reporter: Catherine Palmer >Assignee: Catherine Palmer >Priority: Trivial > Labels: newbie > Fix For: 3.0.0, 2.8.0 > > Attachments: hdfs-9191.patch > > > Line 241 NoSuchElementException has a typo -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HDFS-9191) Typo in Hdfs.java. NoSuchElementException is misspelled
[ https://issues.apache.org/jira/browse/HDFS-9191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira AJISAKA updated HDFS-9191: Fix Version/s: 2.8.0 Backported this to branch-2 since the fix is compatible change. > Typo in Hdfs.java. NoSuchElementException is misspelled > - > > Key: HDFS-9191 > URL: https://issues.apache.org/jira/browse/HDFS-9191 > Project: Hadoop HDFS > Issue Type: Bug > Components: HDFS >Reporter: Catherine Palmer >Assignee: Catherine Palmer >Priority: Trivial > Labels: newbie > Fix For: 3.0.0, 2.8.0 > > Attachments: hdfs-9191.patch > > > Line 241 NoSuchElementException has a typo -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9191) Typo in Hdfs.java. NoSuchElementException is misspelled
[ https://issues.apache.org/jira/browse/HDFS-9191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14942233#comment-14942233 ] Hudson commented on HDFS-9191: -- SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #483 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/483/]) Move HDFS-9191 to 2.8.0. (aajisaka: rev ed5f51faa8e4b37bbe0fd86de7fef10eab449b6e) * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt > Typo in Hdfs.java. NoSuchElementException is misspelled > - > > Key: HDFS-9191 > URL: https://issues.apache.org/jira/browse/HDFS-9191 > Project: Hadoop HDFS > Issue Type: Bug > Components: HDFS >Reporter: Catherine Palmer >Assignee: Catherine Palmer >Priority: Trivial > Labels: newbie > Fix For: 3.0.0, 2.8.0 > > Attachments: hdfs-9191.patch > > > Line 241 NoSuchElementException has a typo -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7899) Improve EOF error message
[
https://issues.apache.org/jira/browse/HDFS-7899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14942281#comment-14942281
]
Harsh J commented on HDFS-7899:
---
Sorry on delay here!
Perhaps it could be fixed to:
s/Unexpected EOF while trying read response/Unexpected EOF while trying to read
response from server
(With hope that it would get some users to look above it and spot the
server/block identifiers to investigate forward.)
> Improve EOF error message
> -
>
> Key: HDFS-7899
> URL: https://issues.apache.org/jira/browse/HDFS-7899
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs-client
>Affects Versions: 2.6.0
>Reporter: Harsh J
>Assignee: Jagadesh Kiran N
>Priority: Minor
> Attachments: HDFS-7899-00.patch, HDFS-7899-01.patch
>
>
> Currently, a DN disconnection for reasons other than connection timeout or
> refused messages, such as an EOF message as a result of rejection or other
> network fault, reports in this manner:
> {code}
> WARN org.apache.hadoop.hdfs.DFSClient: Failed to connect to /x.x.x.x: for
> block, add to deadNodes and continue. java.io.EOFException: Premature EOF: no
> length prefix available
> java.io.EOFException: Premature EOF: no length prefix available
> at
> org.apache.hadoop.hdfs.protocol.HdfsProtoUtil.vintPrefixed(HdfsProtoUtil.java:171)
>
> at
> org.apache.hadoop.hdfs.RemoteBlockReader2.newBlockReader(RemoteBlockReader2.java:392)
>
> at
> org.apache.hadoop.hdfs.BlockReaderFactory.newBlockReader(BlockReaderFactory.java:137)
>
> at
> org.apache.hadoop.hdfs.DFSInputStream.getBlockReader(DFSInputStream.java:1103)
>
> at org.apache.hadoop.hdfs.DFSInputStream.blockSeekTo(DFSInputStream.java:538)
> at
> org.apache.hadoop.hdfs.DFSInputStream.readWithStrategy(DFSInputStream.java:750)
>
> at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:794)
> at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:602)
> {code}
> This is not very clear to a user (warn's at the hdfs-client). It could likely
> be improved with a more diagnosable message, or at least the direct reason
> than an EOF.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HDFS-9194) AlreadyBeingCreatedException ... because pendingCreates is non-null but no leases found.
Tsz Wo Nicholas Sze created HDFS-9194:
-
Summary: AlreadyBeingCreatedException ... because pendingCreates
is non-null but no leases found.
Key: HDFS-9194
URL: https://issues.apache.org/jira/browse/HDFS-9194
Project: Hadoop HDFS
Issue Type: Bug
Components: namenode
Reporter: Tsz Wo Nicholas Sze
Assignee: Tsz Wo Nicholas Sze
There is a possible bug in FSDirectory.addFile causing no leases found for
under construction files.
{code}
//FSDirectory
INodeFile addFile(String path, PermissionStatus permissions,
short replication, long preferredBlockSize,
String clientName, String clientMachine)
throws FileAlreadyExistsException, QuotaExceededException,
UnresolvedLinkException, SnapshotAccessControlException, AclException {
long modTime = now();
INodeFile newNode = newINodeFile(namesystem.allocateNewInodeId(),
permissions, modTime, modTime, replication, preferredBlockSize);
newNode.toUnderConstruction(clientName, clientMachine);
boolean added = false;
writeLock();
try {
added = addINode(path, newNode);
} finally {
writeUnlock();
}
...
}
{code}
- newNode.toUnderConstruction(clientName, clientMachine) adds
FileUnderConstructionFeature to the INode, i.e. the file becomes an under
construction file. At this moment, there is no lease for this file yet. The
lease will be added later in FSNamesystem.startFileInternal(..).
- It is possible that addINode(path, newNode) adds the inode to the namespace
tree but throws QuotaExceededException later on when calling
updateModificationTime. (i.e. addINode -> addLastINode -> addChild ->
parent.addChild -> updateModificationTime throws QuotaExceededException) Then,
the newly added uc file is left in namespace but the corresponding lease won't
be added.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-9191) Typo in Hdfs.java. NoSuchElementException is misspelled
[ https://issues.apache.org/jira/browse/HDFS-9191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14942216#comment-14942216 ] Hudson commented on HDFS-9191: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #2419 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2419/]) Move HDFS-9191 to 2.8.0. (aajisaka: rev ed5f51faa8e4b37bbe0fd86de7fef10eab449b6e) * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt > Typo in Hdfs.java. NoSuchElementException is misspelled > - > > Key: HDFS-9191 > URL: https://issues.apache.org/jira/browse/HDFS-9191 > Project: Hadoop HDFS > Issue Type: Bug > Components: HDFS >Reporter: Catherine Palmer >Assignee: Catherine Palmer >Priority: Trivial > Labels: newbie > Fix For: 3.0.0, 2.8.0 > > Attachments: hdfs-9191.patch > > > Line 241 NoSuchElementException has a typo -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9191) Typo in Hdfs.java. NoSuchElementException is misspelled
[ https://issues.apache.org/jira/browse/HDFS-9191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14942290#comment-14942290 ] Hudson commented on HDFS-9191: -- FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #449 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/449/]) Move HDFS-9191 to 2.8.0. (aajisaka: rev ed5f51faa8e4b37bbe0fd86de7fef10eab449b6e) * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt > Typo in Hdfs.java. NoSuchElementException is misspelled > - > > Key: HDFS-9191 > URL: https://issues.apache.org/jira/browse/HDFS-9191 > Project: Hadoop HDFS > Issue Type: Bug > Components: HDFS >Reporter: Catherine Palmer >Assignee: Catherine Palmer >Priority: Trivial > Labels: newbie > Fix For: 3.0.0, 2.8.0 > > Attachments: hdfs-9191.patch > > > Line 241 NoSuchElementException has a typo -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HDFS-4015) Safemode should count and report orphaned blocks
[ https://issues.apache.org/jira/browse/HDFS-4015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer updated HDFS-4015: --- Status: Open (was: Patch Available) > Safemode should count and report orphaned blocks > > > Key: HDFS-4015 > URL: https://issues.apache.org/jira/browse/HDFS-4015 > Project: Hadoop HDFS > Issue Type: Improvement > Components: namenode >Affects Versions: 3.0.0 >Reporter: Todd Lipcon >Assignee: Anu Engineer > Attachments: HDFS-4015.001.patch, HDFS-4015.002.patch, > dfsAdmin-report_with_forceExit.png, dfsHealth.html.message.png > > > The safemode status currently reports the number of unique reported blocks > compared to the total number of blocks referenced by the namespace. However, > it does not report the inverse: blocks which are reported by datanodes but > not referenced by the namespace. > In the case that an admin accidentally starts up from an old image, this can > be confusing: safemode and fsck will show "corrupt files", which are the > files which actually have been deleted but got resurrected by restarting from > the old image. This will convince them that they can safely force leave > safemode and remove these files -- after all, they know that those files > should really have been deleted. However, they're not aware that leaving > safemode will also unrecoverably delete a bunch of other block files which > have been orphaned due to the namespace rollback. > I'd like to consider reporting something like: "90 of expected 100 > blocks have been reported. Additionally, 1 blocks have been reported > which do not correspond to any file in the namespace. Forcing exit of > safemode will unrecoverably remove those data blocks" > Whether this statistic is also used for some kind of "inverse safe mode" is > the logical next step, but just reporting it as a warning seems easy enough > to accomplish and worth doing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HDFS-4015) Safemode should count and report orphaned blocks
[ https://issues.apache.org/jira/browse/HDFS-4015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer updated HDFS-4015: --- Status: Patch Available (was: Open) > Safemode should count and report orphaned blocks > > > Key: HDFS-4015 > URL: https://issues.apache.org/jira/browse/HDFS-4015 > Project: Hadoop HDFS > Issue Type: Improvement > Components: namenode >Affects Versions: 3.0.0 >Reporter: Todd Lipcon >Assignee: Anu Engineer > Attachments: HDFS-4015.001.patch, HDFS-4015.002.patch, > dfsAdmin-report_with_forceExit.png, dfsHealth.html.message.png > > > The safemode status currently reports the number of unique reported blocks > compared to the total number of blocks referenced by the namespace. However, > it does not report the inverse: blocks which are reported by datanodes but > not referenced by the namespace. > In the case that an admin accidentally starts up from an old image, this can > be confusing: safemode and fsck will show "corrupt files", which are the > files which actually have been deleted but got resurrected by restarting from > the old image. This will convince them that they can safely force leave > safemode and remove these files -- after all, they know that those files > should really have been deleted. However, they're not aware that leaving > safemode will also unrecoverably delete a bunch of other block files which > have been orphaned due to the namespace rollback. > I'd like to consider reporting something like: "90 of expected 100 > blocks have been reported. Additionally, 1 blocks have been reported > which do not correspond to any file in the namespace. Forcing exit of > safemode will unrecoverably remove those data blocks" > Whether this statistic is also used for some kind of "inverse safe mode" is > the logical next step, but just reporting it as a warning seems easy enough > to accomplish and worth doing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9191) Typo in Hdfs.java. NoSuchElementException is misspelled
[ https://issues.apache.org/jira/browse/HDFS-9191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14942208#comment-14942208 ] Hudson commented on HDFS-9191: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #475 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/475/]) Move HDFS-9191 to 2.8.0. (aajisaka: rev ed5f51faa8e4b37bbe0fd86de7fef10eab449b6e) * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt > Typo in Hdfs.java. NoSuchElementException is misspelled > - > > Key: HDFS-9191 > URL: https://issues.apache.org/jira/browse/HDFS-9191 > Project: Hadoop HDFS > Issue Type: Bug > Components: HDFS >Reporter: Catherine Palmer >Assignee: Catherine Palmer >Priority: Trivial > Labels: newbie > Fix For: 3.0.0, 2.8.0 > > Attachments: hdfs-9191.patch > > > Line 241 NoSuchElementException has a typo -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9191) Typo in Hdfs.java. NoSuchElementException is misspelled
[ https://issues.apache.org/jira/browse/HDFS-9191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14942240#comment-14942240 ] Hudson commented on HDFS-9191: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #2389 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/2389/]) Move HDFS-9191 to 2.8.0. (aajisaka: rev ed5f51faa8e4b37bbe0fd86de7fef10eab449b6e) * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt > Typo in Hdfs.java. NoSuchElementException is misspelled > - > > Key: HDFS-9191 > URL: https://issues.apache.org/jira/browse/HDFS-9191 > Project: Hadoop HDFS > Issue Type: Bug > Components: HDFS >Reporter: Catherine Palmer >Assignee: Catherine Palmer >Priority: Trivial > Labels: newbie > Fix For: 3.0.0, 2.8.0 > > Attachments: hdfs-9191.patch > > > Line 241 NoSuchElementException has a typo -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HDFS-9193) Fix incorrect references the usages of the DN in dfshealth.js
[ https://issues.apache.org/jira/browse/HDFS-9193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haohui Mai updated HDFS-9193: - Summary: Fix incorrect references the usages of the DN in dfshealth.js (was: when a datanode's usage go above 70 percent, we can't open datanodes tab in NN UI) > Fix incorrect references the usages of the DN in dfshealth.js > - > > Key: HDFS-9193 > URL: https://issues.apache.org/jira/browse/HDFS-9193 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Chang Li >Assignee: Chang Li >Priority: Minor > Attachments: HDFS-9193.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9193) Fix incorrect references the usages of the DN in dfshealth.js
[ https://issues.apache.org/jira/browse/HDFS-9193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14942409#comment-14942409 ] Haohui Mai commented on HDFS-9193: -- Thanks for reporting. +1. Will commit shortly. > Fix incorrect references the usages of the DN in dfshealth.js > - > > Key: HDFS-9193 > URL: https://issues.apache.org/jira/browse/HDFS-9193 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Chang Li >Assignee: Chang Li >Priority: Minor > Attachments: HDFS-9193.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8979) Clean up checkstyle warnings in hadoop-hdfs-client module
[ https://issues.apache.org/jira/browse/HDFS-8979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14942463#comment-14942463 ] Hudson commented on HDFS-8979: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #477 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/477/]) HDFS-8979. Clean up checkstyle warnings in hadoop-hdfs-client module. (wheat9: rev 7136e8c5582dc4061b566cb9f11a0d6a6d08bb93) * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/PacketHeader.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/ExcludeDatanodesParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/protocol/DatanodeStorageReport.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/GroupParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/CredentialBasedAccessTokenProvider.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocolPB/ClientNamenodeProtocolPB.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/ConfCredentialBasedAccessTokenProvider.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/HdfsDataOutputStream.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/GetOpParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/util/StripedBlockUtil.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocolPB/ClientNamenodeProtocolTranslatorPB.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/KeyProviderCache.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/AccessTokenTimer.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/RemoteBlockReader2.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/BlockStoragePolicy.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/CacheDirectiveIterator.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/XAttrNameParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/AccessTokenProvider.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/LocatedBlocks.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/CacheDirectiveInfo.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/SnapshotDiffReport.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/HttpOpParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/ConcatSourcesParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/ExternalBlockReader.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/DatanodeInfo.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/EncryptionZoneIterator.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/RenewerParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/security/token/delegation/DelegationTokenIdentifier.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/HdfsLocatedFileStatus.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/util/ByteBufferOutputStream.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/RemotePeerFactory.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/fs/CacheFlag.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/CreateParentParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/ClientProtocol.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/NewLengthParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/LengthParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/PutOpParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/impl/LeaseRenewer.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/ReplicationParam.java *
[jira] [Commented] (HDFS-8979) Clean up checkstyle warnings in hadoop-hdfs-client module
[
https://issues.apache.org/jira/browse/HDFS-8979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14942403#comment-14942403
]
Haohui Mai commented on HDFS-8979:
--
+1. Committing
> Clean up checkstyle warnings in hadoop-hdfs-client module
> -
>
> Key: HDFS-8979
> URL: https://issues.apache.org/jira/browse/HDFS-8979
> Project: Hadoop HDFS
> Issue Type: Task
>Reporter: Mingliang Liu
>Assignee: Mingliang Liu
> Attachments: HDFS-8979.000.patch, HDFS-8979.001.patch,
> HDFS-8979.002.patch
>
>
> This jira tracks the effort of cleaning up checkstyle warnings in
> {{hadoop-hdfs-client}} module.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8979) Clean up checkstyle warnings in hadoop-hdfs-client module
[ https://issues.apache.org/jira/browse/HDFS-8979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14942410#comment-14942410 ] Hudson commented on HDFS-8979: -- FAILURE: Integrated in Hadoop-trunk-Commit #8563 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/8563/]) HDFS-8979. Clean up checkstyle warnings in hadoop-hdfs-client module. (wheat9: rev 7136e8c5582dc4061b566cb9f11a0d6a6d08bb93) * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/shortcircuit/ClientMmap.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/DatanodeLocalInfo.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/NSQuotaExceededException.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/PostOpParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/GroupParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/package-info.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/protocol/DatanodeStorage.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/URLConnectionFactory.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/security/token/delegation/DelegationTokenSelector.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/datanode/CachingStrategy.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/HdfsDataOutputStream.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/GetOpParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/LengthParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/DatanodeInfo.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/net/Peer.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/DataTransferProtocol.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/AccessTokenProvider.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/LastBlockWithStatus.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/RecursiveParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/ByteRangeInputStream.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/DoAsParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/ConfRefreshTokenBasedAccessTokenProvider.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/DelegationParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/Sender.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/BlockReaderUtil.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/CreateParentParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/BlockLocalPathInfo.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocolPB/ClientNamenodeProtocolTranslatorPB.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/LocatedStripedBlock.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/fs/XAttr.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/PacketHeader.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSClient.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/EnumParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/AccessTimeParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/inotify/Event.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/OffsetParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/impl/LeaseRenewer.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/CachePoolIterator.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/PermissionParam.java *
[jira] [Commented] (HDFS-8979) Clean up checkstyle warnings in hadoop-hdfs-client module
[ https://issues.apache.org/jira/browse/HDFS-8979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14942446#comment-14942446 ] Hudson commented on HDFS-8979: -- SUCCESS: Integrated in Hadoop-Yarn-trunk #1216 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/1216/]) HDFS-8979. Clean up checkstyle warnings in hadoop-hdfs-client module. (wheat9: rev 7136e8c5582dc4061b566cb9f11a0d6a6d08bb93) * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/Block.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/RecursiveParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/XAttrHelper.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/sasl/DataTransferSaslUtil.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/impl/DfsClientConf.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/util/package-info.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/fs/CacheFlag.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/BlockReaderFactory.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/JsonUtilClient.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/sasl/SaslResponseWithNegotiatedCipherOption.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/Op.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/ModificationTimeParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/GetOpParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocolPB/ClientDatanodeProtocolTranslatorPB.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/ExtendedBlockId.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocolPB/PBHelperClient.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/DeleteOpParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/StripedDataStreamer.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/fs/XAttr.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/net/NioInetPeer.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/LocatedStripedBlock.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/HdfsDataInputStream.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/inotify/Event.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocolPB/ClientNamenodeProtocolTranslatorPB.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/XAttrEncodingParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSStripedInputStream.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/EnumSetParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/PostOpParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/XAttrSetFlagParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/ReplicationParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/DataTransferProtoUtil.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/BlockLocalPathInfo.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/GroupParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/CacheDirectiveInfo.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/CreateParentParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/datanode/ReplicaNotFoundException.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockMetadataHeader.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/PipelineAck.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/BlockReaderLocal.java *
[jira] [Commented] (HDFS-9193) Fix incorrect references the usages of the DN in dfshealth.js
[ https://issues.apache.org/jira/browse/HDFS-9193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14942445#comment-14942445 ] Hudson commented on HDFS-9193: -- SUCCESS: Integrated in Hadoop-Yarn-trunk #1216 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/1216/]) HDFS-9193. Fix incorrect references the usages of the DN in (wheat9: rev 3b85bd7c3391719624a261f061a67cc4356be477) * hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/dfshealth.js * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt > Fix incorrect references the usages of the DN in dfshealth.js > - > > Key: HDFS-9193 > URL: https://issues.apache.org/jira/browse/HDFS-9193 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Chang Li >Assignee: Chang Li >Priority: Minor > Attachments: HDFS-9193.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9193) Fix incorrect references the usages of the DN in dfshealth.js
[ https://issues.apache.org/jira/browse/HDFS-9193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14942462#comment-14942462 ] Hudson commented on HDFS-9193: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #477 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/477/]) HDFS-9193. Fix incorrect references the usages of the DN in (wheat9: rev 3b85bd7c3391719624a261f061a67cc4356be477) * hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/dfshealth.js * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt > Fix incorrect references the usages of the DN in dfshealth.js > - > > Key: HDFS-9193 > URL: https://issues.apache.org/jira/browse/HDFS-9193 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Chang Li >Assignee: Chang Li >Priority: Minor > Attachments: HDFS-9193.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HDFS-9170) Move libhdfs / fuse-dfs / libwebhdfs to hdfs-client
[ https://issues.apache.org/jira/browse/HDFS-9170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haohui Mai updated HDFS-9170: - Attachment: HDFS-9170.002.patch > Move libhdfs / fuse-dfs / libwebhdfs to hdfs-client > --- > > Key: HDFS-9170 > URL: https://issues.apache.org/jira/browse/HDFS-9170 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Haohui Mai >Assignee: Haohui Mai > Attachments: HDFS-9170.000.patch, HDFS-9170.001.patch, > HDFS-9170.002.patch > > > After HDFS-6200 the Java implementation of hdfs-client has be moved to a > separate hadoop-hdfs-client module. > libhdfs, fuse-dfs and libwebhdfs still reside in the hadoop-hdfs module. > Ideally these modules should reside in the hadoop-hdfs-client. However, to > write unit tests for these components, it is often necessary to run > MiniDFSCluster which resides in the hadoop-hdfs module. > This jira is to discuss how these native modules should layout after > HDFS-6200. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9181) Better handling of exceptions thrown during upgrade shutdown
[ https://issues.apache.org/jira/browse/HDFS-9181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14942421#comment-14942421 ] Eric Payne commented on HDFS-9181: -- Thanks [~jojochuang] and [~templedf] for bringing this to my attention. I am fine with putting out a warning in the case of an exception or throwable during shutdown. I want to emphasize that whatever resolution is decided upon for this JIRA, the functionality of allowing the shutdown thread to finish without hanging should be maintained. [~jojochuang] stated in [this comment of HDFS-7533|https://issues.apache.org/jira/browse/HDFS-7533?focusedCommentId=14936261=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14936261], "It looks to me that catching Throwable and ignore it may cause other issues, for example, ignoring a OOME can be pretty bad." Meanwhile, [~kihwal] pointed out in [this comment of HDFS-7533|https://issues.apache.org/jira/browse/HDFS-7533?focusedCommentId=14271049=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14271049] that "We can check whether a responder is running, but it may be in the process of shutting down. Therefore, a proper check requires additional locking." In my opinion, putting in the logic to do additional locking and handling of exceptions is more complicated than is warranted during the datanode shutdown thread. The locking could also cause additional risk and complexity. I do agree that ignoring an OOM may cause a total datanode crash. However, that would be more desirable in this case than a datanode hang, which was happening before HDFS-7533. Will ignoring other throwables also cause the datanode to hang? > Better handling of exceptions thrown during upgrade shutdown > > > Key: HDFS-9181 > URL: https://issues.apache.org/jira/browse/HDFS-9181 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Wei-Chiu Chuang >Assignee: Wei-Chiu Chuang >Priority: Minor > > Previously in HDFS-7533, a bug was fixed by suppressing exceptions during > upgrade shutdown. It may be appropriate as a temporary fix, but it would be > better if the exception is handled in some other ways. > One way to handle it is by emitting a warning message. There could exist > other ways to handle it. This lira is created to discuss how to handle this > case better. > Thanks to [~templedf] for bringing this up. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9170) Move libhdfs / fuse-dfs / libwebhdfs to hdfs-client
[ https://issues.apache.org/jira/browse/HDFS-9170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14942402#comment-14942402 ] Haohui Mai commented on HDFS-9170: -- After some trials it turns out that option (1) is quite nontrivial to get all the dependency right. The v2 patch implements option (2). > Move libhdfs / fuse-dfs / libwebhdfs to hdfs-client > --- > > Key: HDFS-9170 > URL: https://issues.apache.org/jira/browse/HDFS-9170 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Haohui Mai >Assignee: Haohui Mai > Attachments: HDFS-9170.000.patch, HDFS-9170.001.patch, > HDFS-9170.002.patch > > > After HDFS-6200 the Java implementation of hdfs-client has be moved to a > separate hadoop-hdfs-client module. > libhdfs, fuse-dfs and libwebhdfs still reside in the hadoop-hdfs module. > Ideally these modules should reside in the hadoop-hdfs-client. However, to > write unit tests for these components, it is often necessary to run > MiniDFSCluster which resides in the hadoop-hdfs module. > This jira is to discuss how these native modules should layout after > HDFS-6200. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9193) Fix incorrect references the usages of the DN in dfshealth.js
[ https://issues.apache.org/jira/browse/HDFS-9193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14942415#comment-14942415 ] Hudson commented on HDFS-9193: -- FAILURE: Integrated in Hadoop-trunk-Commit #8564 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/8564/]) HDFS-9193. Fix incorrect references the usages of the DN in (wheat9: rev 3b85bd7c3391719624a261f061a67cc4356be477) * hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/dfshealth.js * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt > Fix incorrect references the usages of the DN in dfshealth.js > - > > Key: HDFS-9193 > URL: https://issues.apache.org/jira/browse/HDFS-9193 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Chang Li >Assignee: Chang Li >Priority: Minor > Attachments: HDFS-9193.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9170) Move libhdfs / fuse-dfs / libwebhdfs to hdfs-client
[
https://issues.apache.org/jira/browse/HDFS-9170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14942406#comment-14942406
]
Hadoop QA commented on HDFS-9170:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | pre-patch | 21m 34s | Pre-patch trunk has 7 extant
Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 20 new or modified test files. |
| {color:red}-1{color} | javac | 0m 26s | The patch appears to cause the
build to fail. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12764913/HDFS-9170.002.patch |
| Optional Tests | javadoc javac unit shellcheck findbugs checkstyle |
| git revision | trunk / c918f7b |
| Pre-patch Findbugs warnings |
https://builds.apache.org/job/PreCommit-HDFS-Build/12783/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs-client.html
|
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/12783/console |
This message was automatically generated.
> Move libhdfs / fuse-dfs / libwebhdfs to hdfs-client
> ---
>
> Key: HDFS-9170
> URL: https://issues.apache.org/jira/browse/HDFS-9170
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Haohui Mai
>Assignee: Haohui Mai
> Attachments: HDFS-9170.000.patch, HDFS-9170.001.patch,
> HDFS-9170.002.patch
>
>
> After HDFS-6200 the Java implementation of hdfs-client has be moved to a
> separate hadoop-hdfs-client module.
> libhdfs, fuse-dfs and libwebhdfs still reside in the hadoop-hdfs module.
> Ideally these modules should reside in the hadoop-hdfs-client. However, to
> write unit tests for these components, it is often necessary to run
> MiniDFSCluster which resides in the hadoop-hdfs module.
> This jira is to discuss how these native modules should layout after
> HDFS-6200.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HDFS-8979) Clean up checkstyle warnings in hadoop-hdfs-client module
[
https://issues.apache.org/jira/browse/HDFS-8979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Haohui Mai updated HDFS-8979:
-
Resolution: Fixed
Hadoop Flags: Reviewed
Fix Version/s: 2.8.0
Status: Resolved (was: Patch Available)
I've committed the patch to trunk and branch-2. Thanks [~liuml07] for the
contribution.
> Clean up checkstyle warnings in hadoop-hdfs-client module
> -
>
> Key: HDFS-8979
> URL: https://issues.apache.org/jira/browse/HDFS-8979
> Project: Hadoop HDFS
> Issue Type: Task
>Reporter: Mingliang Liu
>Assignee: Mingliang Liu
> Fix For: 2.8.0
>
> Attachments: HDFS-8979.000.patch, HDFS-8979.001.patch,
> HDFS-8979.002.patch
>
>
> This jira tracks the effort of cleaning up checkstyle warnings in
> {{hadoop-hdfs-client}} module.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HDFS-9193) when a datanode's usage go above 70 percent, we can't open datanodes tab in NN UI
[ https://issues.apache.org/jira/browse/HDFS-9193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haohui Mai updated HDFS-9193: - Priority: Minor (was: Blocker) > when a datanode's usage go above 70 percent, we can't open datanodes tab in > NN UI > - > > Key: HDFS-9193 > URL: https://issues.apache.org/jira/browse/HDFS-9193 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Chang Li >Assignee: Chang Li >Priority: Minor > Attachments: HDFS-9193.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8979) Clean up checkstyle warnings in hadoop-hdfs-client module
[ https://issues.apache.org/jira/browse/HDFS-8979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14942457#comment-14942457 ] Hudson commented on HDFS-8979: -- SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #485 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/485/]) HDFS-8979. Clean up checkstyle warnings in hadoop-hdfs-client module. (wheat9: rev 7136e8c5582dc4061b566cb9f11a0d6a6d08bb93) * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/ClientProtocol.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/NSQuotaExceededException.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/Op.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/protocol/DatanodeStorageReport.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/RenameOptionSetParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/shortcircuit/ShortCircuitReplicaInfo.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/DSQuotaExceededException.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/PeerCache.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/inotify/Event.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/impl/CorruptFileBlockIterator.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/fs/HdfsBlockLocation.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/XAttrSetFlagParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/protocol/StorageReport.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/ExtendedBlockId.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/RollingUpgradeInfo.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/LongParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/DatanodeInfoWithStorage.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/namenode/SafeModeException.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/shortcircuit/ClientMmap.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/ModificationTimeParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSPacket.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/RemotePeerFactory.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/sasl/SaslParticipant.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/AclPermissionParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/XAttrNameParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/DatanodeLocalInfo.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocolPB/ClientNamenodeProtocolPB.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/LengthParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/DeleteOpParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/ExcludeDatanodesParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSStripedOutputStream.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/URLConnectionFactory.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/security/token/delegation/DelegationTokenSelector.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/PostOpParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/util/package-info.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/net/NioInetPeer.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/ReplaceDatanodeOnFailure.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/Param.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/package-info.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java *
[jira] [Commented] (HDFS-9193) Fix incorrect references the usages of the DN in dfshealth.js
[ https://issues.apache.org/jira/browse/HDFS-9193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14942456#comment-14942456 ] Hudson commented on HDFS-9193: -- SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #485 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/485/]) HDFS-9193. Fix incorrect references the usages of the DN in (wheat9: rev 3b85bd7c3391719624a261f061a67cc4356be477) * hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/dfshealth.js * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt > Fix incorrect references the usages of the DN in dfshealth.js > - > > Key: HDFS-9193 > URL: https://issues.apache.org/jira/browse/HDFS-9193 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Chang Li >Assignee: Chang Li >Priority: Minor > Attachments: HDFS-9193.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8979) Clean up checkstyle warnings in hadoop-hdfs-client module
[ https://issues.apache.org/jira/browse/HDFS-8979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14942468#comment-14942468 ] Hudson commented on HDFS-8979: -- FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #451 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/451/]) HDFS-8979. Clean up checkstyle warnings in hadoop-hdfs-client module. (wheat9: rev 7136e8c5582dc4061b566cb9f11a0d6a6d08bb93) * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/BlockLocalPathInfo.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/net/NioInetPeer.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/util/StripedBlockUtil.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/datanode/CachingStrategy.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/DatanodeInfo.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/DatanodeLocalInfo.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/OAuth2ConnectionConfigurator.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/Block.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/CacheDirectiveInfo.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/ClientProtocol.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/BlockReaderLocal.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/LocatedStripedBlock.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/LengthParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSStripedOutputStream.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/AccessTimeParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/ExtendedBlockId.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/CacheDirectiveIterator.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/CachePoolInfo.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockMetadataHeader.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/ClientContext.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/BlockMissingException.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/CreateFlagParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/util/ExactSizeInputStream.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/shortcircuit/DomainSocketFactory.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/fs/XAttr.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/sasl/DataTransferSaslUtil.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/CachePoolIterator.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/AccessTokenProvider.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/CacheDirectiveStats.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/DatanodeInfoWithStorage.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/OwnerParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/ConcatSourcesParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/OverwriteParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/ConfRefreshTokenBasedAccessTokenProvider.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/util/LongBitFormat.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/ErasureCodingPolicy.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/AccessTokenTimer.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/fs/HdfsBlockLocation.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/LocatedBlocks.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/security/token/delegation/DelegationTokenIdentifier.java *
[jira] [Commented] (HDFS-8979) Clean up checkstyle warnings in hadoop-hdfs-client module
[ https://issues.apache.org/jira/browse/HDFS-8979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14942493#comment-14942493 ] Hudson commented on HDFS-8979: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #2391 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/2391/]) HDFS-8979. Clean up checkstyle warnings in hadoop-hdfs-client module. (wheat9: rev 7136e8c5582dc4061b566cb9f11a0d6a6d08bb93) * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/RecursiveParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/Param.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/shortcircuit/ShortCircuitReplicaInfo.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/ConfRefreshTokenBasedAccessTokenProvider.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/CachePoolIterator.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/shortcircuit/ShortCircuitCache.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/EnumSetParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/impl/CorruptFileBlockIterator.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/DelegationParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSPacket.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/util/package-info.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockMetadataHeader.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocolPB/ClientDatanodeProtocolTranslatorPB.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/package-info.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/HdfsDataInputStream.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/ErasureCodingPolicy.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/impl/LeaseRenewer.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/BufferSizeParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/DatanodeID.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/BlockLocalPathInfo.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/NameNodeProxiesClient.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/security/token/delegation/DelegationTokenSelector.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/protocol/StorageReport.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/security/token/delegation/DelegationTokenIdentifier.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/fs/HdfsBlockLocation.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/HdfsLocatedFileStatus.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/CachePoolInfo.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/DatanodeInfo.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/CreateParentParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/CreateFlagParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/BlockReader.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/net/EncryptedPeer.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/LastBlockWithStatus.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/BlockConstructionStage.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/OwnerParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/IOStreamPair.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSStripedInputStream.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/util/ByteBufferOutputStream.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/PeerCache.java *
[jira] [Commented] (HDFS-9193) Fix incorrect references the usages of the DN in dfshealth.js
[ https://issues.apache.org/jira/browse/HDFS-9193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14942471#comment-14942471 ] Hudson commented on HDFS-9193: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #2421 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2421/]) HDFS-9193. Fix incorrect references the usages of the DN in (wheat9: rev 3b85bd7c3391719624a261f061a67cc4356be477) * hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/dfshealth.js * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt > Fix incorrect references the usages of the DN in dfshealth.js > - > > Key: HDFS-9193 > URL: https://issues.apache.org/jira/browse/HDFS-9193 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Chang Li >Assignee: Chang Li >Priority: Minor > Attachments: HDFS-9193.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8979) Clean up checkstyle warnings in hadoop-hdfs-client module
[ https://issues.apache.org/jira/browse/HDFS-8979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14942472#comment-14942472 ] Hudson commented on HDFS-8979: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #2421 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2421/]) HDFS-8979. Clean up checkstyle warnings in hadoop-hdfs-client module. (wheat9: rev 7136e8c5582dc4061b566cb9f11a0d6a6d08bb93) * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/UnresolvedPathException.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/AccessTimeParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/net/NioInetPeer.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSClient.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/BlockMissingException.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/impl/DfsClientConf.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/ExternalBlockReader.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/shortcircuit/ShortCircuitShm.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/fs/CacheFlag.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/ExtendedBlockId.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/PacketReceiver.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/protocol/StorageReport.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSPacket.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/RemotePeerFactory.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/LocatedStripedBlock.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/DelegationParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/ReplaceDatanodeOnFailure.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/OAuth2ConnectionConfigurator.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/GetOpParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/OverwriteParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/XAttrSetFlagParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/StripedDataStreamer.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/UserParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/PermissionParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/BufferSizeParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/CacheDirectiveInfo.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/shortcircuit/ClientMmap.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/XAttrEncodingParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/ConfRefreshTokenBasedAccessTokenProvider.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/shortcircuit/ShortCircuitReplica.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/protocol/DatanodeStorage.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/sasl/SaslResponseWithNegotiatedCipherOption.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/RenewerParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/DeleteOpParam.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/XAttrHelper.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/EncryptionZoneIterator.java * hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/ClientDatanodeProtocol.java *
[jira] [Updated] (HDFS-9193) Fix incorrect references the usages of the DN in dfshealth.js
[ https://issues.apache.org/jira/browse/HDFS-9193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chang Li updated HDFS-9193: --- Resolution: Fixed Status: Resolved (was: Patch Available) > Fix incorrect references the usages of the DN in dfshealth.js > - > > Key: HDFS-9193 > URL: https://issues.apache.org/jira/browse/HDFS-9193 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Chang Li >Assignee: Chang Li >Priority: Minor > Attachments: HDFS-9193.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9193) Fix incorrect references the usages of the DN in dfshealth.js
[ https://issues.apache.org/jira/browse/HDFS-9193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14942530#comment-14942530 ] Hudson commented on HDFS-9193: -- FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #452 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/452/]) HDFS-9193. Fix incorrect references the usages of the DN in (wheat9: rev 3b85bd7c3391719624a261f061a67cc4356be477) * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/dfshealth.js > Fix incorrect references the usages of the DN in dfshealth.js > - > > Key: HDFS-9193 > URL: https://issues.apache.org/jira/browse/HDFS-9193 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Chang Li >Assignee: Chang Li >Priority: Minor > Attachments: HDFS-9193.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HDFS-8344) NameNode doesn't recover lease for files with missing blocks
[
https://issues.apache.org/jira/browse/HDFS-8344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ravi Prakash updated HDFS-8344:
---
Status: Open (was: Patch Available)
> NameNode doesn't recover lease for files with missing blocks
>
>
> Key: HDFS-8344
> URL: https://issues.apache.org/jira/browse/HDFS-8344
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
>Affects Versions: 2.7.0
>Reporter: Ravi Prakash
>Assignee: Ravi Prakash
> Fix For: 2.8.0
>
> Attachments: HDFS-8344.01.patch, HDFS-8344.02.patch,
> HDFS-8344.03.patch, HDFS-8344.04.patch, HDFS-8344.05.patch,
> HDFS-8344.06.patch, HDFS-8344.07.patch, HDFS-8344.08.patch, HDFS-8344.09.patch
>
>
> I found another\(?) instance in which the lease is not recovered. This is
> reproducible easily on a pseudo-distributed single node cluster
> # Before you start it helps if you set. This is not necessary, but simply
> reduces how long you have to wait
> {code}
> public static final long LEASE_SOFTLIMIT_PERIOD = 30 * 1000;
> public static final long LEASE_HARDLIMIT_PERIOD = 2 *
> LEASE_SOFTLIMIT_PERIOD;
> {code}
> # Client starts to write a file. (could be less than 1 block, but it hflushed
> so some of the data has landed on the datanodes) (I'm copying the client code
> I am using. I generate a jar and run it using $ hadoop jar TestHadoop.jar)
> # Client crashes. (I simulate this by kill -9 the $(hadoop jar
> TestHadoop.jar) process after it has printed "Wrote to the bufferedWriter"
> # Shoot the datanode. (Since I ran on a pseudo-distributed cluster, there was
> only 1)
> I believe the lease should be recovered and the block should be marked
> missing. However this is not happening. The lease is never recovered.
> The effect of this bug for us was that nodes could not be decommissioned
> cleanly. Although we knew that the client had crashed, the Namenode never
> released the leases (even after restarting the Namenode) (even months
> afterwards). There are actually several other cases too where we don't
> consider what happens if ALL the datanodes die while the file is being
> written, but I am going to punt on that for another time.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-9137) DeadLock between DataNode#refreshVolumes and BPOfferService#registrationSucceeded
[
https://issues.apache.org/jira/browse/HDFS-9137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14942520#comment-14942520
]
Yi Liu commented on HDFS-9137:
--
Thanks Uma for the patch. I agree we don't need synchronization of {{DataNode}}
for Datanode#triggerBlockReport, so the current fix looks good overall. My
comment is:
{code}
+ IOException exception = null;
try {
LOG.info("Reconfiguring " + property + " to " + newVal);
this.refreshVolumes(newVal);
} catch (IOException e) {
-throw new ReconfigurationException(property, newVal,
-getConf().get(property), e);
+exception = e;
+ } finally {
+// Send a full block report to let NN acknowledge the volume changes.
+try {
+ triggerBlockReport(
+ new BlockReportOptions.Factory().setIncremental(false).build());
+} catch (IOException e) {
+ LOG.warn("Exception while sending the block report after refresh"
+ + " volumes " + property + " to " + newVal, e);
+} finally {
+ if (exception != null) {
+throw new ReconfigurationException(property, newVal,
+getConf().get(property), exception);
+ }
+}
{code}
I think for the IOException of {{refreshVolumes}}, we just need to wrap it to
ReconfigurationException then throw as original. For the exception of
{{triggerBlockReport}}, we also need to wrap it and throw. I see you just log
warn for the IOException of {{triggerBlockReport}} in the patch and ignore it,
any reason to do this? If we want to do this, we don't need to save the
IOException of refreshVolumns and throw it in the finally clause of
triggerBlockReport, we just need to throw it directly, since the {{finally }}
cause is executed before the throwing happens.
> DeadLock between DataNode#refreshVolumes and
> BPOfferService#registrationSucceeded
> --
>
> Key: HDFS-9137
> URL: https://issues.apache.org/jira/browse/HDFS-9137
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: datanode
>Affects Versions: 3.0.0, 2.7.1
>Reporter: Uma Maheswara Rao G
>Assignee: Uma Maheswara Rao G
> Attachments: HDFS-9137.00.patch
>
>
> I can see this code flows between DataNode#refreshVolumes and
> BPOfferService#registrationSucceeded could cause deadLock.
> In practice situation may be rare as user calling refreshVolumes at the time
> DN registration with NN. But seems like issue can happen.
> Reason for deadLock:
> 1) refreshVolumes will be called with DN lock and after at the end it will
> also trigger Block report. In the Block report call,
> BPServiceActor#triggerBlockReport calls toString on bpos. Here it takes
> readLock on bpos.
> DN lock then boos lock
> 2) BPOfferSetrvice#registrationSucceeded call is taking writeLock on bpos and
> calling dn.bpRegistrationSucceeded which is again synchronized call on DN.
> bpos lock and then DN lock.
> So, this can clearly create dead lock.
> I think simple fix could be to move triggerBlockReport call outside out DN
> lock and I feel that call may not be really needed inside DN lock.
> Thoughts?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-9193) Fix incorrect references the usages of the DN in dfshealth.js
[ https://issues.apache.org/jira/browse/HDFS-9193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14942524#comment-14942524 ] Hudson commented on HDFS-9193: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #2392 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/2392/]) HDFS-9193. Fix incorrect references the usages of the DN in (wheat9: rev 3b85bd7c3391719624a261f061a67cc4356be477) * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/dfshealth.js > Fix incorrect references the usages of the DN in dfshealth.js > - > > Key: HDFS-9193 > URL: https://issues.apache.org/jira/browse/HDFS-9193 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Chang Li >Assignee: Chang Li >Priority: Minor > Attachments: HDFS-9193.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
