[jira] [Commented] (HDFS-8432) Introduce a minimum compatible layout version to allow downgrade in more rolling upgrade use cases.
[ https://issues.apache.org/jira/browse/HDFS-8432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14576257#comment-14576257 ] Hudson commented on HDFS-8432: -- SUCCESS: Integrated in Hadoop-Hdfs-trunk #2149 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/2149/]) HDFS-8432. Introduce a minimum compatible layout version to allow downgrade in more rolling upgrade use cases. Contributed by Chris Nauroth. (cnauroth: rev 71de367c5e80ea76d1e8d21f0216cd6b879dcee5) * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeLayoutVersion.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/CreateEditsLog.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/BackupImage.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/protocol/TestLayoutVersion.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/SecondaryNameNode.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/Checkpointer.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImageFormatProtobuf.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSEditLogLoader.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/LayoutVersion.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImage.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLog.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/FSImageTestUtil.java Introduce a minimum compatible layout version to allow downgrade in more rolling upgrade use cases. --- Key: HDFS-8432 URL: https://issues.apache.org/jira/browse/HDFS-8432 Project: Hadoop HDFS Issue Type: Improvement Components: namenode, rolling upgrades Reporter: Chris Nauroth Assignee: Chris Nauroth Fix For: 2.8.0 Attachments: HDFS-8432-HDFS-Downgrade-Extended-Support.pdf, HDFS-8432-branch-2.002.patch, HDFS-8432-branch-2.003.patch, HDFS-8432.001.patch, HDFS-8432.002.patch Maintain the prior layout version during the upgrade window and reject attempts to use new features until after the upgrade has been finalized. This guarantees that the prior software version can read the fsimage and edit logs if the administrator decides to downgrade. This will make downgrade usable for the majority of NameNode layout version changes, which just involve introduction of new edit log operations. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8337) httpfs doesn't work with creates from a jar with kerberos
[
https://issues.apache.org/jira/browse/HDFS-8337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14576357#comment-14576357
]
Hadoop QA commented on HDFS-8337:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 16m 54s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:red}-1{color} | tests included | 0m 0s | The patch doesn't appear
to include any new or modified tests. Please justify why no new tests are
needed for this patch. Also please list what manual steps were performed to
verify this patch. |
| {color:green}+1{color} | javac | 7m 33s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 39s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 23s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 1m 21s | The applied patch generated 1
new checkstyle issues (total was 5, now 6). |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 35s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 32s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 2m 43s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | common tests | 23m 24s | Tests passed in
hadoop-common. |
| {color:green}+1{color} | hdfs tests | 3m 36s | Tests passed in
hadoop-hdfs-httpfs. |
| | | 67m 51s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12738256/HDFS-8337.002.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / b61b489 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HDFS-Build/11262/artifact/patchprocess/diffcheckstylehadoop-common.txt
|
| hadoop-common test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/11262/artifact/patchprocess/testrun_hadoop-common.txt
|
| hadoop-hdfs-httpfs test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/11262/artifact/patchprocess/testrun_hadoop-hdfs-httpfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/11262/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/11262/console |
This message was automatically generated.
httpfs doesn't work with creates from a jar with kerberos
-
Key: HDFS-8337
URL: https://issues.apache.org/jira/browse/HDFS-8337
Project: Hadoop HDFS
Issue Type: Bug
Components: HDFS, hdfs-client, security, webhdfs
Reporter: Yongjun Zhang
Assignee: Yongjun Zhang
Attachments: HDFS-8337.001.patch, HDFS-8337.002.patch
In a secure cluster, running a simple program:
{code}
import org.apache.hadoop.conf.*;
import org.apache.hadoop.fs.*;
import org.apache.hadoop.security.*;
class Foo {
public static void main(String args[]) throws Exception {
FileSystem fs = FileSystem.get(new
java.net.URI(webhdfs://host:14000/), new Configuration());
System.out.println(fs.listStatus(new Path(/))[0]);
java.io.OutputStream os = fs.create(new Path(/tmp/foo));
os.write('a');
os.close();
}
}
{code}
Basically to access httpfs via webhdfs, the following exception is thrown:
{code}
[systest@yj52s ~]$ /usr/java/jdk1.7.0_67-cloudera/bin/java -cp $(hadoop
classpath):. Foo
15/05/06 23:51:38 WARN ssl.FileBasedKeyStoresFactory: The property
'ssl.client.truststore.location' has not been set, no TrustStore will be
loaded
Exception in thread main
org.apache.hadoop.ipc.RemoteException(com.sun.jersey.api.ParamException$QueryParamException):
java.lang.IllegalArgumentException: No enum constant
org.apache.hadoop.fs.http.client.HttpFSFileSystem.Operation.GETDELEGATIONTOKEN
at
org.apache.hadoop.hdfs.web.JsonUtil.toRemoteException(JsonUtil.java:163)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem.validateResponse(WebHdfsFileSystem.java:354)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem.access$200(WebHdfsFileSystem.java:91)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.runWithRetry(WebHdfsFileSystem.java:608)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.access$100(WebHdfsFileSystem.java:458)
at
[jira] [Updated] (HDFS-8539) Hdfs doesnt have class 'debug' in windows
[
https://issues.apache.org/jira/browse/HDFS-8539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Nauroth updated HDFS-8539:
Resolution: Fixed
Fix Version/s: 2.8.0
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
+1 for the patch. I have committed this to trunk and branch-2. Anu, thank you
for the patch.
Hdfs doesnt have class 'debug' in windows
-
Key: HDFS-8539
URL: https://issues.apache.org/jira/browse/HDFS-8539
Project: Hadoop HDFS
Issue Type: Bug
Components: scripts
Affects Versions: 2.7.0
Environment: Windows only
Reporter: Sumana Sathish
Assignee: Anu Engineer
Fix For: 2.8.0
Attachments: hdfs-8539.001.patch
Hdfs debug in windows says the following
{code}
Error:Could not find or load main class debug
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HDFS-8539) Hdfs doesnt have class 'debug' in windows
[
https://issues.apache.org/jira/browse/HDFS-8539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Nauroth updated HDFS-8539:
Component/s: (was: HDFS)
scripts
Hdfs doesnt have class 'debug' in windows
-
Key: HDFS-8539
URL: https://issues.apache.org/jira/browse/HDFS-8539
Project: Hadoop HDFS
Issue Type: Bug
Components: scripts
Affects Versions: 2.7.0
Environment: Windows only
Reporter: Sumana Sathish
Assignee: Anu Engineer
Fix For: 2.8.0
Attachments: hdfs-8539.001.patch
Hdfs debug in windows says the following
{code}
Error:Could not find or load main class debug
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8539) Hdfs doesnt have class 'debug' in windows
[
https://issues.apache.org/jira/browse/HDFS-8539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14576423#comment-14576423
]
Hudson commented on HDFS-8539:
--
FAILURE: Integrated in Hadoop-trunk-Commit #7985 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/7985/])
HDFS-8539. Hdfs doesnt have class debug in windows. Contributed by Anu
Engineer. (cnauroth: rev a6cb4894b45b0b909aca716058590dd6f0e3a6f9)
* hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs.cmd
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
Hdfs doesnt have class 'debug' in windows
-
Key: HDFS-8539
URL: https://issues.apache.org/jira/browse/HDFS-8539
Project: Hadoop HDFS
Issue Type: Bug
Components: scripts
Affects Versions: 2.7.0
Environment: Windows only
Reporter: Sumana Sathish
Assignee: Anu Engineer
Fix For: 2.8.0
Attachments: hdfs-8539.001.patch
Hdfs debug in windows says the following
{code}
Error:Could not find or load main class debug
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8116) Cleanup uncessary if LOG.isDebugEnabled() from RollingWindowManager
[ https://issues.apache.org/jira/browse/HDFS-8116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14576302#comment-14576302 ] Hudson commented on HDFS-8116: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #2167 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2167/]) HDFS-8116. Cleanup uncessary if LOG.isDebugEnabled() from RollingWindowManager. Contributed by Brahma Reddy Battula. (xyao: rev b61b48999530851da177db2e8998858ffc01f77e) * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/top/window/RollingWindowManager.java Cleanup uncessary if LOG.isDebugEnabled() from RollingWindowManager --- Key: HDFS-8116 URL: https://issues.apache.org/jira/browse/HDFS-8116 Project: Hadoop HDFS Issue Type: Improvement Components: namenode Affects Versions: 2.7.0 Reporter: Xiaoyu Yao Assignee: Brahma Reddy Battula Priority: Trivial Fix For: 2.8.0 Attachments: HDFS-8116-002.patch, HDFS-8116.patch RollingWindowManager#getTopUserForMetric should check if LOG.isDebugEnabled() before LOG.debug() -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8432) Introduce a minimum compatible layout version to allow downgrade in more rolling upgrade use cases.
[ https://issues.apache.org/jira/browse/HDFS-8432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14576304#comment-14576304 ] Hudson commented on HDFS-8432: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #2167 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2167/]) HDFS-8432. Introduce a minimum compatible layout version to allow downgrade in more rolling upgrade use cases. Contributed by Chris Nauroth. (cnauroth: rev 71de367c5e80ea76d1e8d21f0216cd6b879dcee5) * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLog.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/LayoutVersion.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/Checkpointer.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/CreateEditsLog.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/FSImageTestUtil.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeLayoutVersion.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/BackupImage.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImage.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/SecondaryNameNode.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/protocol/TestLayoutVersion.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSEditLogLoader.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImageFormatProtobuf.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java Introduce a minimum compatible layout version to allow downgrade in more rolling upgrade use cases. --- Key: HDFS-8432 URL: https://issues.apache.org/jira/browse/HDFS-8432 Project: Hadoop HDFS Issue Type: Improvement Components: namenode, rolling upgrades Reporter: Chris Nauroth Assignee: Chris Nauroth Fix For: 2.8.0 Attachments: HDFS-8432-HDFS-Downgrade-Extended-Support.pdf, HDFS-8432-branch-2.002.patch, HDFS-8432-branch-2.003.patch, HDFS-8432.001.patch, HDFS-8432.002.patch Maintain the prior layout version during the upgrade window and reject attempts to use new features until after the upgrade has been finalized. This guarantees that the prior software version can read the fsimage and edit logs if the administrator decides to downgrade. This will make downgrade usable for the majority of NameNode layout version changes, which just involve introduction of new edit log operations. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8116) Cleanup uncessary if LOG.isDebugEnabled() from RollingWindowManager
[ https://issues.apache.org/jira/browse/HDFS-8116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14576286#comment-14576286 ] Hudson commented on HDFS-8116: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #219 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/219/]) HDFS-8116. Cleanup uncessary if LOG.isDebugEnabled() from RollingWindowManager. Contributed by Brahma Reddy Battula. (xyao: rev b61b48999530851da177db2e8998858ffc01f77e) * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/top/window/RollingWindowManager.java Cleanup uncessary if LOG.isDebugEnabled() from RollingWindowManager --- Key: HDFS-8116 URL: https://issues.apache.org/jira/browse/HDFS-8116 Project: Hadoop HDFS Issue Type: Improvement Components: namenode Affects Versions: 2.7.0 Reporter: Xiaoyu Yao Assignee: Brahma Reddy Battula Priority: Trivial Fix For: 2.8.0 Attachments: HDFS-8116-002.patch, HDFS-8116.patch RollingWindowManager#getTopUserForMetric should check if LOG.isDebugEnabled() before LOG.debug() -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8432) Introduce a minimum compatible layout version to allow downgrade in more rolling upgrade use cases.
[ https://issues.apache.org/jira/browse/HDFS-8432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14576288#comment-14576288 ] Hudson commented on HDFS-8432: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #219 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/219/]) HDFS-8432. Introduce a minimum compatible layout version to allow downgrade in more rolling upgrade use cases. Contributed by Chris Nauroth. (cnauroth: rev 71de367c5e80ea76d1e8d21f0216cd6b879dcee5) * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/SecondaryNameNode.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImageFormatProtobuf.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/Checkpointer.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSEditLogLoader.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/LayoutVersion.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImage.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/CreateEditsLog.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/BackupImage.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeLayoutVersion.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLog.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/protocol/TestLayoutVersion.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/FSImageTestUtil.java Introduce a minimum compatible layout version to allow downgrade in more rolling upgrade use cases. --- Key: HDFS-8432 URL: https://issues.apache.org/jira/browse/HDFS-8432 Project: Hadoop HDFS Issue Type: Improvement Components: namenode, rolling upgrades Reporter: Chris Nauroth Assignee: Chris Nauroth Fix For: 2.8.0 Attachments: HDFS-8432-HDFS-Downgrade-Extended-Support.pdf, HDFS-8432-branch-2.002.patch, HDFS-8432-branch-2.003.patch, HDFS-8432.001.patch, HDFS-8432.002.patch Maintain the prior layout version during the upgrade window and reject attempts to use new features until after the upgrade has been finalized. This guarantees that the prior software version can read the fsimage and edit logs if the administrator decides to downgrade. This will make downgrade usable for the majority of NameNode layout version changes, which just involve introduction of new edit log operations. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8116) Cleanup uncessary if LOG.isDebugEnabled() from RollingWindowManager
[ https://issues.apache.org/jira/browse/HDFS-8116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14576189#comment-14576189 ] Hudson commented on HDFS-8116: -- FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #221 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/221/]) HDFS-8116. Cleanup uncessary if LOG.isDebugEnabled() from RollingWindowManager. Contributed by Brahma Reddy Battula. (xyao: rev b61b48999530851da177db2e8998858ffc01f77e) * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/top/window/RollingWindowManager.java Cleanup uncessary if LOG.isDebugEnabled() from RollingWindowManager --- Key: HDFS-8116 URL: https://issues.apache.org/jira/browse/HDFS-8116 Project: Hadoop HDFS Issue Type: Improvement Components: namenode Affects Versions: 2.7.0 Reporter: Xiaoyu Yao Assignee: Brahma Reddy Battula Priority: Trivial Fix For: 2.8.0 Attachments: HDFS-8116-002.patch, HDFS-8116.patch RollingWindowManager#getTopUserForMetric should check if LOG.isDebugEnabled() before LOG.debug() -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8432) Introduce a minimum compatible layout version to allow downgrade in more rolling upgrade use cases.
[ https://issues.apache.org/jira/browse/HDFS-8432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14576191#comment-14576191 ] Hudson commented on HDFS-8432: -- FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #221 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/221/]) HDFS-8432. Introduce a minimum compatible layout version to allow downgrade in more rolling upgrade use cases. Contributed by Chris Nauroth. (cnauroth: rev 71de367c5e80ea76d1e8d21f0216cd6b879dcee5) * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/FSImageTestUtil.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/Checkpointer.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/BackupImage.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/SecondaryNameNode.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImage.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeLayoutVersion.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSEditLogLoader.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImageFormatProtobuf.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLog.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/protocol/TestLayoutVersion.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/LayoutVersion.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/CreateEditsLog.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java Introduce a minimum compatible layout version to allow downgrade in more rolling upgrade use cases. --- Key: HDFS-8432 URL: https://issues.apache.org/jira/browse/HDFS-8432 Project: Hadoop HDFS Issue Type: Improvement Components: namenode, rolling upgrades Reporter: Chris Nauroth Assignee: Chris Nauroth Fix For: 2.8.0 Attachments: HDFS-8432-HDFS-Downgrade-Extended-Support.pdf, HDFS-8432-branch-2.002.patch, HDFS-8432-branch-2.003.patch, HDFS-8432.001.patch, HDFS-8432.002.patch Maintain the prior layout version during the upgrade window and reject attempts to use new features until after the upgrade has been finalized. This guarantees that the prior software version can read the fsimage and edit logs if the administrator decides to downgrade. This will make downgrade usable for the majority of NameNode layout version changes, which just involve introduction of new edit log operations. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14576226#comment-14576226
]
Hadoop QA commented on HDFS-8450:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | pre-patch | 15m 0s | Findbugs (version ) appears to
be broken on HDFS-7285. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:red}-1{color} | tests included | 0m 0s | The patch doesn't appear
to include any new or modified tests. Please justify why no new tests are
needed for this patch. Also please list what manual steps were performed to
verify this patch. |
| {color:green}+1{color} | javac | 7m 32s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 39s | There were no new javadoc
warning messages. |
| {color:red}-1{color} | release audit | 0m 15s | The applied patch generated
1 release audit warnings. |
| {color:green}+1{color} | checkstyle | 0m 37s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 37s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:red}-1{color} | findbugs | 3m 25s | The patch appears to introduce 4
new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | native | 3m 14s | Pre-build of native portion |
| {color:red}-1{color} | hdfs tests | 230m 17s | Tests failed in hadoop-hdfs. |
| | | 272m 14s | |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-hdfs |
| Failed unit tests | hadoop.hdfs.TestEncryptedTransfer |
| | hadoop.hdfs.server.namenode.ha.TestInitializeSharedEdits |
| | hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFS |
| | hadoop.hdfs.server.namenode.ha.TestPipelinesFailover |
| | hadoop.hdfs.server.namenode.ha.TestHAMetrics |
| | hadoop.hdfs.server.namenode.ha.TestEditLogTailer |
| | hadoop.hdfs.server.namenode.ha.TestEditLogsDuringFailover |
| | hadoop.hdfs.server.namenode.TestAuditLogs |
| | hadoop.hdfs.server.namenode.TestFsck |
| | hadoop.hdfs.TestDecommission |
| | hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits |
| | hadoop.hdfs.server.namenode.ha.TestPendingCorruptDnMessages |
| | hadoop.hdfs.server.blockmanagement.TestBlockInfo |
| | hadoop.hdfs.server.namenode.ha.TestHAAppend |
| | hadoop.hdfs.server.namenode.TestFileTruncate |
| | hadoop.hdfs.server.namenode.ha.TestHAStateTransitions |
| Timed out tests | org.apache.hadoop.hdfs.server.namenode.ha.TestDNFencing |
| | org.apache.hadoop.hdfs.server.namenode.ha.TestHASafeMode |
| | org.apache.hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA |
| | org.apache.hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12738228/HDFS-8450-HDFS-7285-09.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | HDFS-7285 / 2eee19c |
| Release Audit |
https://builds.apache.org/job/PreCommit-HDFS-Build/11261/artifact/patchprocess/patchReleaseAuditProblems.txt
|
| Findbugs warnings |
https://builds.apache.org/job/PreCommit-HDFS-Build/11261/artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html
|
| hadoop-hdfs test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/11261/artifact/patchprocess/testrun_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/11261/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/11261/console |
This message was automatically generated.
Erasure Coding: Consolidate erasure coding zone related implementation into a
single class
--
Key: HDFS-8450
URL: https://issues.apache.org/jira/browse/HDFS-8450
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: Rakesh R
Attachments: HDFS-8450-FYI.patch, HDFS-8450-HDFS-7285-00.patch,
HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch,
HDFS-8450-HDFS-7285-03.patch, HDFS-8450-HDFS-7285-04.patch,
HDFS-8450-HDFS-7285-05.patch, HDFS-8450-HDFS-7285-07.patch,
HDFS-8450-HDFS-7285-08.patch, HDFS-8450-HDFS-7285-09.patch
The idea is to follow the same pattern suggested by HDFS-7416. It is good to
consolidate all the erasure coding zone related implementations of
{{FSNamesystem}}. Here, proposing
[jira] [Commented] (HDFS-8298) HA: NameNode should not shut down completely without quorum, doesn't recover from temporary network outages
[
https://issues.apache.org/jira/browse/HDFS-8298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14576319#comment-14576319
]
Rohit Krishnan commented on HDFS-8298:
--
We are facing this issue every week at the same time might be due to a network
glitch but is there a workaround that can be put in place?
2015-06-07 09:03:11,670 FATAL namenode.FSEditLog
(JournalSet.java:mapJournalsAndReportErrors(398)) - Error: flush failed for
required journal (JournalAndStream(mgr=QJM to [X.X.X.X:8485, X.X.X.X:8485,
X.X.X.X:8485], stream=QuorumOutputStream starting at txid 738153893))
java.io.IOException: Timed out waiting 2ms for a quorum of nodes to respond.
at
org.apache.hadoop.hdfs.qjournal.client.AsyncLoggerSet.waitForWriteQuorum(AsyncLoggerSet.java:137)
at
org.apache.hadoop.hdfs.qjournal.client.QuorumOutputStream.flushAndSync(QuorumOutputStream.java:107)
at
org.apache.hadoop.hdfs.server.namenode.EditLogOutputStream.flush(EditLogOutputStream.java:113)
at
org.apache.hadoop.hdfs.server.namenode.EditLogOutputStream.flush(EditLogOutputStream.java:107)
at
org.apache.hadoop.hdfs.server.namenode.JournalSet$JournalSetOutputStream$8.apply(JournalSet.java:533)
at
org.apache.hadoop.hdfs.server.namenode.JournalSet.mapJournalsAndReportErrors(JournalSet.java:393)
at
org.apache.hadoop.hdfs.server.namenode.JournalSet.access$100(JournalSet.java:57)
at
org.apache.hadoop.hdfs.server.namenode.JournalSet$JournalSetOutputStream.flush(JournalSet.java:529)
at
org.apache.hadoop.hdfs.server.namenode.FSEditLog.logSync(FSEditLog.java:639)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:3252)
at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:641)
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:482)
at
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:619)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:962)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2039)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2035)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2033)
2015-06-07 09:03:11,672 WARN client.QuorumJournalManager
(QuorumOutputStream.java:abort(72)) - Aborting QuorumOutputStream starting at
txid 738153893
2015-06-07 09:03:11,683 INFO util.ExitUtil (ExitUtil.java:terminate(124)) -
Exiting with status 1
2015-06-07 09:03:12,111 INFO namenode.NameNode (StringUtils.java:run(659)) -
SHUTDOWN_MSG:
/
SHUTDOWN_MSG: Shutting down NameNode at X.X.X.X/X.X.X.X
/
HA: NameNode should not shut down completely without quorum, doesn't recover
from temporary network outages
---
Key: HDFS-8298
URL: https://issues.apache.org/jira/browse/HDFS-8298
Project: Hadoop HDFS
Issue Type: Improvement
Components: ha, HDFS, namenode, qjm
Affects Versions: 2.6.0
Environment: HDP 2.2
Reporter: Hari Sekhon
In an HDFS HA setup if there is a temporary problem with contacting journal
nodes (eg. network interruption), the NameNode shuts down entirely, when it
should instead go in to a standby mode so that it can stay online and retry
to achieve quorum later.
If both NameNodes shut themselves off like this then even after the temporary
network outage is resolved, the entire cluster remains offline indefinitely
until operator intervention, whereas it could have self-repaired after
re-contacting the journalnodes and re-achieving quorum.
{code}2015-04-15 15:59:26,900 FATAL namenode.FSEditLog
(JournalSet.java:mapJournalsAndReportErrors(398)) - Error: flush failed for
required journal (JournalAndStre
am(mgr=QJM to [ip:8485, ip:8485, ip:8485], stream=QuorumOutputStream
starting at txid 54270281))
java.io.IOException: Interrupted waiting 2ms for a quorum of nodes to
respond.
at
org.apache.hadoop.hdfs.qjournal.client.AsyncLoggerSet.waitForWriteQuorum(AsyncLoggerSet.java:134)
at
[jira] [Updated] (HDFS-8337) httpfs doesn't work with creates from a jar with kerberos
[
https://issues.apache.org/jira/browse/HDFS-8337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yongjun Zhang updated HDFS-8337:
Attachment: HDFS-8337.002.patch
Uploaded rev 002 to address test failures.
httpfs doesn't work with creates from a jar with kerberos
-
Key: HDFS-8337
URL: https://issues.apache.org/jira/browse/HDFS-8337
Project: Hadoop HDFS
Issue Type: Bug
Components: HDFS, hdfs-client, security, webhdfs
Reporter: Yongjun Zhang
Assignee: Yongjun Zhang
Attachments: HDFS-8337.001.patch, HDFS-8337.002.patch
In a secure cluster, running a simple program:
{code}
import org.apache.hadoop.conf.*;
import org.apache.hadoop.fs.*;
import org.apache.hadoop.security.*;
class Foo {
public static void main(String args[]) throws Exception {
FileSystem fs = FileSystem.get(new
java.net.URI(webhdfs://host:14000/), new Configuration());
System.out.println(fs.listStatus(new Path(/))[0]);
java.io.OutputStream os = fs.create(new Path(/tmp/foo));
os.write('a');
os.close();
}
}
{code}
Basically to access httpfs via webhdfs, the following exception is thrown:
{code}
[systest@yj52s ~]$ /usr/java/jdk1.7.0_67-cloudera/bin/java -cp $(hadoop
classpath):. Foo
15/05/06 23:51:38 WARN ssl.FileBasedKeyStoresFactory: The property
'ssl.client.truststore.location' has not been set, no TrustStore will be
loaded
Exception in thread main
org.apache.hadoop.ipc.RemoteException(com.sun.jersey.api.ParamException$QueryParamException):
java.lang.IllegalArgumentException: No enum constant
org.apache.hadoop.fs.http.client.HttpFSFileSystem.Operation.GETDELEGATIONTOKEN
at
org.apache.hadoop.hdfs.web.JsonUtil.toRemoteException(JsonUtil.java:163)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem.validateResponse(WebHdfsFileSystem.java:354)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem.access$200(WebHdfsFileSystem.java:91)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.runWithRetry(WebHdfsFileSystem.java:608)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.access$100(WebHdfsFileSystem.java:458)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner$1.run(WebHdfsFileSystem.java:487)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.run(WebHdfsFileSystem.java:483)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem.getDelegationToken(WebHdfsFileSystem.java:1299)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem.getDelegationToken(WebHdfsFileSystem.java:237)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem.getAuthParameters(WebHdfsFileSystem.java:423)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem.toUrl(WebHdfsFileSystem.java:444)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractFsPathRunner.getUrl(WebHdfsFileSystem.java:691)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.runWithRetry(WebHdfsFileSystem.java:603)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.access$100(WebHdfsFileSystem.java:458)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner$1.run(WebHdfsFileSystem.java:487)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.run(WebHdfsFileSystem.java:483)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem.listStatus(WebHdfsFileSystem.java:1277)
at Foo.main(Foo.java:7)
{code}
Thanks [~qwertymaniac] and [~caseyjbrotherton] for reporting the issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8116) Cleanup uncessary if LOG.isDebugEnabled() from RollingWindowManager
[ https://issues.apache.org/jira/browse/HDFS-8116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14576192#comment-14576192 ] Hudson commented on HDFS-8116: -- FAILURE: Integrated in Hadoop-Yarn-trunk #951 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/951/]) HDFS-8116. Cleanup uncessary if LOG.isDebugEnabled() from RollingWindowManager. Contributed by Brahma Reddy Battula. (xyao: rev b61b48999530851da177db2e8998858ffc01f77e) * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/top/window/RollingWindowManager.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt Cleanup uncessary if LOG.isDebugEnabled() from RollingWindowManager --- Key: HDFS-8116 URL: https://issues.apache.org/jira/browse/HDFS-8116 Project: Hadoop HDFS Issue Type: Improvement Components: namenode Affects Versions: 2.7.0 Reporter: Xiaoyu Yao Assignee: Brahma Reddy Battula Priority: Trivial Fix For: 2.8.0 Attachments: HDFS-8116-002.patch, HDFS-8116.patch RollingWindowManager#getTopUserForMetric should check if LOG.isDebugEnabled() before LOG.debug() -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8432) Introduce a minimum compatible layout version to allow downgrade in more rolling upgrade use cases.
[ https://issues.apache.org/jira/browse/HDFS-8432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14576194#comment-14576194 ] Hudson commented on HDFS-8432: -- FAILURE: Integrated in Hadoop-Yarn-trunk #951 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/951/]) HDFS-8432. Introduce a minimum compatible layout version to allow downgrade in more rolling upgrade use cases. Contributed by Chris Nauroth. (cnauroth: rev 71de367c5e80ea76d1e8d21f0216cd6b879dcee5) * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImage.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/LayoutVersion.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/BackupImage.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSEditLogLoader.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLog.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/CreateEditsLog.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImageFormatProtobuf.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/FSImageTestUtil.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/Checkpointer.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/SecondaryNameNode.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeLayoutVersion.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/protocol/TestLayoutVersion.java Introduce a minimum compatible layout version to allow downgrade in more rolling upgrade use cases. --- Key: HDFS-8432 URL: https://issues.apache.org/jira/browse/HDFS-8432 Project: Hadoop HDFS Issue Type: Improvement Components: namenode, rolling upgrades Reporter: Chris Nauroth Assignee: Chris Nauroth Fix For: 2.8.0 Attachments: HDFS-8432-HDFS-Downgrade-Extended-Support.pdf, HDFS-8432-branch-2.002.patch, HDFS-8432-branch-2.003.patch, HDFS-8432.001.patch, HDFS-8432.002.patch Maintain the prior layout version during the upgrade window and reject attempts to use new features until after the upgrade has been finalized. This guarantees that the prior software version can read the fsimage and edit logs if the administrator decides to downgrade. This will make downgrade usable for the majority of NameNode layout version changes, which just involve introduction of new edit log operations. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8116) Cleanup uncessary if LOG.isDebugEnabled() from RollingWindowManager
[ https://issues.apache.org/jira/browse/HDFS-8116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14576255#comment-14576255 ] Hudson commented on HDFS-8116: -- SUCCESS: Integrated in Hadoop-Hdfs-trunk #2149 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/2149/]) HDFS-8116. Cleanup uncessary if LOG.isDebugEnabled() from RollingWindowManager. Contributed by Brahma Reddy Battula. (xyao: rev b61b48999530851da177db2e8998858ffc01f77e) * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/top/window/RollingWindowManager.java Cleanup uncessary if LOG.isDebugEnabled() from RollingWindowManager --- Key: HDFS-8116 URL: https://issues.apache.org/jira/browse/HDFS-8116 Project: Hadoop HDFS Issue Type: Improvement Components: namenode Affects Versions: 2.7.0 Reporter: Xiaoyu Yao Assignee: Brahma Reddy Battula Priority: Trivial Fix For: 2.8.0 Attachments: HDFS-8116-002.patch, HDFS-8116.patch RollingWindowManager#getTopUserForMetric should check if LOG.isDebugEnabled() before LOG.debug() -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8116) Cleanup uncessary if LOG.isDebugEnabled() from RollingWindowManager
[ https://issues.apache.org/jira/browse/HDFS-8116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14576267#comment-14576267 ] Hudson commented on HDFS-8116: -- SUCCESS: Integrated in Hadoop-Hdfs-trunk-Java8 #210 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/210/]) HDFS-8116. Cleanup uncessary if LOG.isDebugEnabled() from RollingWindowManager. Contributed by Brahma Reddy Battula. (xyao: rev b61b48999530851da177db2e8998858ffc01f77e) * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/top/window/RollingWindowManager.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt Cleanup uncessary if LOG.isDebugEnabled() from RollingWindowManager --- Key: HDFS-8116 URL: https://issues.apache.org/jira/browse/HDFS-8116 Project: Hadoop HDFS Issue Type: Improvement Components: namenode Affects Versions: 2.7.0 Reporter: Xiaoyu Yao Assignee: Brahma Reddy Battula Priority: Trivial Fix For: 2.8.0 Attachments: HDFS-8116-002.patch, HDFS-8116.patch RollingWindowManager#getTopUserForMetric should check if LOG.isDebugEnabled() before LOG.debug() -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8432) Introduce a minimum compatible layout version to allow downgrade in more rolling upgrade use cases.
[ https://issues.apache.org/jira/browse/HDFS-8432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14576269#comment-14576269 ] Hudson commented on HDFS-8432: -- SUCCESS: Integrated in Hadoop-Hdfs-trunk-Java8 #210 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/210/]) HDFS-8432. Introduce a minimum compatible layout version to allow downgrade in more rolling upgrade use cases. Contributed by Chris Nauroth. (cnauroth: rev 71de367c5e80ea76d1e8d21f0216cd6b879dcee5) * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/CreateEditsLog.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/BackupImage.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImageFormatProtobuf.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/LayoutVersion.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeLayoutVersion.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/protocol/TestLayoutVersion.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImage.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/Checkpointer.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLog.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSEditLogLoader.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/FSImageTestUtil.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/SecondaryNameNode.java Introduce a minimum compatible layout version to allow downgrade in more rolling upgrade use cases. --- Key: HDFS-8432 URL: https://issues.apache.org/jira/browse/HDFS-8432 Project: Hadoop HDFS Issue Type: Improvement Components: namenode, rolling upgrades Reporter: Chris Nauroth Assignee: Chris Nauroth Fix For: 2.8.0 Attachments: HDFS-8432-HDFS-Downgrade-Extended-Support.pdf, HDFS-8432-branch-2.002.patch, HDFS-8432-branch-2.003.patch, HDFS-8432.001.patch, HDFS-8432.002.patch Maintain the prior layout version during the upgrade window and reject attempts to use new features until after the upgrade has been finalized. This guarantees that the prior software version can read the fsimage and edit logs if the administrator decides to downgrade. This will make downgrade usable for the majority of NameNode layout version changes, which just involve introduction of new edit log operations. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HDFS-8540) Mover should exit with NO_MOVE_BLOCK if no block can be moved
[
https://issues.apache.org/jira/browse/HDFS-8540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tsz Wo Nicholas Sze updated HDFS-8540:
--
Issue Type: Sub-task (was: Bug)
Parent: HDFS-7197
Mover should exit with NO_MOVE_BLOCK if no block can be moved
-
Key: HDFS-8540
URL: https://issues.apache.org/jira/browse/HDFS-8540
Project: Hadoop HDFS
Issue Type: Sub-task
Components: balancer mover
Reporter: Tsz Wo Nicholas Sze
Assignee: surendra singh lilhore
Attachments: HDFS-8540.patch
When there are files not satisfying their storage policy and no move is
possible, Mover exits with SUCCESS. It should exit with NO_MOVE_BLOCK.
The bug seems in the following code. When StorageTypeDiff is not empty and
scheduleMoves4Block return false, it does not update hasRemaining. Also,
there is no indication of No block can be moved for the entire iteration.
{code}
//Mover.processFile(..)
if (!diff.removeOverlap(true)) {
if (scheduleMoves4Block(diff, lb, ecSchema)) {
hasRemaining |= (diff.existing.size() 1
diff.expected.size() 1);
}
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HDFS-8541) Mover should exit with NO_MOVE_PROGRESS if there is no move progress
[ https://issues.apache.org/jira/browse/HDFS-8541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-8541: -- Issue Type: Sub-task (was: Improvement) Parent: HDFS-7197 Mover should exit with NO_MOVE_PROGRESS if there is no move progress Key: HDFS-8541 URL: https://issues.apache.org/jira/browse/HDFS-8541 Project: Hadoop HDFS Issue Type: Sub-task Components: balancer mover Reporter: Tsz Wo Nicholas Sze Assignee: surendra singh lilhore Priority: Minor HDFS-8143 changed Mover to exit after some retry when failed to move blocks. Two additional suggestions: # Mover retry counter should be incremented only if all moves fail. If there are some successful moves, the counter should be reset. # Mover should exit with NO_MOVE_PROGRESS instead of IO_EXCEPTION in case of failure. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14576594#comment-14576594
]
Rakesh R commented on HDFS-8450:
I've attached patch addressing [~drankye]'s comments. Please review!
Note: It seems the FindBug warning and test case failures are unrelated to the
patch.
Erasure Coding: Consolidate erasure coding zone related implementation into a
single class
--
Key: HDFS-8450
URL: https://issues.apache.org/jira/browse/HDFS-8450
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: Rakesh R
Attachments: HDFS-8450-FYI.patch, HDFS-8450-HDFS-7285-00.patch,
HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch,
HDFS-8450-HDFS-7285-03.patch, HDFS-8450-HDFS-7285-04.patch,
HDFS-8450-HDFS-7285-05.patch, HDFS-8450-HDFS-7285-07.patch,
HDFS-8450-HDFS-7285-08.patch, HDFS-8450-HDFS-7285-09.patch
The idea is to follow the same pattern suggested by HDFS-7416. It is good to
consolidate all the erasure coding zone related implementations of
{{FSNamesystem}}. Here, proposing {{FSDirErasureCodingZoneOp}} class to have
functions to perform related erasure coding zone operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HDFS-7310) Mover can give first priority to local DN if it has target storage type available in local DN
[ https://issues.apache.org/jira/browse/HDFS-7310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-7310: -- Issue Type: Sub-task (was: Improvement) Parent: HDFS-7197 Mover can give first priority to local DN if it has target storage type available in local DN - Key: HDFS-7310 URL: https://issues.apache.org/jira/browse/HDFS-7310 Project: Hadoop HDFS Issue Type: Sub-task Components: balancer mover Affects Versions: 3.0.0 Reporter: Uma Maheswara Rao G Assignee: Vinayakumar B Fix For: 2.7.0 Attachments: HDFS-7310-001.patch, HDFS-7310-002.patch, HDFS-7310-003.patch, HDFS-7310-004.patch Currently Mover logic may move blocks to any DN which had target storage type. But if the src DN has target storage type then mover can give highest priority to local DN. If local DN does not contains target storage type, then it can assign to any DN as the current logic does. This is a thought, have not go through the code fully yet. Thoughts? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HDFS-6261) Document for enabling node group layer in HDFS
[
https://issues.apache.org/jira/browse/HDFS-6261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Binglin Chang updated HDFS-6261:
Attachment: HDFS-6261.010.patch
Thanks for the review Junping!
I change nodes to node groups, that should have not conflict with No
duplicated replicas are on the same node or node group, and still simple
enough to avoid misunderstanding?
{noformat}
The remaining replicas are placed randomly across other node groups
{noformat}
Document for enabling node group layer in HDFS
--
Key: HDFS-6261
URL: https://issues.apache.org/jira/browse/HDFS-6261
Project: Hadoop HDFS
Issue Type: Task
Components: documentation
Reporter: Wenwu Peng
Assignee: Binglin Chang
Labels: documentation
Attachments: 2-layer-topology.png, 3-layer-topology.png,
3layer-topology.png, 4layer-topology.png, HDFS-6261.004.patch,
HDFS-6261.005.patch, HDFS-6261.006.patch, HDFS-6261.007.patch,
HDFS-6261.008.patch, HDFS-6261.009.patch, HDFS-6261.010.patch,
HDFS-6261.v1.patch, HDFS-6261.v1.patch, HDFS-6261.v2.patch, HDFS-6261.v3.patch
Most of patches from Umbrella JIRA HADOOP-8468 have committed, However there
is no site to introduce NodeGroup-aware(HADOOP Virtualization Extensisons)
and how to do configuration. so we need to doc it.
1. Doc NodeGroup-aware relate in http://hadoop.apache.org/docs/current
2. Doc NodeGroup-aware properties in core-default.xml.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HDFS-8555) Random read support on HDFS files using Indexed Namenode feature
[ https://issues.apache.org/jira/browse/HDFS-8555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] amit sehga updated HDFS-8555: - Summary: Random read support on HDFS files using Indexed Namenode feature (was: Random access support on HDFS files using Indexed Namenode feature) Random read support on HDFS files using Indexed Namenode feature Key: HDFS-8555 URL: https://issues.apache.org/jira/browse/HDFS-8555 Project: Hadoop HDFS Issue Type: Improvement Components: HDFS, hdfs-client, namenode Affects Versions: 2.5.2 Environment: Linux Reporter: amit sehga Assignee: amit sehga Fix For: 3.0.0 Original Estimate: 720h Remaining Estimate: 720h Currently Namenode does not provide support to do random reads. With so many tools built on top of HDFS solving the use case of Exploratory BI and providing SQL over HDFS. The need of hour is to reduce the number of blocks read for a Random read. E.g. extracting say 10 lines worth of information out of 100GB files should be reading only those block which can potentially have those 10 lines. This can be achieved by adding a tagging feature per block in name node, each block written to HDFS will have tags associated to it stored in index. Namednode when access via the Indexing feature will use this index native to reduce the no. of block returned to the client. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HDFS-8555) Random access support on HDFS files using Indexed Namenode feature
amit sehga created HDFS-8555: Summary: Random access support on HDFS files using Indexed Namenode feature Key: HDFS-8555 URL: https://issues.apache.org/jira/browse/HDFS-8555 Project: Hadoop HDFS Issue Type: Improvement Components: HDFS, hdfs-client, namenode Affects Versions: 2.5.2 Environment: Linux Reporter: amit sehga Assignee: amit sehga Fix For: 3.0.0 Currently Namenode does not provide support to do random reads. With so many tools built on top of HDFS solving the use case of Exploratory BI and providing SQL over HDFS. The need of hour is to reduce the number of blocks read for a Random read. E.g. extracting say 10 lines worth of information out of 100GB files should be reading only those block which can potentially have those 10 lines. This can be achieved by adding a tagging feature per block in name node, each block written to HDFS will have tags associated to it stored in index. Namednode when access via the Indexing feature will use this index native to reduce the no. of block returned to the client. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14576596#comment-14576596
]
Kai Zheng commented on HDFS-8450:
-
Rakesh,
1. Please rename {{FSDirErasureCodingZoneOp.java}} to
{{FSDirErasureCodingOp.java}};
2. Please correct comment in below:
{code}
+ /**
+ * Get erasure coding zone.
+ *
+ * @param fsn namespace
+ * @param iip inodes in the path containing the file
+ * @return {@link ErasureCodingZoneInfo}
+ * @throws IOException
+ */
+ static ErasureCodingZone getErasureCodingZone(final FSNamesystem fsn,
+ final INodesInPath iip) throws IOException {
{code}
3. Maybe we could move all the {{checkOperation}} calls to {{FSNamesystem}}
like it does in other {{*Op}} classes?
Erasure Coding: Consolidate erasure coding zone related implementation into a
single class
--
Key: HDFS-8450
URL: https://issues.apache.org/jira/browse/HDFS-8450
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: Rakesh R
Attachments: HDFS-8450-FYI.patch, HDFS-8450-HDFS-7285-00.patch,
HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch,
HDFS-8450-HDFS-7285-03.patch, HDFS-8450-HDFS-7285-04.patch,
HDFS-8450-HDFS-7285-05.patch, HDFS-8450-HDFS-7285-07.patch,
HDFS-8450-HDFS-7285-08.patch, HDFS-8450-HDFS-7285-09.patch
The idea is to follow the same pattern suggested by HDFS-7416. It is good to
consolidate all the erasure coding zone related implementations of
{{FSNamesystem}}. Here, proposing {{FSDirErasureCodingZoneOp}} class to have
functions to perform related erasure coding zone operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14576632#comment-14576632
]
Rakesh R commented on HDFS-8450:
Thanks [~drankye] for the quick reviews. My env had a local copy of
{{FSDirErasureCodingZoneOp}} that created the mistake:( Yes, I agree to move
{{checkOperation}} calls outside. In java comment, I think
{{ErasureCodingZoneInfo}} is the mistake. I will correct all the three comments
and prepare a patch. Thanks again!
Erasure Coding: Consolidate erasure coding zone related implementation into a
single class
--
Key: HDFS-8450
URL: https://issues.apache.org/jira/browse/HDFS-8450
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: Rakesh R
Attachments: HDFS-8450-FYI.patch, HDFS-8450-HDFS-7285-00.patch,
HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch,
HDFS-8450-HDFS-7285-03.patch, HDFS-8450-HDFS-7285-04.patch,
HDFS-8450-HDFS-7285-05.patch, HDFS-8450-HDFS-7285-07.patch,
HDFS-8450-HDFS-7285-08.patch, HDFS-8450-HDFS-7285-09.patch
The idea is to follow the same pattern suggested by HDFS-7416. It is good to
consolidate all the erasure coding zone related implementations of
{{FSNamesystem}}. Here, proposing {{FSDirErasureCodingZoneOp}} class to have
functions to perform related erasure coding zone operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HDFS-7197) Enhancements to Mover
[ https://issues.apache.org/jira/browse/HDFS-7197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-7197: -- Component/s: balancer mover Summary: Enhancements to Mover (was: Enhancements to Data Migration Tool ) Enhancements to Mover - Key: HDFS-7197 URL: https://issues.apache.org/jira/browse/HDFS-7197 Project: Hadoop HDFS Issue Type: Improvement Components: balancer mover Reporter: Benoy Antony Assignee: Benoy Antony Data migration tool (mover) was added as part of HDFS-6584. We have been using Archival storage tier in our clusters. We have implemented a similar data migration tool (Mover) to migrate data to and from Archival storage. This is an umbrella jira to contribute the features and improvements identified based on our experience. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-6261) Document for enabling node group layer in HDFS
[
https://issues.apache.org/jira/browse/HDFS-6261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14576569#comment-14576569
]
Hadoop QA commented on HDFS-6261:
-
\\
\\
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 3m 47s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | release audit | 0m 20s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | site | 2m 57s | Site still builds. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| | | 7m 7s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12738290/HDFS-6261.010.patch |
| Optional Tests | site |
| git revision | trunk / a6cb489 |
| Java | 1.7.0_55 |
| uname | Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/11264/console |
This message was automatically generated.
Document for enabling node group layer in HDFS
--
Key: HDFS-6261
URL: https://issues.apache.org/jira/browse/HDFS-6261
Project: Hadoop HDFS
Issue Type: Task
Components: documentation
Reporter: Wenwu Peng
Assignee: Binglin Chang
Labels: documentation
Attachments: 2-layer-topology.png, 3-layer-topology.png,
3layer-topology.png, 4layer-topology.png, HDFS-6261.004.patch,
HDFS-6261.005.patch, HDFS-6261.006.patch, HDFS-6261.007.patch,
HDFS-6261.008.patch, HDFS-6261.009.patch, HDFS-6261.010.patch,
HDFS-6261.v1.patch, HDFS-6261.v1.patch, HDFS-6261.v2.patch, HDFS-6261.v3.patch
Most of patches from Umbrella JIRA HADOOP-8468 have committed, However there
is no site to introduce NodeGroup-aware(HADOOP Virtualization Extensisons)
and how to do configuration. so we need to doc it.
1. Doc NodeGroup-aware relate in http://hadoop.apache.org/docs/current
2. Doc NodeGroup-aware properties in core-default.xml.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HDFS-8540) Mover should exit with NO_MOVE_BLOCK if no block can be moved
[
https://issues.apache.org/jira/browse/HDFS-8540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
surendra singh lilhore updated HDFS-8540:
-
Status: Patch Available (was: Open)
Mover should exit with NO_MOVE_BLOCK if no block can be moved
-
Key: HDFS-8540
URL: https://issues.apache.org/jira/browse/HDFS-8540
Project: Hadoop HDFS
Issue Type: Sub-task
Components: balancer mover
Reporter: Tsz Wo Nicholas Sze
Assignee: surendra singh lilhore
Attachments: HDFS-8540.patch
When there are files not satisfying their storage policy and no move is
possible, Mover exits with SUCCESS. It should exit with NO_MOVE_BLOCK.
The bug seems in the following code. When StorageTypeDiff is not empty and
scheduleMoves4Block return false, it does not update hasRemaining. Also,
there is no indication of No block can be moved for the entire iteration.
{code}
//Mover.processFile(..)
if (!diff.removeOverlap(true)) {
if (scheduleMoves4Block(diff, lb, ecSchema)) {
hasRemaining |= (diff.existing.size() 1
diff.expected.size() 1);
}
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Reopened] (HDFS-8505) Truncate should not be success when Truncate Size and Current Size are equal.
[
https://issues.apache.org/jira/browse/HDFS-8505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinayakumar B reopened HDFS-8505:
-
Truncate should not be success when Truncate Size and Current Size are equal.
-
Key: HDFS-8505
URL: https://issues.apache.org/jira/browse/HDFS-8505
Project: Hadoop HDFS
Issue Type: Bug
Reporter: Archana T
Assignee: Brahma Reddy Battula
Priority: Minor
Attachments: HDFS-8505.patch
Truncate should not be success when Truncate Size and Current Size are equal.
$ ./hdfs dfs -cat /file
abcdefgh
$ ./hdfs dfs -truncate -w 2 /file
Waiting for /file ...
Truncated /file to length: 2
$ ./hdfs dfs -cat /file
ab
{color:red}
$ ./hdfs dfs -truncate -w 2 /file
Truncated /file to length: 2
{color}
$ ./hdfs dfs -cat /file
ab
Expecting to throw Truncate Error:
-truncate: Cannot truncate to a larger file size. Current size: 2, truncate
size: 2
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HDFS-8505) Truncate should not be success when Truncate Size and Current Size are equal.
[
https://issues.apache.org/jira/browse/HDFS-8505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinayakumar B resolved HDFS-8505.
-
Resolution: Not A Problem
Resolved as 'Not a Problem' as suggested.
Truncate should not be success when Truncate Size and Current Size are equal.
-
Key: HDFS-8505
URL: https://issues.apache.org/jira/browse/HDFS-8505
Project: Hadoop HDFS
Issue Type: Bug
Reporter: Archana T
Assignee: Brahma Reddy Battula
Priority: Minor
Attachments: HDFS-8505.patch
Truncate should not be success when Truncate Size and Current Size are equal.
$ ./hdfs dfs -cat /file
abcdefgh
$ ./hdfs dfs -truncate -w 2 /file
Waiting for /file ...
Truncated /file to length: 2
$ ./hdfs dfs -cat /file
ab
{color:red}
$ ./hdfs dfs -truncate -w 2 /file
Truncated /file to length: 2
{color}
$ ./hdfs dfs -cat /file
ab
Expecting to throw Truncate Error:
-truncate: Cannot truncate to a larger file size. Current size: 2, truncate
size: 2
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-6708) StorageType should be encoded in the block token
[ https://issues.apache.org/jira/browse/HDFS-6708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14576584#comment-14576584 ] Tsz Wo Nicholas Sze commented on HDFS-6708: --- Hi Arpit, do we still need this JIRA? StorageType should be encoded in the block token Key: HDFS-6708 URL: https://issues.apache.org/jira/browse/HDFS-6708 Project: Hadoop HDFS Issue Type: Sub-task Components: datanode, namenode Affects Versions: 2.4.1 Reporter: Arpit Agarwal HDFS-6702 is adding support for file creation based on StorageType. The block token is used as a tamper-proof channel for communicating block parameters from the NN to the DN during block creation. The StorageType should be included in this block token. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HDFS-8337) httpfs doesn't work with creates from a jar with kerberos
[
https://issues.apache.org/jira/browse/HDFS-8337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yongjun Zhang updated HDFS-8337:
Attachment: HDFS-8337.003.patch
Upload rev 003 to address checkstyle failure.
httpfs doesn't work with creates from a jar with kerberos
-
Key: HDFS-8337
URL: https://issues.apache.org/jira/browse/HDFS-8337
Project: Hadoop HDFS
Issue Type: Bug
Components: HDFS, hdfs-client, security, webhdfs
Reporter: Yongjun Zhang
Assignee: Yongjun Zhang
Attachments: HDFS-8337.001.patch, HDFS-8337.002.patch,
HDFS-8337.003.patch
In a secure cluster, running a simple program:
{code}
import org.apache.hadoop.conf.*;
import org.apache.hadoop.fs.*;
import org.apache.hadoop.security.*;
class Foo {
public static void main(String args[]) throws Exception {
FileSystem fs = FileSystem.get(new
java.net.URI(webhdfs://host:14000/), new Configuration());
System.out.println(fs.listStatus(new Path(/))[0]);
java.io.OutputStream os = fs.create(new Path(/tmp/foo));
os.write('a');
os.close();
}
}
{code}
Basically to access httpfs via webhdfs, the following exception is thrown:
{code}
[systest@yj52s ~]$ /usr/java/jdk1.7.0_67-cloudera/bin/java -cp $(hadoop
classpath):. Foo
15/05/06 23:51:38 WARN ssl.FileBasedKeyStoresFactory: The property
'ssl.client.truststore.location' has not been set, no TrustStore will be
loaded
Exception in thread main
org.apache.hadoop.ipc.RemoteException(com.sun.jersey.api.ParamException$QueryParamException):
java.lang.IllegalArgumentException: No enum constant
org.apache.hadoop.fs.http.client.HttpFSFileSystem.Operation.GETDELEGATIONTOKEN
at
org.apache.hadoop.hdfs.web.JsonUtil.toRemoteException(JsonUtil.java:163)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem.validateResponse(WebHdfsFileSystem.java:354)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem.access$200(WebHdfsFileSystem.java:91)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.runWithRetry(WebHdfsFileSystem.java:608)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.access$100(WebHdfsFileSystem.java:458)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner$1.run(WebHdfsFileSystem.java:487)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.run(WebHdfsFileSystem.java:483)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem.getDelegationToken(WebHdfsFileSystem.java:1299)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem.getDelegationToken(WebHdfsFileSystem.java:237)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem.getAuthParameters(WebHdfsFileSystem.java:423)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem.toUrl(WebHdfsFileSystem.java:444)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractFsPathRunner.getUrl(WebHdfsFileSystem.java:691)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.runWithRetry(WebHdfsFileSystem.java:603)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.access$100(WebHdfsFileSystem.java:458)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner$1.run(WebHdfsFileSystem.java:487)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.run(WebHdfsFileSystem.java:483)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem.listStatus(WebHdfsFileSystem.java:1277)
at Foo.main(Foo.java:7)
{code}
Thanks [~qwertymaniac] and [~caseyjbrotherton] for reporting the issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8337) httpfs doesn't work with creates from a jar with kerberos
[
https://issues.apache.org/jira/browse/HDFS-8337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14576506#comment-14576506
]
Hadoop QA commented on HDFS-8337:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 16m 57s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:red}-1{color} | tests included | 0m 0s | The patch doesn't appear
to include any new or modified tests. Please justify why no new tests are
needed for this patch. Also please list what manual steps were performed to
verify this patch. |
| {color:green}+1{color} | javac | 7m 34s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 41s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 23s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 1m 31s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 32s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 2m 41s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | common tests | 23m 10s | Tests passed in
hadoop-common. |
| {color:green}+1{color} | hdfs tests | 3m 38s | Tests passed in
hadoop-hdfs-httpfs. |
| | | 67m 44s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12738279/HDFS-8337.003.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / a6cb489 |
| hadoop-common test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/11263/artifact/patchprocess/testrun_hadoop-common.txt
|
| hadoop-hdfs-httpfs test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/11263/artifact/patchprocess/testrun_hadoop-hdfs-httpfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/11263/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/11263/console |
This message was automatically generated.
httpfs doesn't work with creates from a jar with kerberos
-
Key: HDFS-8337
URL: https://issues.apache.org/jira/browse/HDFS-8337
Project: Hadoop HDFS
Issue Type: Bug
Components: HDFS, hdfs-client, security, webhdfs
Reporter: Yongjun Zhang
Assignee: Yongjun Zhang
Attachments: HDFS-8337.001.patch, HDFS-8337.002.patch,
HDFS-8337.003.patch
In a secure cluster, running a simple program:
{code}
import org.apache.hadoop.conf.*;
import org.apache.hadoop.fs.*;
import org.apache.hadoop.security.*;
class Foo {
public static void main(String args[]) throws Exception {
FileSystem fs = FileSystem.get(new
java.net.URI(webhdfs://host:14000/), new Configuration());
System.out.println(fs.listStatus(new Path(/))[0]);
java.io.OutputStream os = fs.create(new Path(/tmp/foo));
os.write('a');
os.close();
}
}
{code}
Basically to access httpfs via webhdfs, the following exception is thrown:
{code}
[systest@yj52s ~]$ /usr/java/jdk1.7.0_67-cloudera/bin/java -cp $(hadoop
classpath):. Foo
15/05/06 23:51:38 WARN ssl.FileBasedKeyStoresFactory: The property
'ssl.client.truststore.location' has not been set, no TrustStore will be
loaded
Exception in thread main
org.apache.hadoop.ipc.RemoteException(com.sun.jersey.api.ParamException$QueryParamException):
java.lang.IllegalArgumentException: No enum constant
org.apache.hadoop.fs.http.client.HttpFSFileSystem.Operation.GETDELEGATIONTOKEN
at
org.apache.hadoop.hdfs.web.JsonUtil.toRemoteException(JsonUtil.java:163)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem.validateResponse(WebHdfsFileSystem.java:354)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem.access$200(WebHdfsFileSystem.java:91)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.runWithRetry(WebHdfsFileSystem.java:608)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.access$100(WebHdfsFileSystem.java:458)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner$1.run(WebHdfsFileSystem.java:487)
at java.security.AccessController.doPrivileged(Native Method)
at
[jira] [Commented] (HDFS-8551) Fix hdfs datanode CLI usage message
[
https://issues.apache.org/jira/browse/HDFS-8551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14576125#comment-14576125
]
Hadoop QA commented on HDFS-8551:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 18m 7s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:red}-1{color} | tests included | 0m 0s | The patch doesn't appear
to include any new or modified tests. Please justify why no new tests are
needed for this patch. Also please list what manual steps were performed to
verify this patch. |
| {color:green}+1{color} | javac | 7m 31s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 37s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 23s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 2m 17s | The applied patch generated 2
new checkstyle issues (total was 142, now 143). |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 35s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 32s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 3m 17s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | native | 3m 14s | Pre-build of native portion |
| {color:green}+1{color} | hdfs tests | 162m 2s | Tests passed in hadoop-hdfs.
|
| | | 208m 38s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12738221/HDFS-8551.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / b61b489 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HDFS-Build/11259/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt
|
| hadoop-hdfs test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/11259/artifact/patchprocess/testrun_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/11259/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/11259/console |
This message was automatically generated.
Fix hdfs datanode CLI usage message
---
Key: HDFS-8551
URL: https://issues.apache.org/jira/browse/HDFS-8551
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 2.6.0
Reporter: Xiaoyu Yao
Assignee: Brahma Reddy Battula
Attachments: HDFS-8551.patch
There are two issues with current hdfs datanode usage message below.
{code}
Usage: java DataNode [-regular | -rollback]
-regular : Normal DataNode startup (default).
-rollback: Rollback a standard or rolling upgrade.
Refer to HDFS documentation for the difference between standard
and rolling upgrades.
{code}
1. java DataNode should be hdfs datanode
2. rollingupgrace option is missing but it is document correctly in the
[link|http://hadoop.apache.org/docs/r2.7.0/hadoop-project-dist/hadoop-hdfs/HDFSCommands.html#datanode].
{code}
Usage: hdfs datanode [-regular | -rollback | -rollingupgrace rollback]
COMMAND_OPTIONDescription
-regular Normal datanode startup (default).
-rollback Rollback the datanode to the previous version. This should be
used after stopping the datanode and distributing the old hadoop version.
-rollingupgrade rollback Rollback a rolling upgrade operation.
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8552) Fix hdfs namenode CLI usage message and its document
[
https://issues.apache.org/jira/browse/HDFS-8552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14576127#comment-14576127
]
Hadoop QA commented on HDFS-8552:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 17m 56s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:red}-1{color} | tests included | 0m 0s | The patch doesn't appear
to include any new or modified tests. Please justify why no new tests are
needed for this patch. Also please list what manual steps were performed to
verify this patch. |
| {color:green}+1{color} | javac | 7m 33s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 39s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 22s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 2m 14s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 36s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 32s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 3m 14s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | native | 3m 14s | Pre-build of native portion |
| {color:red}-1{color} | hdfs tests | 160m 42s | Tests failed in hadoop-hdfs. |
| | | 207m 6s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests | hadoop.hdfs.server.namenode.TestFileTruncate |
| | hadoop.hdfs.server.namenode.TestClusterId |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12738222/HDFS-8552.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / b61b489 |
| hadoop-hdfs test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/11260/artifact/patchprocess/testrun_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/11260/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/11260/console |
This message was automatically generated.
Fix hdfs namenode CLI usage message and its document
Key: HDFS-8552
URL: https://issues.apache.org/jira/browse/HDFS-8552
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 2.6.0
Reporter: Xiaoyu Yao
Assignee: Brahma Reddy Battula
Attachments: HDFS-8552.patch
The usage message from code does not match with the one documented.
1. java NameNode should be hdfs namenode
2. Many of options are not documented.
Usage message from code:
{code}
private static final String USAGE = Usage: java NameNode [
+ StartupOption.BACKUP.getName() + ] | \n\t[
+ StartupOption.CHECKPOINT.getName() + ] | \n\t[
+ StartupOption.FORMAT.getName() + [
+ StartupOption.CLUSTERID.getName() + cid ] [
+ StartupOption.FORCE.getName() + ] [
+ StartupOption.NONINTERACTIVE.getName() + ] ] | \n\t[
+ StartupOption.UPGRADE.getName() +
[ + StartupOption.CLUSTERID.getName() + cid] +
[ + StartupOption.RENAMERESERVED.getName() + k-v pairs] ] |
\n\t[
+ StartupOption.UPGRADEONLY.getName() +
[ + StartupOption.CLUSTERID.getName() + cid] +
[ + StartupOption.RENAMERESERVED.getName() + k-v pairs] ] |
\n\t[
+ StartupOption.ROLLBACK.getName() + ] | \n\t[
+ StartupOption.ROLLINGUPGRADE.getName() +
+ RollingUpgradeStartupOption.getAllOptionString() + ] | \n\t[
+ StartupOption.IMPORT.getName() + ] | \n\t[
+ StartupOption.INITIALIZESHAREDEDITS.getName() + ] | \n\t[
+ StartupOption.BOOTSTRAPSTANDBY.getName() + ] | \n\t[
+ StartupOption.RECOVER.getName() + [
+ StartupOption.FORCE.getName() + ] ] | \n\t[
+ StartupOption.METADATAVERSION.getName() + ]
+ ];
{code}
Usage from document:
hdfs namenode [-backup] |
[-checkpoint] |
[-format [-clusterid cid ] [-force] [-nonInteractive] ] |
[-upgrade [-clusterid cid] [-renameReservedk-v pairs] ] |
[-upgradeOnly [-clusterid cid] [-renameReservedk-v pairs] ] |
[-rollback] |
[-rollingUpgrade downgrade |rollback ] |
[-finalize] |
[-importCheckpoint] |
[-initializeSharedEdits] |
[jira] [Updated] (HDFS-8450) Erasure Coding: Consolidate erasure coding zone related implementation into a single class
[
https://issues.apache.org/jira/browse/HDFS-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rakesh R updated HDFS-8450:
---
Attachment: HDFS-8450-HDFS-7285-09.patch
Erasure Coding: Consolidate erasure coding zone related implementation into a
single class
--
Key: HDFS-8450
URL: https://issues.apache.org/jira/browse/HDFS-8450
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: Rakesh R
Attachments: HDFS-8450-FYI.patch, HDFS-8450-HDFS-7285-00.patch,
HDFS-8450-HDFS-7285-01.patch, HDFS-8450-HDFS-7285-02.patch,
HDFS-8450-HDFS-7285-03.patch, HDFS-8450-HDFS-7285-04.patch,
HDFS-8450-HDFS-7285-05.patch, HDFS-8450-HDFS-7285-07.patch,
HDFS-8450-HDFS-7285-08.patch, HDFS-8450-HDFS-7285-09.patch
The idea is to follow the same pattern suggested by HDFS-7416. It is good to
consolidate all the erasure coding zone related implementations of
{{FSNamesystem}}. Here, proposing {{FSDirErasureCodingZoneOp}} class to have
functions to perform related erasure coding zone operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
