[jira] [Updated] (HDFS-3185) Setup configuration for Journal Manager and Journal Services
[ https://issues.apache.org/jira/browse/HDFS-3185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-3185: -- Resolution: Fixed Fix Version/s: Shared journals (HDFS-3092) Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) I committed this. Thank you Hari. Setup configuration for Journal Manager and Journal Services Key: HDFS-3185 URL: https://issues.apache.org/jira/browse/HDFS-3185 Project: Hadoop HDFS Issue Type: Sub-task Components: ha, name-node Reporter: Hari Mankude Assignee: Hari Mankude Fix For: Shared journals (HDFS-3092) Attachments: hdfs-3185-3.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3204) Minor modification to JournalProtocol.proto to make it generic
[ https://issues.apache.org/jira/browse/HDFS-3204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-3204: -- Resolution: Fixed Fix Version/s: 3.0.0 2.0.0 Assignee: Suresh Srinivas Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) I committed the patch to trunk and branch-2. Thanks you Nicholas for the review. Minor modification to JournalProtocol.proto to make it generic -- Key: HDFS-3204 URL: https://issues.apache.org/jira/browse/HDFS-3204 Project: Hadoop HDFS Issue Type: Improvement Components: name-node Affects Versions: 0.24.0 Reporter: Suresh Srinivas Assignee: Suresh Srinivas Fix For: 2.0.0, 3.0.0 Attachments: HDFS-3204.txt, HDFS-3204.txt, HDFS-3204.txt JournalProtocol.proto uses NamenodeRegistration in methods such as journal() for identifying the source. I want to make it generic so that the method can be called with journal information to identify the journal. I plan to use the protocol also for sync purposes, where the source of the journal can be some thing other than namenode. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3136) Multiple SLF4J binding warning
[ https://issues.apache.org/jira/browse/HDFS-3136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-3136: -- Resolution: Fixed Fix Version/s: 3.0.0 2.0.0 Target Version/s: 2.0.0 (was: 0.23.4, 0.23.3, 0.23.2) Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Multiple SLF4J binding warning -- Key: HDFS-3136 URL: https://issues.apache.org/jira/browse/HDFS-3136 Project: Hadoop HDFS Issue Type: Bug Components: build Affects Versions: 0.23.0 Reporter: Jason Lowe Assignee: Jason Lowe Fix For: 2.0.0, 3.0.0 Attachments: HDFS-3136.patch This is the HDFS portion of HADOOP-8005. HDFS no longer depends upon slf4j, so removing it from the assembly will eliminate the HDFS-portion of the multiple SLF4J warnings. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3211) JournalProtocol changes required for introducing epoch and fencing
[ https://issues.apache.org/jira/browse/HDFS-3211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-3211: -- Attachment: HDFS-3211.txt The patch is complete and read for review. I added a test for fence mechanism and checking epoch. JournalProtocol changes required for introducing epoch and fencing -- Key: HDFS-3211 URL: https://issues.apache.org/jira/browse/HDFS-3211 Project: Hadoop HDFS Issue Type: Sub-task Components: ha, name-node Affects Versions: Shared journals (HDFS-3092) Reporter: Suresh Srinivas Assignee: Suresh Srinivas Attachments: HDFS-3211.txt, HDFS-3211.txt, HDFS-3211.txt, HDFS-3211.txt, HDFS-3211.txt JournalProtocol changes to introduce epoch in every request. Adding new method fence for fencing a JournalService. On BackupNode fence is a no-op. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3121) hdfs tests for HADOOP-8014
[ https://issues.apache.org/jira/browse/HDFS-3121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-3121: -- Resolution: Fixed Fix Version/s: 3.0.0 2.0.0 Target Version/s: 2.0.0 (was: 0.23.3) Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) I committed the change to trunk and 2.0. Thanks you John. hdfs tests for HADOOP-8014 -- Key: HDFS-3121 URL: https://issues.apache.org/jira/browse/HDFS-3121 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 0.23.2, 0.23.3 Reporter: John George Assignee: John George Fix For: 2.0.0, 3.0.0 Attachments: hdfs-3121.patch, hdfs-3121.patch, hdfs-3121.patch, hdfs-3121.patch, hdfs-3121.patch, hdfs-3121.patch This JIRA is to write tests for viewing quota using viewfs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3204) Minor modification to JournalProtocol.proto to make it generic
[ https://issues.apache.org/jira/browse/HDFS-3204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-3204: -- Attachment: HDFS-3204.txt Minor modification to JournalProtocol.proto to make it generic -- Key: HDFS-3204 URL: https://issues.apache.org/jira/browse/HDFS-3204 Project: Hadoop HDFS Issue Type: Improvement Components: name-node Affects Versions: 0.24.0 Reporter: Suresh Srinivas Attachments: HDFS-3204.txt JournalProtocol.proto uses NamenodeRegistration in methods such as journal() for identifying the source. I want to make it generic so that the method can be called with journal information to identify the journal. I plan to use the protocol also for sync purposes, where the source of the journal can be some thing other than namenode. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3204) Minor modification to JournalProtocol.proto to make it generic
[ https://issues.apache.org/jira/browse/HDFS-3204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-3204: -- Attachment: HDFS-3204.txt Minor modification to JournalProtocol.proto to make it generic -- Key: HDFS-3204 URL: https://issues.apache.org/jira/browse/HDFS-3204 Project: Hadoop HDFS Issue Type: Improvement Components: name-node Affects Versions: 0.24.0 Reporter: Suresh Srinivas Attachments: HDFS-3204.txt JournalProtocol.proto uses NamenodeRegistration in methods such as journal() for identifying the source. I want to make it generic so that the method can be called with journal information to identify the journal. I plan to use the protocol also for sync purposes, where the source of the journal can be some thing other than namenode. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3204) Minor modification to JournalProtocol.proto to make it generic
[ https://issues.apache.org/jira/browse/HDFS-3204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-3204: -- Attachment: (was: HDFS-3204.txt) Minor modification to JournalProtocol.proto to make it generic -- Key: HDFS-3204 URL: https://issues.apache.org/jira/browse/HDFS-3204 Project: Hadoop HDFS Issue Type: Improvement Components: name-node Affects Versions: 0.24.0 Reporter: Suresh Srinivas Attachments: HDFS-3204.txt JournalProtocol.proto uses NamenodeRegistration in methods such as journal() for identifying the source. I want to make it generic so that the method can be called with journal information to identify the journal. I plan to use the protocol also for sync purposes, where the source of the journal can be some thing other than namenode. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3204) Minor modification to JournalProtocol.proto to make it generic
[ https://issues.apache.org/jira/browse/HDFS-3204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-3204: -- Status: Patch Available (was: Open) Minor modification to JournalProtocol.proto to make it generic -- Key: HDFS-3204 URL: https://issues.apache.org/jira/browse/HDFS-3204 Project: Hadoop HDFS Issue Type: Improvement Components: name-node Affects Versions: 0.24.0 Reporter: Suresh Srinivas Attachments: HDFS-3204.txt JournalProtocol.proto uses NamenodeRegistration in methods such as journal() for identifying the source. I want to make it generic so that the method can be called with journal information to identify the journal. I plan to use the protocol also for sync purposes, where the source of the journal can be some thing other than namenode. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3109) Remove hsqldb exclusions from pom.xml
[ https://issues.apache.org/jira/browse/HDFS-3109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-3109: -- Resolution: Fixed Fix Version/s: 3.0.0 2.0.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) I committed the patch to trunk and branch-2. Thank you Ravi. Also changing the Target Version to 2.0. If it is really needed for 23.x, let me know. Remove hsqldb exclusions from pom.xml - Key: HDFS-3109 URL: https://issues.apache.org/jira/browse/HDFS-3109 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 0.23.2 Reporter: Ravi Prakash Assignee: Ravi Prakash Fix For: 2.0.0, 3.0.0 Attachments: HDFS-3109.patch Related to MAPREDUCE-3621 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3204) Minor modification to JournalProtocol.proto to make it generic
[ https://issues.apache.org/jira/browse/HDFS-3204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-3204: -- Attachment: HDFS-3204.txt Updated patch with comments addressed. Additional changes: # Removed printing exception trace in BackupImage.java. While debugging it threw me off, as I interpreted as error. # Failure is due to BackupNode checking the rpc server address. I do not think it is useful for couple of reasons. There is already an extensive test for checking clusterID, namespaceID etc. Also the removed check was only made in journal() method call and not in startLogSegment() # Given the changes, journal method calls are verified for namespaceID, cluster and version match. Dropping cTime matching. Minor modification to JournalProtocol.proto to make it generic -- Key: HDFS-3204 URL: https://issues.apache.org/jira/browse/HDFS-3204 Project: Hadoop HDFS Issue Type: Improvement Components: name-node Affects Versions: 0.24.0 Reporter: Suresh Srinivas Attachments: HDFS-3204.txt, HDFS-3204.txt JournalProtocol.proto uses NamenodeRegistration in methods such as journal() for identifying the source. I want to make it generic so that the method can be called with journal information to identify the journal. I plan to use the protocol also for sync purposes, where the source of the journal can be some thing other than namenode. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3204) Minor modification to JournalProtocol.proto to make it generic
[ https://issues.apache.org/jira/browse/HDFS-3204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-3204: -- Attachment: HDFS-3204.txt Minor modification to JournalProtocol.proto to make it generic -- Key: HDFS-3204 URL: https://issues.apache.org/jira/browse/HDFS-3204 Project: Hadoop HDFS Issue Type: Improvement Components: name-node Affects Versions: 0.24.0 Reporter: Suresh Srinivas Attachments: HDFS-3204.txt, HDFS-3204.txt, HDFS-3204.txt JournalProtocol.proto uses NamenodeRegistration in methods such as journal() for identifying the source. I want to make it generic so that the method can be called with journal information to identify the journal. I plan to use the protocol also for sync purposes, where the source of the journal can be some thing other than namenode. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3211) JournalProtocol changes required for introducing epoch and fencing
[ https://issues.apache.org/jira/browse/HDFS-3211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-3211: -- Affects Version/s: Shared journals (HDFS-3092) JournalProtocol changes required for introducing epoch and fencing -- Key: HDFS-3211 URL: https://issues.apache.org/jira/browse/HDFS-3211 Project: Hadoop HDFS Issue Type: Sub-task Components: ha, name-node Affects Versions: Shared journals (HDFS-3092) Reporter: Suresh Srinivas Assignee: Suresh Srinivas JournalProtocol changes to introduce epoch in every request. Adding new method fence for fencing a JournalService. On BackupNode fence is a no-op. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3211) JournalProtocol changes required for introducing epoch and fencing
[ https://issues.apache.org/jira/browse/HDFS-3211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-3211: -- Attachment: HDFS-3211.txt JournalProtocol changes required for introducing epoch and fencing -- Key: HDFS-3211 URL: https://issues.apache.org/jira/browse/HDFS-3211 Project: Hadoop HDFS Issue Type: Sub-task Components: ha, name-node Affects Versions: Shared journals (HDFS-3092) Reporter: Suresh Srinivas Assignee: Suresh Srinivas Attachments: HDFS-3211.txt JournalProtocol changes to introduce epoch in every request. Adding new method fence for fencing a JournalService. On BackupNode fence is a no-op. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3211) JournalProtocol changes required for introducing epoch and fencing
[ https://issues.apache.org/jira/browse/HDFS-3211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-3211: -- Attachment: HDFS-3211.txt Early version of the patch with changes: # Changed JournalProtocol to use JournalInfo instead of NamenodeRegistration - continuing the work from HDFS-3204. # All Journal methods are changed to include epoch # Added fence method to JournalProtocol. #* fence is allowed only if epoch is higher #* In JournalService, in fence method the epoch is saved. Subsequently only the requests with that epoch succeeds. Will add unit tests. JournalProtocol changes required for introducing epoch and fencing -- Key: HDFS-3211 URL: https://issues.apache.org/jira/browse/HDFS-3211 Project: Hadoop HDFS Issue Type: Sub-task Components: ha, name-node Affects Versions: Shared journals (HDFS-3092) Reporter: Suresh Srinivas Assignee: Suresh Srinivas Attachments: HDFS-3211.txt, HDFS-3211.txt JournalProtocol changes to introduce epoch in every request. Adding new method fence for fencing a JournalService. On BackupNode fence is a no-op. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3211) JournalProtocol changes required for introducing epoch and fencing
[ https://issues.apache.org/jira/browse/HDFS-3211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-3211: -- Attachment: HDFS-3211.txt Updated patch with missing files. JournalProtocol changes required for introducing epoch and fencing -- Key: HDFS-3211 URL: https://issues.apache.org/jira/browse/HDFS-3211 Project: Hadoop HDFS Issue Type: Sub-task Components: ha, name-node Affects Versions: Shared journals (HDFS-3092) Reporter: Suresh Srinivas Assignee: Suresh Srinivas Attachments: HDFS-3211.txt, HDFS-3211.txt, HDFS-3211.txt JournalProtocol changes to introduce epoch in every request. Adding new method fence for fencing a JournalService. On BackupNode fence is a no-op. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3211) JournalProtocol changes required for introducing epoch and fencing
[ https://issues.apache.org/jira/browse/HDFS-3211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-3211: -- Attachment: HDFS-3211.txt Updated patch with fixes to test failure. JournalProtocol changes required for introducing epoch and fencing -- Key: HDFS-3211 URL: https://issues.apache.org/jira/browse/HDFS-3211 Project: Hadoop HDFS Issue Type: Sub-task Components: ha, name-node Affects Versions: Shared journals (HDFS-3092) Reporter: Suresh Srinivas Assignee: Suresh Srinivas Attachments: HDFS-3211.txt, HDFS-3211.txt, HDFS-3211.txt, HDFS-3211.txt JournalProtocol changes to introduce epoch in every request. Adding new method fence for fencing a JournalService. On BackupNode fence is a no-op. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3211) JournalProtocol changes required for introducing epoch and fencing
[ https://issues.apache.org/jira/browse/HDFS-3211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-3211: -- Attachment: HDFS-3211.txt JournalProtocol changes required for introducing epoch and fencing -- Key: HDFS-3211 URL: https://issues.apache.org/jira/browse/HDFS-3211 Project: Hadoop HDFS Issue Type: Sub-task Components: ha, name-node Affects Versions: Shared journals (HDFS-3092) Reporter: Suresh Srinivas Assignee: Suresh Srinivas Attachments: HDFS-3211.txt, HDFS-3211.txt, HDFS-3211.txt, HDFS-3211.txt JournalProtocol changes to introduce epoch in every request. Adding new method fence for fencing a JournalService. On BackupNode fence is a no-op. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3211) JournalProtocol changes required for introducing epoch and fencing
[ https://issues.apache.org/jira/browse/HDFS-3211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-3211: -- Status: Patch Available (was: Open) JournalProtocol changes required for introducing epoch and fencing -- Key: HDFS-3211 URL: https://issues.apache.org/jira/browse/HDFS-3211 Project: Hadoop HDFS Issue Type: Sub-task Components: ha, name-node Affects Versions: Shared journals (HDFS-3092) Reporter: Suresh Srinivas Assignee: Suresh Srinivas Attachments: HDFS-3211.txt, HDFS-3211.txt, HDFS-3211.txt, HDFS-3211.txt JournalProtocol changes to introduce epoch in every request. Adding new method fence for fencing a JournalService. On BackupNode fence is a no-op. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3183) Add JournalManager implementation to JournalDaemons for storing edits
[ https://issues.apache.org/jira/browse/HDFS-3183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-3183: -- Description: The JournalManager is used in HA configuration and uses the following journal daemons on: - Local namenode - Other namenode - A configured JournalDaemon target from the configuration was: The JournalManager is used in HA configuration and uses the following journal targets: - local namenode - Other namenode - A configured JournalDaemon target from configuration Summary: Add JournalManager implementation to JournalDaemons for storing edits (was: Add JournalManager implementation to use local namenode, remote namenode and a configured JournalDaemon for storing editlogs) Add JournalManager implementation to JournalDaemons for storing edits - Key: HDFS-3183 URL: https://issues.apache.org/jira/browse/HDFS-3183 Project: Hadoop HDFS Issue Type: Sub-task Components: ha, name-node Reporter: Suresh Srinivas The JournalManager is used in HA configuration and uses the following journal daemons on: - Local namenode - Other namenode - A configured JournalDaemon target from the configuration -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3197) Incorrect class comments in a few tests
[
https://issues.apache.org/jira/browse/HDFS-3197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Suresh Srinivas updated HDFS-3197:
--
Description:
TestFileCreationClient, TestDatanodeDeath, TestReplaceDatanodeOnFailure, and
TestDatanodeRegistration all have the following class comment:
{noformat}
/**
* This class tests that a file need not be closed before its
* data can be read by another client.
*/
{noformat}
The javadoc for the classes need to be updated to describe the tests.
was:
Somewhat hilariously, TestFileCreationClient, TestDatanodeDeath,
TestReplaceDatanodeOnFailure, and TestDatanodeRegistration all have the
following class comment:
{noformat}
/**
* This class tests that a file need not be closed before its
* data can be read by another client.
*/
{noformat}
Summary: Incorrect class comments in a few tests (was: Duplicate (and
incorrect) class comments in a few tests)
Incorrect class comments in a few tests
---
Key: HDFS-3197
URL: https://issues.apache.org/jira/browse/HDFS-3197
Project: Hadoop HDFS
Issue Type: Bug
Components: test
Affects Versions: 0.24.0
Reporter: Aaron T. Myers
Priority: Trivial
Labels: newbie
TestFileCreationClient, TestDatanodeDeath, TestReplaceDatanodeOnFailure, and
TestDatanodeRegistration all have the following class comment:
{noformat}
/**
* This class tests that a file need not be closed before its
* data can be read by another client.
*/
{noformat}
The javadoc for the classes need to be updated to describe the tests.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3092) Enable journal protocol based editlog streaming for standby namenode
[ https://issues.apache.org/jira/browse/HDFS-3092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-3092: -- Attachment: MultipleSharedJournals.pdf Design document. Enable journal protocol based editlog streaming for standby namenode Key: HDFS-3092 URL: https://issues.apache.org/jira/browse/HDFS-3092 Project: Hadoop HDFS Issue Type: Improvement Components: ha, name-node Affects Versions: 0.24.0, 0.23.3 Reporter: Suresh Srinivas Assignee: Suresh Srinivas Attachments: MultipleSharedJournals.pdf Currently standby namenode relies on reading shared editlogs to stay current with the active namenode, for namespace changes. BackupNode used streaming edits from active namenode for doing the same. This jira is to explore using journal protocol based editlog streams for the standby namenode. A daemon in standby will get the editlogs from the active and write it to local edits. To begin with, the existing standby mechanism of reading from a file, will continue to be used, instead of from shared edits, from the local edits. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3126) Journal stream from the namenode to backup needs to have a timeout
[ https://issues.apache.org/jira/browse/HDFS-3126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-3126: -- Resolution: Fixed Fix Version/s: 0.24.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) I committed the patch. Thanks you Hari. Journal stream from the namenode to backup needs to have a timeout -- Key: HDFS-3126 URL: https://issues.apache.org/jira/browse/HDFS-3126 Project: Hadoop HDFS Issue Type: Sub-task Components: ha, name-node Affects Versions: 0.24.0 Reporter: Hari Mankude Assignee: Hari Mankude Fix For: 0.24.0 Attachments: hdfs-3126.patch, hdfs-3126.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3125) Add a service that enables JournalDaemon
[ https://issues.apache.org/jira/browse/HDFS-3125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-3125: -- Attachment: HDFS-3125.patch Updated patch with verifyRequest added to startLogSegment method Add a service that enables JournalDaemon Key: HDFS-3125 URL: https://issues.apache.org/jira/browse/HDFS-3125 Project: Hadoop HDFS Issue Type: Sub-task Components: ha, name-node Reporter: Suresh Srinivas Assignee: Suresh Srinivas Attachments: HDFS-3125.patch, HDFS-3125.patch, HDFS-3125.patch, HDFS-3125.patch In this subtask, I plan to add JournalService. It will provide the following functionality: # Starts RPC server with JournalProtocolService or uses the RPC server provided and add JournalProtocol service. # Registers with the namenode. # Receives JournalProtocol related requests and hands it to over to a listener. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3125) Add a service that enables JournalDaemon
[ https://issues.apache.org/jira/browse/HDFS-3125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-3125: -- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) I committed the patch. Add a service that enables JournalDaemon Key: HDFS-3125 URL: https://issues.apache.org/jira/browse/HDFS-3125 Project: Hadoop HDFS Issue Type: Sub-task Components: ha, name-node Reporter: Suresh Srinivas Assignee: Suresh Srinivas Attachments: HDFS-3125.patch, HDFS-3125.patch, HDFS-3125.patch, HDFS-3125.patch In this subtask, I plan to add JournalService. It will provide the following functionality: # Starts RPC server with JournalProtocolService or uses the RPC server provided and add JournalProtocol service. # Registers with the namenode. # Receives JournalProtocol related requests and hands it to over to a listener. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3125) Add a service that enables JournalDaemon
[ https://issues.apache.org/jira/browse/HDFS-3125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-3125: -- Attachment: HDFS-3125.patch Updated patch - ready for review. Add a service that enables JournalDaemon Key: HDFS-3125 URL: https://issues.apache.org/jira/browse/HDFS-3125 Project: Hadoop HDFS Issue Type: Sub-task Components: ha, name-node Reporter: Suresh Srinivas Assignee: Suresh Srinivas Attachments: HDFS-3125.patch, HDFS-3125.patch In this subtask, I plan to add JournalService. It will provide the following functionality: # Starts RPC server with JournalProtocolService or uses the RPC server provided and add JournalProtocol service. # Registers with the namenode. # Receives JournalProtocol related requests and hands it to over to a listener. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3125) Add a service that enables JournalDaemon
[ https://issues.apache.org/jira/browse/HDFS-3125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-3125: -- Attachment: HDFS-3125.patch bq. Question: why service is need in the methods of JournalListener? Makes it convenient to know which listener is sending the call back. Then the listener can stop the service using the reference it gets. bq. Does is make sense to support multiple listeners? If yes, we should implement the observer pattern. I do not have a use case for multiple lister. If needed, we can add support for it later. What do you think? Other comments addressed. I also changed exception handling - the exception from callbacks are thrown all the way to the client. Add a service that enables JournalDaemon Key: HDFS-3125 URL: https://issues.apache.org/jira/browse/HDFS-3125 Project: Hadoop HDFS Issue Type: Sub-task Components: ha, name-node Reporter: Suresh Srinivas Assignee: Suresh Srinivas Attachments: HDFS-3125.patch, HDFS-3125.patch, HDFS-3125.patch In this subtask, I plan to add JournalService. It will provide the following functionality: # Starts RPC server with JournalProtocolService or uses the RPC server provided and add JournalProtocol service. # Registers with the namenode. # Receives JournalProtocol related requests and hands it to over to a listener. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3125) Add a service that enable JournalDaemon
[ https://issues.apache.org/jira/browse/HDFS-3125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-3125: -- Attachment: (was: HDFS-3125.patch) Add a service that enable JournalDaemon --- Key: HDFS-3125 URL: https://issues.apache.org/jira/browse/HDFS-3125 Project: Hadoop HDFS Issue Type: Sub-task Components: ha, name-node Reporter: Suresh Srinivas Assignee: Suresh Srinivas Attachments: HDFS-3125.patch In this subtask, I plan to add JournalService. It will provide the following functionality: # Starts RPC server with JournalProtocolService or uses the RPC server provided and add JournalProtocol service. # Registers with the namenode. # Receives JournalProtocol related requests and hands it to over to a listener. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3125) Add a service that enable JournalDaemon
[ https://issues.apache.org/jira/browse/HDFS-3125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-3125: -- Attachment: (was: HDFS-3125.patch) Add a service that enable JournalDaemon --- Key: HDFS-3125 URL: https://issues.apache.org/jira/browse/HDFS-3125 Project: Hadoop HDFS Issue Type: Sub-task Components: ha, name-node Reporter: Suresh Srinivas Assignee: Suresh Srinivas Attachments: HDFS-3125.patch In this subtask, I plan to add JournalService. It will provide the following functionality: # Starts RPC server with JournalProtocolService or uses the RPC server provided and add JournalProtocol service. # Registers with the namenode. # Receives JournalProtocol related requests and hands it to over to a listener. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3125) Add a service that enable JournalDaemon
[ https://issues.apache.org/jira/browse/HDFS-3125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-3125: -- Attachment: HDFS-3125.patch Patch with tests Add a service that enable JournalDaemon --- Key: HDFS-3125 URL: https://issues.apache.org/jira/browse/HDFS-3125 Project: Hadoop HDFS Issue Type: Sub-task Components: ha, name-node Reporter: Suresh Srinivas Assignee: Suresh Srinivas Attachments: HDFS-3125.patch In this subtask, I plan to add JournalService. It will provide the following functionality: # Starts RPC server with JournalProtocolService or uses the RPC server provided and add JournalProtocol service. # Registers with the namenode. # Receives JournalProtocol related requests and hands it to over to a listener. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3125) Add a service that enables JournalDaemon
[ https://issues.apache.org/jira/browse/HDFS-3125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-3125: -- Summary: Add a service that enables JournalDaemon (was: Add a service that enable JournalDaemon) Add a service that enables JournalDaemon Key: HDFS-3125 URL: https://issues.apache.org/jira/browse/HDFS-3125 Project: Hadoop HDFS Issue Type: Sub-task Components: ha, name-node Reporter: Suresh Srinivas Assignee: Suresh Srinivas Attachments: HDFS-3125.patch In this subtask, I plan to add JournalService. It will provide the following functionality: # Starts RPC server with JournalProtocolService or uses the RPC server provided and add JournalProtocol service. # Registers with the namenode. # Receives JournalProtocol related requests and hands it to over to a listener. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3125) Add a service that enables JournalDaemon
[ https://issues.apache.org/jira/browse/HDFS-3125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-3125: -- Status: Patch Available (was: Open) Add a service that enables JournalDaemon Key: HDFS-3125 URL: https://issues.apache.org/jira/browse/HDFS-3125 Project: Hadoop HDFS Issue Type: Sub-task Components: ha, name-node Reporter: Suresh Srinivas Assignee: Suresh Srinivas Attachments: HDFS-3125.patch In this subtask, I plan to add JournalService. It will provide the following functionality: # Starts RPC server with JournalProtocolService or uses the RPC server provided and add JournalProtocol service. # Registers with the namenode. # Receives JournalProtocol related requests and hands it to over to a listener. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3125) Add a service that enable JournalDaemon
[ https://issues.apache.org/jira/browse/HDFS-3125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-3125: -- Attachment: HDFS-3125.patch I will create a separate jira to add tests, as this work is still under development Add a service that enable JournalDaemon --- Key: HDFS-3125 URL: https://issues.apache.org/jira/browse/HDFS-3125 Project: Hadoop HDFS Issue Type: Sub-task Components: ha, name-node Reporter: Suresh Srinivas Attachments: HDFS-3125.patch In this subtask, I plan to add JournalService. It will provide the following functionality: # Starts RPC server with JournalProtocolService or uses the RPC server provided and add JournalProtocol service. # Registers with the namenode. # Receives JournalProtocol related requests and hands it to over to a listener. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3125) Add a service that enable JournalDaemon
[ https://issues.apache.org/jira/browse/HDFS-3125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-3125: -- Attachment: HDFS-3125.patch More cleanup. Supports two modes: # Start daemon with an existing rpc server - for daemons embedded into Namenode process. # Mode which creates RPC server for stand alone daemon. Add a service that enable JournalDaemon --- Key: HDFS-3125 URL: https://issues.apache.org/jira/browse/HDFS-3125 Project: Hadoop HDFS Issue Type: Sub-task Components: ha, name-node Reporter: Suresh Srinivas Assignee: Suresh Srinivas Attachments: HDFS-3125.patch, HDFS-3125.patch In this subtask, I plan to add JournalService. It will provide the following functionality: # Starts RPC server with JournalProtocolService or uses the RPC server provided and add JournalProtocol service. # Registers with the namenode. # Receives JournalProtocol related requests and hands it to over to a listener. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2701) Cleanup FS* processIOError methods
[ https://issues.apache.org/jira/browse/HDFS-2701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2701: -- Target Version/s: 1.0.2 Fix Version/s: 1.0.2 Cleanup FS* processIOError methods -- Key: HDFS-2701 URL: https://issues.apache.org/jira/browse/HDFS-2701 Project: Hadoop HDFS Issue Type: Improvement Components: name-node Affects Versions: 1.0.0 Reporter: Eli Collins Assignee: Eli Collins Fix For: 1.1.0, 1.0.2 Attachments: hdfs-2701.txt, hdfs-2701.txt, hdfs-2701.txt, hdfs-2701.txt Let's rename the various processIOError methods to be more descriptive. The current code makes it difficult to identify and reason about bug fixes. While we're at it let's remove Fatal from the Unable to sync the edit log log since it's not actually a fatal error (this is confusing to users). And 2NN Checkpoint done should be info, not a warning (also confusing to users). Thanks to HDFS-1073 these issues don't exist on trunk or 23. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2702) A single failed name dir can cause the NN to exit
[
https://issues.apache.org/jira/browse/HDFS-2702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Suresh Srinivas updated HDFS-2702:
--
Target Version/s: 1.0.2
Fix Version/s: 1.0.2
A single failed name dir can cause the NN to exit
--
Key: HDFS-2702
URL: https://issues.apache.org/jira/browse/HDFS-2702
Project: Hadoop HDFS
Issue Type: Bug
Components: name-node
Affects Versions: 1.0.0
Reporter: Eli Collins
Assignee: Eli Collins
Priority: Critical
Fix For: 1.1.0, 1.0.2
Attachments: hdfs-2702.txt, hdfs-2702.txt, hdfs-2702.txt,
hdfs-2702.txt, hdfs-2702.txt
There's a bug in FSEditLog#rollEditLog which results in the NN process
exiting if a single name dir has failed. Here's the relevant code:
{code}
close() // So editStreams.size() is 0
foreach edits dir {
..
eStream = new ... // Might get an IOE here
editStreams.add(eStream);
} catch (IOException ioe) {
removeEditsForStorageDir(sd); // exits if editStreams.size() = 1
}
{code}
If we get an IOException before we've added two edits streams to the list
we'll exit, eg if there's an error processing the 1st name dir we'll exit
even if there are 4 valid name dirs. The fix is to move the checking out of
removeEditsForStorageDir (nee processIOError) or modify it so it can be
disabled in some cases, eg here where we don't yet know how many streams are
valid.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3075) Backport HADOOP-4885 to branch-1
[ https://issues.apache.org/jira/browse/HDFS-3075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-3075: -- Target Version/s: 1.0.2 Affects Version/s: 1.0.0 Fix Version/s: 1.0.2 Backport HADOOP-4885 to branch-1 Key: HDFS-3075 URL: https://issues.apache.org/jira/browse/HDFS-3075 Project: Hadoop HDFS Issue Type: Improvement Components: name-node Affects Versions: 1.0.0 Reporter: Brandon Li Assignee: Brandon Li Fix For: 1.1.0, 1.0.2 When a storage directory is inaccessible, namenode removes it from the valid storage dir list to a removedStorageDirs list. Those storage directories will not be restored when they become healthy again. The proposed solution is to restore the previous failed directories at the beginning of checkpointing, say, rollEdits, by copying necessary metadata files from healthy directory to unhealthy ones. In this way, whenever a failed storage directory is recovered by the administrator, he/she can immediately force a checkpointing to restored a failed directory. See also HADOOP-4885. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2703) removedStorageDirs is not updated everywhere we remove a storage dir
[ https://issues.apache.org/jira/browse/HDFS-2703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2703: -- Target Version/s: 1.0.2 Fix Version/s: 1.0.2 removedStorageDirs is not updated everywhere we remove a storage dir Key: HDFS-2703 URL: https://issues.apache.org/jira/browse/HDFS-2703 Project: Hadoop HDFS Issue Type: Bug Components: name-node Affects Versions: 1.0.0 Reporter: Eli Collins Assignee: Eli Collins Fix For: 1.1.0, 1.0.2 Attachments: hdfs-2703.txt There are a number of places (FSEditLog#open, purgeEditLog, and rollEditLog) where we remove a storage directory but don't add it to the removedStorageDirs list. This means a storage dir may have been removed but we don't see it in the log or Web UI. This doesn't affect trunk/23 since the code there is totally different. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1623) High Availability Framework for HDFS NN
[ https://issues.apache.org/jira/browse/HDFS-1623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-1623: -- Target Version/s: 0.24.0, 0.23.3 (was: 0.24.0) Fix Version/s: 0.23.3 High Availability Framework for HDFS NN --- Key: HDFS-1623 URL: https://issues.apache.org/jira/browse/HDFS-1623 Project: Hadoop HDFS Issue Type: New Feature Reporter: Sanjay Radia Fix For: 0.24.0, 0.23.3 Attachments: HA-tests.pdf, HDFS-1623.rel23.patch, HDFS-1623.trunk.patch, HDFS-High-Availability.pdf, NameNode HA_v2.pdf, NameNode HA_v2_1.pdf, Namenode HA Framework.pdf, dfsio-results.tsv, ha-testplan.pdf, ha-testplan.tex -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2507) HA: Allow saveNamespace operations to be canceled
[ https://issues.apache.org/jira/browse/HDFS-2507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2507: -- Target Version/s: HA branch (HDFS-1623), 0.24.0, 0.23.3 (was: 0.24.0, HA branch (HDFS-1623)) Fix Version/s: 0.23.3 HA: Allow saveNamespace operations to be canceled - Key: HDFS-2507 URL: https://issues.apache.org/jira/browse/HDFS-2507 Project: Hadoop HDFS Issue Type: Improvement Components: name-node Affects Versions: HA branch (HDFS-1623), 0.23.0 Reporter: Todd Lipcon Assignee: Todd Lipcon Fix For: HA branch (HDFS-1623), 0.24.0, 0.23.3 Attachments: hdfs-2507.txt For HDFS-2291 (checkpointing in HA setup) we need to allow the StandbyNode to handle a failover while it may be in the middle of saving a checkpoint. This implies that checkpoints need to be abortable. This JIRA is to implement this functionality. Even though this is only needed for HA, I think we should do the infrastructure in trunk to simplify future merges. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2285) BackupNode should reject requests trying to modify namespace
[
https://issues.apache.org/jira/browse/HDFS-2285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Suresh Srinivas updated HDFS-2285:
--
Target Version/s: 0.23.3
Affects Version/s: 0.23.0
Fix Version/s: 0.23.3
BackupNode should reject requests trying to modify namespace
Key: HDFS-2285
URL: https://issues.apache.org/jira/browse/HDFS-2285
Project: Hadoop HDFS
Issue Type: Bug
Components: name-node
Affects Versions: 0.22.0, 0.23.0, 0.24.0
Reporter: Konstantin Shvachko
Assignee: Konstantin Shvachko
Fix For: 0.22.0, 0.24.0, 0.23.3
Attachments: BNsafemode.patch, HDFS-2285.patch, HDFS-2285.patch
I am trying to remove file from BackupNode using
{code}hadoop fs -fs hdfs://backup.node.com:50100 -rm /README.txt{code}
which is supposed to fail. But it seems to be hanging forever.
Needs some investigation. It used to throw SafeModeException if I remember
correctly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2410) Further clean up hard-coded configuration keys
[ https://issues.apache.org/jira/browse/HDFS-2410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2410: -- Target Version/s: 0.23.3 (was: 0.23.1) Fix Version/s: 0.23.3 Further clean up hard-coded configuration keys -- Key: HDFS-2410 URL: https://issues.apache.org/jira/browse/HDFS-2410 Project: Hadoop HDFS Issue Type: Improvement Components: data-node, name-node, test Reporter: Suresh Srinivas Assignee: Suresh Srinivas Priority: Minor Fix For: 0.24.0, 0.23.3 Attachments: HDFS-2410.txt, HDFS-2410.txt HDFS code is littered with hardcoded config key names. This jira changes to use DFSConfigKeys constants. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2430) The number of failed or low-resource volumes the NN can tolerate should be configurable
[ https://issues.apache.org/jira/browse/HDFS-2430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2430: -- Target Version/s: 0.23.3 Fix Version/s: 0.23.3 The number of failed or low-resource volumes the NN can tolerate should be configurable --- Key: HDFS-2430 URL: https://issues.apache.org/jira/browse/HDFS-2430 Project: Hadoop HDFS Issue Type: New Feature Components: name-node Affects Versions: 0.24.0 Reporter: Aaron T. Myers Assignee: Aaron T. Myers Fix For: 0.24.0, 0.23.3 Attachments: HDFS-2430.patch, HDFS-2430.patch, HDFS-2430.patch, HDFS-2430.patch, HDFS-2430.patch, HDFS-2430.patch Currently the number of failed or low-resource volumes the NN can tolerate is effectively hard-coded at 1. It would be nice if this were configurable. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-208) name node should warn if only one dir is listed in dfs.name.dir
[ https://issues.apache.org/jira/browse/HDFS-208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-208: - Target Version/s: 0.23.3 Fix Version/s: 0.23.3 name node should warn if only one dir is listed in dfs.name.dir --- Key: HDFS-208 URL: https://issues.apache.org/jira/browse/HDFS-208 Project: Hadoop HDFS Issue Type: Improvement Components: name-node Reporter: Allen Wittenauer Assignee: Uma Maheswara Rao G Priority: Minor Labels: newbie Fix For: 0.24.0, 0.23.3 Attachments: HDFS-208.patch, hdfs-208.patch The name node should warn that corruption may occur if only one directory is listed in the dfs.name.dir setting. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2878) TestBlockRecovery does not compile
[ https://issues.apache.org/jira/browse/HDFS-2878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2878: -- Target Version/s: 0.24.0, 0.23.3 (was: 0.24.0) Fix Version/s: 0.23.3 TestBlockRecovery does not compile -- Key: HDFS-2878 URL: https://issues.apache.org/jira/browse/HDFS-2878 Project: Hadoop HDFS Issue Type: Bug Components: test Affects Versions: 0.23.1 Reporter: Eli Collins Assignee: Todd Lipcon Priority: Blocker Fix For: 0.24.0, 0.23.3 Attachments: hdfs-2878.txt Looks like HDFS-2563 introduced a compilation error in TestBlockRecovery. We didn't catch this because of HDFS-2876. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3003) Remove getHostPortString() from NameNode, replace it with NetUtils.getHostPortString()
[ https://issues.apache.org/jira/browse/HDFS-3003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-3003: -- Target Version/s: 0.24.0, 0.23.3 (was: 0.24.0) Fix Version/s: 0.23.3 Remove getHostPortString() from NameNode, replace it with NetUtils.getHostPortString() -- Key: HDFS-3003 URL: https://issues.apache.org/jira/browse/HDFS-3003 Project: Hadoop HDFS Issue Type: Improvement Components: name-node Affects Versions: 0.24.0 Reporter: Brandon Li Assignee: Brandon Li Priority: Trivial Fix For: 0.24.0, 0.23.3 Attachments: HDFS-3003.patch, HDFS-3003.patch, HDFS-3003.patch The method getHostPortString() is not directly related with NameNode, and it's more of a utility method. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3014) FSEditLogOp and its subclasses should have toString() method
[ https://issues.apache.org/jira/browse/HDFS-3014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-3014: -- Target Version/s: 0.24.0, 0.23.3 (was: 0.24.0) Fix Version/s: 0.23.3 FSEditLogOp and its subclasses should have toString() method Key: HDFS-3014 URL: https://issues.apache.org/jira/browse/HDFS-3014 Project: Hadoop HDFS Issue Type: Improvement Components: name-node Affects Versions: 0.24.0 Reporter: Sho Shimauchi Assignee: Sho Shimauchi Fix For: 0.24.0, 0.23.3 Attachments: HDFS-3014.txt, HDFS-3014.txt FSEditLogOp and its subclasses should have toString() method. It's helpful for investigating editlog related issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3036) Remove unused method DFSUtil#isDefaultNamenodeAddress
[
https://issues.apache.org/jira/browse/HDFS-3036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Suresh Srinivas updated HDFS-3036:
--
Target Version/s: 0.24.0, 0.23.3 (was: 0.24.0)
Fix Version/s: 0.23.3
Remove unused method DFSUtil#isDefaultNamenodeAddress
-
Key: HDFS-3036
URL: https://issues.apache.org/jira/browse/HDFS-3036
Project: Hadoop HDFS
Issue Type: Improvement
Components: name-node
Affects Versions: 0.24.0
Reporter: Aaron T. Myers
Assignee: Aaron T. Myers
Priority: Trivial
Fix For: 0.24.0, 0.23.3
Attachments: HDFS-3036.patch
{{DFSUtil#isDefaultNamenodeAddress}} is a private interface but is only
referenced in a test for the behavior of this method.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1623) High Availability Framework for HDFS NN
[ https://issues.apache.org/jira/browse/HDFS-1623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-1623: -- Attachment: HDFS-1623.rel23.patch Patch attached for branch-0.23. High Availability Framework for HDFS NN --- Key: HDFS-1623 URL: https://issues.apache.org/jira/browse/HDFS-1623 Project: Hadoop HDFS Issue Type: New Feature Reporter: Sanjay Radia Fix For: 0.24.0 Attachments: HA-tests.pdf, HDFS-1623.rel23.patch, HDFS-1623.trunk.patch, HDFS-High-Availability.pdf, NameNode HA_v2.pdf, NameNode HA_v2_1.pdf, Namenode HA Framework.pdf, dfsio-results.tsv, ha-testplan.pdf, ha-testplan.tex -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2477) Optimize computing the diff between a block report and the namenode state.
[ https://issues.apache.org/jira/browse/HDFS-2477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2477: -- Target Version/s: 0.23.3 Affects Version/s: 0.23.0 Fix Version/s: 0.23.3 Optimize computing the diff between a block report and the namenode state. -- Key: HDFS-2477 URL: https://issues.apache.org/jira/browse/HDFS-2477 Project: Hadoop HDFS Issue Type: Sub-task Components: name-node Affects Versions: 0.23.0 Reporter: Tomasz Nykiel Assignee: Tomasz Nykiel Fix For: 0.24.0, 0.23.3 Attachments: reportDiff.patch, reportDiff.patch-2, reportDiff.patch-3, reportDiff.patch-4, reportDiff.patch-5 When a block report is processed at the NN, the BlockManager.reportDiff traverses all blocks contained in the report, and for each one block, which is also present in the corresponding datanode descriptor, the block is moved to the head of the list of the blocks in this datanode descriptor. With HDFS-395 the huge majority of the blocks in the report, are also present in the datanode descriptor, which means that almost every block in the report will have to be moved to the head of the list. Currently this operation is performed by DatanodeDescriptor.moveBlockToHead, which removes a block from a list and then inserts it. In this process, we call findDatanode several times (afair 6 times for each moveBlockToHead call). findDatanode is relatively expensive, since it linearly goes through the triplets to locate the given datanode. With this patch, we do some memoization of findDatanode, so we can reclaim 2 findDatanode calls. Our experiments show that this can improve the reportDiff (which is executed under write lock) by around 15%. Currently with HDFS-395, reportDiff is responsible for almost 100% of the block report processing time. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2495) Increase granularity of write operations in ReplicationMonitor thus reducing contention for write lock
[ https://issues.apache.org/jira/browse/HDFS-2495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2495: -- Target Version/s: 0.23.3 Affects Version/s: 0.23.0 Fix Version/s: 0.23.3 Increase granularity of write operations in ReplicationMonitor thus reducing contention for write lock -- Key: HDFS-2495 URL: https://issues.apache.org/jira/browse/HDFS-2495 Project: Hadoop HDFS Issue Type: Sub-task Components: name-node Affects Versions: 0.23.0 Reporter: Tomasz Nykiel Assignee: Tomasz Nykiel Fix For: 0.24.0, 0.23.3 Attachments: replicationMon.patch, replicationMon.patch-1 For processing blocks in ReplicationMonitor (BlockManager.computeReplicationWork), we first obtain a list of blocks to be replicated by calling chooseUnderReplicatedBlocks, and then for each block which was found, we call computeReplicationWorkForBlock. The latter processes a block in three stages, acquiring the writelock twice per call: 1. obtaining block related info (livenodes, srcnode, etc.) under lock 2. choosing target for replication 3. scheduling replication (under lock) We would like to change this behaviour and decrease contention for the write lock, by batching blocks and executing 1,2,3, for sets of blocks, rather than for each one separately. This would decrease the number of writeLock to 2, from 2*numberofblocks. Also, the info level logging can be pushed outside the writelock. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2476) More CPU efficient data structure for under-replicated/over-replicated/invalidate blocks
[ https://issues.apache.org/jira/browse/HDFS-2476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2476: -- Target Version/s: 0.24.0, 0.23.3 (was: 0.24.0) Affects Version/s: 0.23.0 Fix Version/s: 0.23.3 More CPU efficient data structure for under-replicated/over-replicated/invalidate blocks Key: HDFS-2476 URL: https://issues.apache.org/jira/browse/HDFS-2476 Project: Hadoop HDFS Issue Type: Sub-task Components: name-node Affects Versions: 0.23.0 Reporter: Tomasz Nykiel Assignee: Tomasz Nykiel Fix For: 0.24.0, 0.23.3 Attachments: hashStructures.patch, hashStructures.patch-2, hashStructures.patch-3, hashStructures.patch-4, hashStructures.patch-5, hashStructures.patch-6, hashStructures.patch-7, hashStructures.patch-8, hashStructures.patch-9 This patch introduces two hash data structures for storing under-replicated, over-replicated and invalidated blocks. 1. LightWeightHashSet 2. LightWeightLinkedSet Currently in all these cases we are using java.util.TreeSet which adds unnecessary overhead. The main bottlenecks addressed by this patch are: -cluster instability times, when these queues (especially under-replicated) tend to grow quite drastically, -initial cluster startup, when the queues are initialized, after leaving safemode, -block reports, -explicit acks for block addition and deletion 1. The introduced structures are CPU-optimized. 2. They shrink and expand according to current capacity. 3. Add/contains/delete ops are performed in O(1) time (unlike current log n for TreeSet). 4. The sets are equipped with fast access methods for polling a number of elements (get+remove), which are used for handling the queues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1580) Add interface for generic Write Ahead Logging mechanisms
[ https://issues.apache.org/jira/browse/HDFS-1580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-1580: -- Component/s: name-node Target Version/s: 0.23.3 Fix Version/s: 0.23.3 Add interface for generic Write Ahead Logging mechanisms Key: HDFS-1580 URL: https://issues.apache.org/jira/browse/HDFS-1580 Project: Hadoop HDFS Issue Type: Improvement Components: name-node Reporter: Ivan Kelly Assignee: Jitendra Nath Pandey Fix For: HA branch (HDFS-1623), 0.24.0, 0.23.3 Attachments: EditlogInterface.1.pdf, EditlogInterface.2.pdf, EditlogInterface.3.pdf, HDFS-1580+1521.diff, HDFS-1580.diff, HDFS-1580.diff, HDFS-1580.diff, HDFS-1580.diff, HDFS-1580.diff, HDFS-1580.diff, HDFS-1580.diff, HDFS-1580.diff, HDFS-1580.diff, HDFS-1580.diff, HDFS-1580.diff, HDFS-1580.diff, generic_wal_iface.pdf, generic_wal_iface.pdf, generic_wal_iface.pdf, generic_wal_iface.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1765) Block Replication should respect under-replication block priority
[ https://issues.apache.org/jira/browse/HDFS-1765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-1765: -- Target Version/s: 0.24.0, 0.23.3 (was: 0.23.3, 0.24.0) Fix Version/s: 0.23.3 Block Replication should respect under-replication block priority - Key: HDFS-1765 URL: https://issues.apache.org/jira/browse/HDFS-1765 Project: Hadoop HDFS Issue Type: Bug Components: name-node Affects Versions: 0.23.0 Reporter: Hairong Kuang Assignee: Uma Maheswara Rao G Fix For: 0.24.0, 0.23.3 Attachments: HDFS-1765.patch, HDFS-1765.patch, HDFS-1765.patch, HDFS-1765.patch, HDFS-1765.pdf, underReplicatedQueue.pdf Time Spent: 0.5h Remaining Estimate: 0h Currently under-replicated blocks are assigned different priorities depending on how many replicas a block has. However the replication monitor works on blocks in a round-robin fashion. So the newly added high priority blocks won't get replicated until all low-priority blocks are done. One example is that on decommissioning datanode WebUI we often observe that blocks with only decommissioning replicas do not get scheduled to replicate before other blocks, so risking data availability if the node is shutdown for repair before decommission completes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3030) Remove getProtocolVersion and getProtocolSignature from translators
[ https://issues.apache.org/jira/browse/HDFS-3030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-3030: -- Target Version/s: 0.23.3 Affects Version/s: 0.23.3 0.24.0 Status: Patch Available (was: Open) Remove getProtocolVersion and getProtocolSignature from translators --- Key: HDFS-3030 URL: https://issues.apache.org/jira/browse/HDFS-3030 Project: Hadoop HDFS Issue Type: Improvement Affects Versions: 0.24.0, 0.23.3 Reporter: Jitendra Nath Pandey Assignee: Jitendra Nath Pandey Attachments: HDFS-3030.patch This is hdfs part of HADOOP-7994. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3009) DFSClient islocaladdress() can use similar routine in netutils
[ https://issues.apache.org/jira/browse/HDFS-3009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-3009: -- Resolution: Fixed Fix Version/s: 0.24.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed the patch. Thank you Hari. DFSClient islocaladdress() can use similar routine in netutils -- Key: HDFS-3009 URL: https://issues.apache.org/jira/browse/HDFS-3009 Project: Hadoop HDFS Issue Type: Bug Components: hdfs client Affects Versions: 0.23.0, 0.24.0 Reporter: Hari Mankude Assignee: Hari Mankude Priority: Trivial Fix For: 0.24.0 Attachments: HDFS-3009.patch isLocalAddress() in dfsclient can use similar function in netutils -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2966) TestNameNodeMetrics tests can fail under load
[
https://issues.apache.org/jira/browse/HDFS-2966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Suresh Srinivas updated HDFS-2966:
--
Attachment: HDFS-2966.patch
Updated patch post HDFS-3002 changes.
TestNameNodeMetrics tests can fail under load
-
Key: HDFS-2966
URL: https://issues.apache.org/jira/browse/HDFS-2966
Project: Hadoop HDFS
Issue Type: Bug
Components: test
Affects Versions: 0.24.0
Environment: OS/X running intellij IDEA, firefox, winxp in a
virtualbox.
Reporter: Steve Loughran
Assignee: Steve Loughran
Priority: Minor
Fix For: 0.24.0, 0.23.2
Attachments: HDFS-2966.patch, HDFS-2966.patch, HDFS-2966.patch
I've managed to recreate HDFS-540 and HDFS-2434 by the simple technique of
running the HDFS tests on a desktop with out enough memory for all the
programs trying to run. Things got swapped out and the tests failed as the DN
heartbeats didn't come in on time.
the tests both rely on {{waitForDeletion()}} to block the tests until the
delete operation has completed, but all it does is sleep for the same number
of seconds as there are datanodes. This is too brittle -it may work on a
lightly-loaded system, but not on a system under heavy load where it is
taking longer to replicate than expect.
Immediate fix: double, triple, the sleep time?
Better fix: have the thread block until all the DN heartbeats have finished.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3002) TestNameNodeMetrics need not wait for metrics update with new metrics framework
[ https://issues.apache.org/jira/browse/HDFS-3002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-3002: -- Attachment: HDFS-3002.patch TestNameNodeMetrics need not wait for metrics update with new metrics framework --- Key: HDFS-3002 URL: https://issues.apache.org/jira/browse/HDFS-3002 Project: Hadoop HDFS Issue Type: Improvement Components: test Affects Versions: 0.23.0, 0.24.0 Reporter: Suresh Srinivas Assignee: Suresh Srinivas Priority: Trivial Attachments: HDFS-3002.patch With older metrics framework, the namenode metrics was updated by replication thread. This required test having to wait for replication interval. This is no longer necessary with metrics2 framework. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3002) TestNameNodeMetrics need not wait for metrics update with new metrics framework
[ https://issues.apache.org/jira/browse/HDFS-3002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-3002: -- Status: Patch Available (was: Open) TestNameNodeMetrics need not wait for metrics update with new metrics framework --- Key: HDFS-3002 URL: https://issues.apache.org/jira/browse/HDFS-3002 Project: Hadoop HDFS Issue Type: Improvement Components: test Affects Versions: 0.23.0, 0.24.0 Reporter: Suresh Srinivas Assignee: Suresh Srinivas Priority: Trivial Attachments: HDFS-3002.patch With older metrics framework, the namenode metrics was updated by replication thread. This required test having to wait for replication interval. This is no longer necessary with metrics2 framework. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3001) dfsadmin -refreshServiceAcl fails Kerb authentication with valid Kerb ticket, other subcommands succeed
[ https://issues.apache.org/jira/browse/HDFS-3001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-3001: -- Description: With a valid hdfs kerberos ticket, the dfsadmin subcommand '-refreshServiceAcl' still fails on Kerb authentication. Please see the comment for more details. was: With a valid hdfs kerberos ticket, the dfsadmin subcommand '-refreshServiceAcl' still fails on Kerb authentication with the following error: bash-3.2$ /home/share/hadoop/bin/hdfs --config /home/conf/hadoop/ dfsadmin -refreshServiceAcl refreshServiceAcl: User hdfs/USER@DOMAIN (auth:KERBEROS) is not authorized for protocol interface org.apache.hadoop.security.authorize.Refresh AuthorizationPolicyProtocol, expected client Kerberos principal is null However, other dfsadmin commands like '-printTopology', '-refreshNamenodes', '-safemode', '-report', which should use the same privilege level, do not give authentication errors and work successfully: -- kerb ticket -- bash-3.2$ klist -5 Ticket cache: FILE:/tmp/path/kbtickets/hdfs.kerberos.ticket Default principal: hdfs/USER@DOMAIN Valid starting ExpiresService principal 01/18/12 23:59:53 01/19/12 23:59:53 krbtgt/USER@DOMAIN renew until 01/25/12 23:59:53 -- -printTopology subcommand -- bash-3.2$ /home/share/hadoop/bin/hdfs --config /home/conf/hadoop/ dfsadmin -printTopology Rack: /IPADDR1.0 IPADDR2.43:1004 (HOST1.com) IPADDR3.44:1004 (HOST2.com) IPADDRn.60:1004 (HOSTn.com) Rack: /default-rack HOSTr.com -- -refreshNamenodes subcommand -- bash-3.2$ /home/share/hadoop/bin/hdfs --config /home/conf/hadoop/ dfsadmin -fs hdfs://NNHOST:8020 -refreshNamenodes DNHOST:8020 bash-3.2$ echo $? 0 -- -safemode subcommand -- bash-3.2$ /home/share/hadoop/bin/hdfs --config /home/conf/hadoop/ dfsadmin -fs hdfs://NNHOST:8020 -safemode get Safe mode is OFF With a valid hdfs kerberos ticket, the dfsadmin subcommand '-refreshServiceAcl' still fails on Kerb authentication with the following error: bash-3.2$ /home/share/hadoop/bin/hdfs --config /home/conf/hadoop/ dfsadmin -refreshServiceAcl refreshServiceAcl: User hdfs/USER@DOMAIN (auth:KERBEROS) is not authorized for protocol interface org.apache.hadoop.security.authorize.Refresh AuthorizationPolicyProtocol, expected client Kerberos principal is null However, other dfsadmin commands like '-printTopology', '-refreshNamenodes', '-safemode', '-report', which should use the same privilege level, do not give authentication errors and work successfully: - kerb ticket - bash-3.2$ klist -5 Ticket cache: FILE:/tmp/path/kbtickets/hdfs.kerberos.ticket Default principal: hdfs/USER@DOMAIN Valid starting Expires Service principal 01/18/12 23:59:53 01/19/12 23:59:53 krbtgt/USER@DOMAIN renew until 01/25/12 23:59:53 - -printTopology subcommand - bash-3.2$ /home/share/hadoop/bin/hdfs --config /home/conf/hadoop/ dfsadmin -printTopology Rack: /IPADDR1.0 IPADDR2.43:1004 (HOST1.com) IPADDR3.44:1004 (HOST2.com) IPADDRn.60:1004 (HOSTn.com) Rack: /default-rack HOSTr.com - -refreshNamenodes subcommand - bash-3.2$ /home/share/hadoop/bin/hdfs --config /home/conf/hadoop/ dfsadmin -fs hdfs://NNHOST:8020 -refreshNamenodes DNHOST:8020 bash-3.2$ echo $? 0 - -safemode subcommand - bash-3.2$ /home/share/hadoop/bin/hdfs --config /home/conf/hadoop/ dfsadmin -fs hdfs://NNHOST:8020 -safemode get Safe mode is OFF dfsadmin -refreshServiceAcl fails Kerb authentication with valid Kerb ticket, other subcommands succeed --- Key: HDFS-3001 URL: https://issues.apache.org/jira/browse/HDFS-3001 Project: Hadoop HDFS Issue Type: Bug Components: hdfs client Affects Versions: 0.23.1 Reporter: patrick white With a valid hdfs kerberos ticket, the dfsadmin subcommand '-refreshServiceAcl' still fails on Kerb authentication. Please see the comment for more details. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2937) HA: TestDFSHAAdmin needs tests with MiniDFSCluster
[ https://issues.apache.org/jira/browse/HDFS-2937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2937: -- Attachment: HDFS-2937.txt +1 for the patch. I added/modified some comments. HA: TestDFSHAAdmin needs tests with MiniDFSCluster -- Key: HDFS-2937 URL: https://issues.apache.org/jira/browse/HDFS-2937 Project: Hadoop HDFS Issue Type: Sub-task Components: ha, test Affects Versions: HA branch (HDFS-1623) Reporter: Suresh Srinivas Assignee: Brandon Li Attachments: HDFS-2937.HDFS-1623.patch, HDFS-2937.HDFS-1623.patch, HDFS-2937.HDFS-1623.patch, HDFS-2937.txt TestDFSHAAdmin currently works with Mockito base HAServiceProtocol. Tests are needed with real namenodes using MiniDFSCluster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2586) Add protobuf service and implementation for HAServiceProtocol
[ https://issues.apache.org/jira/browse/HDFS-2586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2586: -- Attachment: HDFS-2586.txt Good catch Todd. I tested the old patch with test from HDFS-2937 and it failed as expected. The new patch passes that test. Add protobuf service and implementation for HAServiceProtocol - Key: HDFS-2586 URL: https://issues.apache.org/jira/browse/HDFS-2586 Project: Hadoop HDFS Issue Type: Sub-task Components: ha, name-node Reporter: Suresh Srinivas Assignee: Suresh Srinivas Fix For: HA branch (HDFS-1623) Attachments: HDFS-2586.txt, HDFS-2586.txt, HDFS-2586.txt When the trunk moves to protobuf based RPC, HAServiceProtocol should have equivalent protobuf implementation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2937) HA: TestDFSHAAdmin needs tests with MiniDFSCluster
[ https://issues.apache.org/jira/browse/HDFS-2937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2937: -- Attachment: HDFS-2937.txt New patch addresses comments... HA: TestDFSHAAdmin needs tests with MiniDFSCluster -- Key: HDFS-2937 URL: https://issues.apache.org/jira/browse/HDFS-2937 Project: Hadoop HDFS Issue Type: Sub-task Components: ha, test Affects Versions: HA branch (HDFS-1623) Reporter: Suresh Srinivas Assignee: Brandon Li Attachments: HDFS-2937.HDFS-1623.patch, HDFS-2937.HDFS-1623.patch, HDFS-2937.HDFS-1623.patch, HDFS-2937.txt, HDFS-2937.txt TestDFSHAAdmin currently works with Mockito base HAServiceProtocol. Tests are needed with real namenodes using MiniDFSCluster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2937) HA: TestDFSHAAdmin needs tests with MiniDFSCluster
[ https://issues.apache.org/jira/browse/HDFS-2937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2937: -- Resolution: Fixed Fix Version/s: HA branch (HDFS-1623) Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) I committed the patch. Thank you Brandon. HA: TestDFSHAAdmin needs tests with MiniDFSCluster -- Key: HDFS-2937 URL: https://issues.apache.org/jira/browse/HDFS-2937 Project: Hadoop HDFS Issue Type: Sub-task Components: ha, test Affects Versions: HA branch (HDFS-1623) Reporter: Suresh Srinivas Assignee: Brandon Li Fix For: HA branch (HDFS-1623) Attachments: HDFS-2937.HDFS-1623.patch, HDFS-2937.HDFS-1623.patch, HDFS-2937.HDFS-1623.patch, HDFS-2937.txt, HDFS-2937.txt TestDFSHAAdmin currently works with Mockito base HAServiceProtocol. Tests are needed with real namenodes using MiniDFSCluster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2586) Add protobuf service and implementation for HAServiceProtocol
[ https://issues.apache.org/jira/browse/HDFS-2586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2586: -- Status: Patch Available (was: Open) Add protobuf service and implementation for HAServiceProtocol - Key: HDFS-2586 URL: https://issues.apache.org/jira/browse/HDFS-2586 Project: Hadoop HDFS Issue Type: Sub-task Components: ha, name-node Reporter: Suresh Srinivas Assignee: Suresh Srinivas Fix For: HA branch (HDFS-1623) Attachments: HDFS-2586.txt, HDFS-2586.txt, HDFS-2586.txt When the trunk moves to protobuf based RPC, HAServiceProtocol should have equivalent protobuf implementation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2586) Add protobuf service and implementation for HAServiceProtocol
[ https://issues.apache.org/jira/browse/HDFS-2586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2586: -- Status: Open (was: Patch Available) Add protobuf service and implementation for HAServiceProtocol - Key: HDFS-2586 URL: https://issues.apache.org/jira/browse/HDFS-2586 Project: Hadoop HDFS Issue Type: Sub-task Components: ha, name-node Reporter: Suresh Srinivas Assignee: Suresh Srinivas Fix For: HA branch (HDFS-1623) Attachments: HDFS-2586.txt, HDFS-2586.txt, HDFS-2586.txt When the trunk moves to protobuf based RPC, HAServiceProtocol should have equivalent protobuf implementation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2938) Recursive delete of a large directory makes namenode unresponsive
[ https://issues.apache.org/jira/browse/HDFS-2938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2938: -- Resolution: Fixed Target Version/s: 0.24.0, 0.23.2 (was: 0.23.2, 0.24.0) Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed the patch to trunk and 0.23. Thank you Hari. Recursive delete of a large directory makes namenode unresponsive - Key: HDFS-2938 URL: https://issues.apache.org/jira/browse/HDFS-2938 Project: Hadoop HDFS Issue Type: Bug Components: name-node Affects Versions: 0.22.0 Reporter: Suresh Srinivas Assignee: Hari Mankude Attachments: HDFS-2938.patch, HDFS-2938.patch When deleting a large directory with millions of files, namenode holding FSNamesystem lock will make it unresponsive for other request. In this scenario HDFS-173 added a mechanism to delete blocks in smaller chunks holding the locks. With new read/write lock changes, the mechanism from HDFS-173 is lost. Need to resurrect the mechanism back. Also a good unit test/update to existing unit test is needed to catch future errors with this functionality. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2923) Namenode IPC handler count uses the wrong configuration key
[ https://issues.apache.org/jira/browse/HDFS-2923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2923: -- Target Version/s: 0.23.1 (was: 0.23.2) Namenode IPC handler count uses the wrong configuration key --- Key: HDFS-2923 URL: https://issues.apache.org/jira/browse/HDFS-2923 Project: Hadoop HDFS Issue Type: Bug Components: name-node Affects Versions: 0.23.0, 0.23.1 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Critical Fix For: 0.24.0, 0.23.2 Attachments: hdfs-2923.txt In HDFS-1763, there was a typo introduced which causes the namenode to use dfs.datanode.handler.count to set the number of IPC threads instead of the correct dfs.namenode.handler.count. This results in bad performance under high load, since there are not nearly enough handlers. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2869) Error in Webhdfs documentation for mkdir
[
https://issues.apache.org/jira/browse/HDFS-2869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Suresh Srinivas updated HDFS-2869:
--
Target Version/s: 0.23.1
Error in Webhdfs documentation for mkdir
Key: HDFS-2869
URL: https://issues.apache.org/jira/browse/HDFS-2869
Project: Hadoop HDFS
Issue Type: Bug
Components: documentation
Affects Versions: 0.23.1, 1.0.0
Reporter: Harsh J
Assignee: Harsh J
Priority: Minor
Fix For: 0.24.0, 1.1.0, 0.23.2
Attachments: HDFS-2869-branch-1.patch, HDFS-2869.patch
Reported over the lists by user Stuti Awasthi:
{quote}
I have tried the webhdfs functionality of Hadoop-1.0.0 and it is working fine.
Just a small change is required in the documentation :
Make a Directory declaration in documentation:
curl -i -X PUT http://HOST:PORT/PATH?op=MKDIRS[permission=OCTAL]
Gives following error :
HTTP/1.1 405 HTTP method PUT is not supported by this URL
Content-Length: 0
Server: Jetty(6.1.26)
Correction Required : This works for me
curl -i -X PUT http://host:port/*webhdfs/v1/*PATH?op=MKDIRS
{quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2815) Namenode is not coming out of safemode when we perform ( NN crash + restart ) . Also FSCK report shows blocks missed.
[ https://issues.apache.org/jira/browse/HDFS-2815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2815: -- Target Version/s: 0.24.0, 1.1.0, 0.23.2 (was: 0.23.2, 0.24.0) Fix Version/s: 0.23.2 0.24.0 I committed the patch to 0.24 and 0.23. Thank you Uma. We should fix this for 1.1.0 release. However that is non-trivial since it requires parts of the functionality from HDFS-173. @Uma do you want to take a stab at it? Namenode is not coming out of safemode when we perform ( NN crash + restart ) . Also FSCK report shows blocks missed. -- Key: HDFS-2815 URL: https://issues.apache.org/jira/browse/HDFS-2815 Project: Hadoop HDFS Issue Type: Bug Components: name-node Affects Versions: 0.22.0, 0.24.0, 0.23.1, 1.0.0, 1.1.0 Reporter: Uma Maheswara Rao G Assignee: Uma Maheswara Rao G Priority: Critical Fix For: 0.24.0, 0.23.2 Attachments: HDFS-2815.patch, HDFS-2815.patch When tested the HA(internal) with continuous switch with some 5mins gap, found some *blocks missed* and namenode went into safemode after next switch. After the analysis, i found that this files already deleted by clients. But i don't see any delete commands logs namenode log files. But namenode added that blocks to invalidateSets and DNs deleted the blocks. When restart of the namenode, it went into safemode and expecting some more blocks to come out of safemode. Here the reason could be that, file has been deleted in memory and added into invalidates after this it is trying to sync the edits into editlog file. By that time NN asked DNs to delete that blocks. Now namenode shuts down before persisting to editlogs.( log behind) Due to this reason, we may not get the INFO logs about delete, and when we restart the Namenode (in my scenario it is again switch), Namenode expects this deleted blocks also, as delete request is not persisted into editlog before. I reproduced this scenario with bedug points. *I feel, We should not add the blocks to invalidates before persisting into Editlog*. Note: for switch, we used kill -9 (force kill) I am currently in 0.20.2 version. Same verified in 0.23 as well in normal crash + restart scenario. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2815) Namenode is not coming out of safemode when we perform ( NN crash + restart ) . Also FSCK report shows blocks missed.
[ https://issues.apache.org/jira/browse/HDFS-2815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2815: -- Target Version/s: 0.24.0, 0.23.2 (was: 0.23.2, 0.24.0) Affects Version/s: 1.1.0 Namenode is not coming out of safemode when we perform ( NN crash + restart ) . Also FSCK report shows blocks missed. -- Key: HDFS-2815 URL: https://issues.apache.org/jira/browse/HDFS-2815 Project: Hadoop HDFS Issue Type: Bug Components: name-node Affects Versions: 0.22.0, 0.24.0, 0.23.1, 1.0.0, 1.1.0 Reporter: Uma Maheswara Rao G Assignee: Uma Maheswara Rao G Priority: Critical Attachments: HDFS-2815.patch When tested the HA(internal) with continuous switch with some 5mins gap, found some *blocks missed* and namenode went into safemode after next switch. After the analysis, i found that this files already deleted by clients. But i don't see any delete commands logs namenode log files. But namenode added that blocks to invalidateSets and DNs deleted the blocks. When restart of the namenode, it went into safemode and expecting some more blocks to come out of safemode. Here the reason could be that, file has been deleted in memory and added into invalidates after this it is trying to sync the edits into editlog file. By that time NN asked DNs to delete that blocks. Now namenode shuts down before persisting to editlogs.( log behind) Due to this reason, we may not get the INFO logs about delete, and when we restart the Namenode (in my scenario it is again switch), Namenode expects this deleted blocks also, as delete request is not persisted into editlog before. I reproduced this scenario with bedug points. *I feel, We should not add the blocks to invalidates before persisting into Editlog*. Note: for switch, we used kill -9 (force kill) I am currently in 0.20.2 version. Same verified in 0.23 as well in normal crash + restart scenario. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2938) Recursive delete of a large directory makes namenode unresponsive
[ https://issues.apache.org/jira/browse/HDFS-2938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2938: -- Target Version/s: 0.23.2 Recursive delete of a large directory makes namenode unresponsive - Key: HDFS-2938 URL: https://issues.apache.org/jira/browse/HDFS-2938 Project: Hadoop HDFS Issue Type: Bug Components: name-node Affects Versions: 0.22.0, 0.24.0, 0.23.1 Reporter: Suresh Srinivas Assignee: Suresh Srinivas When deleting a large directory with millions of files, namenode holding FSNamesystem lock will make it unresponsive for other request. In this scenario HDFS-173 added a mechanism to delete blocks in smaller chunks holding the locks. With new read/write lock changes, the mechanism from HDFS-173 is lost. Need to resurrect the mechanism back. Also a good unit test/update to existing unit test is needed to catch future errors with this functionality. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2899) Service protocol change to support multiple storages added in HDFS-2880
[ https://issues.apache.org/jira/browse/HDFS-2899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2899: -- Attachment: HDFS-2899.txt Thanks for the comments Jitendra. Reverted the accidental proto changes. Service protocol change to support multiple storages added in HDFS-2880 --- Key: HDFS-2899 URL: https://issues.apache.org/jira/browse/HDFS-2899 Project: Hadoop HDFS Issue Type: Sub-task Affects Versions: 0.24.0 Reporter: Suresh Srinivas Assignee: Suresh Srinivas Attachments: HDFS-2899.txt, HDFS-2899.txt, HDFS-2899.txt, HDFS-2899.txt This jira makes changes in the service protocol corresponding to protobuf protocol changes made in HDFS-2880. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2899) Service protocol change to support multiple storages added in HDFS-2880
[ https://issues.apache.org/jira/browse/HDFS-2899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2899: -- Resolution: Fixed Fix Version/s: 0.24.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) I committed the patch. Service protocol change to support multiple storages added in HDFS-2880 --- Key: HDFS-2899 URL: https://issues.apache.org/jira/browse/HDFS-2899 Project: Hadoop HDFS Issue Type: Sub-task Affects Versions: 0.24.0 Reporter: Suresh Srinivas Assignee: Suresh Srinivas Fix For: 0.24.0 Attachments: HDFS-2899.txt, HDFS-2899.txt, HDFS-2899.txt, HDFS-2899.txt This jira makes changes in the service protocol corresponding to protobuf protocol changes made in HDFS-2880. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2907) Make FSDataset in Datanode Pluggable
[ https://issues.apache.org/jira/browse/HDFS-2907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2907: -- Target Version/s: 0.24.0 Fix Version/s: (was: 0.24.0) Make FSDataset in Datanode Pluggable Key: HDFS-2907 URL: https://issues.apache.org/jira/browse/HDFS-2907 Project: Hadoop HDFS Issue Type: Improvement Reporter: Sanjay Radia Assignee: Sanjay Radia Priority: Minor -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2594) webhdfs HTTP API should implement getDelegationTokens() instead getDelegationToken()
[ https://issues.apache.org/jira/browse/HDFS-2594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2594: -- Fix Version/s: 0.24.0 webhdfs HTTP API should implement getDelegationTokens() instead getDelegationToken() Key: HDFS-2594 URL: https://issues.apache.org/jira/browse/HDFS-2594 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 0.24.0, 0.23.1 Reporter: Alejandro Abdelnur Assignee: Tsz Wo (Nicholas), SZE Priority: Critical Fix For: 0.24.0, 0.23.1 Attachments: h2594_2030.patch, h2594_2030_no_apt.patch, h2594_20111201.patch The current API returns a single delegation token, that method from the FileSystem API is deprecated in favor of the one that returns a list of tokens. The HTTP API should implement the new/undeprecated signature getDelegationTokens(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2543) HADOOP_PREFIX cannot be overriden
[ https://issues.apache.org/jira/browse/HDFS-2543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2543: -- Fix Version/s: 0.24.0 HADOOP_PREFIX cannot be overriden - Key: HDFS-2543 URL: https://issues.apache.org/jira/browse/HDFS-2543 Project: Hadoop HDFS Issue Type: Bug Components: scripts Affects Versions: 0.23.0 Reporter: Bruno Mahé Assignee: Bruno Mahé Labels: bigtop Fix For: 0.24.0, 0.23.1 Attachments: HDFS-2543.patch hadoop-config.sh forces HADOOP_prefix to a specific value: export HADOOP_PREFIX=`dirname $this`/.. It would be nice to make this overridable. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2570) Add descriptions for dfs.*.https.address in hdfs-default.xml
[ https://issues.apache.org/jira/browse/HDFS-2570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2570: -- Fix Version/s: 0.24.0 Add descriptions for dfs.*.https.address in hdfs-default.xml Key: HDFS-2570 URL: https://issues.apache.org/jira/browse/HDFS-2570 Project: Hadoop HDFS Issue Type: Improvement Components: documentation Affects Versions: 0.23.0 Reporter: Eli Collins Assignee: Eli Collins Priority: Trivial Fix For: 0.24.0, 0.23.1 Attachments: hdfs-2570-1.patch, hdfs-2570-2.patch Let's add descriptions for dfs.*.https.address in hdfs-default.xml. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2654) Make BlockReaderLocal not extend RemoteBlockReader2
[ https://issues.apache.org/jira/browse/HDFS-2654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2654: -- Target Version/s: 0.23.1, 1.1.0 (was: 1.1.0, 0.23.1) Fix Version/s: 0.23.1 0.24.0 Make BlockReaderLocal not extend RemoteBlockReader2 --- Key: HDFS-2654 URL: https://issues.apache.org/jira/browse/HDFS-2654 Project: Hadoop HDFS Issue Type: Improvement Components: data-node Affects Versions: 0.23.1, 1.0.0 Reporter: Eli Collins Assignee: Eli Collins Fix For: 0.24.0, 0.23.1 Attachments: hdfs-2654-1.patch, hdfs-2654-2.patch, hdfs-2654-2.patch, hdfs-2654-2.patch, hdfs-2654-3.patch, hdfs-2654-b1-1.patch, hdfs-2654-b1-2.patch, hdfs-2654-b1-3.patch, hdfs-2654-b1-4-fix.patch, hdfs-2654-b1-4.patch The BlockReaderLocal code paths are easier to understand (especially true on branch-1 where BlockReaderLocal inherits code from BlockerReader and FSInputChecker) if the local and remote block reader implementations are independent, and they're not really sharing much code anyway. If for some reason they start to share significant code we can make the BlockReader interface an abstract class. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2572) Unnecessary double-check in DN#getHostName
[ https://issues.apache.org/jira/browse/HDFS-2572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2572: -- Fix Version/s: 0.23.1 Unnecessary double-check in DN#getHostName -- Key: HDFS-2572 URL: https://issues.apache.org/jira/browse/HDFS-2572 Project: Hadoop HDFS Issue Type: Improvement Components: data-node Affects Versions: 0.24.0 Reporter: Harsh J Assignee: Harsh J Priority: Trivial Fix For: 0.24.0, 0.23.1 Attachments: HDFS-2572.patch, HDFS-2572.patch We do a double config.get unnecessarily inside DN#getHostName(...). Can be removed by this patch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2788) HdfsServerConstants#DN_KEEPALIVE_TIMEOUT is dead code
[ https://issues.apache.org/jira/browse/HDFS-2788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2788: -- Fix Version/s: 0.23.1 HdfsServerConstants#DN_KEEPALIVE_TIMEOUT is dead code - Key: HDFS-2788 URL: https://issues.apache.org/jira/browse/HDFS-2788 Project: Hadoop HDFS Issue Type: Improvement Components: data-node Affects Versions: 0.22.0, 0.23.0 Reporter: Eli Collins Assignee: Eli Collins Fix For: 0.23.1 Attachments: hdfs-2788.txt HDFS-941 introduced HdfsServerConstants#DN_KEEPALIVE_TIMEOUT but its never used. Perhaps was renamed to DFSConfigKeys#DFS_DATANODE_SOCKET_REUSE_KEEPALIVE_DEFAULT while the patch was written and the old one wasn't deleted. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2614) hadoop dist tarball is missing hdfs headers
[ https://issues.apache.org/jira/browse/HDFS-2614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2614: -- Affects Version/s: (was: 0.24.0) hadoop dist tarball is missing hdfs headers --- Key: HDFS-2614 URL: https://issues.apache.org/jira/browse/HDFS-2614 Project: Hadoop HDFS Issue Type: Bug Components: build Affects Versions: 0.23.1 Reporter: Bruno Mahé Assignee: Alejandro Abdelnur Labels: bigtop Fix For: 0.23.1 Attachments: HDFS-2614.patch It would be nice to provide hdfs header so one could easily write programs to be linked against that library and access HDFS -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2895) Remove Writable wire protocol related code that is no longer necessary
[ https://issues.apache.org/jira/browse/HDFS-2895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2895: -- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) I committed the patch to trunk. Thanks for the review Sanjay. Remove Writable wire protocol related code that is no longer necessary -- Key: HDFS-2895 URL: https://issues.apache.org/jira/browse/HDFS-2895 Project: Hadoop HDFS Issue Type: Improvement Components: data-node, name-node Affects Versions: 0.24.0 Reporter: Suresh Srinivas Assignee: Suresh Srinivas Attachments: HDFS-2895.txt With recent changes to move wire protocol to protocol buffers, the existing writable, related translators are no longer necessary. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2895) Remove Writable wire protocol related code that is no longer necessary
[ https://issues.apache.org/jira/browse/HDFS-2895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2895: -- Fix Version/s: 0.24.0 Remove Writable wire protocol related code that is no longer necessary -- Key: HDFS-2895 URL: https://issues.apache.org/jira/browse/HDFS-2895 Project: Hadoop HDFS Issue Type: Improvement Components: data-node, name-node Affects Versions: 0.24.0 Reporter: Suresh Srinivas Assignee: Suresh Srinivas Fix For: 0.24.0 Attachments: HDFS-2895.txt With recent changes to move wire protocol to protocol buffers, the existing writable, related translators are no longer necessary. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2899) Service protocol change to support multiple storages added in HDFS-2880
[ https://issues.apache.org/jira/browse/HDFS-2899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2899: -- Attachment: HDFS-2899.txt Service protocol change to support multiple storages added in HDFS-2880 --- Key: HDFS-2899 URL: https://issues.apache.org/jira/browse/HDFS-2899 Project: Hadoop HDFS Issue Type: Sub-task Affects Versions: 0.24.0 Reporter: Suresh Srinivas Assignee: Suresh Srinivas Attachments: HDFS-2899.txt This jira makes changes in the service protocol corresponding to protobuf protocol changes made in HDFS-2880. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2899) Service protocol change to support multiple storages added in HDFS-2880
[ https://issues.apache.org/jira/browse/HDFS-2899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2899: -- Attachment: HDFS-2899.txt patch with missing files added. Service protocol change to support multiple storages added in HDFS-2880 --- Key: HDFS-2899 URL: https://issues.apache.org/jira/browse/HDFS-2899 Project: Hadoop HDFS Issue Type: Sub-task Affects Versions: 0.24.0 Reporter: Suresh Srinivas Assignee: Suresh Srinivas Attachments: HDFS-2899.txt, HDFS-2899.txt This jira makes changes in the service protocol corresponding to protobuf protocol changes made in HDFS-2880. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2899) Service protocol change to support multiple storages added in HDFS-2880
[ https://issues.apache.org/jira/browse/HDFS-2899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2899: -- Attachment: HDFS-2899.txt Patch that addresses the failure flagged by Jenkins. Service protocol change to support multiple storages added in HDFS-2880 --- Key: HDFS-2899 URL: https://issues.apache.org/jira/browse/HDFS-2899 Project: Hadoop HDFS Issue Type: Sub-task Affects Versions: 0.24.0 Reporter: Suresh Srinivas Assignee: Suresh Srinivas Attachments: HDFS-2899.txt, HDFS-2899.txt, HDFS-2899.txt This jira makes changes in the service protocol corresponding to protobuf protocol changes made in HDFS-2880. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2586) Add protobuf service and implementation for HAServiceProtocol
[ https://issues.apache.org/jira/browse/HDFS-2586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2586: -- Attachment: HDFS-2586.txt Attached patch implements protobuf implementation for HAServiceProtocol Add protobuf service and implementation for HAServiceProtocol - Key: HDFS-2586 URL: https://issues.apache.org/jira/browse/HDFS-2586 Project: Hadoop HDFS Issue Type: Sub-task Components: ha, name-node Reporter: Suresh Srinivas Assignee: Suresh Srinivas Fix For: HA branch (HDFS-1623) Attachments: HDFS-2586.txt When the trunk moves to protobuf based RPC, HAServiceProtocol should have equivalent protobuf implementation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2586) Add protobuf service and implementation for HAServiceProtocol
[ https://issues.apache.org/jira/browse/HDFS-2586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2586: -- Attachment: HDFS-2586.txt Thanks Jitendra for catching the unnecessary changes. Updated patch removes thems. Aarong I manually ran the tests that use HAServiceProtocol, TestFailOverController and TestHAAdmin. Will test it manually on a real cluster, when I get time. Add protobuf service and implementation for HAServiceProtocol - Key: HDFS-2586 URL: https://issues.apache.org/jira/browse/HDFS-2586 Project: Hadoop HDFS Issue Type: Sub-task Components: ha, name-node Reporter: Suresh Srinivas Assignee: Suresh Srinivas Fix For: HA branch (HDFS-1623) Attachments: HDFS-2586.txt, HDFS-2586.txt When the trunk moves to protobuf based RPC, HAServiceProtocol should have equivalent protobuf implementation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2895) Remove Writable wire protocol related code that is no longer necessary
[ https://issues.apache.org/jira/browse/HDFS-2895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2895: -- Attachment: HDFS-2895.txt Remove Writable wire protocol related code that is no longer necessary -- Key: HDFS-2895 URL: https://issues.apache.org/jira/browse/HDFS-2895 Project: Hadoop HDFS Issue Type: Improvement Components: data-node, name-node Affects Versions: 0.24.0 Reporter: Suresh Srinivas Assignee: Suresh Srinivas Attachments: HDFS-2895.txt With recent changes to move wire protocol to protocol buffers, the existing writable, related translators are no longer necessary. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2895) Remove Writable wire protocol related code that is no longer necessary
[ https://issues.apache.org/jira/browse/HDFS-2895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2895: -- Status: Patch Available (was: Open) Remove Writable wire protocol related code that is no longer necessary -- Key: HDFS-2895 URL: https://issues.apache.org/jira/browse/HDFS-2895 Project: Hadoop HDFS Issue Type: Improvement Components: data-node, name-node Affects Versions: 0.24.0 Reporter: Suresh Srinivas Assignee: Suresh Srinivas Attachments: HDFS-2895.txt With recent changes to move wire protocol to protocol buffers, the existing writable, related translators are no longer necessary. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2880) Protocol buffer changes to add support multiple storages
[ https://issues.apache.org/jira/browse/HDFS-2880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2880: -- Attachment: HDFS-2880.txt Thanks for the review Jitendra. New patch addresses all the comments. Protocol buffer changes to add support multiple storages Key: HDFS-2880 URL: https://issues.apache.org/jira/browse/HDFS-2880 Project: Hadoop HDFS Issue Type: Sub-task Components: data-node, name-node Reporter: Suresh Srinivas Assignee: Suresh Srinivas Attachments: HDFS-2880.txt, HDFS-2880.txt, HDFS-2880.txt, HDFS-2880.txt This jira makes protocol buffers protocol changes required for HDFS-2832. HDFS-2832 will have the following subtasks: protocol buffer changes, protocol changes, followed by jiras for implementation changes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2880) Protocol buffer changes to add support multiple storages
[ https://issues.apache.org/jira/browse/HDFS-2880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2880: -- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) This is only protocol layer change. The existing tests cover this. I committed the patch. Protocol buffer changes to add support multiple storages Key: HDFS-2880 URL: https://issues.apache.org/jira/browse/HDFS-2880 Project: Hadoop HDFS Issue Type: Sub-task Components: data-node, name-node Reporter: Suresh Srinivas Assignee: Suresh Srinivas Attachments: HDFS-2880.txt, HDFS-2880.txt, HDFS-2880.txt, HDFS-2880.txt This jira makes protocol buffers protocol changes required for HDFS-2832. HDFS-2832 will have the following subtasks: protocol buffer changes, protocol changes, followed by jiras for implementation changes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2880) Protocol buffer changes to add support multiple storages
[ https://issues.apache.org/jira/browse/HDFS-2880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2880: -- Attachment: HDFS-2880.txt Patch makes the following changes protocol buffers: # Registration request can send list of storages each with state and stroage ID. # Heartbeat request is changed to send utilization per storage. # Block report request is changed to send report per storage. # Block received and deleted list is sent per storage. The change is limited to protocol buffers and translators alone. The rest of the implementation remains the same with only one storage per datanode. Protocol buffer changes to add support multiple storages Key: HDFS-2880 URL: https://issues.apache.org/jira/browse/HDFS-2880 Project: Hadoop HDFS Issue Type: Sub-task Components: data-node, name-node Reporter: Suresh Srinivas Assignee: Suresh Srinivas Attachments: HDFS-2880.txt This jira makes protocol buffers protocol changes required for HDFS-2832. HDFS-2832 will have the following subtasks: protocol buffer changes, protocol changes, followed by jiras for implementation changes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2880) Protocol buffer changes to add support multiple storages
[ https://issues.apache.org/jira/browse/HDFS-2880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2880: -- Status: Patch Available (was: Open) Protocol buffer changes to add support multiple storages Key: HDFS-2880 URL: https://issues.apache.org/jira/browse/HDFS-2880 Project: Hadoop HDFS Issue Type: Sub-task Components: data-node, name-node Reporter: Suresh Srinivas Assignee: Suresh Srinivas Attachments: HDFS-2880.txt This jira makes protocol buffers protocol changes required for HDFS-2832. HDFS-2832 will have the following subtasks: protocol buffer changes, protocol changes, followed by jiras for implementation changes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2880) Protocol buffer changes to add support multiple storages
[ https://issues.apache.org/jira/browse/HDFS-2880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2880: -- Attachment: HDFS-2880.txt HDFS-2880.txt Updated patch to fix test issue. Protocol buffer changes to add support multiple storages Key: HDFS-2880 URL: https://issues.apache.org/jira/browse/HDFS-2880 Project: Hadoop HDFS Issue Type: Sub-task Components: data-node, name-node Reporter: Suresh Srinivas Assignee: Suresh Srinivas Attachments: HDFS-2880.txt, HDFS-2880.txt, HDFS-2880.txt This jira makes protocol buffers protocol changes required for HDFS-2832. HDFS-2832 will have the following subtasks: protocol buffer changes, protocol changes, followed by jiras for implementation changes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2835) Fix org.apache.hadoop.hdfs.tools.GetConf$Command Findbug issue
[ https://issues.apache.org/jira/browse/HDFS-2835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2835: -- Attachment: HDFS-2835.txt New patch with javadoc warnings fixes Fix org.apache.hadoop.hdfs.tools.GetConf$Command Findbug issue -- Key: HDFS-2835 URL: https://issues.apache.org/jira/browse/HDFS-2835 Project: Hadoop HDFS Issue Type: Bug Components: tools Affects Versions: 0.24.0 Reporter: Robert Joseph Evans Assignee: Robert Joseph Evans Attachments: HDFS-2835.txt, HDFS-2835.txt, HDFS-2835.txt https://builds.apache.org/job/PreCommit-HDFS-Build/1804//artifact/trunk/hadoop-hdfs-project/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html shows a findbugs warning. It is unrelated to the patch being tested, and has shown up on a few other JIRAS as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2857) Cleanup BlockInfo class
[ https://issues.apache.org/jira/browse/HDFS-2857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2857: -- Issue Type: Improvement (was: Bug) Cleanup BlockInfo class --- Key: HDFS-2857 URL: https://issues.apache.org/jira/browse/HDFS-2857 Project: Hadoop HDFS Issue Type: Improvement Components: name-node Affects Versions: 0.24.0 Reporter: Suresh Srinivas Assignee: Suresh Srinivas Fix For: 0.24.0 Attachments: HDFS-2857.txt Following are some of the cleanup required: # Remove unnecessary methods # Add interface annotation # Make some of the method private -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2857) Cleanup BlockInfo class
[ https://issues.apache.org/jira/browse/HDFS-2857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2857: -- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) This patch cleaned up BlockInfo class (see description). I did not add new tests. Committed the patch. Cleanup BlockInfo class --- Key: HDFS-2857 URL: https://issues.apache.org/jira/browse/HDFS-2857 Project: Hadoop HDFS Issue Type: Improvement Components: name-node Affects Versions: 0.24.0 Reporter: Suresh Srinivas Assignee: Suresh Srinivas Fix For: 0.24.0 Attachments: HDFS-2857.txt Following are some of the cleanup required: # Remove unnecessary methods # Add interface annotation # Make some of the method private -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2857) Cleanup BlockInfo class
[ https://issues.apache.org/jira/browse/HDFS-2857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2857: -- Attachment: HDFS-2857.23.txt 0.23 version of the patch, given merge from trunk cannot happen cleanly. Cleanup BlockInfo class --- Key: HDFS-2857 URL: https://issues.apache.org/jira/browse/HDFS-2857 Project: Hadoop HDFS Issue Type: Improvement Components: name-node Affects Versions: 0.24.0 Reporter: Suresh Srinivas Assignee: Suresh Srinivas Fix For: 0.24.0 Attachments: HDFS-2857.23.txt, HDFS-2857.txt Following are some of the cleanup required: # Remove unnecessary methods # Add interface annotation # Make some of the method private -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
