[jira] [Assigned] (HDFS-12918) EC Policy defaults incorrectly to enabled in protobufs

2017-12-12 Thread Manoj Govindassamy (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Govindassamy reassigned HDFS-12918: - Assignee: Manoj Govindassamy I can take a look at this if you haven't already

[jira] [Created] (HDFS-12918) EC Policy defaults incorrectly to enabled in protobufs

2017-12-12 Thread Zach Amsden (JIRA)
Zach Amsden created HDFS-12918: -- Summary: EC Policy defaults incorrectly to enabled in protobufs Key: HDFS-12918 URL: https://issues.apache.org/jira/browse/HDFS-12918 Project: Hadoop HDFS Issue

[jira] [Updated] (HDFS-12881) Output streams closed with IOUtils suppressing write errors

2017-12-12 Thread Ajay Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Kumar updated HDFS-12881: -- Status: Open (was: Patch Available) > Output streams closed with IOUtils suppressing write errors >

[jira] [Commented] (HDFS-12751) Ozone: SCM: update container allocated size to container db for all the open containers in ContainerStateManager#close

2017-12-12 Thread Chen Liang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288193#comment-16288193 ] Chen Liang commented on HDFS-12751: --- Thanks [~nandakumar131] for the clarification! I totally missed

[jira] [Updated] (HDFS-12000) Ozone: Container : Add key versioning support-1

2017-12-12 Thread Chen Liang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chen Liang updated HDFS-12000: -- Attachment: HDFS-12000-HDFS-7240.011.patch Fix the checkstyle warning. The javadoc and the asf license

[jira] [Commented] (HDFS-12910) Secure Datanode Starter should log the port when it

2017-12-12 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288198#comment-16288198 ] genericqa commented on HDFS-12910: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-12000) Ozone: Container : Add key versioning support-1

2017-12-12 Thread Xiaoyu Yao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288265#comment-16288265 ] Xiaoyu Yao commented on HDFS-12000: --- Thanks [~vagarychen] for the update. Can you take a look at the

[jira] [Commented] (HDFS-12881) Output streams closed with IOUtils suppressing write errors

2017-12-12 Thread Jason Lowe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288320#comment-16288320 ] Jason Lowe commented on HDFS-12881: --- Thanks for updating the patch! The patch looks much better, but it

[jira] [Comment Edited] (HDFS-12881) Output streams closed with IOUtils suppressing write errors

2017-12-12 Thread Jason Lowe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288320#comment-16288320 ] Jason Lowe edited comment on HDFS-12881 at 12/12/17 9:58 PM: - Thanks for

[jira] [Resolved] (HDFS-4411) Asserts are disabled in unit tests

2017-12-12 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee resolved HDFS-4411. -- Resolution: Invalid > Asserts are disabled in unit tests > -- > >

[jira] [Commented] (HDFS-12918) EC Policy defaults incorrectly to enabled in protobufs

2017-12-12 Thread Manoj Govindassamy (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288378#comment-16288378 ] Manoj Govindassamy commented on HDFS-12918: --- [~zamsden], There is an addendum patch HDFS-12682

[jira] [Commented] (HDFS-12626) Ozone : delete open key entries that will no longer be closed

2017-12-12 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288380#comment-16288380 ] genericqa commented on HDFS-12626: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (HDFS-5926) Documentation should clarify dfs.datanode.du.reserved impact from reserved disk capacity

2017-12-12 Thread Manoj Govindassamy (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Govindassamy updated HDFS-5926: - Summary: Documentation should clarify dfs.datanode.du.reserved impact from reserved disk

[jira] [Created] (HDFS-12919) RBF: MR sets erasure coding by default

2017-12-12 Thread JIRA
Íñigo Goiri created HDFS-12919: -- Summary: RBF: MR sets erasure coding by default Key: HDFS-12919 URL: https://issues.apache.org/jira/browse/HDFS-12919 Project: Hadoop HDFS Issue Type: Sub-task

[jira] [Commented] (HDFS-12919) RBF: MR sets erasure coding by default

2017-12-12 Thread JIRA
[ https://issues.apache.org/jira/browse/HDFS-12919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288404#comment-16288404 ] Íñigo Goiri commented on HDFS-12919: We have a few options: # Make the Router log the error and not

[jira] [Commented] (HDFS-12919) RBF: MR sets erasure coding by default

2017-12-12 Thread JIRA
[ https://issues.apache.org/jira/browse/HDFS-12919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288406#comment-16288406 ] Íñigo Goiri commented on HDFS-12919: [~pbacsko] and [~rkanter], you guys were involved in

[jira] [Commented] (HDFS-12919) RBF: MR sets erasure coding by default

2017-12-12 Thread Robert Kanter (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288411#comment-16288411 ] Robert Kanter commented on HDFS-12919: -- I don't know enough about the {{RouterRpcServer}}, but

[jira] [Updated] (HDFS-12751) Ozone: SCM: update container allocated size to container db for all the open containers in ContainerStateManager#close

2017-12-12 Thread Chen Liang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chen Liang updated HDFS-12751: -- Attachment: HDFS-12751-HDFS-7240.001.patch Post v001 patch. > Ozone: SCM: update container allocated

[jira] [Updated] (HDFS-12751) Ozone: SCM: update container allocated size to container db for all the open containers in ContainerStateManager#close

2017-12-12 Thread Chen Liang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chen Liang updated HDFS-12751: -- Status: Patch Available (was: In Progress) > Ozone: SCM: update container allocated size to container

[jira] [Updated] (HDFS-12919) RBF: MR sets erasure coding by default

2017-12-12 Thread JIRA
[ https://issues.apache.org/jira/browse/HDFS-12919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Íñigo Goiri updated HDFS-12919: --- Labels: RBF (was: ) > RBF: MR sets erasure coding by default >

[jira] [Updated] (HDFS-12919) RBF: MR sets erasure coding by default

2017-12-12 Thread JIRA
[ https://issues.apache.org/jira/browse/HDFS-12919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Íñigo Goiri updated HDFS-12919: --- Affects Version/s: 3.0.0 > RBF: MR sets erasure coding by default >

[jira] [Commented] (HDFS-12919) RBF: MR sets erasure coding by default

2017-12-12 Thread JIRA
[ https://issues.apache.org/jira/browse/HDFS-12919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288418#comment-16288418 ] Íñigo Goiri commented on HDFS-12919: Correct, the {{Router}} implements the same interfaces as the

[jira] [Commented] (HDFS-12000) Ozone: Container : Add key versioning support-1

2017-12-12 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288437#comment-16288437 ] genericqa commented on HDFS-12000: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-12626) Ozone : delete open key entries that will no longer be closed

2017-12-12 Thread Xiaoyu Yao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288466#comment-16288466 ] Xiaoyu Yao commented on HDFS-12626: --- +1 for the latest patch. I will commit it shortly. > Ozone :

[jira] [Updated] (HDFS-12818) Support multiple storages in DataNodeCluster / SimulatedFSDataset

2017-12-12 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-12818: --- Attachment: HDFS-12818.007.patch Argh, {{TestBalancerWithMultipleNameNodes}} snuck by me because it

[jira] [Updated] (HDFS-12919) RBF: MR sets erasure coding by default

2017-12-12 Thread JIRA
[ https://issues.apache.org/jira/browse/HDFS-12919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Íñigo Goiri updated HDFS-12919: --- Attachment: HDFS-12919.000.patch > RBF: MR sets erasure coding by default >

[jira] [Updated] (HDFS-12919) RBF: MR sets erasure coding by default

2017-12-12 Thread JIRA
[ https://issues.apache.org/jira/browse/HDFS-12919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Íñigo Goiri updated HDFS-12919: --- Status: Patch Available (was: Open) > RBF: MR sets erasure coding by default >

[jira] [Commented] (HDFS-12919) RBF: MR sets erasure coding by default

2017-12-12 Thread JIRA
[ https://issues.apache.org/jira/browse/HDFS-12919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288479#comment-16288479 ] Íñigo Goiri commented on HDFS-12919: I added a very quick implementation of the EC methods. I don't

[jira] [Updated] (HDFS-12919) RBF: support erasure coding methods in RouterRpcServer

2017-12-12 Thread JIRA
[ https://issues.apache.org/jira/browse/HDFS-12919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Íñigo Goiri updated HDFS-12919: --- Summary: RBF: support erasure coding methods in RouterRpcServer (was: RBF: MR sets erasure coding by

[jira] [Commented] (HDFS-12918) EC Policy defaults incorrectly to enabled in protobufs

2017-12-12 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288485#comment-16288485 ] Xiao Chen commented on HDFS-12918: -- Thanks Zach for reporting this and Manoj for investigating. As Manoj

[jira] [Updated] (HDFS-12626) Ozone : delete open key entries that will no longer be closed

2017-12-12 Thread Xiaoyu Yao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoyu Yao updated HDFS-12626: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: HDFS-7240 Status:

[jira] [Commented] (HDFS-12000) Ozone: Container : Add key versioning support-1

2017-12-12 Thread Chen Liang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288500#comment-16288500 ] Chen Liang commented on HDFS-12000: --- Thansk [~xyao] for checking the tests! I ran locally the tests you

[jira] [Commented] (HDFS-12918) EC Policy defaults incorrectly to enabled in protobufs

2017-12-12 Thread Manoj Govindassamy (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288521#comment-16288521 ] Manoj Govindassamy commented on HDFS-12918: --- A new check added in the convert seems to be not

[jira] [Updated] (HDFS-12907) Allow read-only access to reserved raw for non-superusers

2017-12-12 Thread Rushabh S Shah (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rushabh S Shah updated HDFS-12907: -- Attachment: HDFS-12907.003.patch Attaching a new patch. Following comments are addressed. 1.

[jira] [Commented] (HDFS-12918) EC Policy defaults incorrectly to enabled in protobufs

2017-12-12 Thread Zach Amsden (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288537#comment-16288537 ] Zach Amsden commented on HDFS-12918: Maybe that is the real bug then. I got this exception when

[jira] [Updated] (HDFS-12881) Output streams closed with IOUtils suppressing write errors

2017-12-12 Thread Ajay Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Kumar updated HDFS-12881: -- Attachment: HDFS-12881.004.patch > Output streams closed with IOUtils suppressing write errors >

[jira] [Commented] (HDFS-12895) RBF: Add ACL support for mount table

2017-12-12 Thread Anbang Hu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288550#comment-16288550 ] Anbang Hu commented on HDFS-12895: -- Thanks Yiqun's patch. Per Inigo's request, a few comments are listed:

[jira] [Comment Edited] (HDFS-12895) RBF: Add ACL support for mount table

2017-12-12 Thread Anbang Hu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288550#comment-16288550 ] Anbang Hu edited comment on HDFS-12895 at 12/13/17 12:57 AM: - Thanks Yiqun's

[jira] [Comment Edited] (HDFS-12895) RBF: Add ACL support for mount table

2017-12-12 Thread Anbang Hu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288550#comment-16288550 ] Anbang Hu edited comment on HDFS-12895 at 12/13/17 12:56 AM: - Thanks Yiqun's

[jira] [Created] (HDFS-12920) HDFS default value change (with adding time unit) breaks old version MR tarball work with new version (3.0) of hadoop

2017-12-12 Thread Junping Du (JIRA)
Junping Du created HDFS-12920: - Summary: HDFS default value change (with adding time unit) breaks old version MR tarball work with new version (3.0) of hadoop Key: HDFS-12920 URL:

[jira] [Updated] (HDFS-12920) HDFS default value change (with adding time unit) breaks old version MR tarball work with new version (3.0) of hadoop

2017-12-12 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-12920: -- Description: After HADOOP-15059 get resolved. I tried to deploy 2.9.0 tar ball with 3.0.0 RC1, and run

[jira] [Updated] (HDFS-12920) HDFS default value change (with adding time unit) breaks old version MR tarball work with new version (3.0) of hadoop

2017-12-12 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-12920: -- Description: After HADOOP-15059 get resolved. I tried to deploy 2.9.0 tar ball with 3.0.0 RC1, and run

[jira] [Commented] (HDFS-12920) HDFS default value change (with adding time unit) breaks old version MR tarball work with new version (3.0) of hadoop

2017-12-12 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288568#comment-16288568 ] Junping Du commented on HDFS-12920: --- CC [~andrew.wang], [~linyiqun], [~chris.douglas]. > HDFS default

[jira] [Updated] (HDFS-12773) RBF: Improve State Store FS implementation

2017-12-12 Thread JIRA
[ https://issues.apache.org/jira/browse/HDFS-12773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Íñigo Goiri updated HDFS-12773: --- Status: Patch Available (was: Open) > RBF: Improve State Store FS implementation >

[jira] [Updated] (HDFS-12773) RBF: Improve State Store FS implementation

2017-12-12 Thread JIRA
[ https://issues.apache.org/jira/browse/HDFS-12773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Íñigo Goiri updated HDFS-12773: --- Status: In Progress (was: Patch Available) > RBF: Improve State Store FS implementation >

[jira] [Work stopped] (HDFS-12773) RBF: Improve State Store FS implementation

2017-12-12 Thread JIRA
[ https://issues.apache.org/jira/browse/HDFS-12773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-12773 stopped by Íñigo Goiri. -- > RBF: Improve State Store FS implementation >

[jira] [Commented] (HDFS-12920) HDFS default value change (with adding time unit) breaks old version MR tarball work with new version (3.0) of hadoop

2017-12-12 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288574#comment-16288574 ] Andrew Wang commented on HDFS-12920: Particularly since there's a workaround, let's bump this to

[jira] [Updated] (HDFS-12920) HDFS default value change (with adding time unit) breaks old version MR tarball work with new version (3.0) of hadoop

2017-12-12 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-12920: --- Target Version/s: 3.0.1 (was: 3.0.0) > HDFS default value change (with adding time unit) breaks old

[jira] [Commented] (HDFS-12881) Output streams closed with IOUtils suppressing write errors

2017-12-12 Thread Ajay Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288576#comment-16288576 ] Ajay Kumar commented on HDFS-12881: --- [~jlowe], removed yarn and hadoop-common related changes in patch

[jira] [Commented] (HDFS-10453) ReplicationMonitor thread could stuck for long time due to the race between replication and delete of same file in a large cluster.

2017-12-12 Thread Xiang Li (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288578#comment-16288578 ] Xiang Li commented on HDFS-10453: - Agree. Thanks xiaoqiao! > ReplicationMonitor thread could stuck for

[jira] [Created] (HDFS-12921) DFS.setReplication should throw IOE on EC files

2017-12-12 Thread Lei (Eddy) Xu (JIRA)
Lei (Eddy) Xu created HDFS-12921: Summary: DFS.setReplication should throw IOE on EC files Key: HDFS-12921 URL: https://issues.apache.org/jira/browse/HDFS-12921 Project: Hadoop HDFS Issue

[jira] [Updated] (HDFS-12921) DFS.setReplication should throw exception on EC files

2017-12-12 Thread Lei (Eddy) Xu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lei (Eddy) Xu updated HDFS-12921: - Summary: DFS.setReplication should throw exception on EC files (was: DFS.setReplication should

[jira] [Updated] (HDFS-12918) NameNode fails to start after upgrade - Missing state in ECPolicy Proto

2017-12-12 Thread Manoj Govindassamy (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Govindassamy updated HDFS-12918: -- Summary: NameNode fails to start after upgrade - Missing state in ECPolicy Proto

[jira] [Updated] (HDFS-12918) NameNode fails to start after upgrade - Missing state in ECPolicy Proto

2017-12-12 Thread Manoj Govindassamy (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Govindassamy updated HDFS-12918: -- Affects Version/s: 3.0.0-beta1 Component/s: hdfs > NameNode fails to start

[jira] [Resolved] (HDFS-12918) NameNode fails to start after upgrade - Missing state in ECPolicy Proto

2017-12-12 Thread Manoj Govindassamy (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Govindassamy resolved HDFS-12918. --- Resolution: Won't Fix We have an upgrade incompatible fix landed in C6 at

[jira] [Commented] (HDFS-12895) RBF: Add ACL support for mount table

2017-12-12 Thread JIRA
[ https://issues.apache.org/jira/browse/HDFS-12895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288594#comment-16288594 ] Íñigo Goiri commented on HDFS-12895: [~huanbang1993], the commented code is in our internal PR not in

[jira] [Created] (HDFS-12922) Arrays of length 1 cause 9.2% memory overhead

2017-12-12 Thread Misha Dmitriev (JIRA)
Misha Dmitriev created HDFS-12922: - Summary: Arrays of length 1 cause 9.2% memory overhead Key: HDFS-12922 URL: https://issues.apache.org/jira/browse/HDFS-12922 Project: Hadoop HDFS Issue

[jira] [Updated] (HDFS-12922) Arrays of length 1 cause 9.2% memory overhead

2017-12-12 Thread Misha Dmitriev (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misha Dmitriev updated HDFS-12922: -- Attachment: screenshot-1.png > Arrays of length 1 cause 9.2% memory overhead >

[jira] [Commented] (HDFS-12818) Support multiple storages in DataNodeCluster / SimulatedFSDataset

2017-12-12 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288624#comment-16288624 ] genericqa commented on HDFS-12818: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (HDFS-9806) Allow HDFS block replicas to be provided by an external storage system

2017-12-12 Thread Virajith Jalaparti (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virajith Jalaparti updated HDFS-9806: - Attachment: (was: HDFS-9806.001.patch) > Allow HDFS block replicas to be provided by

[jira] [Updated] (HDFS-9806) Allow HDFS block replicas to be provided by an external storage system

2017-12-12 Thread Virajith Jalaparti (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virajith Jalaparti updated HDFS-9806: - Attachment: HDFS-9806.001.patch > Allow HDFS block replicas to be provided by an external

[jira] [Updated] (HDFS-9806) Allow HDFS block replicas to be provided by an external storage system

2017-12-12 Thread Virajith Jalaparti (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virajith Jalaparti updated HDFS-9806: - Status: Open (was: Patch Available) > Allow HDFS block replicas to be provided by an

[jira] [Updated] (HDFS-9806) Allow HDFS block replicas to be provided by an external storage system

2017-12-12 Thread Virajith Jalaparti (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virajith Jalaparti updated HDFS-9806: - Status: Patch Available (was: Open) > Allow HDFS block replicas to be provided by an

[jira] [Commented] (HDFS-12751) Ozone: SCM: update container allocated size to container db for all the open containers in ContainerStateManager#close

2017-12-12 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288636#comment-16288636 ] genericqa commented on HDFS-12751: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-12920) HDFS default value change (with adding time unit) breaks old version MR tarball work with new version (3.0) of hadoop

2017-12-12 Thread Yiqun Lin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288638#comment-16288638 ] Yiqun Lin commented on HDFS-12920: -- Hi [~djp], thanks for reporting this. bq. A quick workaround is to

[jira] [Commented] (HDFS-12920) HDFS default value change (with adding time unit) breaks old version MR tarball work with new version (3.0) of hadoop

2017-12-12 Thread Chris Douglas (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288641#comment-16288641 ] Chris Douglas commented on HDFS-12920: -- /cc [~arpitagarwal] If we're not going to [change the config

[jira] [Commented] (HDFS-12919) RBF: support erasure coding methods in RouterRpcServer

2017-12-12 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288645#comment-16288645 ] genericqa commented on HDFS-12919: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-12910) Secure Datanode Starter should log the port when it

2017-12-12 Thread Stephen O'Donnell (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16287940#comment-16287940 ] Stephen O'Donnell commented on HDFS-12910: -- One minor change from patch 003 to 004 - I remove the

[jira] [Updated] (HDFS-12626) Ozone : delete open key entries that will no longer be closed

2017-12-12 Thread Chen Liang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chen Liang updated HDFS-12626: -- Attachment: HDFS-12626-HDFS-7240.006.patch Thanks [~xyao] for the review! Post v006 patch to fix the

[jira] [Commented] (HDFS-12626) Ozone : delete open key entries that will no longer be closed

2017-12-12 Thread Xiaoyu Yao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16287887#comment-16287887 ] Xiaoyu Yao commented on HDFS-12626: --- Thanks [~vagarychen] for the update. It looks good to me now. +1

[jira] [Updated] (HDFS-12910) Secure Datanode Starter should log the port when it

2017-12-12 Thread Stephen O'Donnell (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephen O'Donnell updated HDFS-12910: - Attachment: HDFS-12910.004.patch > Secure Datanode Starter should log the port when it >

[jira] [Commented] (HDFS-12910) Secure Datanode Starter should log the port when it

2017-12-12 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288000#comment-16288000 ] genericqa commented on HDFS-12910: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-12818) Support multiple storages in DataNodeCluster / SimulatedFSDataset

2017-12-12 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16287892#comment-16287892 ] Erik Krogen commented on HDFS-12818: Looks like the test failures on the last run were

[jira] [Commented] (HDFS-12574) Add CryptoInputStream to WebHdfsFileSystem read call.

2017-12-12 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288121#comment-16288121 ] genericqa commented on HDFS-12574: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-12895) RBF: Add ACL support for mount table

2017-12-12 Thread JIRA
[ https://issues.apache.org/jira/browse/HDFS-12895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288226#comment-16288226 ] Íñigo Goiri commented on HDFS-12895: [~linyiqun], thanks for [^HDFS-12895.003.patch]; I tested in our

[jira] [Updated] (HDFS-12881) Output streams closed with IOUtils suppressing write errors

2017-12-12 Thread Ajay Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Kumar updated HDFS-12881: -- Status: Patch Available (was: Open) > Output streams closed with IOUtils suppressing write errors >

[jira] [Commented] (HDFS-12910) Secure Datanode Starter should log the port when it

2017-12-12 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288725#comment-16288725 ] Xiao Chen commented on HDFS-12910: -- Thanks for moving this forward, Nanda and Stephen! Latest patch looks

[jira] [Comment Edited] (HDFS-12910) Secure Datanode Starter should log the port when it

2017-12-12 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288725#comment-16288725 ] Xiao Chen edited comment on HDFS-12910 at 12/13/17 5:15 AM: Thanks for moving

[jira] [Comment Edited] (HDFS-12833) Distcp : Update the usage of delete option for dependency with update and overwrite option

2017-12-12 Thread Surendra Singh Lilhore (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288728#comment-16288728 ] Surendra Singh Lilhore edited comment on HDFS-12833 at 12/13/17 5:29 AM:

[jira] [Updated] (HDFS-12923) DFS.concat should throw exception if files have different EC policies.

2017-12-12 Thread Lei (Eddy) Xu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lei (Eddy) Xu updated HDFS-12923: - Component/s: erasure-coding > DFS.concat should throw exception if files have different EC

[jira] [Created] (HDFS-12923) DFS.concat should throw exception if files have different EC policies.

2017-12-12 Thread Lei (Eddy) Xu (JIRA)
Lei (Eddy) Xu created HDFS-12923: Summary: DFS.concat should throw exception if files have different EC policies. Key: HDFS-12923 URL: https://issues.apache.org/jira/browse/HDFS-12923 Project:

[jira] [Comment Edited] (HDFS-12918) NameNode fails to start after upgrade - Missing state in ECPolicy Proto

2017-12-12 Thread Manoj Govindassamy (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288591#comment-16288591 ] Manoj Govindassamy edited comment on HDFS-12918 at 12/13/17 5:46 AM: -

[jira] [Resolved] (HDFS-12921) DFS.setReplication should throw exception on EC files

2017-12-12 Thread Lei (Eddy) Xu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lei (Eddy) Xu resolved HDFS-12921. -- Resolution: Won't Fix Fix Version/s: 3.0.0 It was a no-op in

[jira] [Commented] (HDFS-12907) Allow read-only access to reserved raw for non-superusers

2017-12-12 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288688#comment-16288688 ] genericqa commented on HDFS-12907: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-12881) Output streams closed with IOUtils suppressing write errors

2017-12-12 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288698#comment-16288698 ] genericqa commented on HDFS-12881: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (HDFS-12833) Distcp : Update the usage of delete option for dependency with update and overwrite option

2017-12-12 Thread Surendra Singh Lilhore (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Surendra Singh Lilhore updated HDFS-12833: -- Attachment: HDFS-12833-branch-2.committed.patch > Distcp : Update the usage of

[jira] [Updated] (HDFS-12833) Distcp : Update the usage of delete option for dependency with update and overwrite option

2017-12-12 Thread Surendra Singh Lilhore (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Surendra Singh Lilhore updated HDFS-12833: -- Resolution: Fixed Fix Version/s: 2.8.4 2.10.0

[jira] [Updated] (HDFS-12833) Distcp : Update the usage of delete option for dependency with update and overwrite option

2017-12-12 Thread Surendra Singh Lilhore (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Surendra Singh Lilhore updated HDFS-12833: -- Affects Version/s: (was: 3.0.0-alpha1) 2.8.0

[jira] [Updated] (HDFS-12917) Fix description errors in testErasureCodingConf.xml

2017-12-12 Thread chencan (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chencan updated HDFS-12917: --- Status: Patch Available (was: Open) > Fix description errors in testErasureCodingConf.xml >

[jira] [Updated] (HDFS-12308) Erasure Coding: Provide DistributedFileSystem & DFSClient API to return the effective EC policy on a directory or file, including the replication policy

2017-12-12 Thread chencan (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chencan updated HDFS-12308: --- Status: Patch Available (was: Open) > Erasure Coding: Provide DistributedFileSystem & DFSClient API to

[jira] [Commented] (HDFS-12917) Fix description errors in testErasureCodingConf.xml

2017-12-12 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288769#comment-16288769 ] genericqa commented on HDFS-12917: -- | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem

[jira] [Commented] (HDFS-12918) EC Policy defaults incorrectly to enabled in protobufs

2017-12-12 Thread Zach Amsden (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288158#comment-16288158 ] Zach Amsden commented on HDFS-12918: [~manojg] thanks, we encountered this during testing. I wasn't

[jira] [Comment Edited] (HDFS-12895) RBF: Add ACL support for mount table

2017-12-12 Thread JIRA
[ https://issues.apache.org/jira/browse/HDFS-12895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288226#comment-16288226 ] Íñigo Goiri edited comment on HDFS-12895 at 12/12/17 8:35 PM: -- [~linyiqun],

[jira] [Comment Edited] (HDFS-12918) NameNode fails to start after upgrade - Missing state in ECPolicy Proto

2017-12-12 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288591#comment-16288591 ] Xiao Chen edited comment on HDFS-12918 at 12/13/17 6:58 AM: We have an upgrade

[jira] [Commented] (HDFS-9806) Allow HDFS block replicas to be provided by an external storage system

2017-12-12 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288834#comment-16288834 ] genericqa commented on HDFS-9806: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-12308) Erasure Coding: Provide DistributedFileSystem & DFSClient API to return the effective EC policy on a directory or file, including the replication policy

2017-12-12 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288849#comment-16288849 ] genericqa commented on HDFS-12308: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-12862) CacheDirective may invalidata,when NN restart or make a transition to Active.

2017-12-12 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16287285#comment-16287285 ] genericqa commented on HDFS-12862: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-12910) Secure Datanode Starter should log the port when it

2017-12-12 Thread Nanda kumar (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16287325#comment-16287325 ] Nanda kumar commented on HDFS-12910: Thanks [~xiaochen] for the review and suggestions. bq. We usually

[jira] [Commented] (HDFS-12891) Do not invalidate blocks if toInvalidate is empty

2017-12-12 Thread Zsolt Venczel (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16287350#comment-16287350 ] Zsolt Venczel commented on HDFS-12891: -- Thansk [~jojochuang] for managing this issue! > Do not

[jira] [Comment Edited] (HDFS-12891) Do not invalidate blocks if toInvalidate is empty

2017-12-12 Thread Zsolt Venczel (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16280479#comment-16280479 ] Zsolt Venczel edited comment on HDFS-12891 at 12/12/17 9:41 AM: * Check

  1   2   >