[jira] [Commented] (HDFS-10453) ReplicationMonitor thread could stuck for long time due to the race between replication and delete of same file in a large cluster.

2017-12-06 Thread Xiang Li (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16281396#comment-16281396 ] Xiang Li commented on HDFS-10453: - Hi [~hexiaoqiao], we met the same issue and thanks for the patch! I

[jira] [Commented] (HDFS-12840) Creating a file with non-default EC policy in a EC zone is not correctly serialized in the editlog

2017-12-06 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16281379#comment-16281379 ] SammiChen commented on HDFS-12840: -- Thanks [~eddyxu] for the contribution! The latest patch LGTM and +1.

[jira] [Commented] (HDFS-12905) [READ] Handle decommissioning and under-maintenance Datanodes with Provided storage.

2017-12-06 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16281380#comment-16281380 ] genericqa commented on HDFS-12905: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (HDFS-12874) [READ] Documentation for provided storage

2017-12-06 Thread Chris Douglas (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Douglas updated HDFS-12874: - Attachment: HDFS-12874-HDFS-9806.00.patch First draft of user docs > [READ] Documentation for

[jira] [Commented] (HDFS-12840) Creating a file with non-default EC policy in a EC zone is not correctly serialized in the editlog

2017-12-06 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16281332#comment-16281332 ] genericqa commented on HDFS-12840: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-12889) Router UI is missing robots.txt file

2017-12-06 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16281327#comment-16281327 ] Hudson commented on HDFS-12889: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #13336 (See

[jira] [Commented] (HDFS-12889) Router UI is missing robots.txt file

2017-12-06 Thread Akira Ajisaka (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16281319#comment-16281319 ] Akira Ajisaka commented on HDFS-12889: -- Sorry I forgot to git add the robot.txt when committing this.

[jira] [Commented] (HDFS-12905) [READ] Handle decommissioning and under-maintenance Datanodes with Provided storage.

2017-12-06 Thread Virajith Jalaparti (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16281269#comment-16281269 ] Virajith Jalaparti commented on HDFS-12905: --- All operations are done under the FSNamesytem write

[jira] [Commented] (HDFS-12905) [READ] Handle decommissioning and under-maintenance Datanodes with Provided storage.

2017-12-06 Thread JIRA
[ https://issues.apache.org/jira/browse/HDFS-12905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16281267#comment-16281267 ] Íñigo Goiri commented on HDFS-12905: Do you need any synchronization when adding and removing the

[jira] [Comment Edited] (HDFS-12875) RBF: Complete logic for -readonly option of dfsrouteradmin add command

2017-12-06 Thread Yiqun Lin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16281247#comment-16281247 ] Yiqun Lin edited comment on HDFS-12875 at 12/7/17 2:58 AM: --- Thanks for taking

[jira] [Comment Edited] (HDFS-12875) RBF: Complete logic for -readonly option of dfsrouteradmin add command

2017-12-06 Thread Yiqun Lin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16281247#comment-16281247 ] Yiqun Lin edited comment on HDFS-12875 at 12/7/17 2:54 AM: --- Thanks for taking

[jira] [Updated] (HDFS-12905) [READ] Handle decommissioning and under-maintenance Datanodes with Provided storage.

2017-12-06 Thread Virajith Jalaparti (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virajith Jalaparti updated HDFS-12905: -- Attachment: HDFS-12905-HDFS-9806.001.patch Attached patch adds a

[jira] [Updated] (HDFS-12905) [READ] Handle decommissioning and under-maintenance Datanodes with Provided storage.

2017-12-06 Thread Virajith Jalaparti (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virajith Jalaparti updated HDFS-12905: -- Status: Patch Available (was: Open) > [READ] Handle decommissioning and

[jira] [Commented] (HDFS-12875) RBF: Complete logic for -readonly option of dfsrouteradmin add command

2017-12-06 Thread Yiqun Lin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16281247#comment-16281247 ] Yiqun Lin commented on HDFS-12875: -- Thanks for taking over this, [~elgoiri]. The updated patch looks good

[jira] [Commented] (HDFS-12896) when set replicate EC policy for a directory or file,it's EC policy cannot be querying by getPolicy command.

2017-12-06 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16281229#comment-16281229 ] SammiChen commented on HDFS-12896: -- Hi [~candychencan], HDFS-12308 is about to provide a new API other

[jira] [Commented] (HDFS-12904) Add DataTransferThrottler to the Datanode transfers

2017-12-06 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16281200#comment-16281200 ] genericqa commented on HDFS-12904: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (HDFS-12840) Creating a file with non-default EC policy in a EC zone is not correctly serialized in the editlog

2017-12-06 Thread Lei (Eddy) Xu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lei (Eddy) Xu updated HDFS-12840: - Attachment: HDFS-12840.05.patch editsStored.05 Thanks for the suggestions,

[jira] [Updated] (HDFS-12899) Ozone: SCM: BlockManagerImpl close is called twice during StorageContainerManager#stop

2017-12-06 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer updated HDFS-12899: Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: HDFS-7240 Status:

[jira] [Assigned] (HDFS-12907) Allow read-only access to reserved raw for non-superusers

2017-12-06 Thread Rushabh S Shah (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rushabh S Shah reassigned HDFS-12907: - Assignee: Rushabh S Shah > Allow read-only access to reserved raw for non-superusers >

[jira] [Commented] (HDFS-12907) Allow read-only access to reserved raw for non-superusers

2017-12-06 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16281129#comment-16281129 ] Andrew Wang commented on HDFS-12907: SGTM, we locked it down originally since we didn't know of a

[jira] [Commented] (HDFS-12899) Ozone: SCM: BlockManagerImpl close is called twice during StorageContainerManager#stop

2017-12-06 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16281101#comment-16281101 ] Anu Engineer commented on HDFS-12899: - +1, Thank for catching this. [~vagarychen] Thanks for the

[jira] [Commented] (HDFS-12869) Ozone: Service Discovery: RPC endpoint in KSM for getServiceList

2017-12-06 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16281096#comment-16281096 ] Anu Engineer commented on HDFS-12869: - [~nandakumar131] The patch looks very good and it will

[jira] [Commented] (HDFS-12818) Support multiple storages in DataNodeCluster / SimulatedFSDataset

2017-12-06 Thread Konstantin Shvachko (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16281093#comment-16281093 ] Konstantin Shvachko commented on HDFS-12818: Looks like a good rework of

[jira] [Commented] (HDFS-12907) Allow read-only access to reserved raw for non-superusers

2017-12-06 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16281094#comment-16281094 ] Daryn Sharp commented on HDFS-12907: [~andrew.wang], does this sound reasonable? > Allow read-only

[jira] [Created] (HDFS-12907) Allow read-only access to reserved raw for non-superusers

2017-12-06 Thread Daryn Sharp (JIRA)
Daryn Sharp created HDFS-12907: -- Summary: Allow read-only access to reserved raw for non-superusers Key: HDFS-12907 URL: https://issues.apache.org/jira/browse/HDFS-12907 Project: Hadoop HDFS

[jira] [Assigned] (HDFS-12905) [READ] Handle decommissioning and under-maintenance Datanodes with Provided storage.

2017-12-06 Thread Virajith Jalaparti (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virajith Jalaparti reassigned HDFS-12905: - Assignee: Virajith Jalaparti > [READ] Handle decommissioning and

[jira] [Commented] (HDFS-11923) Stress test of DFSNetworkTopology

2017-12-06 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16281023#comment-16281023 ] genericqa commented on HDFS-11923: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (HDFS-12904) Add DataTransferThrottler to the Datanode transfers

2017-12-06 Thread JIRA
[ https://issues.apache.org/jira/browse/HDFS-12904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Íñigo Goiri updated HDFS-12904: --- Assignee: Íñigo Goiri Status: Patch Available (was: Open) > Add DataTransferThrottler to the

[jira] [Commented] (HDFS-12875) RBF: Complete logic for -readonly option of dfsrouteradmin add command

2017-12-06 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16281007#comment-16281007 ] genericqa commented on HDFS-12875: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Created] (HDFS-12906) hedged point read in DFSInputStream sends only 1 hedge read request

2017-12-06 Thread Tao Zhang (JIRA)
Tao Zhang created HDFS-12906: Summary: hedged point read in DFSInputStream sends only 1 hedge read request Key: HDFS-12906 URL: https://issues.apache.org/jira/browse/HDFS-12906 Project: Hadoop HDFS

[jira] [Updated] (HDFS-12901) Ozone: SCM: Expose StorageContainerManager#getScmId through container protocol

2017-12-06 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer updated HDFS-12901: Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: HDFS-7240 Target

[jira] [Commented] (HDFS-12901) Ozone: SCM: Expose StorageContainerManager#getScmId through container protocol

2017-12-06 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16280985#comment-16280985 ] Anu Engineer commented on HDFS-12901: - +1, I will commit this shortly. > Ozone: SCM: Expose

[jira] [Commented] (HDFS-12903) [READ] Fix closing streams in ImageWriter

2017-12-06 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16280976#comment-16280976 ] genericqa commented on HDFS-12903: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-12890) Ozone: XceiverClient should have upper bound on async requests

2017-12-06 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16280975#comment-16280975 ] Anu Engineer commented on HDFS-12890: - [~shashikant] Thanks for getting this patch done. It looks good

[jira] [Commented] (HDFS-12510) RBF: Add security to UI

2017-12-06 Thread JIRA
[ https://issues.apache.org/jira/browse/HDFS-12510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16280953#comment-16280953 ] Íñigo Goiri commented on HDFS-12510: Thanks [~ajayydv] for offering. In HDFS-12273, I added a simple

[jira] [Updated] (HDFS-12510) Most of th

2017-12-06 Thread JIRA
[ https://issues.apache.org/jira/browse/HDFS-12510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Íñigo Goiri updated HDFS-12510: --- Summary: Most of th (was: RBF: Add security to UI) > Most of th > -- > >

[jira] [Updated] (HDFS-12510) RBF: Add security to UI

2017-12-06 Thread JIRA
[ https://issues.apache.org/jira/browse/HDFS-12510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Íñigo Goiri updated HDFS-12510: --- Summary: RBF: Add security to UI (was: Most of th) > RBF: Add security to UI >

[jira] [Commented] (HDFS-12901) Ozone: SCM: Expose StorageContainerManager#getScmId through container protocol

2017-12-06 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16280926#comment-16280926 ] genericqa commented on HDFS-12901: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-12903) [READ] Fix closing streams in ImageWriter

2017-12-06 Thread Virajith Jalaparti (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16280894#comment-16280894 ] Virajith Jalaparti commented on HDFS-12903: --- Patch catches {{Throwable}} instead of

[jira] [Updated] (HDFS-12903) [READ] Fix closing streams in ImageWriter

2017-12-06 Thread Virajith Jalaparti (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virajith Jalaparti updated HDFS-12903: -- Attachment: HDFS-12903-HDFS-9806.001.patch > [READ] Fix closing streams in ImageWriter

[jira] [Updated] (HDFS-12903) [READ] Fix closing streams in ImageWriter

2017-12-06 Thread Virajith Jalaparti (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virajith Jalaparti updated HDFS-12903: -- Status: Patch Available (was: Open) > [READ] Fix closing streams in ImageWriter >

[jira] [Commented] (HDFS-12869) Ozone: Service Discovery: RPC endpoint in KSM for getServiceList

2017-12-06 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16280886#comment-16280886 ] genericqa commented on HDFS-12869: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-12899) Ozone: SCM: BlockManagerImpl close is called twice during StorageContainerManager#stop

2017-12-06 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16280879#comment-16280879 ] genericqa commented on HDFS-12899: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-12510) RBF: Add security to UI

2017-12-06 Thread Ajay Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16280868#comment-16280868 ] Ajay Kumar commented on HDFS-12510: --- Hi [~elgoiri], Could you please add more details to what we plan to

[jira] [Updated] (HDFS-12905) [READ] Handle decommissioning and under-maintenance Datanodes with Provided storage.

2017-12-06 Thread Virajith Jalaparti (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virajith Jalaparti updated HDFS-12905: -- Description: {{ProvidedStorageMap}} doesn't keep track of the state of the datanodes

[jira] [Updated] (HDFS-12905) [READ] Handle decommissioning and under-maintenance Datanodes with Provided storage.

2017-12-06 Thread Virajith Jalaparti (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virajith Jalaparti updated HDFS-12905: -- Description: {{ProvidedStorageMap}} doesn't keep track of the state of the datanodes

[jira] [Updated] (HDFS-12905) [READ] Handle decommissioning and under-maintance Datanodes with Provided storage.

2017-12-06 Thread Virajith Jalaparti (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virajith Jalaparti updated HDFS-12905: -- Description: {{ProvidedStorageMap}} doesn't keep track of the state of the datanodes

[jira] [Updated] (HDFS-12905) [READ] Handle decommissioning and under-maintenance Datanodes with Provided storage.

2017-12-06 Thread Virajith Jalaparti (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virajith Jalaparti updated HDFS-12905: -- Summary: [READ] Handle decommissioning and under-maintenance Datanodes with Provided

[jira] [Created] (HDFS-12905) [READ] Handle decommissioning and under-maintance Datanodes with Provided storage.

2017-12-06 Thread Virajith Jalaparti (JIRA)
Virajith Jalaparti created HDFS-12905: - Summary: [READ] Handle decommissioning and under-maintance Datanodes with Provided storage. Key: HDFS-12905 URL: https://issues.apache.org/jira/browse/HDFS-12905

[jira] [Commented] (HDFS-12899) Ozone: SCM: BlockManagerImpl close is called twice during StorageContainerManager#stop

2017-12-06 Thread Chen Liang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16280836#comment-16280836 ] Chen Liang commented on HDFS-12899: --- Thanks [~nandakumar131] for the catch! the patch LGTM, pending

[jira] [Commented] (HDFS-12891) TestDNFencingWithReplication.testFencingStress: java.lang.AssertionError

2017-12-06 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16280798#comment-16280798 ] genericqa commented on HDFS-12891: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-12838) Ozone: Optimize number of allocated block rpc by aggregating multiple block allocation requests

2017-12-06 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16280795#comment-16280795 ] Anu Engineer commented on HDFS-12838: - bq. I tried implementing a block cache but there are issues

[jira] [Updated] (HDFS-12902) IOException when running MiniDfsCluster on Windows: Failed to save in any storage directories while saving namespace

2017-12-06 Thread Jeff Saremi (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Saremi updated HDFS-12902: --- Description: Error when running MiniDfsCluster on windows. I tried this first on 2.7.1. THen I

[jira] [Commented] (HDFS-12904) Add DataTransferThrottler to the Datanode transfers

2017-12-06 Thread JIRA
[ https://issues.apache.org/jira/browse/HDFS-12904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16280789#comment-16280789 ] Íñigo Goiri commented on HDFS-12904: Internally, we are using this to cap the network traffic. > Add

[jira] [Updated] (HDFS-12904) Add DataTransferThrottler to the Datanode transfers

2017-12-06 Thread JIRA
[ https://issues.apache.org/jira/browse/HDFS-12904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Íñigo Goiri updated HDFS-12904: --- Attachment: HDFS-12904.000.patch > Add DataTransferThrottler to the Datanode transfers >

[jira] [Created] (HDFS-12904) Add DataTransferThrottler to the Datanode transfers

2017-12-06 Thread JIRA
Íñigo Goiri created HDFS-12904: -- Summary: Add DataTransferThrottler to the Datanode transfers Key: HDFS-12904 URL: https://issues.apache.org/jira/browse/HDFS-12904 Project: Hadoop HDFS Issue

[jira] [Commented] (HDFS-12894) [READ] Skip setting block count of ProvidedDatanodeStorageInfo on DN registration update

2017-12-06 Thread JIRA
[ https://issues.apache.org/jira/browse/HDFS-12894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16280748#comment-16280748 ] Íñigo Goiri commented on HDFS-12894: Filed HDFS-12903 for the FindBug in the branch. > [READ] Skip

[jira] [Work started] (HDFS-12751) Ozone: SCM: update container allocated size to container db for all the open containers in ContainerStateManager#close

2017-12-06 Thread Chen Liang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-12751 started by Chen Liang. - > Ozone: SCM: update container allocated size to container db for all the open >

[jira] [Updated] (HDFS-11923) Stress test of DFSNetworkTopology

2017-12-06 Thread Chen Liang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chen Liang updated HDFS-11923: -- Attachment: HDFS-11923.003.patch Instead of running the test indefinitely, v003 patch changes to

[jira] [Commented] (HDFS-12838) Ozone: Optimize number of allocated block rpc by aggregating multiple block allocation requests

2017-12-06 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16280721#comment-16280721 ] genericqa commented on HDFS-12838: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (HDFS-12903) [READ] Fix closing streams in ImageWriter

2017-12-06 Thread JIRA
[ https://issues.apache.org/jira/browse/HDFS-12903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Íñigo Goiri updated HDFS-12903: --- Description: HDFS-12894 showed a FindBug in HDFS-9806. This seems related to HDFS-12881 when using

[jira] [Created] (HDFS-12903) [READ] Fix closing streams in ImageWriter

2017-12-06 Thread JIRA
Íñigo Goiri created HDFS-12903: -- Summary: [READ] Fix closing streams in ImageWriter Key: HDFS-12903 URL: https://issues.apache.org/jira/browse/HDFS-12903 Project: Hadoop HDFS Issue Type:

[jira] [Commented] (HDFS-12894) [READ] Skip setting block count of ProvidedDatanodeStorageInfo on DN registration update

2017-12-06 Thread JIRA
[ https://issues.apache.org/jira/browse/HDFS-12894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16280663#comment-16280663 ] Íñigo Goiri commented on HDFS-12894: {{TestUnbuffer}} is already being tracked in HADOOP-15056. For

[jira] [Commented] (HDFS-12875) RBF: Complete logic for -readonly option of dfsrouteradmin add command

2017-12-06 Thread JIRA
[ https://issues.apache.org/jira/browse/HDFS-12875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16280653#comment-16280653 ] Íñigo Goiri commented on HDFS-12875: Attached [^HDFS-12875.002.patch] with the Router funcitonality.

[jira] [Updated] (HDFS-12875) RBF: Complete logic for -readonly option of dfsrouteradmin add command

2017-12-06 Thread JIRA
[ https://issues.apache.org/jira/browse/HDFS-12875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Íñigo Goiri updated HDFS-12875: --- Attachment: HDFS-12875.002.patch > RBF: Complete logic for -readonly option of dfsrouteradmin add

[jira] [Comment Edited] (HDFS-10285) Storage Policy Satisfier in Namenode

2017-12-06 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16280634#comment-16280634 ] Anu Engineer edited comment on HDFS-10285 at 12/6/17 6:36 PM: --

[jira] [Commented] (HDFS-10285) Storage Policy Satisfier in Namenode

2017-12-06 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16280634#comment-16280634 ] Anu Engineer commented on HDFS-10285: - @[~andrew.wang] Thanks for the comments. bq. Adding a new

[jira] [Commented] (HDFS-12890) Ozone: XceiverClient should have upper bound on async requests

2017-12-06 Thread Mukul Kumar Singh (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16280628#comment-16280628 ] Mukul Kumar Singh commented on HDFS-12890: -- Hi [~shashikant], the values in ozone-default.xml and

[jira] [Updated] (HDFS-12902) IOException when running MiniDfsCluster on Windows: Failed to save in any storage directories while saving namespace

2017-12-06 Thread Jeff Saremi (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Saremi updated HDFS-12902: --- Description: Error when running MiniDfsCluster on windows. I tried this first on 2.7.1. THen I

[jira] [Updated] (HDFS-12902) IOException when running MiniDfsCluster on Windows: Failed to save in any storage directories while saving namespace

2017-12-06 Thread Jeff Saremi (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Saremi updated HDFS-12902: --- Description: Error when running MiniDfsCluster on windows. I tried this first on 2.7.1. THen I

[jira] [Updated] (HDFS-12902) IOException when running MiniDfsCluster on Windows: Failed to save in any storage directories while saving namespace

2017-12-06 Thread Jeff Saremi (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Saremi updated HDFS-12902: --- Description: Error when running MiniDfsCluster on windows. I tried this first on 2.7.1. THen I

[jira] [Created] (HDFS-12902) IOException when running MiniDfsCluster on Windows: Failed to save in any storage directories while saving namespace

2017-12-06 Thread Jeff Saremi (JIRA)
Jeff Saremi created HDFS-12902: -- Summary: IOException when running MiniDfsCluster on Windows: Failed to save in any storage directories while saving namespace Key: HDFS-12902 URL:

[jira] [Updated] (HDFS-12898) Ozone: TestSCMCli#testHelp and TestSCMCli#testNonExistCommand fail consistently

2017-12-06 Thread Nanda kumar (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nanda kumar updated HDFS-12898: --- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) >

[jira] [Commented] (HDFS-12898) Ozone: TestSCMCli#testHelp and TestSCMCli#testNonExistCommand fail consistently

2017-12-06 Thread Nanda kumar (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16280604#comment-16280604 ] Nanda kumar commented on HDFS-12898: Thanks [~shashikant] for the contribution, I have committed it to

[jira] [Commented] (HDFS-12898) Ozone: TestSCMCli#testHelp and TestSCMCli#testNonExistCommand fail consistently

2017-12-06 Thread Nanda kumar (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16280593#comment-16280593 ] Nanda kumar commented on HDFS-12898: I will commit this shortly. > Ozone: TestSCMCli#testHelp and

[jira] [Updated] (HDFS-12901) Ozone: SCM: Expose StorageContainerManager#getScmId through container protocol

2017-12-06 Thread Nanda kumar (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nanda kumar updated HDFS-12901: --- Status: Patch Available (was: Open) > Ozone: SCM: Expose StorageContainerManager#getScmId through

[jira] [Updated] (HDFS-12901) Ozone: SCM: Expose StorageContainerManager#getScmId through container protocol

2017-12-06 Thread Nanda kumar (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nanda kumar updated HDFS-12901: --- Attachment: HDFS-12901-HDFS-7240.000.patch > Ozone: SCM: Expose StorageContainerManager#getScmId

[jira] [Created] (HDFS-12901) Ozone: SCM: Expose StorageContainerManager#getScmId through container protocol

2017-12-06 Thread Nanda kumar (JIRA)
Nanda kumar created HDFS-12901: -- Summary: Ozone: SCM: Expose StorageContainerManager#getScmId through container protocol Key: HDFS-12901 URL: https://issues.apache.org/jira/browse/HDFS-12901 Project:

[jira] [Created] (HDFS-12900) Ozone: Add client Block cache for SCM

2017-12-06 Thread Mukul Kumar Singh (JIRA)
Mukul Kumar Singh created HDFS-12900: Summary: Ozone: Add client Block cache for SCM Key: HDFS-12900 URL: https://issues.apache.org/jira/browse/HDFS-12900 Project: Hadoop HDFS Issue

[jira] [Updated] (HDFS-12887) [READ] Allow Datanodes with Provided volumes to start when blocks with the same id exist locally

2017-12-06 Thread Virajith Jalaparti (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virajith Jalaparti updated HDFS-12887: -- Resolution: Fixed Status: Resolved (was: Patch Available) > [READ] Allow

[jira] [Commented] (HDFS-12887) [READ] Allow Datanodes with Provided volumes to start when blocks with the same id exist locally

2017-12-06 Thread Virajith Jalaparti (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16280565#comment-16280565 ] Virajith Jalaparti commented on HDFS-12887: --- Thanks for taking a look [~chris.douglas]. Will

[jira] [Updated] (HDFS-11640) [READ] Datanodes should use a unique identifier when reading from external stores

2017-12-06 Thread Virajith Jalaparti (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virajith Jalaparti updated HDFS-11640: -- Resolution: Fixed Status: Resolved (was: Patch Available) > [READ] Datanodes

[jira] [Commented] (HDFS-11640) [READ] Datanodes should use a unique identifier when reading from external stores

2017-12-06 Thread Virajith Jalaparti (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16280562#comment-16280562 ] Virajith Jalaparti commented on HDFS-11640: --- The failed tests are unrelated. Committing patch v5

[jira] [Updated] (HDFS-12899) Ozone: SCM: BlockManagerImpl close is called twice during StorageContainerManager#stop

2017-12-06 Thread Nanda kumar (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nanda kumar updated HDFS-12899: --- Attachment: HDFS-12899-HDFS-7240.000.patch > Ozone: SCM: BlockManagerImpl close is called twice

[jira] [Updated] (HDFS-12899) Ozone: SCM: BlockManagerImpl close is called twice during StorageContainerManager#stop

2017-12-06 Thread Nanda kumar (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nanda kumar updated HDFS-12899: --- Status: Patch Available (was: Open) > Ozone: SCM: BlockManagerImpl close is called twice during >

[jira] [Created] (HDFS-12899) Ozone: SCM: BlockManagerImpl close is called twice during StorageContainerManager#stop

2017-12-06 Thread Nanda kumar (JIRA)
Nanda kumar created HDFS-12899: -- Summary: Ozone: SCM: BlockManagerImpl close is called twice during StorageContainerManager#stop Key: HDFS-12899 URL: https://issues.apache.org/jira/browse/HDFS-12899

[jira] [Comment Edited] (HDFS-12891) TestDNFencingWithReplication.testFencingStress: java.lang.AssertionError

2017-12-06 Thread Zsolt Venczel (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16280479#comment-16280479 ] Zsolt Venczel edited comment on HDFS-12891 at 12/6/17 5:01 PM: --- * Remove

[jira] [Updated] (HDFS-12869) Ozone: Service Discovery: RPC endpoint in KSM for getServiceList

2017-12-06 Thread Nanda kumar (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nanda kumar updated HDFS-12869: --- Status: Patch Available (was: Open) > Ozone: Service Discovery: RPC endpoint in KSM for

[jira] [Updated] (HDFS-12869) Ozone: Service Discovery: RPC endpoint in KSM for getServiceList

2017-12-06 Thread Nanda kumar (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nanda kumar updated HDFS-12869: --- Attachment: HDFS-12869-HDFS-7240.000.patch > Ozone: Service Discovery: RPC endpoint in KSM for

[jira] [Updated] (HDFS-12891) TestDNFencingWithReplication.testFencingStress: java.lang.AssertionError

2017-12-06 Thread Zsolt Venczel (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zsolt Venczel updated HDFS-12891: - Status: Patch Available (was: In Progress) * Remove double read operation check where the first

[jira] [Updated] (HDFS-12891) TestDNFencingWithReplication.testFencingStress: java.lang.AssertionError

2017-12-06 Thread Zsolt Venczel (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zsolt Venczel updated HDFS-12891: - Attachment: HDFS-12891.01.patch > TestDNFencingWithReplication.testFencingStress:

[jira] [Work started] (HDFS-12891) TestDNFencingWithReplication.testFencingStress: java.lang.AssertionError

2017-12-06 Thread Zsolt Venczel (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-12891 started by Zsolt Venczel. > TestDNFencingWithReplication.testFencingStress: java.lang.AssertionError >

[jira] [Updated] (HDFS-12891) TestDNFencingWithReplication.testFencingStress: java.lang.AssertionError

2017-12-06 Thread Zsolt Venczel (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zsolt Venczel updated HDFS-12891: - Status: Open (was: Patch Available) > TestDNFencingWithReplication.testFencingStress:

[jira] [Updated] (HDFS-12838) Ozone: Optimize number of allocated block rpc by aggregating multiple block allocation requests

2017-12-06 Thread Mukul Kumar Singh (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mukul Kumar Singh updated HDFS-12838: - Attachment: HDFS-12838-HDFS-7240.004.patch Thanks for the review [~anu] & [~vagarychen].

[jira] [Updated] (HDFS-12891) TestDNFencingWithReplication.testFencingStress: java.lang.AssertionError

2017-12-06 Thread Zsolt Venczel (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zsolt Venczel updated HDFS-12891: - Status: Patch Available (was: Reopened) > TestDNFencingWithReplication.testFencingStress:

[jira] [Commented] (HDFS-12898) Ozone: TestSCMCli#testHelp and TestSCMCli#testNonExistCommand fail consistently

2017-12-06 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16280086#comment-16280086 ] genericqa commented on HDFS-12898: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-12898) Ozone: TestSCMCli#testHelp and TestSCMCli#testNonExistCommand fail consistently

2017-12-06 Thread Nanda kumar (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16279886#comment-16279886 ] Nanda kumar commented on HDFS-12898: Thanks [~shashikant] for catching and fixing this. LGTM +1,

[jira] [Updated] (HDFS-12898) Ozone: TestSCMCli#testHelp and TestSCMCli#testNonExistCommand fail consistently

2017-12-06 Thread Shashikant Banerjee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shashikant Banerjee updated HDFS-12898: --- Attachment: HDFS-12898-HDFS-7240.001.patch Patch v1 addresses the test failures.

[jira] [Updated] (HDFS-12898) Ozone: TestSCMCli#testHelp and TestSCMCli#testNonExistCommand fail consistently

2017-12-06 Thread Shashikant Banerjee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shashikant Banerjee updated HDFS-12898: --- Status: Patch Available (was: Open) > Ozone: TestSCMCli#testHelp and

[jira] [Updated] (HDFS-12898) Ozone: TestSCMCli#testHelp and TestSCMCli#testNonExistCommand fail consistently

2017-12-06 Thread Shashikant Banerjee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shashikant Banerjee updated HDFS-12898: --- Summary: Ozone: TestSCMCli#testHelp and TestSCMCli#testNonExistCommand fail

  1   2   >