[jira] [Commented] (HDFS-10169) TestEditLog.testBatchedSyncWithClosedLogs with useAsyncEditLog sometimes fails

2016-07-18 Thread Rakesh R (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383590#comment-15383590 ] Rakesh R commented on HDFS-10169: - [~cnauroth], I think the test case failure reason is different from

[jira] [Commented] (HDFS-8065) Erasure coding: Support truncate at striped group boundary

2016-07-18 Thread Rakesh R (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383586#comment-15383586 ] Rakesh R commented on HDFS-8065: Agreed, will come back to this jira once HDFS-7622 design is concluded.

[jira] [Commented] (HDFS-10425) Clean up NNStorage and TestSaveNamespace

2016-07-18 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383567#comment-15383567 ] Hadoop QA commented on HDFS-10425: -- | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem

[jira] [Commented] (HDFS-10633) DiskBalancer : Add the description for the new setting dfs.disk.balancer.plan.threshold.percent in HDFSDiskbalancer.md

2016-07-18 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383535#comment-15383535 ] Hadoop QA commented on HDFS-10633: -- | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem

[jira] [Commented] (HDFS-10633) DiskBalancer : Add the description for the new setting dfs.disk.balancer.plan.threshold.percent in HDFSDiskbalancer.md

2016-07-18 Thread Yiqun Lin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383530#comment-15383530 ] Yiqun Lin commented on HDFS-10633: -- Thanks [~ajisakaa] and [~eddyxu] for the review. Post a new patch to

[jira] [Updated] (HDFS-10633) DiskBalancer : Add the description for the new setting dfs.disk.balancer.plan.threshold.percent in HDFSDiskbalancer.md

2016-07-18 Thread Yiqun Lin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HDFS-10633: - Attachment: HDFS-10633.003.patch > DiskBalancer : Add the description for the new setting >

[jira] [Commented] (HDFS-10647) Add a link to HDFS disk balancer document in site.xml

2016-07-18 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383519#comment-15383519 ] Hadoop QA commented on HDFS-10647: -- | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem

[jira] [Commented] (HDFS-10626) VolumeScanner prints incorrect IOException in reportBadBlocks operation

2016-07-18 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383512#comment-15383512 ] Hadoop QA commented on HDFS-10626: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (HDFS-10647) Add a link to HDFS disk balancer document in site.xml

2016-07-18 Thread Yiqun Lin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HDFS-10647: - Attachment: HDFS-10647.001.patch > Add a link to HDFS disk balancer document in site.xml >

[jira] [Updated] (HDFS-10647) Add a link to HDFS disk balancer document in site.xml

2016-07-18 Thread Yiqun Lin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HDFS-10647: - Status: Patch Available (was: Open) > Add a link to HDFS disk balancer document in site.xml >

[jira] [Commented] (HDFS-10647) Add a link to HDFS disk balancer document in site.xml

2016-07-18 Thread Yiqun Lin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383501#comment-15383501 ] Yiqun Lin commented on HDFS-10647: -- Thanks for reporting, [~ajisakaa]. Attach a patch for this. > Add a

[jira] [Assigned] (HDFS-10647) Add a link to HDFS disk balancer document in site.xml

2016-07-18 Thread Yiqun Lin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin reassigned HDFS-10647: Assignee: Yiqun Lin > Add a link to HDFS disk balancer document in site.xml >

[jira] [Commented] (HDFS-10603) Flaky test org.apache.hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot.testWithCheckpoint

2016-07-18 Thread Akira Ajisaka (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383488#comment-15383488 ] Akira Ajisaka commented on HDFS-10603: -- Nice catch [~linyiqun], +1 for the latest patch. I'll commit

[jira] [Commented] (HDFS-10620) StringBuilder created and appended even if logging is disabled

2016-07-18 Thread Akira Ajisaka (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383485#comment-15383485 ] Akira Ajisaka commented on HDFS-10620: -- +1 for the 002 patch. I ran the failed tests locally and all

[jira] [Commented] (HDFS-10643) HDFS namenode should always use service user (hdfs) to generateEncryptedKey

2016-07-18 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383468#comment-15383468 ] Hadoop QA commented on HDFS-10643: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-10301) BlockReport retransmissions may lead to storages falsely being declared zombie if storage report processing happens out of order

2016-07-18 Thread Konstantin Shvachko (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383461#comment-15383461 ] Konstantin Shvachko commented on HDFS-10301: > adding a list of storage IDs to the block

[jira] [Commented] (HDFS-10620) StringBuilder created and appended even if logging is disabled

2016-07-18 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383454#comment-15383454 ] Hadoop QA commented on HDFS-10620: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-10301) BlockReport retransmissions may lead to storages falsely being declared zombie if storage report processing happens out of order

2016-07-18 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383447#comment-15383447 ] Hadoop QA commented on HDFS-10301: -- | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem

[jira] [Commented] (HDFS-10643) HDFS namenode should always use service user (hdfs) to generateEncryptedKey

2016-07-18 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383432#comment-15383432 ] Xiao Chen commented on HDFS-10643: -- Thanks [~xyao] for opening the issue and the patch. I think the idea

[jira] [Commented] (HDFS-10301) BlockReport retransmissions may lead to storages falsely being declared zombie if storage report processing happens out of order

2016-07-18 Thread Colin P. McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383413#comment-15383413 ] Colin P. McCabe commented on HDFS-10301: ---

[jira] [Commented] (HDFS-8901) Use ByteBuffer in striping positional read

2016-07-18 Thread Kai Zheng (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383408#comment-15383408 ] Kai Zheng commented on HDFS-8901: - Since HDFS-10548 was in this will need to be rebased. [~hayabusa], would

[jira] [Commented] (HDFS-10619) Cache path in InodesInPath

2016-07-18 Thread Yiqun Lin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383404#comment-15383404 ] Yiqun Lin commented on HDFS-10619: -- Hi, [~daryn], I have one another minor comment. Can we also replace

[jira] [Created] (HDFS-10651) Clean up some configuration related codes about legacy block reader

2016-07-18 Thread Kai Zheng (JIRA)
Kai Zheng created HDFS-10651: Summary: Clean up some configuration related codes about legacy block reader Key: HDFS-10651 URL: https://issues.apache.org/jira/browse/HDFS-10651 Project: Hadoop HDFS

[jira] [Updated] (HDFS-10626) VolumeScanner prints incorrect IOException in reportBadBlocks operation

2016-07-18 Thread Yiqun Lin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HDFS-10626: - Attachment: HDFS-10626.004.patch Thanks [~yzhangal] for the review, post a new patch for addressing the

[jira] [Commented] (HDFS-10650) DFSClient#mkdirs and DFSClient#primitiveMkdir should use default directory permission

2016-07-18 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383376#comment-15383376 ] John Zhuge commented on HDFS-10650: --- In {{trunk}} branch, things were consolidated into this private

[jira] [Created] (HDFS-10650) DFSClient#mkdirs and DFSClient#primitiveMkdir should use default directory permission

2016-07-18 Thread John Zhuge (JIRA)
John Zhuge created HDFS-10650: - Summary: DFSClient#mkdirs and DFSClient#primitiveMkdir should use default directory permission Key: HDFS-10650 URL: https://issues.apache.org/jira/browse/HDFS-10650

[jira] [Updated] (HDFS-10649) Remove unused PermissionStatus#applyUMask

2016-07-18 Thread Jitendra Nath Pandey (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jitendra Nath Pandey updated HDFS-10649: Assignee: Chen Liang > Remove unused PermissionStatus#applyUMask >

[jira] [Updated] (HDFS-10649) Remove unused PermissionStatus#applyUMask

2016-07-18 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-10649: -- Description: Class {{PermissionStatus}} is LimitedPrivate("HDFS", "MapReduce") and Unstable. (was:

[jira] [Created] (HDFS-10649) Remove unused PermissionStatus#applyUMask

2016-07-18 Thread John Zhuge (JIRA)
John Zhuge created HDFS-10649: - Summary: Remove unused PermissionStatus#applyUMask Key: HDFS-10649 URL: https://issues.apache.org/jira/browse/HDFS-10649 Project: Hadoop HDFS Issue Type:

[jira] [Commented] (HDFS-6962) ACLs inheritance conflict with umaskmode

2016-07-18 Thread Chris Nauroth (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383330#comment-15383330 ] Chris Nauroth commented on HDFS-6962: - bq. One additional question before responding to your comments.

[jira] [Updated] (HDFS-10643) HDFS namenode should always use service user (hdfs) to generateEncryptedKey

2016-07-18 Thread Xiaoyu Yao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoyu Yao updated HDFS-10643: -- Attachment: HDFS-10643.01.patch {{checkTGTAndReloginFromKeytab}} is not needed with HADOOP-13255 per

[jira] [Commented] (HDFS-10636) Modify ReplicaInfo to remove the assumption that replica metadata and data are stored in java.io.File.

2016-07-18 Thread Virajith Jalaparti (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383319#comment-15383319 ] Virajith Jalaparti commented on HDFS-10636: --- Uploading a new patch which contains some classes

[jira] [Updated] (HDFS-6962) ACLs inheritance conflict with umaskmode

2016-07-18 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-6962: - Target Version/s: 3.0.0-alpha2 (was: 2.8.0) Hadoop Flags: Incompatible change > ACLs inheritance

[jira] [Commented] (HDFS-6962) ACLs inheritance conflict with umaskmode

2016-07-18 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383312#comment-15383312 ] John Zhuge commented on HDFS-6962: -- Thank you very much [~cnauroth]. Valid points. One additional

[jira] [Updated] (HDFS-10636) Modify ReplicaInfo to remove the assumption that replica metadata and data are stored in java.io.File.

2016-07-18 Thread Virajith Jalaparti (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virajith Jalaparti updated HDFS-10636: -- Attachment: HDFS-10636.002.patch > Modify ReplicaInfo to remove the assumption that

[jira] [Updated] (HDFS-10636) Modify ReplicaInfo to remove the assumption that replica metadata and data are stored in java.io.File.

2016-07-18 Thread Virajith Jalaparti (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virajith Jalaparti updated HDFS-10636: -- Attachment: (was: HDFS-10636.002.patch) > Modify ReplicaInfo to remove the

[jira] [Updated] (HDFS-10636) Modify ReplicaInfo to remove the assumption that replica metadata and data are stored in java.io.File.

2016-07-18 Thread Virajith Jalaparti (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virajith Jalaparti updated HDFS-10636: -- Attachment: HDFS-10636.002.patch > Modify ReplicaInfo to remove the assumption that

[jira] [Assigned] (HDFS-4176) EditLogTailer should call rollEdits with a timeout

2016-07-18 Thread Lei (Eddy) Xu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lei (Eddy) Xu reassigned HDFS-4176: --- Assignee: Lei (Eddy) Xu (was: Zhe Zhang) > EditLogTailer should call rollEdits with a

[jira] [Commented] (HDFS-10301) BlockReport retransmissions may lead to storages falsely being declared zombie if storage report processing happens out of order

2016-07-18 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383300#comment-15383300 ] Hadoop QA commented on HDFS-10301: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-10519) Add a configuration option to enable in-progress edit log tailing

2016-07-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383298#comment-15383298 ] Andrew Wang commented on HDFS-10519: Hi Jiayi, thanks for working on this. I took a look at the patch

[jira] [Commented] (HDFS-10626) VolumeScanner prints incorrect IOException in reportBadBlocks operation

2016-07-18 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383261#comment-15383261 ] Yongjun Zhang commented on HDFS-10626: -- HI [~linyiqun], Thanks for the new rev. What about

[jira] [Commented] (HDFS-6962) ACLs inheritance conflict with umaskmode

2016-07-18 Thread Chris Nauroth (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383249#comment-15383249 ] Chris Nauroth commented on HDFS-6962: - Hello [~jzhuge]. I apologize for my delayed response. Thank

[jira] [Commented] (HDFS-10301) BlockReport retransmissions may lead to storages falsely being declared zombie if storage report processing happens out of order

2016-07-18 Thread Vinitha Reddy Gankidi (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383224#comment-15383224 ] Vinitha Reddy Gankidi commented on HDFS-10301: -- I have made STORAGE_REPORT {{static final}}

[jira] [Updated] (HDFS-10301) BlockReport retransmissions may lead to storages falsely being declared zombie if storage report processing happens out of order

2016-07-18 Thread Vinitha Reddy Gankidi (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinitha Reddy Gankidi updated HDFS-10301: - Attachment: HDFS-10301.010.patch > BlockReport retransmissions may lead to

[jira] [Updated] (HDFS-10620) StringBuilder created and appended even if logging is disabled

2016-07-18 Thread Akira Ajisaka (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated HDFS-10620: - Attachment: HDFS-10620.002.patch Attaching the latter patch. > StringBuilder created and

[jira] [Commented] (HDFS-10620) StringBuilder created and appended even if logging is disabled

2016-07-18 Thread Akira Ajisaka (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383187#comment-15383187 ] Akira Ajisaka commented on HDFS-10620: -- I'm in favor of the latter patch to avoid creating

[jira] [Commented] (HDFS-10301) BlockReport retransmissions may lead to storages falsely being declared zombie if storage report processing happens out of order

2016-07-18 Thread Konstantin Shvachko (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383161#comment-15383161 ] Konstantin Shvachko commented on HDFS-10301: All-capital identifiers are reserved for

[jira] [Commented] (HDFS-10645) Make block report size as a metric and add this metric to datanode web ui

2016-07-18 Thread Akira Ajisaka (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383139#comment-15383139 ] Akira Ajisaka commented on HDFS-10645: -- +1 for the idea. Some comments from me: 1. Can we use Set

[jira] [Updated] (HDFS-10648) Expose Balancer metrics through Metrics2

2016-07-18 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-10648: - Labels: metrics (was: ) > Expose Balancer metrics through Metrics2 >

[jira] [Updated] (HDFS-1312) Re-balance disks within a Datanode

2016-07-18 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDFS-1312: Release Note: The Disk Balancer lets administrators rebalance data across multiple disks of a

[jira] [Updated] (HDFS-10648) Expose Balancer metrics through Metrics2

2016-07-18 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-10648: - Component/s: balancer & mover > Expose Balancer metrics through Metrics2 >

[jira] [Created] (HDFS-10648) Expose Balancer metrics through Metrics2

2016-07-18 Thread Mark Wagner (JIRA)
Mark Wagner created HDFS-10648: -- Summary: Expose Balancer metrics through Metrics2 Key: HDFS-10648 URL: https://issues.apache.org/jira/browse/HDFS-10648 Project: Hadoop HDFS Issue Type: New

[jira] [Commented] (HDFS-1312) Re-balance disks within a Datanode

2016-07-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382995#comment-15382995 ] Andrew Wang commented on HDFS-1312: --- Thanks Arpit, I've added similar text over at HADOOP-13383 for the

[jira] [Commented] (HDFS-10301) BlockReport retransmissions may lead to storages falsely being declared zombie if storage report processing happens out of order

2016-07-18 Thread Vinitha Reddy Gankidi (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382959#comment-15382959 ] Vinitha Reddy Gankidi commented on HDFS-10301: -- Attached a new patch (009) addressing

[jira] [Updated] (HDFS-10301) BlockReport retransmissions may lead to storages falsely being declared zombie if storage report processing happens out of order

2016-07-18 Thread Vinitha Reddy Gankidi (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinitha Reddy Gankidi updated HDFS-10301: - Attachment: HDFS-10301.009.patch > BlockReport retransmissions may lead to

[jira] [Commented] (HDFS-10636) Modify ReplicaInfo to remove the assumption that replica metadata and data are stored in java.io.File.

2016-07-18 Thread Joe Pallas (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382860#comment-15382860 ] Joe Pallas commented on HDFS-10636: --- I haven't reviewed every line, but I noticed in

[jira] [Commented] (HDFS-10633) DiskBalancer : Add the description for the new setting dfs.disk.balancer.plan.threshold.percent in HDFSDiskbalancer.md

2016-07-18 Thread Lei (Eddy) Xu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382819#comment-15382819 ] Lei (Eddy) Xu commented on HDFS-10633: -- Thanks a lot for the patch and reviews, [~linyiqun] and

[jira] [Commented] (HDFS-1312) Re-balance disks within a Datanode

2016-07-18 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382793#comment-15382793 ] Arpit Agarwal commented on HDFS-1312: - Done. > Re-balance disks within a Datanode >

[jira] [Updated] (HDFS-1312) Re-balance disks within a Datanode

2016-07-18 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDFS-1312: Release Note: The Disk Balancer lets administrators rebalance data across multiple disks of a

[jira] [Commented] (HDFS-10633) DiskBalancer : Add the description for the new setting dfs.disk.balancer.plan.threshold.percent in HDFSDiskbalancer.md

2016-07-18 Thread Akira Ajisaka (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382784#comment-15382784 ] Akira Ajisaka commented on HDFS-10633: -- +1, thanks [~linyiqun] for the update. Hi [~eddyxu], would

[jira] [Commented] (HDFS-10647) Add a link to HDFS disk balancer document in site.xml

2016-07-18 Thread Akira Ajisaka (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382770#comment-15382770 ] Akira Ajisaka commented on HDFS-10647: -- hadoop-project/src/site/site.xml should be changed. > Add a

[jira] [Created] (HDFS-10647) Add a link to HDFS disk balancer document in site.xml

2016-07-18 Thread Akira Ajisaka (JIRA)
Akira Ajisaka created HDFS-10647: Summary: Add a link to HDFS disk balancer document in site.xml Key: HDFS-10647 URL: https://issues.apache.org/jira/browse/HDFS-10647 Project: Hadoop HDFS

[jira] [Commented] (HDFS-1312) Re-balance disks within a Datanode

2016-07-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382756#comment-15382756 ] Andrew Wang commented on HDFS-1312: --- Could someone (Arpit? Anu? Eddy?) add release notes for this

[jira] [Commented] (HDFS-10635) expected/actual parameters inverted in TestGlobPaths assertEquals

2016-07-18 Thread Chen Liang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382644#comment-15382644 ] Chen Liang commented on HDFS-10635: --- Thanks Steve, I'll keep an eye on that for now then. >

[jira] [Commented] (HDFS-10635) expected/actual parameters inverted in TestGlobPaths assertEquals

2016-07-18 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382633#comment-15382633 ] Steve Loughran commented on HDFS-10635: --- Chen, before you start this, wait until HADOOP-13371 is

[jira] [Commented] (HDFS-10598) DiskBalancer does not execute multi-steps plan.

2016-07-18 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382630#comment-15382630 ] Arpit Agarwal commented on HDFS-10598: -- Hi [~eddyxu], I will review in a couple of days as the fix

[jira] [Assigned] (HDFS-10635) expected/actual parameters inverted in TestGlobPaths assertEquals

2016-07-18 Thread Chen Liang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chen Liang reassigned HDFS-10635: - Assignee: Chen Liang > expected/actual parameters inverted in TestGlobPaths assertEquals >

[jira] [Commented] (HDFS-10598) DiskBalancer does not execute multi-steps plan.

2016-07-18 Thread Lei (Eddy) Xu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382604#comment-15382604 ] Lei (Eddy) Xu commented on HDFS-10598: -- Ping [~arpitagarwal] , would you mind to give a review on

[jira] [Commented] (HDFS-10467) Router-based HDFS federation

2016-07-18 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382547#comment-15382547 ] Inigo Goiri commented on HDFS-10467: Thanks [~vinayrpet] for the comments. bq. Why cant we use

[jira] [Commented] (HDFS-10467) Router-based HDFS federation

2016-07-18 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382535#comment-15382535 ] Inigo Goiri commented on HDFS-10467: [~mingma], thank you for the comments. A few

[jira] [Created] (HDFS-10646) Federation admin tool

2016-07-18 Thread Inigo Goiri (JIRA)
Inigo Goiri created HDFS-10646: -- Summary: Federation admin tool Key: HDFS-10646 URL: https://issues.apache.org/jira/browse/HDFS-10646 Project: Hadoop HDFS Issue Type: Sub-task

[jira] [Commented] (HDFS-10619) Cache path in InodesInPath

2016-07-18 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382519#comment-15382519 ] Daryn Sharp commented on HDFS-10619: The common case is {{IIP#getPath}} is called frequently and

[jira] [Commented] (HDFS-9271) Implement basic NN operations

2016-07-18 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382323#comment-15382323 ] Hadoop QA commented on HDFS-9271: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-10641) Fix intermittently failing TestBlockManager#testBlockReportQueueing

2016-07-18 Thread Rushabh S Shah (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382288#comment-15382288 ] Rushabh S Shah commented on HDFS-10641: --- Copying the stack trace and the test logs in the jira just

[jira] [Updated] (HDFS-9271) Implement basic NN operations

2016-07-18 Thread Anatoli Shein (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anatoli Shein updated HDFS-9271: Attachment: HDFS-9271.HDFS-8707.005.patch Error in doTestGetDefaultBlockSize happened in one more

[jira] [Commented] (HDFS-10645) Make block report size as a metric and add this metric to datanode web ui

2016-07-18 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382185#comment-15382185 ] Hadoop QA commented on HDFS-10645: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-10467) Router-based HDFS federation

2016-07-18 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382072#comment-15382072 ] Vinayakumar B commented on HDFS-10467: -- Hi, This will be nice addition to Federation. Apparently,

[jira] [Updated] (HDFS-10645) Make block report size as a metric and add this metric to datanode web ui

2016-07-18 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-10645: -- Status: Patch Available (was: Open) > Make block report size as a metric and add this metric to

[jira] [Updated] (HDFS-10645) Make block report size as a metric and add this metric to datanode web ui

2016-07-18 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-10645: -- Attachment: HDFS-10645.001.patch > Make block report size as a metric and add this metric to datanode

[jira] [Comment Edited] (HDFS-10645) Make block report size as a metric and add this metric to datanode web ui

2016-07-18 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382053#comment-15382053 ] Yuanbo Liu edited comment on HDFS-10645 at 7/18/16 10:33 AM: - [~cheersyang]

[jira] [Updated] (HDFS-10645) Make block report size as a metric and add this metric to datanode web ui

2016-07-18 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-10645: -- Attachment: Selection_048.png > Make block report size as a metric and add this metric to datanode web

[jira] [Commented] (HDFS-10645) Make block report size as a metric and add this metric to datanode web ui

2016-07-18 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382053#comment-15382053 ] Yuanbo Liu commented on HDFS-10645: --- [~cheersyang] Good point. I submitted a new snapshot and v1 patch.

[jira] [Commented] (HDFS-10645) Make block report size as a metric and add this metric to datanode web ui

2016-07-18 Thread Weiwei Yang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15381938#comment-15381938 ] Weiwei Yang commented on HDFS-10645: It would be good to expose the BR size in datanode metrics, so it

[jira] [Updated] (HDFS-10645) Make block report size as a metric and add this metric to datanode web ui

2016-07-18 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-10645: -- Attachment: (was: Selection_046.png) > Make block report size as a metric and add this metric to

[jira] [Updated] (HDFS-10645) Make block report size as a metric and add this metric to datanode web ui

2016-07-18 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-10645: -- Attachment: Selection_047.png > Make block report size as a metric and add this metric to datanode web

[jira] [Updated] (HDFS-10645) Make block report size as a metric and add this metric to datanode web ui

2016-07-18 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-10645: -- Description: Record block report size as a metric and show it on datanode UI. It's important for

[jira] [Comment Edited] (HDFS-10645) Make block report size as a metric and add this metric to datanode web ui

2016-07-18 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15381802#comment-15381802 ] Yuanbo Liu edited comment on HDFS-10645 at 7/18/16 8:52 AM: If the cluster

[jira] [Updated] (HDFS-10645) Make block report size as a metric and add this metric to datanode web ui

2016-07-18 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-10645: -- Summary: Make block report size as a metric and add this metric to datanode web ui (was: Make max

[jira] [Updated] (HDFS-10645) Make max block report size as a metric and add this metric to datanode web ui

2016-07-18 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-10645: -- Attachment: Selection_046.png > Make max block report size as a metric and add this metric to datanode

[jira] [Updated] (HDFS-10645) Make max block report size as a metric and add this metric to datanode web ui

2016-07-18 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-10645: -- Summary: Make max block report size as a metric and add this metric to datanode web ui (was: Make

[jira] [Commented] (HDFS-10645) Make block report size as a metric and add this metric to datanode web ui

2016-07-18 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15381802#comment-15381802 ] Yuanbo Liu commented on HDFS-10645: --- If the cluster grows big enough, it will hit this error: {noformat}

[jira] [Comment Edited] (HDFS-10645) Make block report size as a metric and add this metric to datanode web ui

2016-07-18 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15381802#comment-15381802 ] Yuanbo Liu edited comment on HDFS-10645 at 7/18/16 6:54 AM: If the cluster

[jira] [Updated] (HDFS-10645) Make block report size as a metric and add this metric to datanode web ui

2016-07-18 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-10645: -- Description: Add a new metric called "Max block report size". It's important for administrators to know

[jira] [Created] (HDFS-10645) Make block report size as a metric and add this metric to datanode web ui

2016-07-18 Thread Yuanbo Liu (JIRA)
Yuanbo Liu created HDFS-10645: - Summary: Make block report size as a metric and add this metric to datanode web ui Key: HDFS-10645 URL: https://issues.apache.org/jira/browse/HDFS-10645 Project: Hadoop