[jira] [Commented] (HDFS-11659) TestDataNodeHotSwapVolumes.testRemoveVolumeBeingWritten fail due to no DataNode available for pipeline recovery.

2017-05-03 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15996163#comment-15996163 ] Yongjun Zhang commented on HDFS-11659: -- Hi [~eddyxu], Thanks for working on this issue. I did a

[jira] [Commented] (HDFS-9807) Add an optional StorageID to writes

2017-05-03 Thread Chris Douglas (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15996161#comment-15996161 ] Chris Douglas commented on HDFS-9807: - bq. Makes sense. May be others can comment if this needs to be

[jira] [Updated] (HDFS-11546) Federation Router RPC server

2017-05-03 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri updated HDFS-11546: --- Attachment: HDFS-11546-HDFS-10467-008.patch * Improved synchronization in the {{ConnectionPool}} *

[jira] [Commented] (HDFS-11708) positional read will fail if replicas moved to different DNs after stream is opened

2017-05-03 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15996063#comment-15996063 ] Vinayakumar B commented on HDFS-11708: -- Hi [~zhz], Can you please take a look at this? > positional

[jira] [Commented] (HDFS-11738) hedged pread takes more time when block moved from initial locations

2017-05-03 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15996062#comment-15996062 ] Vinayakumar B commented on HDFS-11738: -- Hi [~zhz], Can you please take a look at this? > hedged

[jira] [Commented] (HDFS-11643) Balancer fencing fails when writing erasure coded lock file

2017-05-03 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15996052#comment-15996052 ] SammiChen commented on HDFS-11643: -- My consideration is Id lock file is not only used for {{Balancer}},

[jira] [Commented] (HDFS-11687) Add new public encryption APIs required by Hive

2017-05-03 Thread Ferdinand Xu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15996009#comment-15996009 ] Ferdinand Xu commented on HDFS-11687: - +1 ltgm. (non-binding) Cc [~spena] > Add new public

[jira] [Commented] (HDFS-11687) Add new public encryption APIs required by Hive

2017-05-03 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15996007#comment-15996007 ] Hadoop QA commented on HDFS-11687: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-6984) In Hadoop 3, make FileStatus serialize itself via protobuf

2017-05-03 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995986#comment-15995986 ] Andrew Wang commented on HDFS-6984: --- Thanks for the rev Chris. Looks like this latest patch has grown

[jira] [Commented] (HDFS-11722) Change Datanode file IO profiling sampling to percentage

2017-05-03 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995946#comment-15995946 ] Hudson commented on HDFS-11722: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11682 (See

[jira] [Commented] (HDFS-11488) Fix failure of TestDataNodeVolumeFailureReporting#testVolFailureStatsPreservedOnNNRestart

2017-05-03 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995944#comment-15995944 ] Hudson commented on HDFS-11488: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11682 (See

[jira] [Commented] (HDFS-11741) Long running balancer may fail due to expired DataEncryptionKey

2017-05-03 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995913#comment-15995913 ] Hadoop QA commented on HDFS-11741: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-11687) Add new public encryption APIs required by Hive

2017-05-03 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995899#comment-15995899 ] Andrew Wang commented on HDFS-11687: +1 LGTM, thanks Eddy! > Add new public encryption APIs required

[jira] [Updated] (HDFS-11687) Add new public encryption APIs required by Hive

2017-05-03 Thread Lei (Eddy) Xu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lei (Eddy) Xu updated HDFS-11687: - Attachment: HDFS-11687.03.patch > Add new public encryption APIs required by Hive >

[jira] [Updated] (HDFS-11687) Add new public encryption APIs required by Hive

2017-05-03 Thread Lei (Eddy) Xu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lei (Eddy) Xu updated HDFS-11687: - Attachment: (was: HDFS-11687.03.patch) > Add new public encryption APIs required by Hive >

[jira] [Commented] (HDFS-11722) Change Datanode file IO profiling sampling to percentage

2017-05-03 Thread Hanisha Koneru (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995897#comment-15995897 ] Hanisha Koneru commented on HDFS-11722: --- Thank you [~arpiagariu] for committing the patch. >

[jira] [Comment Edited] (HDFS-11722) Change Datanode file IO profiling sampling to percentage

2017-05-03 Thread Hanisha Koneru (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995897#comment-15995897 ] Hanisha Koneru edited comment on HDFS-11722 at 5/3/17 11:24 PM: Thank you

[jira] [Updated] (HDFS-11687) Add new public encryption APIs required by Hive

2017-05-03 Thread Lei (Eddy) Xu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lei (Eddy) Xu updated HDFS-11687: - Attachment: HDFS-11687.03.patch Remove

[jira] [Commented] (HDFS-11687) Add new public encryption APIs required by Hive

2017-05-03 Thread Rushabh S Shah (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995895#comment-15995895 ] Rushabh S Shah commented on HDFS-11687: --- It would be nice if someone from hive team can review it

[jira] [Comment Edited] (HDFS-11448) JN log segment syncing should support HA upgrade

2017-05-03 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995890#comment-15995890 ] Arpit Agarwal edited comment on HDFS-11448 at 5/3/17 11:20 PM: --- Thanks for

[jira] [Commented] (HDFS-11448) JN log segment syncing should support HA upgrade

2017-05-03 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995890#comment-15995890 ] Arpit Agarwal commented on HDFS-11448: -- Thanks for updating the patch [~hanishakoneru]. The v2 patch

[jira] [Commented] (HDFS-11515) -du throws ConcurrentModificationException

2017-05-03 Thread Istvan Fajth (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995884#comment-15995884 ] Istvan Fajth commented on HDFS-11515: - Hello [~jojochuang], I have been thinking about whether the

[jira] [Updated] (HDFS-11687) Add new public encryption APIs required by Hive

2017-05-03 Thread Lei (Eddy) Xu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lei (Eddy) Xu updated HDFS-11687: - Attachment: HDFS-11687.02.patch Hi, [~andrew.wang] Thanks for the review, I added the test in

[jira] [Commented] (HDFS-11488) Fix failure of TestDataNodeVolumeFailureReporting#testVolFailureStatsPreservedOnNNRestart

2017-05-03 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995880#comment-15995880 ] Hudson commented on HDFS-11488: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11681 (See

[jira] [Commented] (HDFS-11644) DFSStripedOutputStream should not implement Syncable

2017-05-03 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995874#comment-15995874 ] Andrew Wang commented on HDFS-11644: Thanks for working on this Manoj, looks good overall, few little

[jira] [Commented] (HDFS-11643) Balancer fencing fails when writing erasure coded lock file

2017-05-03 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995848#comment-15995848 ] Andrew Wang commented on HDFS-11643: I see the check was added to NameNodeConnector, can it be added

[jira] [Updated] (HDFS-11722) Change Datanode file IO profiling sampling to percentage

2017-05-03 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDFS-11722: - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha3

[jira] [Commented] (HDFS-11687) Add new public encryption APIs required by Hive

2017-05-03 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995836#comment-15995836 ] Andrew Wang commented on HDFS-11687: One additional cooment, it'd be nice to also add some additional

[jira] [Commented] (HDFS-11687) Add new public encryption APIs required by Hive

2017-05-03 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995833#comment-15995833 ] Andrew Wang commented on HDFS-11687: LGTM +1, I checked that calling {{HdfsAdmin#getKeyProvider}}

[jira] [Commented] (HDFS-11384) Add option for balancer to disperse getBlocks calls to avoid NameNode's rpc.CallQueueLength spike

2017-05-03 Thread yunjiong zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995826#comment-15995826 ] yunjiong zhao commented on HDFS-11384: -- [~shv] Thanks for the fix. > Add option for balancer to

[jira] [Commented] (HDFS-11448) JN log segment syncing should support HA upgrade

2017-05-03 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995809#comment-15995809 ] Hadoop QA commented on HDFS-11448: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-11695) [SPS]: Namenode failed to start while loading SPS xAttrs from the edits log.

2017-05-03 Thread Uma Maheswara Rao G (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995800#comment-15995800 ] Uma Maheswara Rao G commented on HDFS-11695: Well spotted [~surendrasingh] I want you to fix

[jira] [Comment Edited] (HDFS-11749) Ongoing file write fails when its pipeline DataNode is pulled out for maintenance

2017-05-03 Thread Manoj Govindassamy (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995782#comment-15995782 ] Manoj Govindassamy edited comment on HDFS-11749 at 5/3/17 10:11 PM:

[jira] [Updated] (HDFS-11749) Ongoing file write fails when its pipeline DataNode is pulled out for maintenance

2017-05-03 Thread Manoj Govindassamy (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Govindassamy updated HDFS-11749: -- Attachment: HDFS-11749-test.01.patch Attaching the test case to show ongoing file write

[jira] [Created] (HDFS-11749) Ongoing file write fails when its pipeline DataNode is pulled out for maintenance

2017-05-03 Thread Manoj Govindassamy (JIRA)
Manoj Govindassamy created HDFS-11749: - Summary: Ongoing file write fails when its pipeline DataNode is pulled out for maintenance Key: HDFS-11749 URL: https://issues.apache.org/jira/browse/HDFS-11749

[jira] [Updated] (HDFS-11741) Long running balancer may fail due to expired DataEncryptionKey

2017-05-03 Thread Wei-Chiu Chuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-11741: --- Attachment: HDFS-11741.003.patch Rev03. I've worked on the DEK/BK expiration mismatch issue

[jira] [Commented] (HDFS-11687) Add new public encryption APIs required by Hive

2017-05-03 Thread Lei (Eddy) Xu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995737#comment-15995737 ] Lei (Eddy) Xu commented on HDFS-11687: -- The findbugs warnings are not relevant to this change. Could

[jira] [Commented] (HDFS-11687) Add new public encryption APIs required by Hive

2017-05-03 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995728#comment-15995728 ] Hadoop QA commented on HDFS-11687: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (HDFS-11712) Ozone: Reuse ObjectMapper instance to improve the performance

2017-05-03 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer updated HDFS-11712: Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: HDFS-7240 Status:

[jira] [Comment Edited] (HDFS-11712) Ozone: Reuse ObjectMapper instance to improve the performance

2017-05-03 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995697#comment-15995697 ] Anu Engineer edited comment on HDFS-11712 at 5/3/17 9:21 PM: - +1, I will

[jira] [Commented] (HDFS-11712) Ozone: Reuse ObjectMapper instance to improve the performance

2017-05-03 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995697#comment-15995697 ] Anu Engineer commented on HDFS-11712: - +1, I will commit this shortly. Thank for making this

[jira] [Updated] (HDFS-11748) Deleting an open file should invalidate the block from expected locations.

2017-05-03 Thread Rushabh S Shah (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rushabh S Shah updated HDFS-11748: -- Summary: Deleting an open file should invalidate the block from expected locations. (was:

[jira] [Created] (HDFS-11748) Deleting an open file should invalidates the block from expected locations.

2017-05-03 Thread Rushabh S Shah (JIRA)
Rushabh S Shah created HDFS-11748: - Summary: Deleting an open file should invalidates the block from expected locations. Key: HDFS-11748 URL: https://issues.apache.org/jira/browse/HDFS-11748 Project:

[jira] [Updated] (HDFS-11448) JN log segment syncing should support HA upgrade

2017-05-03 Thread Hanisha Koneru (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hanisha Koneru updated HDFS-11448: -- Attachment: HDFS-11448.002.patch Thank you [~arpitagarwal] and [~jingzhao] for reviewing the

[jira] [Updated] (HDFS-11747) Ozone : need to fix OZONE_SCM_DEFAULT_PORT

2017-05-03 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer updated HDFS-11747: Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: HDFS-7240 Target

[jira] [Commented] (HDFS-11747) Ozone : need to fix OZONE_SCM_DEFAULT_PORT

2017-05-03 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995619#comment-15995619 ] Anu Engineer commented on HDFS-11747: - +1. I will commit this shortly. > Ozone : need to fix

[jira] [Commented] (HDFS-11741) Long running balancer may fail due to expired DataEncryptionKey

2017-05-03 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995578#comment-15995578 ] Hadoop QA commented on HDFS-11741: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-11687) Add new public encryption APIs required by Hive

2017-05-03 Thread Rushabh S Shah (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995571#comment-15995571 ] Rushabh S Shah commented on HDFS-11687: --- +1 ltgm. (non-binding) Thanks [~eddyxu] ! > Add new public

[jira] [Commented] (HDFS-11747) Ozone : need to fix OZONE_SCM_DEFAULT_PORT

2017-05-03 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995568#comment-15995568 ] Hadoop QA commented on HDFS-11747: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-11698) Ozone: SCM: CLI: Add Debug command

2017-05-03 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995565#comment-15995565 ] Hadoop QA commented on HDFS-11698: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-8818) Allow Balancer to run faster

2017-05-03 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995548#comment-15995548 ] Tsz Wo Nicholas Sze commented on HDFS-8818: --- [~kihwal], the metric replaceblockoperationspersec

[jira] [Reopened] (HDFS-11515) -du throws ConcurrentModificationException

2017-05-03 Thread Wei-Chiu Chuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang reopened HDFS-11515: I ran du on a CDH5.11.0 + HDFS-11515 patch NameNode using a production cluster fsimage.

[jira] [Commented] (HDFS-8818) Allow Balancer to run faster

2017-05-03 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995534#comment-15995534 ] Tsz Wo Nicholas Sze commented on HDFS-8818: --- > ... I'm sure the redesign works great on a couple

[jira] [Comment Edited] (HDFS-8818) Allow Balancer to run faster

2017-05-03 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995525#comment-15995525 ] Tsz Wo Nicholas Sze edited comment on HDFS-8818 at 5/3/17 7:58 PM: --- 15*72

[jira] [Commented] (HDFS-8818) Allow Balancer to run faster

2017-05-03 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995525#comment-15995525 ] Tsz Wo Nicholas Sze commented on HDFS-8818: --- 15*72 = 1080. I guess you were using default

[jira] [Updated] (HDFS-11687) Add new public encryption APIs required by Hive

2017-05-03 Thread Lei (Eddy) Xu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lei (Eddy) Xu updated HDFS-11687: - Attachment: HDFS-11687.01.patch Thanks for the suggestion, [~shahrs87]. I reverted the changes in

[jira] [Updated] (HDFS-11747) Ozone : need to fix OZONE_SCM_DEFAULT_PORT

2017-05-03 Thread Chen Liang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chen Liang updated HDFS-11747: -- Status: Patch Available (was: Open) > Ozone : need to fix OZONE_SCM_DEFAULT_PORT >

[jira] [Updated] (HDFS-11747) Ozone : need to fix OZONE_SCM_DEFAULT_PORT

2017-05-03 Thread Chen Liang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chen Liang updated HDFS-11747: -- Attachment: HDFS-11747-HDFS-7240.001.patch > Ozone : need to fix OZONE_SCM_DEFAULT_PORT >

[jira] [Created] (HDFS-11747) Ozone : need to fix OZONE_SCM_DEFAULT_PORT

2017-05-03 Thread Chen Liang (JIRA)
Chen Liang created HDFS-11747: - Summary: Ozone : need to fix OZONE_SCM_DEFAULT_PORT Key: HDFS-11747 URL: https://issues.apache.org/jira/browse/HDFS-11747 Project: Hadoop HDFS Issue Type:

[jira] [Commented] (HDFS-8818) Allow Balancer to run faster

2017-05-03 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995477#comment-15995477 ] Kihwal Lee commented on HDFS-8818: -- I set the dfs.datanode.balance.max.concurrent.moves to 15, which is

[jira] [Comment Edited] (HDFS-8818) Allow Balancer to run faster

2017-05-03 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995477#comment-15995477 ] Kihwal Lee edited comment on HDFS-8818 at 5/3/17 7:17 PM: -- I set

[jira] [Commented] (HDFS-8818) Allow Balancer to run faster

2017-05-03 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995450#comment-15995450 ] Daryn Sharp commented on HDFS-8818: --- bq. The new design is more flexible than the previous one since we

[jira] [Updated] (HDFS-11746) Ozone:SCM: Add support for getContainer in SCM

2017-05-03 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer updated HDFS-11746: Resolution: Not A Problem Status: Resolved (was: Patch Available) A similar change was

[jira] [Commented] (HDFS-11746) Ozone:SCM: Add support for getContainer in SCM

2017-05-03 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995439#comment-15995439 ] Hadoop QA commented on HDFS-11746: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Comment Edited] (HDFS-11740) Ozone: Differentiate time interval for different DatanodeStateMachine state tasks

2017-05-03 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995416#comment-15995416 ] Anu Engineer edited comment on HDFS-11740 at 5/3/17 6:55 PM: - @Weiwei yang,

[jira] [Comment Edited] (HDFS-11740) Ozone: Differentiate time interval for different DatanodeStateMachine state tasks

2017-05-03 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995416#comment-15995416 ] Anu Engineer edited comment on HDFS-11740 at 5/3/17 6:54 PM: - @Weiwei yang,

[jira] [Comment Edited] (HDFS-11740) Ozone: Differentiate time interval for different DatanodeStateMachine state tasks

2017-05-03 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995416#comment-15995416 ] Anu Engineer edited comment on HDFS-11740 at 5/3/17 6:53 PM: - @Weiwei yang,

[jira] [Comment Edited] (HDFS-11740) Ozone: Differentiate time interval for different DatanodeStateMachine state tasks

2017-05-03 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995416#comment-15995416 ] Anu Engineer edited comment on HDFS-11740 at 5/3/17 6:52 PM: - @Weiwei yang,

[jira] [Updated] (HDFS-11746) Ozone:SCM: Add support for getContainer in SCM

2017-05-03 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer updated HDFS-11746: Status: Patch Available (was: Open) > Ozone:SCM: Add support for getContainer in SCM >

[jira] [Comment Edited] (HDFS-8789) Block Placement policy migrator

2017-05-03 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995424#comment-15995424 ] Chris Trezzo edited comment on HDFS-8789 at 5/3/17 6:51 PM: [~kihwal] [~zhz] I

[jira] [Commented] (HDFS-8789) Block Placement policy migrator

2017-05-03 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995424#comment-15995424 ] Chris Trezzo commented on HDFS-8789: [~kihwal] [~zhz] I am not sure what your plans are to support

[jira] [Commented] (HDFS-11745) Increase HDFS test timeouts from 1 second to 10 seconds

2017-05-03 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995417#comment-15995417 ] Arpit Agarwal commented on HDFS-11745: -- +1 > Increase HDFS test timeouts from 1 second to 10 seconds

[jira] [Commented] (HDFS-11740) Ozone: Differentiate time interval for different DatanodeStateMachine state tasks

2017-05-03 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995416#comment-15995416 ] Anu Engineer commented on HDFS-11740: - @Weiwei yang, Thanks for the v2 patch. Sorry for the long

[jira] [Commented] (HDFS-11739) Fix regression in tests caused by YARN-679

2017-05-03 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995411#comment-15995411 ] Hudson commented on HDFS-11739: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11679 (See

[jira] [Commented] (HDFS-11722) Change Datanode file IO profiling sampling to percentage

2017-05-03 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995400#comment-15995400 ] Hadoop QA commented on HDFS-11722: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (HDFS-11661) GetContentSummary uses excessive amounts of memory

2017-05-03 Thread Wei-Chiu Chuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-11661: --- Status: Open (was: Patch Available) > GetContentSummary uses excessive amounts of memory >

[jira] [Commented] (HDFS-11448) JN log segment syncing should support HA upgrade

2017-05-03 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995387#comment-15995387 ] Jing Zhao commented on HDFS-11448: -- Thanks for continuing working on this, [~hanishakoneru]. And thanks

[jira] [Comment Edited] (HDFS-11739) Fix regression in tests caused by YARN-679

2017-05-03 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995381#comment-15995381 ] Mingliang Liu edited comment on HDFS-11739 at 5/3/17 6:24 PM: -- Failing tests

[jira] [Updated] (HDFS-11739) Fix regression in tests caused by YARN-679

2017-05-03 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-11739: - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha3

[jira] [Commented] (HDFS-11448) JN log segment syncing should support HA upgrade

2017-05-03 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995379#comment-15995379 ] Hadoop QA commented on HDFS-11448: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (HDFS-11698) Ozone: SCM: CLI: Add Debug command

2017-05-03 Thread Chen Liang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chen Liang updated HDFS-11698: -- Attachment: HDFS-11698-HDFS-7240.002.patch Thanks [~cheersyang] for the comment! Post v002 patch to

[jira] [Updated] (HDFS-11746) Ozone:SCM: Add support for getContainer in SCM

2017-05-03 Thread Nandakumar (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandakumar updated HDFS-11746: -- Attachment: HDFS-11746-HDFS-7240.000.patch > Ozone:SCM: Add support for getContainer in SCM >

[jira] [Commented] (HDFS-11746) Ozone:SCM: Add support for getContainer in SCM

2017-05-03 Thread Nandakumar (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995376#comment-15995376 ] Nandakumar commented on HDFS-11746: --- Attaching an initial patch for review. > Ozone:SCM: Add support

[jira] [Commented] (HDFS-8818) Allow Balancer to run faster

2017-05-03 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995369#comment-15995369 ] Tsz Wo Nicholas Sze commented on HDFS-8818: --- > ... However, please do realize that it will be a

[jira] [Updated] (HDFS-11741) Long running balancer may fail due to expired DataEncryptionKey

2017-05-03 Thread Wei-Chiu Chuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-11741: --- Attachment: HDFS-11741.002.patch Posting my rev 002 patch to address the other comments: bq.

[jira] [Created] (HDFS-11746) Ozone:SCM: Add support for getContainer in SCM

2017-05-03 Thread Nandakumar (JIRA)
Nandakumar created HDFS-11746: - Summary: Ozone:SCM: Add support for getContainer in SCM Key: HDFS-11746 URL: https://issues.apache.org/jira/browse/HDFS-11746 Project: Hadoop HDFS Issue Type:

[jira] [Commented] (HDFS-11741) Long running balancer may fail due to expired DataEncryptionKey

2017-05-03 Thread Wei-Chiu Chuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995341#comment-15995341 ] Wei-Chiu Chuang commented on HDFS-11741: Hi Andrew, thanks for the review! I just realized a

[jira] [Commented] (HDFS-11740) Ozone: Differentiate time interval for different DatanodeStateMachine state tasks

2017-05-03 Thread Weiwei Yang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995340#comment-15995340 ] Weiwei Yang commented on HDFS-11740: Thanks [~anu], sorry I uploaded a wrong patch the first time,

[jira] [Updated] (HDFS-11740) Ozone: Differentiate time interval for different DatanodeStateMachine state tasks

2017-05-03 Thread Weiwei Yang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang updated HDFS-11740: --- Attachment: HDFS-11740-HDFS-7240.002.patch > Ozone: Differentiate time interval for different

[jira] [Updated] (HDFS-11740) Ozone: Differentiate time interval for different DatanodeStateMachine state tasks

2017-05-03 Thread Weiwei Yang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang updated HDFS-11740: --- Status: In Progress (was: Patch Available) > Ozone: Differentiate time interval for different

[jira] [Updated] (HDFS-11740) Ozone: Differentiate time interval for different DatanodeStateMachine state tasks

2017-05-03 Thread Weiwei Yang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang updated HDFS-11740: --- Attachment: HDFS-11740-HDFS-7240.001.patch > Ozone: Differentiate time interval for different

[jira] [Updated] (HDFS-11740) Ozone: Differentiate time interval for different DatanodeStateMachine state tasks

2017-05-03 Thread Weiwei Yang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang updated HDFS-11740: --- Attachment: (was: HDFS-11725-HDFS-7240.001.patch) > Ozone: Differentiate time interval for

[jira] [Commented] (HDFS-11740) Ozone: Differentiate time interval for different DatanodeStateMachine state tasks

2017-05-03 Thread Weiwei Yang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995325#comment-15995325 ] Weiwei Yang commented on HDFS-11740: Hi [~anu] Thanks for your quick response, appreciate. I read

[jira] [Commented] (HDFS-11740) Ozone: Differentiate time interval for different DatanodeStateMachine state tasks

2017-05-03 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995304#comment-15995304 ] Anu Engineer commented on HDFS-11740: - [~cheersyang] Can you please take a look at the patch. It looks

[jira] [Updated] (HDFS-11740) Ozone: Differentiate time interval for different DatanodeStateMachine state tasks

2017-05-03 Thread Weiwei Yang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang updated HDFS-11740: --- Attachment: statemachine_2.png statemachine_1.png > Ozone: Differentiate time

[jira] [Updated] (HDFS-11740) Ozone: Differentiate time interval for different DatanodeStateMachine state tasks

2017-05-03 Thread Weiwei Yang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang updated HDFS-11740: --- Status: Patch Available (was: Open) > Ozone: Differentiate time interval for different

[jira] [Updated] (HDFS-11740) Ozone: Differentiate time interval for different DatanodeStateMachine state tasks

2017-05-03 Thread Weiwei Yang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang updated HDFS-11740: --- Attachment: HDFS-11725-HDFS-7240.001.patch > Ozone: Differentiate time interval for different

[jira] [Commented] (HDFS-11745) Increase HDFS test timeouts from 1 second to 10 seconds

2017-05-03 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995239#comment-15995239 ] Hadoop QA commented on HDFS-11745: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-10785) libhdfs++: Implement the rest of the tools

2017-05-03 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995204#comment-15995204 ] Hadoop QA commented on HDFS-10785: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-11448) JN log segment syncing should support HA upgrade

2017-05-03 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15995188#comment-15995188 ] Arpit Agarwal commented on HDFS-11448: -- Thanks for this improvement [~hanishakoneru]. # The tmp

  1   2   >