[jira] [Commented] (HDFS-9942) Add an HTrace span when refreshing the groups for a username

2016-03-10 Thread Masatake Iwasaki (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15190489#comment-15190489 ] Masatake Iwasaki commented on HDFS-9942: +1 BTW, this looks like the use case of NullTracer being

[jira] [Commented] (HDFS-9941) Do not log StandbyException on NN, other minor logging fixes

2016-03-10 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15190479#comment-15190479 ] Hadoop QA commented on HDFS-9941: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-9942) Add an HTrace span when refreshing the groups for a username

2016-03-10 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15190475#comment-15190475 ] Hadoop QA commented on HDFS-9942: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-1477) Support reconfiguring dfs.heartbeat.interval and dfs.namenode.heartbeat.recheck-interval without NN restart

2016-03-10 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15190456#comment-15190456 ] Hudson commented on HDFS-1477: -- FAILURE: Integrated in Hadoop-trunk-Commit #9452 (See

[jira] [Commented] (HDFS-9556) libhdfs++: pull Options from default configs by default

2016-03-10 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15190445#comment-15190445 ] Hadoop QA commented on HDFS-9556: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-9427) HDFS should not default to ephemeral ports

2016-03-10 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15190406#comment-15190406 ] Arpit Agarwal commented on HDFS-9427: - So at the risk of bike-shedding this - I think the 907x port

[jira] [Updated] (HDFS-1477) Support reconfiguring dfs.heartbeat.interval and dfs.namenode.heartbeat.recheck-interval without NN restart

2016-03-10 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDFS-1477: Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.9.0 Status: Resolved

[jira] [Updated] (HDFS-9703) DiskBalancer : getBandwidth implementation

2016-03-10 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer updated HDFS-9703: --- Attachment: HDFS-9703-HDFS-1312.003.patch * Fixed a findbug issue * Converted from string to return a

[jira] [Updated] (HDFS-9943) Support reconfiguring namenode replication confs

2016-03-10 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-9943: -- Description: The following confs should be re-configurable in runtime. -

[jira] [Commented] (HDFS-9405) When starting a file, NameNode should generate EDEK in a separate thread

2016-03-10 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15190378#comment-15190378 ] Hadoop QA commented on HDFS-9405: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Comment Edited] (HDFS-9941) Do not log StandbyException on NN, other minor logging fixes

2016-03-10 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15190373#comment-15190373 ] Arpit Agarwal edited comment on HDFS-9941 at 3/11/16 2:29 AM: -- Actually turns

[jira] [Updated] (HDFS-9941) Do not log StandbyException on NN, other minor logging fixes

2016-03-10 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDFS-9941: Attachment: HDFS-9941.03.patch > Do not log StandbyException on NN, other minor logging fixes >

[jira] [Commented] (HDFS-9941) Do not log StandbyException on NN, other minor logging fixes

2016-03-10 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15190373#comment-15190373 ] Arpit Agarwal commented on HDFS-9941: - Actually turns out I was confused. The dispatch doesn't work the

[jira] [Updated] (HDFS-9926) ozone : Add volume commands to CLI

2016-03-10 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer updated HDFS-9926: --- Attachment: HDFS-9926-HDFS-7240.003.patch [~cnauroth] Thanks for the review. bq. I think all of the

[jira] [Updated] (HDFS-9941) Do not log StandbyException on NN, other minor logging fixes

2016-03-10 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDFS-9941: Attachment: HDFS-9941.02.patch Thanks for the quick review [~cnauroth]. bq. The last line looks

[jira] [Updated] (HDFS-9944) Ozone : Add container dispatcher

2016-03-10 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer updated HDFS-9944: --- Status: Patch Available (was: Open) > Ozone : Add container dispatcher >

[jira] [Updated] (HDFS-9944) Ozone : Add container dispatcher

2016-03-10 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer updated HDFS-9944: --- Attachment: HDFS-9944-HDFS-7240.001.patch > Ozone : Add container dispatcher >

[jira] [Created] (HDFS-9944) Ozone : Add container dispatcher

2016-03-10 Thread Anu Engineer (JIRA)
Anu Engineer created HDFS-9944: -- Summary: Ozone : Add container dispatcher Key: HDFS-9944 URL: https://issues.apache.org/jira/browse/HDFS-9944 Project: Hadoop HDFS Issue Type: Sub-task

[jira] [Commented] (HDFS-9941) Do not log StandbyException on NN, other minor logging fixes

2016-03-10 Thread Chris Nauroth (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15190278#comment-15190278 ] Chris Nauroth commented on HDFS-9941: - [~arpitagarwal], thank you for the patch. {code} public void

[jira] [Created] (HDFS-9943) Support reconfiguring namenode replication confs

2016-03-10 Thread Tsz Wo Nicholas Sze (JIRA)
Tsz Wo Nicholas Sze created HDFS-9943: - Summary: Support reconfiguring namenode replication confs Key: HDFS-9943 URL: https://issues.apache.org/jira/browse/HDFS-9943 Project: Hadoop HDFS

[jira] [Commented] (HDFS-3702) Add an option for NOT writing the blocks locally if there is a datanode on the same box as the client

2016-03-10 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15190274#comment-15190274 ] Hadoop QA commented on HDFS-3702: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (HDFS-9942) Add an HTrace span when refreshing the groups for a username

2016-03-10 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-9942: --- Status: Patch Available (was: Open) > Add an HTrace span when refreshing the groups

[jira] [Updated] (HDFS-9942) Add an HTrace span when refreshing the groups for a username

2016-03-10 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-9942: --- Attachment: HDFS-9942.001.patch > Add an HTrace span when refreshing the groups for a

[jira] [Commented] (HDFS-9405) When starting a file, NameNode should generate EDEK in a separate thread

2016-03-10 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15190271#comment-15190271 ] Mingliang Liu commented on HDFS-9405: - Thanks for updating the patch, [~xiaochen]. I was aware that we

[jira] [Created] (HDFS-9942) Add an HTrace span when refreshing the groups for a username

2016-03-10 Thread Colin Patrick McCabe (JIRA)
Colin Patrick McCabe created HDFS-9942: -- Summary: Add an HTrace span when refreshing the groups for a username Key: HDFS-9942 URL: https://issues.apache.org/jira/browse/HDFS-9942 Project: Hadoop

[jira] [Updated] (HDFS-9941) Do not log StandbyException on NN, other minor logging fixes

2016-03-10 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDFS-9941: Attachment: HDFS-9941.01.patch > Do not log StandbyException on NN, other minor logging fixes >

[jira] [Updated] (HDFS-9941) Do not log StandbyException on NN, other minor logging fixes

2016-03-10 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDFS-9941: Status: Patch Available (was: Open) > Do not log StandbyException on NN, other minor logging fixes

[jira] [Commented] (HDFS-9941) Do not log StandbyException on NN, other minor logging fixes

2016-03-10 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15190259#comment-15190259 ] ASF GitHub Bot commented on HDFS-9941: -- GitHub user arp7 opened a pull request:

[jira] [Commented] (HDFS-9709) DiskBalancer : Add tests for disk balancer using a Mock Mover class.

2016-03-10 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15190252#comment-15190252 ] Anu Engineer commented on HDFS-9709: [~eddyxu] Thanks for your suggestions. I agree with all of them, I

[jira] [Updated] (HDFS-9556) libhdfs++: pull Options from default configs by default

2016-03-10 Thread Bob Hansen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bob Hansen updated HDFS-9556: - Attachment: HDFS-9556.HDFS-8707.003.patch HDFS-9556.HDFS-9932.003.patch New patch uses

[jira] [Commented] (HDFS-9709) DiskBalancer : Add tests for disk balancer using a Mock Mover class.

2016-03-10 Thread Lei (Eddy) Xu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15190246#comment-15190246 ] Lei (Eddy) Xu commented on HDFS-9709: - [~anu] Thanks for providing this patch. This patch looks fine

[jira] [Commented] (HDFS-9874) Long living DataXceiver threads cause volume shutdown to block.

2016-03-10 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15190243#comment-15190243 ] Hadoop QA commented on HDFS-9874: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-9941) Do not log StandbyException on NN, other minor logging fixes

2016-03-10 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15190241#comment-15190241 ] Mingliang Liu commented on HDFS-9941: - This proposal makes sense to me. Thanks for reporting this,

[jira] [Commented] (HDFS-9926) ozone : Add volume commands to CLI

2016-03-10 Thread Chris Nauroth (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15190222#comment-15190222 ] Chris Nauroth commented on HDFS-9926: - Hello [~anu]. This is looking good. I think all of the

[jira] [Commented] (HDFS-3702) Add an option for NOT writing the blocks locally if there is a datanode on the same box as the client

2016-03-10 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15190216#comment-15190216 ] Hadoop QA commented on HDFS-3702: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-9924) [umbrella] Asynchronous HDFS Access

2016-03-10 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15190211#comment-15190211 ] Colin Patrick McCabe commented on HDFS-9924: bq. Comparing with a single thread, the

[jira] [Created] (HDFS-9941) Do not log StandbyException on NN, other minor logging fixes

2016-03-10 Thread Arpit Agarwal (JIRA)
Arpit Agarwal created HDFS-9941: --- Summary: Do not log StandbyException on NN, other minor logging fixes Key: HDFS-9941 URL: https://issues.apache.org/jira/browse/HDFS-9941 Project: Hadoop HDFS

[jira] [Updated] (HDFS-6425) Large postponedMisreplicatedBlocks has impact on blockReport latency

2016-03-10 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-6425: -- Component/s: namenode > Large postponedMisreplicatedBlocks has impact on blockReport

[jira] [Commented] (HDFS-9924) [umbrella] Asynchronous HDFS Access

2016-03-10 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15190179#comment-15190179 ] Steve Loughran commented on HDFS-9924: -- So the use cases are one or more rename/delete operations to

[jira] [Updated] (HDFS-9925) Ozone: Add Ozone Client lib for bucket handling

2016-03-10 Thread Chris Nauroth (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Nauroth updated HDFS-9925: Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) +1

[jira] [Created] (HDFS-9940) New property dfs.balancer.max.concurrent.moves for Balancer

2016-03-10 Thread John Zhuge (JIRA)
John Zhuge created HDFS-9940: Summary: New property dfs.balancer.max.concurrent.moves for Balancer Key: HDFS-9940 URL: https://issues.apache.org/jira/browse/HDFS-9940 Project: Hadoop HDFS Issue

[jira] [Updated] (HDFS-9940) Rename dfs.balancer.max.concurrent.moves to avoid confusion

2016-03-10 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-9940: - Summary: Rename dfs.balancer.max.concurrent.moves to avoid confusion (was: New property

[jira] [Commented] (HDFS-9924) [umbrella] Asynchronous HDFS Access

2016-03-10 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15190112#comment-15190112 ] Tsz Wo Nicholas Sze commented on HDFS-9924: --- > Can you quantify what the performance improvements

[jira] [Comment Edited] (HDFS-3702) Add an option for NOT writing the blocks locally if there is a datanode on the same box as the client

2016-03-10 Thread Lei (Eddy) Xu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15190091#comment-15190091 ] Lei (Eddy) Xu edited comment on HDFS-3702 at 3/10/16 10:45 PM: --- Thanks,

[jira] [Commented] (HDFS-3702) Add an option for NOT writing the blocks locally if there is a datanode on the same box as the client

2016-03-10 Thread Lei (Eddy) Xu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15190091#comment-15190091 ] Lei (Eddy) Xu commented on HDFS-3702: - Thanks, [~arpitagarwal]. This patch shares many similarities

[jira] [Commented] (HDFS-9932) libhdfs++: find a URI parsing library

2016-03-10 Thread James Clampffer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15190087#comment-15190087 ] James Clampffer commented on HDFS-9932: --- Looks good to me, +1. > libhdfs++: find a URI parsing

[jira] [Commented] (HDFS-9930) libhdfs++: add hooks to facilitate fault testing

2016-03-10 Thread James Clampffer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15190085#comment-15190085 ] James Clampffer commented on HDFS-9930: --- Duplicate of HDFS-9616? > libhdfs++: add hooks to

[jira] [Commented] (HDFS-9933) ReverseXML should be capitalized in oiv usage message

2016-03-10 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15190005#comment-15190005 ] Hudson commented on HDFS-9933: -- FAILURE: Integrated in Hadoop-trunk-Commit #9450 (See

[jira] [Commented] (HDFS-9934) ReverseXML oiv processor should bail out if the XML file's layoutVersion doesn't match oiv's

2016-03-10 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15190004#comment-15190004 ] Hudson commented on HDFS-9934: -- FAILURE: Integrated in Hadoop-trunk-Commit #9450 (See

[jira] [Commented] (HDFS-9927) Document the new OIV ReverseXML processor

2016-03-10 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15190006#comment-15190006 ] Hudson commented on HDFS-9927: -- FAILURE: Integrated in Hadoop-trunk-Commit #9450 (See

[jira] [Updated] (HDFS-9933) ReverseXML should be capitalized in oiv usage message

2016-03-10 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-9933: --- Resolution: Fixed Fix Version/s: 2.8.0 Status: Resolved (was: Patch

[jira] [Updated] (HDFS-9934) ReverseXML oiv processor should bail out if the XML file's layoutVersion doesn't match oiv's

2016-03-10 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-9934: --- Resolution: Fixed Fix Version/s: 2.8.0 Status: Resolved (was: Patch

[jira] [Commented] (HDFS-3702) Add an option for NOT writing the blocks locally if there is a datanode on the same box as the client

2016-03-10 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15189990#comment-15189990 ] Arpit Agarwal commented on HDFS-3702: - Thanks [~eddyxu]. Before we get into specifics of this approach

[jira] [Commented] (HDFS-3702) Add an option for NOT writing the blocks locally if there is a datanode on the same box as the client

2016-03-10 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15189989#comment-15189989 ] Hadoop QA commented on HDFS-3702: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (HDFS-9927) Document the new OIV ReverseXML processor

2016-03-10 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-9927: --- Resolution: Fixed Fix Version/s: 2.8.0 Status: Resolved (was: Patch

[jira] [Updated] (HDFS-9405) When starting a file, NameNode should generate EDEK in a separate thread

2016-03-10 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-9405: Attachment: HDFS-9405.04.patch > When starting a file, NameNode should generate EDEK in a separate thread >

[jira] [Commented] (HDFS-9405) When starting a file, NameNode should generate EDEK in a separate thread

2016-03-10 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15189964#comment-15189964 ] Xiao Chen commented on HDFS-9405: - Thanks Arun and Mingliang for the review. Mingliang's comments #0 and

[jira] [Commented] (HDFS-9901) Move disk IO out of the heartbeat thread

2016-03-10 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15189962#comment-15189962 ] Inigo Goiri commented on HDFS-9901: --- [~hualiu], we should add comments in the new thread creations

[jira] [Commented] (HDFS-9901) Move disk IO out of the heartbeat thread

2016-03-10 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15189953#comment-15189953 ] Hadoop QA commented on HDFS-9901: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-9924) [umbrella] Asynchronous HDFS Access

2016-03-10 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15189945#comment-15189945 ] Tsz Wo Nicholas Sze commented on HDFS-9924: --- > ... If you can lay down that async operations

[jira] [Commented] (HDFS-9405) When starting a file, NameNode should generate EDEK in a separate thread

2016-03-10 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15189895#comment-15189895 ] Hadoop QA commented on HDFS-9405: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (HDFS-9874) Long living DataXceiver threads cause volume shutdown to block.

2016-03-10 Thread Rushabh S Shah (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rushabh S Shah updated HDFS-9874: - Attachment: HDFS-9874-trunk-2.patch Fixed checkstyle warning. > Long living DataXceiver threads

[jira] [Updated] (HDFS-9874) Long living DataXceiver threads cause volume shutdown to block.

2016-03-10 Thread Rushabh S Shah (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rushabh S Shah updated HDFS-9874: - Status: Patch Available (was: Open) Ran all the failed tests on both jdk7 and jdk8. All of them

[jira] [Updated] (HDFS-9874) Long living DataXceiver threads cause volume shutdown to block.

2016-03-10 Thread Rushabh S Shah (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rushabh S Shah updated HDFS-9874: - Status: Open (was: Patch Available) Cancelling the patch to address checkstyle warning. > Long

[jira] [Commented] (HDFS-9405) When starting a file, NameNode should generate EDEK in a separate thread

2016-03-10 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15189869#comment-15189869 ] Mingliang Liu commented on HDFS-9405: - Nits: 0. {{keyNames}} and {{kp}} seem final. 1. I like the idea

[jira] [Updated] (HDFS-3702) Add an option for NOT writing the blocks locally if there is a datanode on the same box as the client

2016-03-10 Thread Lei (Eddy) Xu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lei (Eddy) Xu updated HDFS-3702: Attachment: HDFS-3702.008.patch Thanks a lot for the suggestions, [~stack]. Updated the patch as

[jira] [Updated] (HDFS-3702) Add an option for NOT writing the blocks locally if there is a datanode on the same box as the client

2016-03-10 Thread Lei (Eddy) Xu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lei (Eddy) Xu updated HDFS-3702: Release Note: This patch will attempt to allocate all replicas to remote DataNodes, by adding local

[jira] [Assigned] (HDFS-9939) Possible performance improvement by increasing buf size in DecompressorStream in HDFS

2016-03-10 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge reassigned HDFS-9939: Assignee: John Zhuge (was: Yongjun Zhang) > Possible performance improvement by increasing buf

[jira] [Updated] (HDFS-3702) Add an option for NOT writing the blocks locally if there is a datanode on the same box as the client

2016-03-10 Thread Lei (Eddy) Xu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lei (Eddy) Xu updated HDFS-3702: Attachment: HDFS-3702_Design.pdf Hey, [~arpitagarwal] . Here is the design doc. The basic idea is

[jira] [Commented] (HDFS-3702) Add an option for NOT writing the blocks locally if there is a datanode on the same box as the client

2016-03-10 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15189771#comment-15189771 ] stack commented on HDFS-3702: - Excellent. +1. Below are nits for if you make a new version of the patch: Be

[jira] [Commented] (HDFS-9924) [umbrella] Asynchronous HDFS Access

2016-03-10 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15189749#comment-15189749 ] Colin Patrick McCabe commented on HDFS-9924: Can you quantify what the performance improvements

[jira] [Commented] (HDFS-9405) When starting a file, NameNode should generate EDEK in a separate thread

2016-03-10 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15189702#comment-15189702 ] Arun Suresh commented on HDFS-9405: --- +1, The latest patch LGTM (pending jenkins) Thanks [~xiaochen] >

[jira] [Commented] (HDFS-9822) Erasure Coding: Avoids scheduling multiple reconstruction tasks for a striped block at the same time

2016-03-10 Thread Rakesh R (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15189704#comment-15189704 ] Rakesh R commented on HDFS-9822: Thanks [~zhz] for the advice. I will consider this in the next patch. >

[jira] [Comment Edited] (HDFS-3702) Add an option for NOT writing the blocks locally if there is a datanode on the same box as the client

2016-03-10 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15189691#comment-15189691 ] Arpit Agarwal edited comment on HDFS-3702 at 3/10/16 6:36 PM: -- [~eddyxu],

[jira] [Commented] (HDFS-3702) Add an option for NOT writing the blocks locally if there is a datanode on the same box as the client

2016-03-10 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15189691#comment-15189691 ] Arpit Agarwal commented on HDFS-3702: - [~eddyxu], given the size of the patch could you please

[jira] [Commented] (HDFS-9822) Erasure Coding: Avoids scheduling multiple reconstruction tasks for a striped block at the same time

2016-03-10 Thread Rakesh R (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15189690#comment-15189690 ] Rakesh R commented on HDFS-9822: Thanks Walter, Li Bo. If I remember correctly, I had observed once in my

[jira] [Updated] (HDFS-3702) Add an option for NOT writing the blocks locally if there is a datanode on the same box as the client

2016-03-10 Thread Lei (Eddy) Xu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lei (Eddy) Xu updated HDFS-3702: Attachment: HDFS-3702.007.patch Updated the comments for {{ClientProtocol}} and {{AddBlockFlag}}.

[jira] [Updated] (HDFS-9405) When starting a file, NameNode should generate EDEK in a separate thread

2016-03-10 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-9405: Attachment: HDFS-9405.03.patch Failed tests seems not related, but I figured we'd need some protection since

[jira] [Commented] (HDFS-9118) Add logging system for libdhfs++

2016-03-10 Thread James Clampffer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15189643#comment-15189643 ] James Clampffer commented on HDFS-9118: --- Disregard the stuff about template instantiations not

[jira] [Commented] (HDFS-9924) [umbrella] Asynchronous HDFS Access

2016-03-10 Thread Chris Nauroth (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15189626#comment-15189626 ] Chris Nauroth commented on HDFS-9924: - bq. If you can't make any guarantees about ordering of calls in

[jira] [Updated] (HDFS-9901) Move disk IO out of the heartbeat thread

2016-03-10 Thread Hua Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hua Liu updated HDFS-9901: -- Status: Patch Available (was: Open) > Move disk IO out of the heartbeat thread >

[jira] [Updated] (HDFS-9901) Move disk IO out of the heartbeat thread

2016-03-10 Thread Hua Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hua Liu updated HDFS-9901: -- Attachment: 0003-HDFS-9901-Move-disk-IO-out-of-the-heartbeat-thread.patch > Move disk IO out of the heartbeat

[jira] [Updated] (HDFS-9901) Move disk IO out of the heartbeat thread

2016-03-10 Thread Hua Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hua Liu updated HDFS-9901: -- Status: Open (was: Patch Available) > Move disk IO out of the heartbeat thread >

[jira] [Updated] (HDFS-9901) Move disk IO out of the heartbeat thread

2016-03-10 Thread Hua Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hua Liu updated HDFS-9901: -- Summary: Move disk IO out of the heartbeat thread (was: Move block validation out of the heartbeat thread) >

[jira] [Commented] (HDFS-9928) Make HDFS commands guide up to date

2016-03-10 Thread Masatake Iwasaki (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15189599#comment-15189599 ] Masatake Iwasaki commented on HDFS-9928: How about targeting 2.9.0 here by just reverting the part

[jira] [Commented] (HDFS-9928) Make HDFS commands guide up to date

2016-03-10 Thread Masatake Iwasaki (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15189595#comment-15189595 ] Masatake Iwasaki commented on HDFS-9928: {noformat} + hdfs storagepolicies +

[jira] [Commented] (HDFS-9895) Remove all cached configuration from DataNode

2016-03-10 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15189591#comment-15189591 ] Haohui Mai commented on HDFS-9895: -- Quick question: how to ensure that different threads have a consistent

[jira] [Created] (HDFS-9939) Possible performance improvement by increasing buf size in DecompressorStream in HDFS

2016-03-10 Thread Yongjun Zhang (JIRA)
Yongjun Zhang created HDFS-9939: --- Summary: Possible performance improvement by increasing buf size in DecompressorStream in HDFS Key: HDFS-9939 URL: https://issues.apache.org/jira/browse/HDFS-9939

[jira] [Commented] (HDFS-9928) Make HDFS commands guide up to date

2016-03-10 Thread Masatake Iwasaki (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15189573#comment-15189573 ] Masatake Iwasaki commented on HDFS-9928: {noformat} hdfs dfsadmin [GENERIC_OPTIONS]

[jira] [Commented] (HDFS-9928) Make HDFS commands guide up to date

2016-03-10 Thread Masatake Iwasaki (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15189527#comment-15189527 ] Masatake Iwasaki commented on HDFS-9928: {noformat} -Usage: `hdfs cacheadmin -addDirective -path

[jira] [Commented] (HDFS-9179) fs.defaultFS should not be used on the server side

2016-03-10 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15189523#comment-15189523 ] Allen Wittenauer commented on HDFS-9179: bq. The justification was that the NN service URI

[jira] [Commented] (HDFS-9928) Make HDFS commands guide up to date

2016-03-10 Thread Masatake Iwasaki (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15189512#comment-15189512 ] Masatake Iwasaki commented on HDFS-9928: {noformat} 264 | `-h`\|`--help` | Display the tool

[jira] [Commented] (HDFS-9118) Add logging system for libdhfs++

2016-03-10 Thread James Clampffer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15189499#comment-15189499 ] James Clampffer commented on HDFS-9118: --- bq. Great. If we can, let's implement the plugins for the

[jira] [Commented] (HDFS-9924) [umbrella] Asynchronous HDFS Access

2016-03-10 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15189497#comment-15189497 ] Steve Loughran commented on HDFS-9924: -- As I note in HADOOP-12910, someone is going to have to very

[jira] [Commented] (HDFS-9719) Refactoring ErasureCodingWorker into smaller reusable constructs

2016-03-10 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15189484#comment-15189484 ] Hadoop QA commented on HDFS-9719: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (HDFS-9938) Sub classing of WebHDFS for the new file system implementation

2016-03-10 Thread Vishwajeet Dusane (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vishwajeet Dusane updated HDFS-9938: Attachment: HDFS-9938-001.patch Changes required in WebHDFS file system i.e. option 2. if

[jira] [Created] (HDFS-9938) Sub classing of WebHDFS for the new file system implementation

2016-03-10 Thread Vishwajeet Dusane (JIRA)
Vishwajeet Dusane created HDFS-9938: --- Summary: Sub classing of WebHDFS for the new file system implementation Key: HDFS-9938 URL: https://issues.apache.org/jira/browse/HDFS-9938 Project: Hadoop

[jira] [Commented] (HDFS-9895) Remove all cached configuration from DataNode

2016-03-10 Thread Masatake Iwasaki (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15189449#comment-15189449 ] Masatake Iwasaki commented on HDFS-9895: Current implementation does not need this because

[jira] [Commented] (HDFS-9922) Upgrade Domain placement policy status marks a good block in violation when there are decommissioned nodes

2016-03-10 Thread Ming Ma (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15189448#comment-15189448 ] Ming Ma commented on HDFS-9922: --- Thanks [~ctrezzo]. The patch makes sense. * Based on how it is

[jira] [Commented] (HDFS-8520) Patch for PPC64 block size

2016-03-10 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15189326#comment-15189326 ] Hadoop QA commented on HDFS-8520: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

  1   2   >