[jira] [Updated] (HDFS-9721) Allow Delimited PB OIV tool to run upon fsimage that contains INodeReference

2016-01-28 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-9721: Attachment: HDFS-9721.02.patch Patch 2 to fix findbugs and test case. > Allow Delimited PB OIV tool to run

[jira] [Commented] (HDFS-9706) Log more details in debug logs in BlockReceiver's constructor

2016-01-28 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15123136#comment-15123136 ] Xiao Chen commented on HDFS-9706: - Thank you Yongjun for the review and commit! > Log more details in

[jira] [Commented] (HDFS-6565) Use jackson instead jetty json in hdfs-client

2016-01-28 Thread Demeter Sztanko (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15121913#comment-15121913 ] Demeter Sztanko commented on HDFS-6565: --- We are experiencing performance degradation of webhdfs, when

[jira] [Commented] (HDFS-9711) Integrate CSRF prevention filter in WebHDFS.

2016-01-28 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15123030#comment-15123030 ] Haohui Mai commented on HDFS-9711: -- bq. Even if there was a way, it would defeat the intent of blocking

[jira] [Updated] (HDFS-9706) Log more details in debug logs in BlockReceiver's constructor

2016-01-28 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yongjun Zhang updated HDFS-9706: Issue Type: Improvement (was: Bug) > Log more details in debug logs in BlockReceiver's constructor

[jira] [Updated] (HDFS-9697) NN fails to restart due to corrupt fsimage caused by snapshot handling

2016-01-28 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HDFS-9697: Attachment: HDFS0-9697-test.patch Still I am able to reproduce on latest trunk.

[jira] [Updated] (HDFS-9706) Log more details in debug logs in BlockReceiver's constructor

2016-01-28 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yongjun Zhang updated HDFS-9706: Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.0 Status: Resolved

[jira] [Commented] (HDFS-247) A tool to plot the locations of the blocks of a directory

2016-01-28 Thread Bob (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15123139#comment-15123139 ] Bob commented on HDFS-247: -- Hi [~Pooja.Gupta], I am interesting at this feature, could you share you idea on it or

[jira] [Commented] (HDFS-9697) NN fails to restart due to corrupt fsimage caused by snapshot handling

2016-01-28 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15123155#comment-15123155 ] Yongjun Zhang commented on HDFS-9697: - Thanks a lot [~vinayrpet]! I really feel the power of our

[jira] [Updated] (HDFS-9697) NN fails to restart due to corrupt fsimage caused by snapshot handling

2016-01-28 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HDFS-9697: Attachment: HDFS0-9697-test.patch Attaching one more test, with fix to previous stack trace. With

[jira] [Commented] (HDFS-9706) Log more details in debug logs in BlockReceiver's constructor

2016-01-28 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15123131#comment-15123131 ] Hudson commented on HDFS-9706: -- FAILURE: Integrated in Hadoop-trunk-Commit #9206 (See

[jira] [Updated] (HDFS-9406) FSImage corruption after taking snapshot

2016-01-28 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yongjun Zhang updated HDFS-9406: Attachment: HDFS-9406.002.patch > FSImage corruption after taking snapshot >

[jira] [Commented] (HDFS-9684) DataNode stopped sending heartbeat after getting OutOfMemoryError form DataTransfer thread.

2016-01-28 Thread Surendra Singh Lilhore (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15123088#comment-15123088 ] Surendra Singh Lilhore commented on HDFS-9684: -- Thanks [~cnauroth]. Yes, it is duplicate of

[jira] [Commented] (HDFS-9598) Use heap buffer in PacketReceiver If hdfs client has no enough free direct buffer.

2016-01-28 Thread Yong Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15123138#comment-15123138 ] Yong Zhang commented on HDFS-9598: -- In a high load of reading hdfs application, such as HBase, direct

[jira] [Commented] (HDFS-9697) NN fails to restart due to corrupt fsimage caused by snapshot handling

2016-01-28 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15121974#comment-15121974 ] Yongjun Zhang commented on HDFS-9697: - Hi [~vinayrpet], Thanks for trying it out. I think your test

[jira] [Commented] (HDFS-7955) Improve naming of classes, methods, and variables related to block replication and recovery

2016-01-28 Thread Rakesh R (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122035#comment-15122035 ] Rakesh R commented on HDFS-7955: Thanks [~drankye]. bq. I guess we don't need to worry about this as the

[jira] [Assigned] (HDFS-9700) DFSClient and DFSOutputStream do not respect TCP_NODELAY config in two spots

2016-01-28 Thread Gary Helmling (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gary Helmling reassigned HDFS-9700: --- Assignee: Gary Helmling > DFSClient and DFSOutputStream do not respect TCP_NODELAY config in

[jira] [Commented] (HDFS-9711) Integrate CSRF prevention filter in WebHDFS.

2016-01-28 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122148#comment-15122148 ] Anu Engineer commented on HDFS-9711: +1, ( Non-binding) Thanks for contributing this patch [~cnauroth].

[jira] [Updated] (HDFS-9679) libhdfs++: Fix inconsistencies with libhdfs C API

2016-01-28 Thread James Clampffer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Clampffer updated HDFS-9679: -- Attachment: HDFS-9679.HDFS-8707.000.patch Simple patch: -only real inconsistency was in

[jira] [Commented] (HDFS-9697) NN fails to restart due to corrupt fsimage caused by snapshot handling

2016-01-28 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122414#comment-15122414 ] Yongjun Zhang commented on HDFS-9697: - Hi [~vinayrpet], Interesting thing here, I tried latest trunk

[jira] [Updated] (HDFS-9720) DiskBalancer : Add configuration parameters

2016-01-28 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer updated HDFS-9720: --- Description: Add the following config params: # Max Disk Throughput - This allows users to control

[jira] [Commented] (HDFS-9679) libhdfs++: Fix inconsistencies with libhdfs C API

2016-01-28 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122498#comment-15122498 ] Hadoop QA commented on HDFS-9679: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-7955) Improve naming of classes, methods, and variables related to block replication and recovery

2016-01-28 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122382#comment-15122382 ] Hadoop QA commented on HDFS-7955: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-9711) Integrate CSRF prevention filter in WebHDFS.

2016-01-28 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122408#comment-15122408 ] Haohui Mai commented on HDFS-9711: -- I don't think it needs to be configurable. The only thing needs to be

[jira] [Commented] (HDFS-9697) NN fails to restart due to corrupt fsimage caused by snapshot handling

2016-01-28 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122419#comment-15122419 ] Yongjun Zhang commented on HDFS-9697: - On the other hand, I did not reproduce the stack trace you

[jira] [Commented] (HDFS-9697) NN fails to restart due to corrupt fsimage caused by snapshot handling

2016-01-28 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122423#comment-15122423 ] Jing Zhao commented on HDFS-9697: - I could reproduce the exception reported by [~vinayrpet] in trunk. The

[jira] [Commented] (HDFS-9260) Improve performance and GC friendliness of startup and FBRs

2016-01-28 Thread Staffan Friberg (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122433#comment-15122433 ] Staffan Friberg commented on HDFS-9260: --- Thanks for the comments Patch 15 (and 16) should address

[jira] [Commented] (HDFS-9406) FSImage corruption after taking snapshot

2016-01-28 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122362#comment-15122362 ] Jing Zhao commented on HDFS-9406: - Thanks a lot for digging into the issue and providing fix, [~yzhangal]!

[jira] [Updated] (HDFS-9260) Improve performance and GC friendliness of startup and FBRs

2016-01-28 Thread Staffan Friberg (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Staffan Friberg updated HDFS-9260: -- Attachment: HDFS-9260.016.patch > Improve performance and GC friendliness of startup and FBRs >

[jira] [Commented] (HDFS-9406) FSImage corruption after taking snapshot

2016-01-28 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122499#comment-15122499 ] Yongjun Zhang commented on HDFS-9406: - Thanks a lot for looking into and your feedback [~jingzhao]! I

[jira] [Commented] (HDFS-8578) On upgrade, Datanode should process all storage/data dirs in parallel

2016-01-28 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122366#comment-15122366 ] Chris Trezzo commented on HDFS-8578: Looking through the patch now. > On upgrade, Datanode should

[jira] [Updated] (HDFS-9679) libhdfs++: Fix inconsistencies with libhdfs C API

2016-01-28 Thread James Clampffer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Clampffer updated HDFS-9679: -- Status: Patch Available (was: Open) > libhdfs++: Fix inconsistencies with libhdfs C API >

[jira] [Commented] (HDFS-9260) Improve performance and GC friendliness of startup and FBRs

2016-01-28 Thread Staffan Friberg (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122460#comment-15122460 ] Staffan Friberg commented on HDFS-9260: --- Hi [~jingzhao], Thank you for your comments! Updated with

[jira] [Created] (HDFS-9720) DiskBalancer : Add configuration parameters

2016-01-28 Thread Anu Engineer (JIRA)
Anu Engineer created HDFS-9720: -- Summary: DiskBalancer : Add configuration parameters Key: HDFS-9720 URL: https://issues.apache.org/jira/browse/HDFS-9720 Project: Hadoop HDFS Issue Type:

[jira] [Commented] (HDFS-9697) NN fails to restart due to corrupt fsimage caused by snapshot handling

2016-01-28 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122478#comment-15122478 ] Jing Zhao commented on HDFS-9697: - Looks like the NPE was caused by some recent changes on quota updating

[jira] [Updated] (HDFS-9720) DiskBalancer : Add configuration parameters

2016-01-28 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer updated HDFS-9720: --- Attachment: HDFS-9720-HDFS-1312.001.patch Adding the patch for code review purpose. This patch is

[jira] [Commented] (HDFS-9697) NN fails to restart due to corrupt fsimage caused by snapshot handling

2016-01-28 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122482#comment-15122482 ] Jing Zhao commented on HDFS-9697: - [~yzhangal], if you can confirm the test case here can be fixed by the

[jira] [Updated] (HDFS-9711) Integrate CSRF prevention filter in WebHDFS.

2016-01-28 Thread Chris Nauroth (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Nauroth updated HDFS-9711: Attachment: HDFS-9711.002.patch [~anu], thank you for the code review. I am attaching patch v002

[jira] [Commented] (HDFS-9711) Integrate CSRF prevention filter in WebHDFS.

2016-01-28 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122282#comment-15122282 ] Anu Engineer commented on HDFS-9711: it looks perfect, thanks > Integrate CSRF prevention filter in

[jira] [Commented] (HDFS-9700) DFSClient and DFSOutputStream do not respect TCP_NODELAY config in two spots

2016-01-28 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122295#comment-15122295 ] Mingliang Liu commented on HDFS-9700: - Nice. +1 (non-binding) > DFSClient and DFSOutputStream do not

[jira] [Updated] (HDFS-9688) Test the effect of nested encryption zones in HDFS downgrade

2016-01-28 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-9688: Attachment: HDFS-9688-branch-2.6.01.patch Thanks Andrew for the review! I initially preferred not modifying

[jira] [Commented] (HDFS-9688) Test the effect of nested encryption zones in HDFS downgrade

2016-01-28 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122524#comment-15122524 ] Zhe Zhang commented on HDFS-9688: - And I don't think we necessarily need HDFS-9244 before this change. They

[jira] [Commented] (HDFS-9688) Test the effect of nested encryption zones in HDFS downgrade

2016-01-28 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122585#comment-15122585 ] Zhe Zhang commented on HDFS-9688: - [~aw] Does the above compile failure look alarming (any infra issue

[jira] [Commented] (HDFS-9260) Improve performance and GC friendliness of startup and FBRs

2016-01-28 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122693#comment-15122693 ] Hadoop QA commented on HDFS-9260: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-9688) Test the effect of nested encryption zones in HDFS downgrade

2016-01-28 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122575#comment-15122575 ] Hadoop QA commented on HDFS-9688: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-9711) Integrate CSRF prevention filter in WebHDFS.

2016-01-28 Thread Chris Nauroth (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122598#comment-15122598 ] Chris Nauroth commented on HDFS-9711: - I'm not aware of any way for an HTTP redirect response to tell

[jira] [Created] (HDFS-9721) Allow Delimited PB OIV tool to run upon fsimage that contains INodeReference

2016-01-28 Thread Xiao Chen (JIRA)
Xiao Chen created HDFS-9721: --- Summary: Allow Delimited PB OIV tool to run upon fsimage that contains INodeReference Key: HDFS-9721 URL: https://issues.apache.org/jira/browse/HDFS-9721 Project: Hadoop HDFS

[jira] [Commented] (HDFS-9698) Long running Balancer should renew TGT

2016-01-28 Thread Chris Nauroth (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122627#comment-15122627 ] Chris Nauroth commented on HDFS-9698: - +1 for Jing's comment about how this is already done in the RPC

[jira] [Comment Edited] (HDFS-9688) Test the effect of nested encryption zones in HDFS downgrade

2016-01-28 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122652#comment-15122652 ] Allen Wittenauer edited comment on HDFS-9688 at 1/29/16 12:54 AM: -- >From

[jira] [Commented] (HDFS-9688) Test the effect of nested encryption zones in HDFS downgrade

2016-01-28 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122535#comment-15122535 ] Andrew Wang commented on HDFS-9688: --- Code LGTM, +1 pending Jenkins. > Test the effect of nested

[jira] [Commented] (HDFS-9684) DataNode stopped sending heartbeat after getting OutOfMemoryError form DataTransfer thread.

2016-01-28 Thread Chris Nauroth (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122615#comment-15122615 ] Chris Nauroth commented on HDFS-9684: - This is likely to be a duplicate of HDFS-9046, though I have not

[jira] [Commented] (HDFS-9406) FSImage corruption after taking snapshot

2016-01-28 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122625#comment-15122625 ] Yongjun Zhang commented on HDFS-9406: - Very nice [~jingzhao], I verified that the change you suggested

[jira] [Commented] (HDFS-9697) NN fails to restart due to corrupt fsimage caused by snapshot handling

2016-01-28 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122658#comment-15122658 ] Yongjun Zhang commented on HDFS-9697: - Hi [~jingzhao], Agree with you. Weird thing, I tried

[jira] [Commented] (HDFS-9503) Replace -namenode option with -fs for NNThroughputBenchmark

2016-01-28 Thread Konstantin Shvachko (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122549#comment-15122549 ] Konstantin Shvachko commented on HDFS-9503: --- Mingliang, I could not apply your last patch to

[jira] [Commented] (HDFS-9688) Test the effect of nested encryption zones in HDFS downgrade

2016-01-28 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122624#comment-15122624 ] Hadoop QA commented on HDFS-9688: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-9688) Test the effect of nested encryption zones in HDFS downgrade

2016-01-28 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122652#comment-15122652 ] Allen Wittenauer commented on HDFS-9688: >From the console link provided above: {code} ERROR:

[jira] [Commented] (HDFS-9406) FSImage corruption after taking snapshot

2016-01-28 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15120981#comment-15120981 ] Yongjun Zhang commented on HDFS-9406: - Hi Guys, I just uploaded a prototype patch to address this

[jira] [Updated] (HDFS-9697) NN fails to restart due to corrupt fsimage caused by snapshot handling

2016-01-28 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yongjun Zhang updated HDFS-9697: Description: This is related to HDFS-9406, but not quite the same symptom. {quote} ERROR

[jira] [Commented] (HDFS-9697) NN fails to restart due to corrupt fsimage caused by snapshot handling

2016-01-28 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15120986#comment-15120986 ] Yongjun Zhang commented on HDFS-9697: - Uploaded a patch to HDFS-9406 that also solves the problem here.

[jira] [Commented] (HDFS-8578) On upgrade, Datanode should process all storage/data dirs in parallel

2016-01-28 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15121070#comment-15121070 ] Vinayakumar B commented on HDFS-8578: - patch looks great. Simple nits. 1. typo,{{doUgrade}}. I think

[jira] [Updated] (HDFS-9718) NameNode#initializeGenericKeys don't unset the property while it is not assign on special nnId

2016-01-28 Thread DENG FEI (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DENG FEI updated HDFS-9718: --- Attachment: HDFS-9718.002.patch > NameNode#initializeGenericKeys don't unset the property while it is not >

[jira] [Updated] (HDFS-2139) Fast copy for HDFS.

2016-01-28 Thread Liu Junhong (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liu Junhong updated HDFS-2139: -- Attachment: HDFS-2139-For-2.7.1.patch Add fastcopy to 2.7.1 base on facebook's code and the patch in

[jira] [Commented] (HDFS-9697) NN fails to restart due to corrupt fsimage caused by snapshot handling

2016-01-28 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15121324#comment-15121324 ] Vinayakumar B commented on HDFS-9697: - I followed the same steps and written one test and got exception

[jira] [Commented] (HDFS-8578) On upgrade, Datanode should process all storage/data dirs in parallel

2016-01-28 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15121337#comment-15121337 ] Vinayakumar B commented on HDFS-8578: - +1. Latest patch looks excellent [~ctrezzo], do you want to take

[jira] [Commented] (HDFS-8578) On upgrade, Datanode should process all storage/data dirs in parallel

2016-01-28 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15121382#comment-15121382 ] Hadoop QA commented on HDFS-8578: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-7955) Improve naming of classes, methods, and variables related to block replication and recovery

2016-01-28 Thread Rakesh R (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15121287#comment-15121287 ] Rakesh R commented on HDFS-7955: Thanks [~drankye] for the interest and comments. With this jira, the idea

[jira] [Commented] (HDFS-7955) Improve naming of classes, methods, and variables related to block replication and recovery

2016-01-28 Thread Kai Zheng (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15121313#comment-15121313 ] Kai Zheng commented on HDFS-7955: - Thanks Rakesh. bq. I could see changing the entire block recovery will

[jira] [Commented] (HDFS-9406) FSImage corruption after taking snapshot

2016-01-28 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15121345#comment-15121345 ] Hadoop QA commented on HDFS-9406: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (HDFS-8578) On upgrade, Datanode should process all storage/data dirs in parallel

2016-01-28 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-8578: -- Attachment: h8578_20160128b.patch > On upgrade, Datanode should process all storage/data

[jira] [Commented] (HDFS-9198) Coalesce IBR processing in the NN

2016-01-28 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15121147#comment-15121147 ] Vinayakumar B commented on HDFS-9198: - [~daryn], Is it worth merging this to 2.7.3? > Coalesce IBR

[jira] [Updated] (HDFS-8578) On upgrade, Datanode should process all storage/data dirs in parallel

2016-01-28 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-8578: -- Attachment: h8578_20160128b.patch Thank Vinay. Here is a new patch addressing your

[jira] [Updated] (HDFS-8578) On upgrade, Datanode should process all storage/data dirs in parallel

2016-01-28 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-8578: -- Attachment: (was: h8578_20160128b.patch) > On upgrade, Datanode should process all

[jira] [Updated] (HDFS-7955) Improve naming of classes, methods, and variables related to block replication and recovery

2016-01-28 Thread Rakesh R (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated HDFS-7955: --- Attachment: HDFS-7955-004.patch > Improve naming of classes, methods, and variables related to block >

[jira] [Commented] (HDFS-9701) DN may deadlock when hot-swapping under load

2016-01-28 Thread Xiaobing Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122049#comment-15122049 ] Xiaobing Zhou commented on HDFS-9701: - Thanks, that lock is indeed in the call stack of reconfig. The

[jira] [Updated] (HDFS-9721) Allow Delimited PB OIV tool to run upon fsimage that contains INodeReference

2016-01-28 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-9721: Attachment: HDFS-9721.01.patch > Allow Delimited PB OIV tool to run upon fsimage that contains

[jira] [Updated] (HDFS-9721) Allow Delimited PB OIV tool to run upon fsimage that contains INodeReference

2016-01-28 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-9721: Status: Patch Available (was: Open) > Allow Delimited PB OIV tool to run upon fsimage that contains

[jira] [Updated] (HDFS-7764) DirectoryScanner shouldn't abort the scan if one directory had an error

2016-01-28 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-7764: --- Resolution: Fixed Fix Version/s: 2.9.0 Status: Resolved (was: Patch

[jira] [Commented] (HDFS-9598) Use heap buffer in PacketReceiver If hdfs client has no enough free direct buffer.

2016-01-28 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122915#comment-15122915 ] Colin Patrick McCabe commented on HDFS-9598: It seems like at least the first time this

[jira] [Commented] (HDFS-7764) DirectoryScanner shouldn't abort the scan if one directory had an error

2016-01-28 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122928#comment-15122928 ] Hudson commented on HDFS-7764: -- FAILURE: Integrated in Hadoop-trunk-Commit #9204 (See

[jira] [Commented] (HDFS-7764) DirectoryScanner shouldn't abort the scan if one directory had an error

2016-01-28 Thread Rakesh R (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122947#comment-15122947 ] Rakesh R commented on HDFS-7764: Thank you [~cmccabe] for the good support in resolving this issue! >

[jira] [Commented] (HDFS-9503) Replace -namenode option with -fs for NNThroughputBenchmark

2016-01-28 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122960#comment-15122960 ] Hadoop QA commented on HDFS-9503: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-9198) Coalesce IBR processing in the NN

2016-01-28 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15121517#comment-15121517 ] Kihwal Lee commented on HDFS-9198: -- We have applied this internally to 2.7 and run for quite some time.

[jira] [Updated] (HDFS-9503) Replace -namenode option with -fs for NNThroughputBenchmark

2016-01-28 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9503: Component/s: test > Replace -namenode option with -fs for NNThroughputBenchmark >

[jira] [Updated] (HDFS-9503) Replace -namenode option with -fs for NNThroughputBenchmark

2016-01-28 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-9503: Attachment: HDFS-9053.003.patch Thanks for your kind remind, [~shv]. The v3 patch is the latest one

[jira] [Updated] (HDFS-9721) Allow Delimited PB OIV tool to run upon fsimage that contains INodeReference

2016-01-28 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-9721: Attachment: HDFS-9721.01.patch > Allow Delimited PB OIV tool to run upon fsimage that contains

[jira] [Commented] (HDFS-9721) Allow Delimited PB OIV tool to run upon fsimage that contains INodeReference

2016-01-28 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122865#comment-15122865 ] Xiao Chen commented on HDFS-9721: - I studied the legacy_oiv Delimited tool, XML format OIV tool, and have

[jira] [Commented] (HDFS-9260) Improve performance and GC friendliness of startup and FBRs

2016-01-28 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122898#comment-15122898 ] Colin Patrick McCabe commented on HDFS-9260: Thanks, [~sfriberg]. bq. 7. The reason for

[jira] [Commented] (HDFS-9711) Integrate CSRF prevention filter in WebHDFS.

2016-01-28 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122773#comment-15122773 ] Hadoop QA commented on HDFS-9711: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (HDFS-9721) Allow Delimited PB OIV tool to run upon fsimage that contains INodeReference

2016-01-28 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-9721: Attachment: (was: HDFS-9721.01.patch) > Allow Delimited PB OIV tool to run upon fsimage that contains

[jira] [Commented] (HDFS-9721) Allow Delimited PB OIV tool to run upon fsimage that contains INodeReference

2016-01-28 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122866#comment-15122866 ] Xiao Chen commented on HDFS-9721: - I think at least we need to separate the snapshots and regular paths in

[jira] [Commented] (HDFS-9611) DiskBalancer : Replace htrace json imports with jackson

2016-01-28 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122875#comment-15122875 ] Colin Patrick McCabe commented on HDFS-9611: Ah. Thanks for clarifying that. > DiskBalancer :

[jira] [Commented] (HDFS-7764) DirectoryScanner shouldn't abort the scan if one directory had an error

2016-01-28 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122902#comment-15122902 ] Colin Patrick McCabe commented on HDFS-7764: +1. Thanks, [~rakeshr]. > DirectoryScanner

[jira] [Comment Edited] (HDFS-9718) NameNode#initializeGenericKeys don't unset the property while it is not assign on special nnId

2016-01-28 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15121507#comment-15121507 ] Vinayakumar B edited comment on HDFS-9718 at 1/28/16 2:03 PM: -- Good find

[jira] [Commented] (HDFS-9718) NameNode#initializeGenericKeys don't unset the property while it is not assign on special nnId

2016-01-28 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15121507#comment-15121507 ] Vinayakumar B commented on HDFS-9718: - Good find [~Deng FEI]. But IMO, best place to unset those props

[jira] [Commented] (HDFS-9721) Allow Delimited PB OIV tool to run upon fsimage that contains INodeReference

2016-01-28 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15123000#comment-15123000 ] Hadoop QA commented on HDFS-9721: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||