[jira] [Commented] (HDFS-8562) HDFS Performance is impacted by FileInputStream Finalizer

2015-12-23 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15069459#comment-15069459 ] Hadoop QA commented on HDFS-8562: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Assigned] (HDFS-9466) TestShortCircuitCache#testDataXceiverCleansUpSlotsOnFailure is flaky

2015-12-23 Thread Wei-Chiu Chuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang reassigned HDFS-9466: - Assignee: Wei-Chiu Chuang > TestShortCircuitCache#testDataXceiverCleansUpSlotsOnFailure

[jira] [Commented] (HDFS-9466) TestShortCircuitCache#testDataXceiverCleansUpSlotsOnFailure is flaky

2015-12-23 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15069620#comment-15069620 ] Hadoop QA commented on HDFS-9466: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-9466) TestShortCircuitCache#testDataXceiverCleansUpSlotsOnFailure is flaky

2015-12-23 Thread Wei-Chiu Chuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15069866#comment-15069866 ] Wei-Chiu Chuang commented on HDFS-9466: --- Test failures appear unrelated. I've run the test locally

[jira] [Created] (HDFS-9594) DataNode threw NullPointerException

2015-12-23 Thread Wei-Chiu Chuang (JIRA)
Wei-Chiu Chuang created HDFS-9594: - Summary: DataNode threw NullPointerException Key: HDFS-9594 URL: https://issues.apache.org/jira/browse/HDFS-9594 Project: Hadoop HDFS Issue Type: Bug

[jira] [Commented] (HDFS-9584) NPE in distcp when ssl configuration file does not exist in class path.

2015-12-23 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15069971#comment-15069971 ] Hadoop QA commented on HDFS-9584: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-9466) TestShortCircuitCache#testDataXceiverCleansUpSlotsOnFailure is flaky

2015-12-23 Thread Wei-Chiu Chuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15070010#comment-15070010 ] Wei-Chiu Chuang commented on HDFS-9466: --- Thank you very much for the review and comments! (1) AFAIK,

[jira] [Updated] (HDFS-9596) Remove "Shuffle" Method From DFSUtil

2015-12-23 Thread BELUGA BEHR (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BELUGA BEHR updated HDFS-9596: -- Attachment: Shuffle.HDFS-9596.patch > Remove "Shuffle" Method From DFSUtil >

[jira] [Updated] (HDFS-9596) Remove "Shuffle" Method From DFSUtil

2015-12-23 Thread BELUGA BEHR (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BELUGA BEHR updated HDFS-9596: -- Status: Patch Available (was: Open) > Remove "Shuffle" Method From DFSUtil >

[jira] [Updated] (HDFS-9584) NPE in distcp when ssl configuration file does not exist in class path.

2015-12-23 Thread Surendra Singh Lilhore (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Surendra Singh Lilhore updated HDFS-9584: - Attachment: HDFS-9584.patch Resubmitting patch to trigger jenkins build. > NPE in

[jira] [Created] (HDFS-9596) Remove "Shuffle" Method From DFSUtil

2015-12-23 Thread BELUGA BEHR (JIRA)
BELUGA BEHR created HDFS-9596: - Summary: Remove "Shuffle" Method From DFSUtil Key: HDFS-9596 URL: https://issues.apache.org/jira/browse/HDFS-9596 Project: Hadoop HDFS Issue Type: Improvement

[jira] [Commented] (HDFS-8562) HDFS Performance is impacted by FileInputStream Finalizer

2015-12-23 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15069932#comment-15069932 ] Colin Patrick McCabe commented on HDFS-8562: Thanks, [~drankye]. bq. I'm worrying about a

[jira] [Updated] (HDFS-9559) Add haadmin command to get HA state of all the namenodes

2015-12-23 Thread Surendra Singh Lilhore (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Surendra Singh Lilhore updated HDFS-9559: - Description: Currently we have one command to get state of namenode. {code}

[jira] [Commented] (HDFS-9590) NPE in Storage$StorageDirectory#unlock()

2015-12-23 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15070032#comment-15070032 ] Xiao Chen commented on HDFS-9590: - This is the full log for run (7,1): {noformat}

[jira] [Commented] (HDFS-9466) TestShortCircuitCache#testDataXceiverCleansUpSlotsOnFailure is flaky

2015-12-23 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15069915#comment-15069915 ] Xiao Chen commented on HDFS-9466: - Thanks [~jojochuang] for working on this. The analysis sounds reasonable

[jira] [Commented] (HDFS-9590) NPE in Storage$StorageDirectory#unlock()

2015-12-23 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15070031#comment-15070031 ] Xiao Chen commented on HDFS-9590: - I think this is how the NPE happened. This looks to be a test specific

[jira] [Updated] (HDFS-9590) NPE in Storage$StorageDirectory#unlock()

2015-12-23 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-9590: Attachment: HDFS-9590.01.patch > NPE in Storage$StorageDirectory#unlock() >

[jira] [Created] (HDFS-9595) DiskBalancer : Add cancelPlan RPC

2015-12-23 Thread Anu Engineer (JIRA)
Anu Engineer created HDFS-9595: -- Summary: DiskBalancer : Add cancelPlan RPC Key: HDFS-9595 URL: https://issues.apache.org/jira/browse/HDFS-9595 Project: Hadoop HDFS Issue Type: Sub-task

[jira] [Created] (HDFS-9597) TestReplicationPolicyConsiderLoad#testChooseTargetWithDecomNodes is failing

2015-12-23 Thread Wei-Chiu Chuang (JIRA)
Wei-Chiu Chuang created HDFS-9597: - Summary: TestReplicationPolicyConsiderLoad#testChooseTargetWithDecomNodes is failing Key: HDFS-9597 URL: https://issues.apache.org/jira/browse/HDFS-9597 Project:

[jira] [Commented] (HDFS-9466) TestShortCircuitCache#testDataXceiverCleansUpSlotsOnFailure is flaky

2015-12-23 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15070354#comment-15070354 ] Hadoop QA commented on HDFS-9466: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (HDFS-9597) TestReplicationPolicyConsiderLoad#testChooseTargetWithDecomNodes is failing

2015-12-23 Thread Wei-Chiu Chuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-9597: -- Affects Version/s: 2.8.0 > TestReplicationPolicyConsiderLoad#testChooseTargetWithDecomNodes is

[jira] [Updated] (HDFS-9597) TestReplicationPolicyConsiderLoad#testChooseTargetWithDecomNodes is failing

2015-12-23 Thread Wei-Chiu Chuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-9597: -- Attachment: HDFS-9597.001.patch Rev01: call HeartbeatManager.updateDnStat() to add stats after

[jira] [Updated] (HDFS-9597) TestReplicationPolicyConsiderLoad#testChooseTargetWithDecomNodes is failing

2015-12-23 Thread Wei-Chiu Chuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-9597: -- Status: Patch Available (was: Open) >

[jira] [Assigned] (HDFS-9376) TestSeveralNameNodes fails occasionally

2015-12-23 Thread Masatake Iwasaki (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki reassigned HDFS-9376: -- Assignee: Masatake Iwasaki > TestSeveralNameNodes fails occasionally >

[jira] [Commented] (HDFS-9470) Encryption zone on root not loaded from fsimage after NN restart

2015-12-23 Thread Vinod Kumar Vavilapalli (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15070340#comment-15070340 ] Vinod Kumar Vavilapalli commented on HDFS-9470: --- This originally never made it to

[jira] [Commented] (HDFS-9597) TestReplicationPolicyConsiderLoad#testChooseTargetWithDecomNodes is failing

2015-12-23 Thread Wei-Chiu Chuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15070552#comment-15070552 ] Wei-Chiu Chuang commented on HDFS-9597: --- The test case failed, because after HDFS-9034, the data node

[jira] [Updated] (HDFS-9597) TestReplicationPolicyConsiderLoad#testChooseTargetWithDecomNodes is failing

2015-12-23 Thread Wei-Chiu Chuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-9597: -- Description: Looks like HDFS-9034 broke this test. This test has been failing since yesterday.

[jira] [Updated] (HDFS-9434) Recommission a datanode with 500k blocks may pause NN for 30 seconds

2015-12-23 Thread Vinod Kumar Vavilapalli (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli updated HDFS-9434: -- Fix Version/s: 2.7.2 > Recommission a datanode with 500k blocks may pause NN for

[jira] [Comment Edited] (HDFS-9094) Add command line option to ask NameNode reload configuration.

2015-12-23 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15070362#comment-15070362 ] Arpit Agarwal edited comment on HDFS-9094 at 12/24/15 1:11 AM: --- Hi

[jira] [Commented] (HDFS-9094) Add command line option to ask NameNode reload configuration.

2015-12-23 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15070362#comment-15070362 ] Arpit Agarwal commented on HDFS-9094: - Hi [~xiaobingo], thanks for the updated patch. My comments

[jira] [Commented] (HDFS-9596) Remove "Shuffle" Method From DFSUtil

2015-12-23 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15070211#comment-15070211 ] Hadoop QA commented on HDFS-9596: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (HDFS-9595) DiskBalancer : Add cancelPlan RPC

2015-12-23 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer updated HDFS-9595: --- Attachment: HDFS-9595-HDFS-1312.001.patch Attaching patch for code review, This patch depends on

[jira] [Updated] (HDFS-9466) TestShortCircuitCache#testDataXceiverCleansUpSlotsOnFailure is flaky

2015-12-23 Thread Wei-Chiu Chuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-9466: -- Attachment: HDFS-9466.002.patch Rev02: set a time out of 10 seconds. use

[jira] [Commented] (HDFS-9470) Encryption zone on root not loaded from fsimage after NN restart

2015-12-23 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15070296#comment-15070296 ] Junping Du commented on HDFS-9470: -- Hi [~vinodkv], this should be cherry-pick to latest 2.7.2 RC too. >

[jira] [Commented] (HDFS-9597) TestReplicationPolicyConsiderLoad#testChooseTargetWithDecomNodes is failing

2015-12-23 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15070636#comment-15070636 ] Hadoop QA commented on HDFS-9597: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-8536) HDFS debug command is missed from top level help message

2015-12-23 Thread Daisuke Kobayashi (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15070659#comment-15070659 ] Daisuke Kobayashi commented on HDFS-8536: - This looks like being fixed per HADOOP-10979. > HDFS

[jira] [Commented] (HDFS-6481) DatanodeManager#getDatanodeStorageInfos() should check the length of storageIDs

2015-12-23 Thread edward.xiao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15070598#comment-15070598 ] edward.xiao commented on HDFS-6481: --- Hi Nicholas , Do you have fixed backporting fot this bug. I want

[jira] [Commented] (HDFS-9597) TestReplicationPolicyConsiderLoad#testChooseTargetWithDecomNodes is failing

2015-12-23 Thread Wei-Chiu Chuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15070645#comment-15070645 ] Wei-Chiu Chuang commented on HDFS-9597: --- Failed test are unrelated. The ASF license warning appears

[jira] [Updated] (HDFS-9598) Use heap buffer in PacketReceiver If hdfs client has no enough free direct buffer.

2015-12-23 Thread Yong Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yong Zhang updated HDFS-9598: - Status: Patch Available (was: Open) > Use heap buffer in PacketReceiver If hdfs client has no enough free

[jira] [Commented] (HDFS-9455) In distcp, Invalid Argument Error thrown in case of filesystem operation failure

2015-12-23 Thread Daisuke Kobayashi (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15070613#comment-15070613 ] Daisuke Kobayashi commented on HDFS-9455: - [~archanat], what do you actually mean by 'Filesystem

[jira] [Updated] (HDFS-9597) BaseReplicationPolicyTest should update data node stats after adding a data node

2015-12-23 Thread Wei-Chiu Chuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-9597: -- Summary: BaseReplicationPolicyTest should update data node stats after adding a data node (was:

[jira] [Updated] (HDFS-9597) BaseReplicationPolicyTest should update data node stats after adding a data node

2015-12-23 Thread Wei-Chiu Chuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-9597: -- Description: Looks like HDFS-9034 broke

[jira] [Commented] (HDFS-8562) HDFS Performance is impacted by FileInputStream Finalizer

2015-12-23 Thread Kai Zheng (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15070677#comment-15070677 ] Kai Zheng commented on HDFS-8562: - I thought this may be pending some while for more reviews as it's

[jira] [Commented] (HDFS-6481) DatanodeManager#getDatanodeStorageInfos() should check the length of storageIDs

2015-12-23 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15070729#comment-15070729 ] Tsz Wo Nicholas Sze commented on HDFS-6481: --- > ... Do you have fixed backporting fot this bug.

[jira] [Created] (HDFS-9598) Use heap buffer in PacketReceiver If hdfs client has no enough free direct buffer.

2015-12-23 Thread Yong Zhang (JIRA)
Yong Zhang created HDFS-9598: Summary: Use heap buffer in PacketReceiver If hdfs client has no enough free direct buffer. Key: HDFS-9598 URL: https://issues.apache.org/jira/browse/HDFS-9598 Project:

[jira] [Updated] (HDFS-9598) Use heap buffer in PacketReceiver If hdfs client has no enough free direct buffer.

2015-12-23 Thread Yong Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yong Zhang updated HDFS-9598: - Attachment: HDFS-9598.001.patch Please help to review > Use heap buffer in PacketReceiver If hdfs client

[jira] [Commented] (HDFS-9466) TestShortCircuitCache#testDataXceiverCleansUpSlotsOnFailure is flaky

2015-12-23 Thread Wei-Chiu Chuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15069486#comment-15069486 ] Wei-Chiu Chuang commented on HDFS-9466: --- Assigning this JIRA to me. I found it is due to race

[jira] [Work started] (HDFS-9466) TestShortCircuitCache#testDataXceiverCleansUpSlotsOnFailure is flaky

2015-12-23 Thread Wei-Chiu Chuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-9466 started by Wei-Chiu Chuang. - > TestShortCircuitCache#testDataXceiverCleansUpSlotsOnFailure is flaky >

[jira] [Updated] (HDFS-9466) TestShortCircuitCache#testDataXceiverCleansUpSlotsOnFailure is flaky

2015-12-23 Thread Wei-Chiu Chuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-9466: -- Status: Patch Available (was: In Progress) >

[jira] [Updated] (HDFS-9466) TestShortCircuitCache#testDataXceiverCleansUpSlotsOnFailure is flaky

2015-12-23 Thread Wei-Chiu Chuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-9466: -- Attachment: HDFS-9466.001.patch Rev01: changed checkNumberOfSegmentsAndSlots. Instead of assert