[jira] [Updated] (HDFS-14649) Add suspect probe for DeadNodeDetector

2019-11-25 Thread Lisheng Sun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14649: --- Attachment: HDFS-14649.004.patch > Add suspect probe for DeadNodeDetector >

[jira] [Commented] (HDFS-15013) Reduce NameNode overview tab response time

2019-11-25 Thread Jira
[ https://issues.apache.org/jira/browse/HDFS-15013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16982200#comment-16982200 ] Íñigo Goiri commented on HDFS-15013: RBF has the same code. Could update federationhealth.js too? >

[jira] [Commented] (HDFS-14998) Update Observer Namenode doc for ZKFC after HDFS-14130

2019-11-25 Thread Hadoop QA (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16982198#comment-16982198 ] Hadoop QA commented on HDFS-14998: -- | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem

[jira] [Commented] (HDFS-14998) Update Observer Namenode doc for ZKFC after HDFS-14130

2019-11-25 Thread Fei Hui (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16982157#comment-16982157 ] Fei Hui commented on HDFS-14998: Upload v003. [~csun]Could you please take a look? Thanks. > Update

[jira] [Updated] (HDFS-14998) Update Observer Namenode doc for ZKFC after HDFS-14130

2019-11-25 Thread Fei Hui (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fei Hui updated HDFS-14998: --- Attachment: HDFS-14998.003.patch > Update Observer Namenode doc for ZKFC after HDFS-14130 >

[jira] [Commented] (HDFS-14908) LeaseManager should check parent-child relationship when filter open files.

2019-11-25 Thread Hadoop QA (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16982130#comment-16982130 ] Hadoop QA commented on HDFS-14908: -- | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem

[jira] [Commented] (HDFS-14901) RBF: Add Encryption Zone related ClientProtocol APIs

2019-11-25 Thread hemanthboyina (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16982119#comment-16982119 ] hemanthboyina commented on HDFS-14901: -- thanks for the review and suggestion [~ayushtkn] [~elgoiri]

[jira] [Comment Edited] (HDFS-15012) NN fails to parse Edit logs after applying HDFS-13101

2019-11-25 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16982106#comment-16982106 ] Wei-Chiu Chuang edited comment on HDFS-15012 at 11/26/19 3:26 AM: --

[jira] [Commented] (HDFS-15012) NN fails to parse Edit logs after applying HDFS-13101

2019-11-25 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16982106#comment-16982106 ] Wei-Chiu Chuang commented on HDFS-15012: We recently received several reports regarding corrupt

[jira] [Commented] (HDFS-15009) FSCK "-list-corruptfileblocks" return Invalid Entries

2019-11-25 Thread Ayush Saxena (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16982100#comment-16982100 ] Ayush Saxena commented on HDFS-15009: - I am not sure {{FederationUtil#isParentEntry}} can be reused

[jira] [Commented] (HDFS-15013) Reduce NameNode overview tab response time

2019-11-25 Thread Hadoop QA (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16982096#comment-16982096 ] Hadoop QA commented on HDFS-15013: -- | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem

[jira] [Updated] (HDFS-15012) NN fails to parse Edit logs after applying HDFS-13101

2019-11-25 Thread Ayush Saxena (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated HDFS-15012: Priority: Critical (was: Major) > NN fails to parse Edit logs after applying HDFS-13101 >

[jira] [Commented] (HDFS-15012) NN fails to parse Edit logs after applying HDFS-13101

2019-11-25 Thread Ayush Saxena (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16982090#comment-16982090 ] Ayush Saxena commented on HDFS-15012: - Thanx [~ericlin] for the report. Do you propose a fix for

[jira] [Commented] (HDFS-15013) Reduce NameNode overview tab response time

2019-11-25 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16982089#comment-16982089 ] HuangTao commented on HDFS-15013: - [~surendrasingh] PTAL > Reduce NameNode overview tab response time >

[jira] [Updated] (HDFS-14908) LeaseManager should check parent-child relationship when filter open files.

2019-11-25 Thread Jinglun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jinglun updated HDFS-14908: --- Attachment: HDFS-14908.006.patch > LeaseManager should check parent-child relationship when filter open

[jira] [Commented] (HDFS-14908) LeaseManager should check parent-child relationship when filter open files.

2019-11-25 Thread Jinglun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16982073#comment-16982073 ] Jinglun commented on HDFS-14908: Rebase and upload v06. > LeaseManager should check parent-child

[jira] [Updated] (HDFS-15013) Reduce NameNode overview tab response time

2019-11-25 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] HuangTao updated HDFS-15013: Attachment: HDFS-15013.001.patch Status: Patch Available (was: Open) > Reduce NameNode overview

[jira] [Commented] (HDFS-14973) Balancer getBlocks RPC dispersal does not function properly

2019-11-25 Thread Konstantin Shvachko (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16982062#comment-16982062 ] Konstantin Shvachko commented on HDFS-14973: Erik, the implementation of {{RateLimiter}}

[jira] [Updated] (HDFS-15013) Reduce NameNode overview tab response time

2019-11-25 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] HuangTao updated HDFS-15013: Description: Now, the overview tab load /conf synchronously as follow picture.

[jira] [Updated] (HDFS-15013) Reduce NameNode overview tab response time

2019-11-25 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] HuangTao updated HDFS-15013: Description: Now, the overview tab load /conf synchronously as follow picture.

[jira] [Created] (HDFS-15013) Reduce NameNode overview tab response time

2019-11-25 Thread HuangTao (Jira)
HuangTao created HDFS-15013: --- Summary: Reduce NameNode overview tab response time Key: HDFS-15013 URL: https://issues.apache.org/jira/browse/HDFS-15013 Project: Hadoop HDFS Issue Type: Improvement

[jira] [Comment Edited] (HDFS-14963) Add HDFS Client machine caching active namenode index mechanism.

2019-11-25 Thread Xudong Cao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16982041#comment-16982041 ] Xudong Cao edited comment on HDFS-14963 at 11/26/19 1:34 AM: - cc [~csun] At 

[jira] [Comment Edited] (HDFS-14963) Add HDFS Client machine caching active namenode index mechanism.

2019-11-25 Thread Xudong Cao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16982041#comment-16982041 ] Xudong Cao edited comment on HDFS-14963 at 11/26/19 1:34 AM: - cc [~csun] At 

[jira] [Commented] (HDFS-14963) Add HDFS Client machine caching active namenode index mechanism.

2019-11-25 Thread Xudong Cao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16982041#comment-16982041 ] Xudong Cao commented on HDFS-14963: --- cc [~csun] At the beginning, the main goal of this jira was to

[jira] [Commented] (HDFS-14998) Update Observer Namenode doc for ZKFC after HDFS-14130

2019-11-25 Thread Fei Hui (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16982028#comment-16982028 ] Fei Hui commented on HDFS-14998: [~csun] Thanks. Try to add more details for this > Update Observer

[jira] [Commented] (HDFS-14998) Update Observer Namenode doc for ZKFC after HDFS-14130

2019-11-25 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16982017#comment-16982017 ] Chao Sun commented on HDFS-14998: - [~ferhui] Yes that's correct. I'm thinking whether we can add more

[jira] [Commented] (HDFS-14998) Update Observer Namenode doc for ZKFC after HDFS-14130

2019-11-25 Thread Fei Hui (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16982008#comment-16982008 ] Fei Hui commented on HDFS-14998: [~csun]Thanks for your comments. I think the main motivation for

[jira] [Updated] (HDFS-15012) NN fails to parse Edit logs after applying HDFS-13101

2019-11-25 Thread Eric Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Lin updated HDFS-15012: Description: After applying HDFS-13101, and deleting and creating large number of snapshots, SNN exited

[jira] [Created] (HDFS-15012) NN fails to parse Edit logs after applying HDFS-13101

2019-11-25 Thread Eric Lin (Jira)
Eric Lin created HDFS-15012: --- Summary: NN fails to parse Edit logs after applying HDFS-13101 Key: HDFS-15012 URL: https://issues.apache.org/jira/browse/HDFS-15012 Project: Hadoop HDFS Issue Type:

[jira] [Commented] (HDFS-14901) RBF: Add Encryption Zone related ClientProtocol APIs

2019-11-25 Thread Ayush Saxena (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16981956#comment-16981956 ] Ayush Saxena commented on HDFS-14901: - I feel, HDFS-14091 can be handled here, its very related. Yes,

[jira] [Commented] (HDFS-14908) LeaseManager should check parent-child relationship when filter open files.

2019-11-25 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16981911#comment-16981911 ] Wei-Chiu Chuang commented on HDFS-14908: Sounds good to me. Thanks doing the extra check >

[jira] [Commented] (HDFS-6874) Add GETFILEBLOCKLOCATIONS operation to HttpFS

2019-11-25 Thread Jira
[ https://issues.apache.org/jira/browse/HDFS-6874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16981922#comment-16981922 ] Íñigo Goiri commented on HDFS-6874: --- It looks we are still stuck with HDFS-11156 and HDFS-13351. But I

[jira] [Updated] (HDFS-15010) BlockPoolSlice#addReplicaThreadPool static pool should be initialized by static method

2019-11-25 Thread Jira
[ https://issues.apache.org/jira/browse/HDFS-15010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Íñigo Goiri updated HDFS-15010: --- Description: {{BlockPoolSlice#initializeAddReplicaPool()}} method currently initialize the static

[jira] [Commented] (HDFS-15010) BlockPoolSlice#addReplicaThreadPool static pool should be initialized by static method

2019-11-25 Thread Jira
[ https://issues.apache.org/jira/browse/HDFS-15010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16981918#comment-16981918 ] Íñigo Goiri commented on HDFS-15010: Is there an easy test we can add to verify that two parallel

[jira] [Commented] (HDFS-14901) RBF: Add Encryption Zone related ClientProtocol APIs

2019-11-25 Thread Jira
[ https://issues.apache.org/jira/browse/HDFS-14901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16981919#comment-16981919 ] Íñigo Goiri commented on HDFS-14901: Can we also take care of HDFS-14091 or do you think is worth a

[jira] [Commented] (HDFS-14908) LeaseManager should check parent-child relationship when filter open files.

2019-11-25 Thread Jira
[ https://issues.apache.org/jira/browse/HDFS-14908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16981902#comment-16981902 ] Íñigo Goiri commented on HDFS-14908: It looks like [^HDFS-14908.005.patch] is better. Unless

[jira] [Commented] (HDFS-14649) Add suspect probe for DeadNodeDetector

2019-11-25 Thread Hadoop QA (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16981894#comment-16981894 ] Hadoop QA commented on HDFS-14649: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-14998) Update Observer Namenode doc for ZKFC after HDFS-14130

2019-11-25 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16981885#comment-16981885 ] Chao Sun commented on HDFS-14998: - I think the main motivation for supporting observer in ZKFC is that

[jira] [Commented] (HDFS-14961) [SBN read] Prevent ZKFC changing Observer Namenode state

2019-11-25 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16981879#comment-16981879 ] Chao Sun commented on HDFS-14961: - Thanks [~ayushtkn]. +1 on this from me (non-binding). > [SBN read]

[jira] [Updated] (HDFS-14649) Add suspect probe for DeadNodeDetector

2019-11-25 Thread Lisheng Sun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14649: --- Attachment: HDFS-14649.003.patch > Add suspect probe for DeadNodeDetector >

[jira] [Comment Edited] (HDFS-14668) Support Fuse with Users from multiple Security Realms

2019-11-25 Thread Siddharth Wagle (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16979471#comment-16979471 ] Siddharth Wagle edited comment on HDFS-14668 at 11/25/19 5:39 PM: -- +

[jira] [Comment Edited] (HDFS-14649) Add suspect probe for DeadNodeDetector

2019-11-25 Thread Yiqun Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16981612#comment-16981612 ] Yiqun Lin edited comment on HDFS-14649 at 11/25/19 2:41 PM: [~leosun08], we

[jira] [Commented] (HDFS-14649) Add suspect probe for DeadNodeDetector

2019-11-25 Thread Yiqun Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16981612#comment-16981612 ] Yiqun Lin commented on HDFS-14649: -- [~leosun08], we can add a corner case that the node is added in

[jira] [Commented] (HDFS-15009) FSCK "-list-corruptfileblocks" return Invalid Entries

2019-11-25 Thread Fei Hui (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16981506#comment-16981506 ] Fei Hui commented on HDFS-15009: [~hemanthboyina] Thanks. The method isParentEntry is the same as

[jira] [Commented] (HDFS-14901) RBF: Add Encryption Zone related ClientProtocol APIs

2019-11-25 Thread hemanthboyina (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16981491#comment-16981491 ] hemanthboyina commented on HDFS-14901: -- attached patch , please review [~ayushtkn] [~elgoiri] >

[jira] [Commented] (HDFS-15009) FSCK "-list-corruptfileblocks" return Invalid Entries

2019-11-25 Thread hemanthboyina (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16981490#comment-16981490 ] hemanthboyina commented on HDFS-15009: -- attached patch , please review [~ayushtkn] [~ferhui] > FSCK

[jira] [Commented] (HDFS-14649) Add suspect probe for DeadNodeDetector

2019-11-25 Thread Lisheng Sun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16981439#comment-16981439 ] Lisheng Sun commented on HDFS-14649: hi [~linyiqun] {quote} We need a additional unit test for

[jira] [Commented] (HDFS-14546) Document block placement policies

2019-11-25 Thread Hadoop QA (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16981437#comment-16981437 ] Hadoop QA commented on HDFS-14546: -- | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem

[jira] [Updated] (HDFS-14546) Document block placement policies

2019-11-25 Thread Amithsha (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amithsha updated HDFS-14546: Attachment: HDFS-14546-07.patch > Document block placement policies > - >

[jira] [Commented] (HDFS-14546) Document block placement policies

2019-11-25 Thread Amithsha (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16981424#comment-16981424 ] Amithsha commented on HDFS-14546: - [~ayushtkn] [~elgoiri] Changes done. > Document block placement

[jira] [Updated] (HDFS-14649) Add suspect probe for DeadNodeDetector

2019-11-25 Thread Lisheng Sun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-14649: --- Description: Add suspect probe for DeadNodeDetector. when  some DataNode of the block is found to

[jira] [Comment Edited] (HDFS-14908) LeaseManager should check parent-child relationship when filter open files.

2019-11-25 Thread Jinglun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16981409#comment-16981409 ] Jinglun edited comment on HDFS-14908 at 11/25/19 9:27 AM: -- Hi [~elgoiri]

[jira] [Commented] (HDFS-14908) LeaseManager should check parent-child relationship when filter open files.

2019-11-25 Thread Jinglun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16981409#comment-16981409 ] Jinglun commented on HDFS-14908: Hi [~elgoiri] [~weichiu], based on the result DFSUtil.isParent() is much

[jira] [Commented] (HDFS-13811) RBF: Race condition between router admin quota update and periodic quota update service

2019-11-25 Thread Jinglun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-13811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16981402#comment-16981402 ] Jinglun commented on HDFS-13811: Hi [~linyiqun], just a minor reminder of this jira. Do you have time to

[jira] [Commented] (HDFS-14568) setStoragePolicy should check quota and update consume on storage type quota.

2019-11-25 Thread Jinglun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16981403#comment-16981403 ] Jinglun commented on HDFS-14568: Hi [~xyao] , just a minor reminder of this jira. Do you have time for