[jira] [Created] (HDFS-11039) Expose more configuration properties to hdfs-default.xml

2016-10-19 Thread Yi Liu (JIRA)
Yi Liu created HDFS-11039: - Summary: Expose more configuration properties to hdfs-default.xml Key: HDFS-11039 URL: https://issues.apache.org/jira/browse/HDFS-11039 Project: Hadoop HDFS Issue Type:

[jira] [Updated] (HDFS-11020) Add more doc for HDFS transparent encryption

2016-10-17 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-11020: -- Assignee: Jennica Pounds (was: Yi Liu) > Add more doc for HDFS transparent encryption >

[jira] [Updated] (HDFS-11020) Add more doc for HDFS transparent encryption

2016-10-17 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-11020: -- Summary: Add more doc for HDFS transparent encryption (was: Enable HDFS transparent encryption doc) > Add

[jira] [Created] (HDFS-11020) Enable HDFS transparent encryption doc

2016-10-17 Thread Yi Liu (JIRA)
Yi Liu created HDFS-11020: - Summary: Enable HDFS transparent encryption doc Key: HDFS-11020 URL: https://issues.apache.org/jira/browse/HDFS-11020 Project: Hadoop HDFS Issue Type: Task

[jira] [Commented] (HDFS-9276) Failed to Update HDFS Delegation Token for long running application in HA mode

2016-04-27 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15261467#comment-15261467 ] Yi Liu commented on HDFS-9276: -- Not only one people tell me they encounter this issue in real cluster, and ask

[jira] [Updated] (HDFS-8786) Erasure coding: DataNode should transfer striped blocks before being decommissioned

2016-02-14 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-8786: - Assignee: Rakesh R (was: Yi Liu) > Erasure coding: DataNode should transfer striped blocks before being >

[jira] [Commented] (HDFS-9713) DataXceiver#copyBlock should return if block is pinned

2016-02-04 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15133453#comment-15133453 ] Yi Liu commented on HDFS-9713: -- +1, good catch. Thanks Uma. > DataXceiver#copyBlock should return if block is

[jira] [Commented] (HDFS-9582) TestLeaseRecoveryStriped file missing Apache License header and not well formatted

2015-12-19 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15065602#comment-15065602 ] Yi Liu commented on HDFS-9582: -- +1 pending Jenkins. Thanks Uma. > TestLeaseRecoveryStriped file missing

[jira] [Issue Comment Deleted] (HDFS-8562) HDFS Performance is impacted by FileInputStream Finalizer

2015-12-13 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-8562: - Comment: was deleted (was: GitHub user hash-X opened a pull request: https://github.com/apache/hadoop/pull/42

[jira] [Commented] (HDFS-9276) Failed to Update HDFS Delegation Token for long running application in HA mode

2015-11-06 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14993370#comment-14993370 ] Yi Liu commented on HDFS-9276: -- In the test, you get the token1/token2 for current login user, suppose it's

[jira] [Commented] (HDFS-4937) ReplicationMonitor can infinite-loop in BlockPlacementPolicyDefault#chooseRandom()

2015-11-05 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14991615#comment-14991615 ] Yi Liu commented on HDFS-4937: -- This time, it's correct now. The logic of current patch is straight. +1 for

[jira] [Commented] (HDFS-9276) Failed to Update HDFS Delegation Token for long running application in HA mode

2015-11-05 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14991686#comment-14991686 ] Yi Liu commented on HDFS-9276: -- Yes, even the security is not enabled, but if there is token in user's

[jira] [Commented] (HDFS-9276) Failed to Update HDFS Delegation Token for long running application in HA mode

2015-11-02 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14986402#comment-14986402 ] Yi Liu commented on HDFS-9276: -- Liangliang, thanks for the update. See following comments. (I forgot to add

[jira] [Commented] (HDFS-9275) Wait previous ErasureCodingWork to finish before schedule another one

2015-11-02 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14986424#comment-14986424 ] Yi Liu commented on HDFS-9275: -- Looks good now, +1. > Wait previous ErasureCodingWork to finish before

[jira] [Updated] (HDFS-9275) Wait previous ErasureCodingWork to finish before schedule another one

2015-11-02 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-9275: - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0 Status: Resolved (was: Patch

[jira] [Commented] (HDFS-9275) Wait previous ErasureCodingWork to finish before schedule another one

2015-11-02 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14986431#comment-14986431 ] Yi Liu commented on HDFS-9275: -- Will commit shortly. > Wait previous ErasureCodingWork to finish before

[jira] [Commented] (HDFS-9348) DFS GetErasureCodingPolicy API on a non-existent file should be handled properly

2015-11-02 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14986476#comment-14986476 ] Yi Liu commented on HDFS-9348: -- Thanks Uma for ping me, and Rakesh for the work. I think it makes sense to

[jira] [Comment Edited] (HDFS-9348) DFS GetErasureCodingPolicy API on a non-existent file should be handled properly

2015-11-02 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14986476#comment-14986476 ] Yi Liu edited comment on HDFS-9348 at 11/3/15 1:41 AM: --- Thanks Uma for ping me, and

[jira] [Comment Edited] (HDFS-4937) ReplicationMonitor can infinite-loop in BlockPlacementPolicyDefault#chooseRandom()

2015-10-31 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14983905#comment-14983905 ] Yi Liu edited comment on HDFS-4937 at 10/31/15 8:46 AM: Revert from trunk,

[jira] [Reopened] (HDFS-4937) ReplicationMonitor can infinite-loop in BlockPlacementPolicyDefault#chooseRandom()

2015-10-31 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu reopened HDFS-4937: -- > ReplicationMonitor can infinite-loop in > BlockPlacementPolicyDefault#chooseRandom() >

[jira] [Commented] (HDFS-4937) ReplicationMonitor can infinite-loop in BlockPlacementPolicyDefault#chooseRandom()

2015-10-31 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14983905#comment-14983905 ] Yi Liu commented on HDFS-4937: -- Revert from trunk, branch-2, branch-2.7. Thanks Brahma. I thought the tests

[jira] [Comment Edited] (HDFS-4937) ReplicationMonitor can infinite-loop in BlockPlacementPolicyDefault#chooseRandom()

2015-10-31 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14983906#comment-14983906 ] Yi Liu edited comment on HDFS-4937 at 10/31/15 8:45 AM: I did consider the

[jira] [Commented] (HDFS-4937) ReplicationMonitor can infinite-loop in BlockPlacementPolicyDefault#chooseRandom()

2015-10-31 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14983906#comment-14983906 ] Yi Liu commented on HDFS-4937: -- I did consider the situation you mentioned, But I thought in real env the NN

[jira] [Commented] (HDFS-4937) ReplicationMonitor can infinite-loop in BlockPlacementPolicyDefault#chooseRandom()

2015-10-29 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14981789#comment-14981789 ] Yi Liu commented on HDFS-4937: -- +1, thanks Kihwal. > ReplicationMonitor can infinite-loop in >

[jira] [Commented] (HDFS-9276) Failed to Update HDFS Delegation Token for long running application in HA mode

2015-10-29 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14981710#comment-14981710 ] Yi Liu commented on HDFS-9276: -- *1.* The code format is not correct. The indent is 2 spaces. If some line

[jira] [Commented] (HDFS-9275) Wait previous ErasureCodingWork to finish before schedule another one

2015-10-29 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14981734#comment-14981734 ] Yi Liu commented on HDFS-9275: -- {code} + if (pendingNum > 0) { +// Wait the previous recovery to

[jira] [Commented] (HDFS-9302) WebHDFS throws NullPointerException if newLength is not provided

2015-10-28 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14977821#comment-14977821 ] Yi Liu commented on HDFS-9302: -- OK, thanks Jagadesh, Akira, will commit shortly. > WebHDFS throws

[jira] [Updated] (HDFS-9302) WebHDFS throws NullPointerException if newLength is not provided

2015-10-28 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-9302: - Target Version/s: 2.8.0 (was: 2.8.0, 2.7.2) > WebHDFS throws NullPointerException if newLength is not provided >

[jira] [Updated] (HDFS-9302) WebHDFS throws NullPointerException if newLength is not provided

2015-10-28 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-9302: - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.0 Status: Resolved (was: Patch

[jira] [Commented] (HDFS-9275) Fix TestRecoverStripedFile

2015-10-27 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14976233#comment-14976233 ] Yi Liu commented on HDFS-9275: -- Walter, sorry I forgot this JIRA :-) For continuous block, if n replicas are

[jira] [Commented] (HDFS-9302) WebHDFS throws NullPointerException if newLength is not provided

2015-10-27 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14977483#comment-14977483 ] Yi Liu commented on HDFS-9302: -- Looks good, Jagadesh. Just a small nit, +1 after addressing - please put the

[jira] [Commented] (HDFS-9276) Failed to Update HDFS Delegation Token for long running application in HA mode

2015-10-27 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14976479#comment-14976479 ] Yi Liu commented on HDFS-9276: -- Now looks good overall. *1.* {code} + + /** + * Create a private token for

[jira] [Comment Edited] (HDFS-9276) Failed to Update HDFS Delegation Token for long running application in HA mode

2015-10-27 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14976479#comment-14976479 ] Yi Liu edited comment on HDFS-9276 at 10/27/15 2:31 PM: Now looks good overall.

[jira] [Comment Edited] (HDFS-7984) webhdfs:// needs to support provided delegation tokens

2015-10-26 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973802#comment-14973802 ] Yi Liu edited comment on HDFS-7984 at 10/26/15 6:27 AM: {quote} there is no way for

[jira] [Commented] (HDFS-7984) webhdfs:// needs to support provided delegation tokens

2015-10-26 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973802#comment-14973802 ] Yi Liu commented on HDFS-7984: -- {quote} there is no way for an end user to create a token file with multiple

[jira] [Commented] (HDFS-9276) Failed to Update HDFS Delegation Token for long running application in HA mode

2015-10-26 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14974277#comment-14974277 ] Yi Liu commented on HDFS-9276: -- So it's the problem of service name of token for HA, there are three entries

[jira] [Reopened] (HDFS-9293) FSEditLog's 'OpInstanceCache' instance of threadLocal cache exists dirty 'rpcId',which may cause standby NN too busy to communicate

2015-10-25 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu reopened HDFS-9293: -- > FSEditLog's 'OpInstanceCache' instance of threadLocal cache exists dirty > 'rpcId',which may cause standby NN

[jira] [Resolved] (HDFS-9293) FSEditLog's 'OpInstanceCache' instance of threadLocal cache exists dirty 'rpcId',which may cause standby NN too busy to communicate

2015-10-25 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu resolved HDFS-9293. -- Resolution: Duplicate > FSEditLog's 'OpInstanceCache' instance of threadLocal cache exists dirty >

[jira] [Commented] (HDFS-9276) Failed to Update HDFS Delegation Token for long running application in HA mode

2015-10-25 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973758#comment-14973758 ] Yi Liu commented on HDFS-9276: -- {quote} To reproduce the bug, please set the following configuration to Name

[jira] [Comment Edited] (HDFS-9276) Failed to Update HDFS Delegation Token for long running application in HA mode

2015-10-25 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973758#comment-14973758 ] Yi Liu edited comment on HDFS-9276 at 10/26/15 5:39 AM: {quote} To reproduce the

[jira] [Comment Edited] (HDFS-9276) Failed to Update HDFS Delegation Token for long running application in HA mode

2015-10-25 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973758#comment-14973758 ] Yi Liu edited comment on HDFS-9276 at 10/26/15 5:41 AM: {quote} To reproduce the

[jira] [Commented] (HDFS-9276) Failed to Update HDFS Delegation Token for long running application in HA mode

2015-10-25 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973249#comment-14973249 ] Yi Liu commented on HDFS-9276: -- [~marsishandsome], Agree with Steve you need to move this to Hadoop common if

[jira] [Comment Edited] (HDFS-9276) Failed to Update HDFS Delegation Token for long running application in HA mode

2015-10-25 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973249#comment-14973249 ] Yi Liu edited comment on HDFS-9276 at 10/25/15 1:57 PM: [~marsishandsome], Agree

[jira] [Commented] (HDFS-7984) webhdfs:// needs to support provided delegation tokens

2015-10-25 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973256#comment-14973256 ] Yi Liu commented on HDFS-7984: -- {quote} String fileLocation = System.getenv(HADOOP_TOKEN_FILE_LOCATION); ...

[jira] [Commented] (HDFS-9302) WebHDFS throws NullPointerException if newLength is not provided

2015-10-24 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973001#comment-14973001 ] Yi Liu commented on HDFS-9302: -- [~Karthik Palaniappan], thanks for reporting this, it's better to return a

[jira] [Commented] (HDFS-9053) Support large directories efficiently using B-Tree

2015-10-22 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14968754#comment-14968754 ] Yi Liu commented on HDFS-9053: -- Thanks for the discussion, Nicholas. {quote} BTW, why do we need two arrays

[jira] [Commented] (HDFS-9275) Fix TestRecoverStripedFile

2015-10-22 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14968797#comment-14968797 ] Yi Liu commented on HDFS-9275: -- Walter, please add some description in the JIRA description part about what

[jira] [Commented] (HDFS-7087) Ability to list /.reserved

2015-10-22 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14970062#comment-14970062 ] Yi Liu commented on HDFS-7087: -- I think there is no problem, thanks Andrew, the cherry-pick shows no

[jira] [Commented] (HDFS-9275) Fix TestRecoverStripedFile

2015-10-22 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14970093#comment-14970093 ] Yi Liu commented on HDFS-9275: -- Thanks Walter for the update, I will review it tomorrow since I'm OOO today if

[jira] [Commented] (HDFS-9053) Support large directories efficiently using B-Tree

2015-10-22 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14969206#comment-14969206 ] Yi Liu commented on HDFS-9053: -- {quote} The advantages of having one array are 1) smaller memory footprint and

[jira] [Commented] (HDFS-9053) Support large directories efficiently using B-Tree

2015-10-22 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14969202#comment-14969202 ] Yi Liu commented on HDFS-9053: -- {quote} Could you give some examples? {quote} https://github.com/google/btree

[jira] [Commented] (HDFS-9053) Support large directories efficiently using B-Tree

2015-10-21 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14968277#comment-14968277 ] Yi Liu commented on HDFS-9053: -- Thanks [~jingzhao]! {quote} My concern is if the number of directories is

[jira] [Comment Edited] (HDFS-9053) Support large directories efficiently using B-Tree

2015-10-21 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14968277#comment-14968277 ] Yi Liu edited comment on HDFS-9053 at 10/22/15 12:51 AM: - Thanks [~jingzhao]!

[jira] [Commented] (HDFS-7087) Ability to list /.reserved

2015-10-21 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14968396#comment-14968396 ] Yi Liu commented on HDFS-7087: -- That's right. I will help to revert it from branch-2 and re-commit it shortly.

[jira] [Updated] (HDFS-7087) Ability to list /.reserved

2015-10-21 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-7087: - Attachment: HDFS-7087.003.branch-2.patch recommit to branch-2, and attach the patch of branch-2 for track. >

[jira] [Commented] (HDFS-9053) Support large directories efficiently using B-Tree

2015-10-21 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14968445#comment-14968445 ] Yi Liu commented on HDFS-9053: -- Got it now. Thanks a lot [~jingzhao]! Let's recalculate again. For original

[jira] [Comment Edited] (HDFS-9053) Support large directories efficiently using B-Tree

2015-10-21 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14968445#comment-14968445 ] Yi Liu edited comment on HDFS-9053 at 10/22/15 3:34 AM: Got it now. Thanks a lot

[jira] [Commented] (HDFS-9053) Support large directories efficiently using B-Tree

2015-10-20 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14964775#comment-14964775 ] Yi Liu commented on HDFS-9053: -- The test failures are not related. I plan to support {{shrinkable}} in future

[jira] [Created] (HDFS-9274) Default value of dfs.datanode.directoryscan.throttle.limit.ms.per.sec should be consistent

2015-10-20 Thread Yi Liu (JIRA)
Yi Liu created HDFS-9274: Summary: Default value of dfs.datanode.directoryscan.throttle.limit.ms.per.sec should be consistent Key: HDFS-9274 URL: https://issues.apache.org/jira/browse/HDFS-9274 Project:

[jira] [Updated] (HDFS-9274) Default value of dfs.datanode.directoryscan.throttle.limit.ms.per.sec should be consistent

2015-10-20 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-9274: - Attachment: HDFS-9274.001.patch > Default value of dfs.datanode.directoryscan.throttle.limit.ms.per.sec should >

[jira] [Updated] (HDFS-9274) Default value of dfs.datanode.directoryscan.throttle.limit.ms.per.sec should be consistent

2015-10-20 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-9274: - Status: Patch Available (was: Open) > Default value of dfs.datanode.directoryscan.throttle.limit.ms.per.sec

[jira] [Updated] (HDFS-9053) Support large directories efficiently using B-Tree

2015-10-19 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-9053: - Attachment: HDFS-9053.007.patch > Support large directories efficiently using B-Tree >

[jira] [Commented] (HDFS-9053) Support large directories efficiently using B-Tree

2015-10-19 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14964433#comment-14964433 ] Yi Liu commented on HDFS-9053: -- Update the patch: # Fix some checkstyles and the whitespace. # Update few

[jira] [Updated] (HDFS-9208) Disabling atime may fail clients like distCp

2015-10-19 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-9208: - Resolution: Fixed Fix Version/s: 2.8.0 Status: Resolved (was: Patch Available) Committed to

[jira] [Commented] (HDFS-7964) Add support for async edit logging

2015-10-18 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14962767#comment-14962767 ] Yi Liu commented on HDFS-7964: -- 1. That's right. 2. You can keep it. > Add support for async edit logging >

[jira] [Resolved] (HDFS-8398) Erasure Coding: Correctly calculate last striped block length in DFSStripedInputStream if it's under construction.

2015-10-16 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu resolved HDFS-8398. -- Resolution: Duplicate > Erasure Coding: Correctly calculate last striped block length in >

[jira] [Resolved] (HDFS-8059) Erasure coding: revisit how to store EC schema and cellSize in NameNode

2015-10-16 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu resolved HDFS-8059. -- Resolution: Duplicate > Erasure coding: revisit how to store EC schema and cellSize in NameNode >

[jira] [Comment Edited] (HDFS-7964) Add support for async edit logging

2015-10-16 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14960114#comment-14960114 ] Yi Liu edited comment on HDFS-7964 at 10/16/15 6:40 AM: Thanks [~daryn] for the

[jira] [Resolved] (HDFS-8087) Erasure Coding: Add more EC zone management APIs (get/list EC zone(s))

2015-10-16 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu resolved HDFS-8087. -- Resolution: Invalid Now, getErasureCodingPolicies and getErasureCodingPolicy are implemented, so close it as

[jira] [Resolved] (HDFS-8376) Erasure Coding: Update last cellsize calculation according to whether the erasure codec has chunk boundary

2015-10-16 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu resolved HDFS-8376. -- Resolution: Invalid > Erasure Coding: Update last cellsize calculation according to whether the > erasure codec

[jira] [Comment Edited] (HDFS-9053) Support large directories efficiently using B-Tree

2015-10-15 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14959955#comment-14959955 ] Yi Liu edited comment on HDFS-9053 at 10/16/15 12:56 AM: - Thanks [~szetszwo] for

[jira] [Updated] (HDFS-7964) Add support for async edit logging

2015-10-15 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-7964: - Status: Patch Available (was: Open) > Add support for async edit logging > -- > >

[jira] [Commented] (HDFS-7964) Add support for async edit logging

2015-10-15 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14960114#comment-14960114 ] Yi Liu commented on HDFS-7964: -- Thanks [~daryn] for the work. Further comments: *1.* In FSEditLogAsync#run

[jira] [Commented] (HDFS-9053) Support large directories efficiently using B-Tree

2015-10-15 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14959988#comment-14959988 ] Yi Liu commented on HDFS-9053: -- Hi Nicholas, sorry, I may bypass some description here. For 2047, I wanted to

[jira] [Commented] (HDFS-9053) Support large directories efficiently using B-Tree

2015-10-15 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14959955#comment-14959955 ] Yi Liu commented on HDFS-9053: -- Thanks [~szetszwo] for the comments. {quote} >> 24 8 Object[]

[jira] [Updated] (HDFS-9221) HdfsServerConstants#ReplicaState#getState should avoid calling values() since it creates a temporary array

2015-10-12 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-9221: - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.0 Status: Resolved (was: Patch

[jira] [Updated] (HDFS-8988) Use LightWeightHashSet instead of LightWeightLinkedSet in BlockManager#excessReplicateMap

2015-10-12 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-8988: - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.0 Status: Resolved (was: Patch

[jira] [Commented] (HDFS-8988) Use LightWeightHashSet instead of LightWeightLinkedSet in BlockManager#excessReplicateMap

2015-10-12 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14952724#comment-14952724 ] Yi Liu commented on HDFS-8988: -- Thanks [~umamaheswararao] for the review, the test failure are not related and

[jira] [Issue Comment Deleted] (HDFS-9053) Support large directories efficiently using B-Tree

2015-10-10 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-9053: - Comment: was deleted (was: \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime ||

[jira] [Updated] (HDFS-9053) Support large directories efficiently using B-Tree

2015-10-09 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-9053: - Attachment: HDFS-9053.005.patch > Support large directories efficiently using B-Tree >

[jira] [Comment Edited] (HDFS-9053) Support large directories efficiently using B-Tree

2015-10-09 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14949847#comment-14949847 ] Yi Liu edited comment on HDFS-9053 at 10/9/15 8:49 AM: --- Hi [~szetszwo], thanks for

[jira] [Commented] (HDFS-9053) Support large directories efficiently using B-Tree

2015-10-09 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14950082#comment-14950082 ] Yi Liu commented on HDFS-9053: -- Update the patch. The new patch uses an array list if the children size is

[jira] [Commented] (HDFS-9053) Support large directories efficiently using B-Tree

2015-10-09 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14950084#comment-14950084 ] Yi Liu commented on HDFS-9053: -- The increased memory usage is only 8 bytes in {{SortedCollection}} comparing

[jira] [Updated] (HDFS-8988) Use LightWeightHashSet instead of LightWeightLinkedSet in BlockManager#excessReplicateMap

2015-10-09 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-8988: - Attachment: HDFS-8988.002.patch Rebase the patch. > Use LightWeightHashSet instead of LightWeightLinkedSet in >

[jira] [Updated] (HDFS-8988) Use LightWeightHashSet instead of LightWeightLinkedSet in BlockManager#excessReplicateMap

2015-10-09 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-8988: - Priority: Major (was: Minor) > Use LightWeightHashSet instead of LightWeightLinkedSet in >

[jira] [Updated] (HDFS-9053) Support large directories efficiently using B-Tree

2015-10-09 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-9053: - Attachment: HDFS-9053.006.patch > Support large directories efficiently using B-Tree >

[jira] [Comment Edited] (HDFS-9053) Support large directories efficiently using B-Tree

2015-10-09 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14950498#comment-14950498 ] Yi Liu edited comment on HDFS-9053 at 10/9/15 2:51 PM: --- I find a good approach to

[jira] [Commented] (HDFS-9053) Support large directories efficiently using B-Tree

2015-10-09 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14950498#comment-14950498 ] Yi Liu commented on HDFS-9053: -- I find a good approach to improve B-Tree overhead to make it only increase *8

[jira] [Commented] (HDFS-9053) Support large directories efficiently using B-Tree

2015-10-08 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14948629#comment-14948629 ] Yi Liu commented on HDFS-9053: -- Hi [~szetszwo], do you have further comments about it? Thanks. > Support

[jira] [Commented] (HDFS-9053) Support large directories efficiently using B-Tree

2015-10-08 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14949847#comment-14949847 ] Yi Liu commented on HDFS-9053: -- Hi [~szetszwo], thanks for your comments. I will do as your suggestion: use

[jira] [Comment Edited] (HDFS-9053) Support large directories efficiently using B-Tree

2015-10-08 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14949847#comment-14949847 ] Yi Liu edited comment on HDFS-9053 at 10/9/15 3:51 AM: --- Hi [~szetszwo], thanks for

[jira] [Commented] (HDFS-9137) DeadLock between DataNode#refreshVolumes and BPOfferService#registrationSucceeded

2015-10-07 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14947963#comment-14947963 ] Yi Liu commented on HDFS-9137: -- +1, thanks Uma, Colin and Vinay. > DeadLock between DataNode#refreshVolumes

[jira] [Updated] (HDFS-9137) DeadLock between DataNode#refreshVolumes and BPOfferService#registrationSucceeded

2015-10-07 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-9137: - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.0 Target Version/s: 2.8.0

[jira] [Resolved] (HDFS-9212) Are there any official performance tests or reports using WebHDFS

2015-10-07 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu resolved HDFS-9212. -- Resolution: Invalid Send email to u...@hadoop.apache.org > Are there any official performance tests or reports

[jira] [Commented] (HDFS-9137) DeadLock between DataNode#refreshVolumes and BPOfferService#registrationSucceeded

2015-10-06 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14946085#comment-14946085 ] Yi Liu commented on HDFS-9137: -- I think it's OK to do the fix using this approach and update the

[jira] [Commented] (HDFS-9182) Cleanup the findbugs and other issues after HDFS EC merged to trunk.

2015-10-06 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14946080#comment-14946080 ] Yi Liu commented on HDFS-9182: -- The patch looks good to me too. Thanks [~umamaheswararao] and [~jingzhao].

[jira] [Commented] (HDFS-9180) Update excluded DataNodes in DFSStripedOutputStream based on failures in data streamers

2015-10-06 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14944873#comment-14944873 ] Yi Liu commented on HDFS-9180: -- +1, thanks [~jingzhao], also thanks Walter for the review. > Update excluded

[jira] [Commented] (HDFS-9137) DeadLock between DataNode#refreshVolumes and BPOfferService#registrationSucceeded

2015-10-03 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14942520#comment-14942520 ] Yi Liu commented on HDFS-9137: -- Thanks Uma for the patch. I agree we don't need synchronization of

[jira] [Commented] (HDFS-9180) Update excluded DataNodes in DFSStripedOutputStream based on failures in data streamers

2015-10-02 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14942000#comment-14942000 ] Yi Liu commented on HDFS-9180: -- {quote} I kicked the Jenkins again. If we all agree the fixes here are valid

[jira] [Comment Edited] (HDFS-9053) Support large directories efficiently using B-Tree

2015-10-02 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14942004#comment-14942004 ] Yi Liu edited comment on HDFS-9053 at 10/3/15 12:49 AM: Thanks [~szetszwo], good

  1   2   3   4   5   6   7   8   9   10   >