[jira] [Commented] (HDFS-12278) LeaseManager operations are inefficient in 2.8.
[
https://issues.apache.org/jira/browse/HDFS-12278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120808#comment-16120808
]
Hadoop QA commented on HDFS-12278:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 12m
33s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} branch-2.8 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
53s{color} | {color:green} branch-2.8 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
47s{color} | {color:green} branch-2.8 passed with JDK v1.8.0_144 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
46s{color} | {color:green} branch-2.8 passed with JDK v1.7.0_131 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s{color} | {color:green} branch-2.8 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
59s{color} | {color:green} branch-2.8 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
11s{color} | {color:green} branch-2.8 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
44s{color} | {color:green} branch-2.8 passed with JDK v1.8.0_144 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} branch-2.8 passed with JDK v1.7.0_131 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
38s{color} | {color:green} the patch passed with JDK v1.8.0_144 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
41s{color} | {color:green} the patch passed with JDK v1.7.0_131 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
35s{color} | {color:green} the patch passed with JDK v1.8.0_144 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} the patch passed with JDK v1.7.0_131 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 52m
11s{color} | {color:green} hadoop-hdfs in the patch passed with JDK v1.7.0_131.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
22s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}150m 38s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.8.0_144 Failed junit tests |
hadoop.hdfs.server.blockmanagement.TestRBWBlockInvalidation |
| | hadoop.hdfs.server.namenode.TestStartup |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:d946387 |
| JIRA Issue | HDFS-12278 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12881080/HDFS-12278-branch-2.8.001.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 060b17dc6b37 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26
18:04:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
[jira] [Commented] (HDFS-12278) LeaseManager operations are inefficient in 2.8.
[
https://issues.apache.org/jira/browse/HDFS-12278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120729#comment-16120729
]
Hudson commented on HDFS-12278:
---
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12156 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/12156/])
HDFS-12278. LeaseManager operations are inefficient in 2.8. Contributed
(kihwal: rev b5c02f95b5a2fcb8931d4a86f8192caa18009ea9)
* (edit)
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/LeaseManager.java
> LeaseManager operations are inefficient in 2.8.
> ---
>
> Key: HDFS-12278
> URL: https://issues.apache.org/jira/browse/HDFS-12278
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
>Affects Versions: 2.8.0
>Reporter: Rushabh S Shah
>Assignee: Rushabh S Shah
>Priority: Blocker
> Fix For: 2.9.0, 3.0.0-beta1, 2.8.2
>
> Attachments: HDFS-12278-branch-2.8.001.patch, HDFS-12278.patch
>
>
> After HDFS-6757, LeaseManager #removeLease became expensive.
> HDFS-6757 changed the {{sortedLeases}} object from TreeSet to PriorityQueue.
> Previously the {{remove(Object)}} operation from {{sortedLeases}} was {{O(log
> n)}} but after the change it became {{O( n)}} since it has to find the object
> first.
> Recently we had an incident in one of our production cluster just hours after
> we upgraded from 2.7 to 2.8
> The {{sortledLeases}} object had approximately 100,000 items within it.
> While removing the lease, it will acquire the LeaseManager lock and that will
> slow down the lookup of lease also.
> HDFS-6757 is a good improvement which replaced the path by inode id.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12278) LeaseManager operations are inefficient in 2.8.
[
https://issues.apache.org/jira/browse/HDFS-12278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120726#comment-16120726
]
Rushabh S Shah commented on HDFS-12278:
---
Thanks [~kihwal] for the review and commit !
Thanks [~daryn] for the benchmarking the change.
> LeaseManager operations are inefficient in 2.8.
> ---
>
> Key: HDFS-12278
> URL: https://issues.apache.org/jira/browse/HDFS-12278
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
>Affects Versions: 2.8.0
>Reporter: Rushabh S Shah
>Assignee: Rushabh S Shah
>Priority: Blocker
> Fix For: 2.9.0, 3.0.0-beta1, 2.8.2
>
> Attachments: HDFS-12278-branch-2.8.001.patch, HDFS-12278.patch
>
>
> After HDFS-6757, LeaseManager #removeLease became expensive.
> HDFS-6757 changed the {{sortedLeases}} object from TreeSet to PriorityQueue.
> Previously the {{remove(Object)}} operation from {{sortedLeases}} was {{O(log
> n)}} but after the change it became {{O( n)}} since it has to find the object
> first.
> Recently we had an incident in one of our production cluster just hours after
> we upgraded from 2.7 to 2.8
> The {{sortledLeases}} object had approximately 100,000 items within it.
> While removing the lease, it will acquire the LeaseManager lock and that will
> slow down the lookup of lease also.
> HDFS-6757 is a good improvement which replaced the path by inode id.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12278) LeaseManager operations are inefficient in 2.8.
[
https://issues.apache.org/jira/browse/HDFS-12278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120702#comment-16120702
]
Rushabh S Shah commented on HDFS-12278:
---
bq. but how did you benchmark TreeSet and PriorityQueue
[~daryn] benchmarked.
He just created lease-like objects and tested renew like methods.
Basically an object with string, int member variable and a comparator.
He created 100,000 such objects and called renew on them and measured via
{{monotonicTime}}.
Daryn: please correct me if I am wrong.
bq. Are you aware of JMH?
I wasn't aware until I read the comment as did web search.
But it is very simple to understand that priority queue is not a good data
structure if you want to remove any object other than the top one.
> LeaseManager operations are inefficient in 2.8.
> ---
>
> Key: HDFS-12278
> URL: https://issues.apache.org/jira/browse/HDFS-12278
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
>Affects Versions: 2.8.0
>Reporter: Rushabh S Shah
>Assignee: Rushabh S Shah
>Priority: Blocker
> Attachments: HDFS-12278-branch-2.8.001.patch, HDFS-12278.patch
>
>
> After HDFS-6757, LeaseManager #removeLease became expensive.
> HDFS-6757 changed the {{sortedLeases}} object from TreeSet to PriorityQueue.
> Previously the {{remove(Object)}} operation from {{sortedLeases}} was {{O(log
> n)}} but after the change it became {{O( n)}} since it has to find the object
> first.
> Recently we had an incident in one of our production cluster just hours after
> we upgraded from 2.7 to 2.8
> The {{sortledLeases}} object had approximately 100,000 items within it.
> While removing the lease, it will acquire the LeaseManager lock and that will
> slow down the lookup of lease also.
> HDFS-6757 is a good improvement which replaced the path by inode id.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12278) LeaseManager operations are inefficient in 2.8.
[
https://issues.apache.org/jira/browse/HDFS-12278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120700#comment-16120700
]
Kihwal Lee commented on HDFS-12278:
---
+1 looks good.
> LeaseManager operations are inefficient in 2.8.
> ---
>
> Key: HDFS-12278
> URL: https://issues.apache.org/jira/browse/HDFS-12278
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
>Affects Versions: 2.8.0
>Reporter: Rushabh S Shah
>Assignee: Rushabh S Shah
>Priority: Blocker
> Attachments: HDFS-12278-branch-2.8.001.patch, HDFS-12278.patch
>
>
> After HDFS-6757, LeaseManager #removeLease became expensive.
> HDFS-6757 changed the {{sortedLeases}} object from TreeSet to PriorityQueue.
> Previously the {{remove(Object)}} operation from {{sortedLeases}} was {{O(log
> n)}} but after the change it became {{O( n)}} since it has to find the object
> first.
> Recently we had an incident in one of our production cluster just hours after
> we upgraded from 2.7 to 2.8
> The {{sortledLeases}} object had approximately 100,000 items within it.
> While removing the lease, it will acquire the LeaseManager lock and that will
> slow down the lookup of lease also.
> HDFS-6757 is a good improvement which replaced the path by inode id.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12278) LeaseManager operations are inefficient in 2.8.
[
https://issues.apache.org/jira/browse/HDFS-12278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120686#comment-16120686
]
Ravi Prakash commented on HDFS-12278:
-
Hi Rushabh! Slightly on a tangent, but how did you benchmark TreeSet and
PriorityQueue? Are you aware of JMH?
> LeaseManager operations are inefficient in 2.8.
> ---
>
> Key: HDFS-12278
> URL: https://issues.apache.org/jira/browse/HDFS-12278
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
>Affects Versions: 2.8.0
>Reporter: Rushabh S Shah
>Assignee: Rushabh S Shah
>Priority: Blocker
> Attachments: HDFS-12278-branch-2.8.001.patch, HDFS-12278.patch
>
>
> After HDFS-6757, LeaseManager #removeLease became expensive.
> HDFS-6757 changed the {{sortedLeases}} object from TreeSet to PriorityQueue.
> Previously the {{remove(Object)}} operation from {{sortedLeases}} was {{O(log
> n)}} but after the change it became {{O( n)}} since it has to find the object
> first.
> Recently we had an incident in one of our production cluster just hours after
> we upgraded from 2.7 to 2.8
> The {{sortledLeases}} object had approximately 100,000 items within it.
> While removing the lease, it will acquire the LeaseManager lock and that will
> slow down the lookup of lease also.
> HDFS-6757 is a good improvement which replaced the path by inode id.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12278) LeaseManager operations are inefficient in 2.8.
[
https://issues.apache.org/jira/browse/HDFS-12278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120620#comment-16120620
]
Rushabh S Shah commented on HDFS-12278:
---
{{TestDFSStripedOutputStreamWithFailure150}} and
{{TestDFSStripedOutputStreamWithFailure080}} are well known flaky tests.
Both have failed number of times. I ran 4 times each test. Fails with a
probability of 50%
{{TestUnderReplicatedBlocks#testSetRepIncWithUnderReplicatedBlocks}} is timing
out. Tracked via HDFS-9243.
There are no new tests since all the existing test covers the correctness.
Will attach a branch-2.8 patch soon.
> LeaseManager operations are inefficient in 2.8.
> ---
>
> Key: HDFS-12278
> URL: https://issues.apache.org/jira/browse/HDFS-12278
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
>Affects Versions: 2.8.0
>Reporter: Rushabh S Shah
>Assignee: Rushabh S Shah
>Priority: Blocker
> Attachments: HDFS-12278.patch
>
>
> After HDFS-6757, LeaseManager #removeLease became expensive.
> HDFS-6757 changed the {{sortedLeases}} object from TreeSet to PriorityQueue.
> Previously the {{remove(Object)}} operation from {{sortedLeases}} was {{O(log
> n)}} but after the change it became {{O( n)}} since it has to find the object
> first.
> Recently we had an incident in one of our production cluster just hours after
> we upgraded from 2.7 to 2.8
> The {{sortledLeases}} object had approximately 100,000 items within it.
> While removing the lease, it will acquire the LeaseManager lock and that will
> slow down the lookup of lease also.
> HDFS-6757 is a good improvement which replaced the path by inode id.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
