subject:"\[jira\] \[Commented\] \(HBASE\-21150\) Avoid delay in first flushes due to overheads in table metrics registration"

[jira] [Commented] (HBASE-21150) Avoid delay in first flushes due to overheads in table metrics registration

2018-09-05 Thread Hadoop QA (JIRA)



[ 
https://issues.apache.org/jira/browse/HBASE-21150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16604875#comment-16604875
 ] 

Hadoop QA commented on HBASE-21150:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
19s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue}  0m  
2s{color} | {color:blue} The patch file was not named according to hbase's 
naming conventions. Please see 
https://yetus.apache.org/documentation/0.7.0/precommit-patchnames for 
instructions. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}  
0m  0s{color} | {color:orange} The patch doesn't appear to include any new or 
modified tests. Please justify why no new tests are needed for this patch. Also 
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
26s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  3m 
50s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  2m  
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  4m 
16s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
27s{color} | {color:blue} hbase-hadoop2-compat in master has 18 extant Findbugs 
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
54s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
18s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  2m  
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  2m  
3s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  1m 
13s{color} | {color:red} hbase-server: The patch generated 1 new + 78 unchanged 
- 0 fixed = 79 total (was 78) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  4m 
11s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}  
7m 37s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4 
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  2m 
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
41s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  0m 
29s{color} | {color:green} hbase-hadoop2-compat in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}195m 
59s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
43s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}237m 48s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21150 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12938496/21150.v7.txt |
| Optional Tests |  asflicense  javac  javadoc  unit

[jira] [Commented] (HBASE-21150) Avoid delay in first flushes due to overheads in table metrics registration

2018-09-05 Thread stack (JIRA)



[ 
https://issues.apache.org/jira/browse/HBASE-21150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16604665#comment-16604665
 ] 

stack commented on HBASE-21150:
---

bq. I will resolve this issue since the comment from other committers was that 
the delay is not significant.

Above is a total mischaracterization of what went on in this 'car crash' of an 
issue.

> Avoid delay in first flushes due to overheads in table metrics registration
> ---
>
> Key: HBASE-21150
> URL: https://issues.apache.org/jira/browse/HBASE-21150
> Project: HBase
>  Issue Type: Improvement
>Reporter: Ted Yu
>Priority: Minor
> Attachments: 21150.v1.txt, 21150.v2.txt, 21150.v3.txt, 21150.v4.txt, 
> 21150.v4.txt, 21150.v5.txt, 21150.v6.txt, 21150.v7.txt
>
>
> After HBASE-15728 is integrated, the lazy table metrics registration results 
> in penalty for the first flushes.
> Excerpt from log shows delay (note the same timestamp 08:18:23,234) :
> {code:java}
> 2018-09-02 08:18:23,232 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(124): Creating new  
> MetricsTableSourceImpl for table 'testtb-1535901500805'
> 2018-09-02 08:18:23,233 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(137): registering metrics for testtb-   
> 1535901500805
> 2018-09-02 08:18:23,234 INFO  
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.HRegion(2822): Finished flush of dataSize ~2.29 KB/2343,   
> heapSize ~5.16 KB/5280, currentSize=0 B/0 for 
> fa403f6a4fb8dbc1a1c389744fce2d58 in 280ms, sequenceid=5, compaction 
> requested=false
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 0 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 5 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2,5,FailOnTimeoutGroup]
> {code}
> This is a regression in that there were multiple (6 ms) delays before the 
> flush can finish, waiting for the metrics table to be registered.
> When first region of the table is opened on region server, we can proactively 
> register table metrics.
> This would avoid the penalty on first flushes for the table.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-21150) Avoid delay in first flushes due to overheads in table metrics registration

2018-09-05 Thread Ted Yu (JIRA)



[ 
https://issues.apache.org/jira/browse/HBASE-21150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16604628#comment-16604628
 ] 

Ted Yu commented on HBASE-21150:


bq. Where is evidence that the patch makes improvement?

As I noted earlier, when adding log in the getOrCreateTableSource method of 
code path for flushing, the log didn't show up with patch.

bq. Is it just the original cited 6ms instance?

That was what I observed, through the additional log.

I will resolve this issue since the comment from other committers was that the 
delay is not significant.

> Avoid delay in first flushes due to overheads in table metrics registration
> ---
>
> Key: HBASE-21150
> URL: https://issues.apache.org/jira/browse/HBASE-21150
> Project: HBase
>  Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Minor
> Fix For: 3.0.0, 2.2.0
>
> Attachments: 21150.v1.txt, 21150.v2.txt, 21150.v3.txt, 21150.v4.txt, 
> 21150.v4.txt, 21150.v5.txt, 21150.v6.txt, 21150.v7.txt
>
>
> After HBASE-15728 is integrated, the lazy table metrics registration results 
> in penalty for the first flushes.
> Excerpt from log shows delay (note the same timestamp 08:18:23,234) :
> {code:java}
> 2018-09-02 08:18:23,232 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(124): Creating new  
> MetricsTableSourceImpl for table 'testtb-1535901500805'
> 2018-09-02 08:18:23,233 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(137): registering metrics for testtb-   
> 1535901500805
> 2018-09-02 08:18:23,234 INFO  
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.HRegion(2822): Finished flush of dataSize ~2.29 KB/2343,   
> heapSize ~5.16 KB/5280, currentSize=0 B/0 for 
> fa403f6a4fb8dbc1a1c389744fce2d58 in 280ms, sequenceid=5, compaction 
> requested=false
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 0 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 5 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2,5,FailOnTimeoutGroup]
> {code}
> This is a regression in that there were multiple (6 ms) delays before the 
> flush can finish, waiting for the metrics table to be registered.
> When first region of the table is opened on region server, we can proactively 
> register table metrics.
> This would avoid the penalty on first flushes for the table.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-21150) Avoid delay in first flushes due to overheads in table metrics registration

2018-09-05 Thread stack (JIRA)



[ 
https://issues.apache.org/jira/browse/HBASE-21150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16604622#comment-16604622
 ] 

stack commented on HBASE-21150:
---

Where is the evidence that this is a problem? Is it just the original cited 6ms 
instance? Where is evidence that the patch makes improvement?

-1 on committing contended code on a minor 'improvement'.



> Avoid delay in first flushes due to overheads in table metrics registration
> ---
>
> Key: HBASE-21150
> URL: https://issues.apache.org/jira/browse/HBASE-21150
> Project: HBase
>  Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Minor
> Fix For: 3.0.0, 2.2.0
>
> Attachments: 21150.v1.txt, 21150.v2.txt, 21150.v3.txt, 21150.v4.txt, 
> 21150.v4.txt, 21150.v5.txt, 21150.v6.txt, 21150.v7.txt
>
>
> After HBASE-15728 is integrated, the lazy table metrics registration results 
> in penalty for the first flushes.
> Excerpt from log shows delay (note the same timestamp 08:18:23,234) :
> {code:java}
> 2018-09-02 08:18:23,232 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(124): Creating new  
> MetricsTableSourceImpl for table 'testtb-1535901500805'
> 2018-09-02 08:18:23,233 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(137): registering metrics for testtb-   
> 1535901500805
> 2018-09-02 08:18:23,234 INFO  
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.HRegion(2822): Finished flush of dataSize ~2.29 KB/2343,   
> heapSize ~5.16 KB/5280, currentSize=0 B/0 for 
> fa403f6a4fb8dbc1a1c389744fce2d58 in 280ms, sequenceid=5, compaction 
> requested=false
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 0 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 5 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2,5,FailOnTimeoutGroup]
> {code}
> This is a regression in that there were multiple (6 ms) delays before the 
> flush can finish, waiting for the metrics table to be registered.
> When first region of the table is opened on region server, we can proactively 
> register table metrics.
> This would avoid the penalty on first flushes for the table.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-21150) Avoid delay in first flushes due to overheads in table metrics registration

2018-09-05 Thread Hadoop QA (JIRA)



[ 
https://issues.apache.org/jira/browse/HBASE-21150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16604614#comment-16604614
 ] 

Hadoop QA commented on HBASE-21150:
---

| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
17s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue}  0m  
1s{color} | {color:blue} The patch file was not named according to hbase's 
naming conventions. Please see 
https://yetus.apache.org/documentation/0.7.0/precommit-patchnames for 
instructions. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}  
0m  0s{color} | {color:orange} The patch doesn't appear to include any new or 
modified tests. Please justify why no new tests are needed for this patch. Also 
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
11s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  3m 
35s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  2m  
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  4m 
15s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
25s{color} | {color:blue} hbase-hadoop2-compat in master has 18 extant Findbugs 
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
42s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  3m 
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  2m  
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  2m  
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  4m 
13s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}  
7m 34s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4 
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  2m 
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
42s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  0m 
29s{color} | {color:green} hbase-hadoop2-compat in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}120m 
37s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
39s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}159m 33s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21150 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12938467/21150.v6.txt |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  shadedjars  
hadoopcheck  hbaseanti  checkstyle

[jira] [Commented] (HBASE-21150) Avoid delay in first flushes due to overheads in table metrics registration

2018-09-05 Thread Hadoop QA (JIRA)



[ 
https://issues.apache.org/jira/browse/HBASE-21150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16604458#comment-16604458
 ] 

Hadoop QA commented on HBASE-21150:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
21s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue}  0m  
2s{color} | {color:blue} The patch file was not named according to hbase's 
naming conventions. Please see 
https://yetus.apache.org/documentation/0.7.0/precommit-patchnames for 
instructions. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}  
0m  0s{color} | {color:orange} The patch doesn't appear to include any new or 
modified tests. Please justify why no new tests are needed for this patch. Also 
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  1m 
17s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  5m 
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  2m 
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  3m 
59s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
23s{color} | {color:blue} hbase-hadoop2-compat in master has 18 extant Findbugs 
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m  
0s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
18s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  3m 
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  2m 
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  2m 
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  4m 
 0s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}  
7m 36s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4 
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  3m  
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
52s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  0m 
27s{color} | {color:green} hbase-hadoop-compat in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  0m 
33s{color} | {color:green} hbase-hadoop2-compat in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}238m  9s{color} 
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
41s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}283m 12s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21150 |
| JIRA Patch URL |

[jira] [Commented] (HBASE-21150) Avoid delay in first flushes due to overheads in table metrics registration

2018-09-05 Thread Ted Yu (JIRA)



[ 
https://issues.apache.org/jira/browse/HBASE-21150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16604443#comment-16604443
 ] 

Ted Yu commented on HBASE-21150:


bq. do not like the ugly code.

Can you tell me which part of the code is ugly ?

> Avoid delay in first flushes due to overheads in table metrics registration
> ---
>
> Key: HBASE-21150
> URL: https://issues.apache.org/jira/browse/HBASE-21150
> Project: HBase
>  Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Minor
> Fix For: 3.0.0, 2.2.0
>
> Attachments: 21150.v1.txt, 21150.v2.txt, 21150.v3.txt, 21150.v4.txt, 
> 21150.v4.txt, 21150.v5.txt, 21150.v6.txt
>
>
> After HBASE-15728 is integrated, the lazy table metrics registration results 
> in penalty for the first flushes.
> Excerpt from log shows delay (note the same timestamp 08:18:23,234) :
> {code:java}
> 2018-09-02 08:18:23,232 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(124): Creating new  
> MetricsTableSourceImpl for table 'testtb-1535901500805'
> 2018-09-02 08:18:23,233 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(137): registering metrics for testtb-   
> 1535901500805
> 2018-09-02 08:18:23,234 INFO  
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.HRegion(2822): Finished flush of dataSize ~2.29 KB/2343,   
> heapSize ~5.16 KB/5280, currentSize=0 B/0 for 
> fa403f6a4fb8dbc1a1c389744fce2d58 in 280ms, sequenceid=5, compaction 
> requested=false
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 0 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 5 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2,5,FailOnTimeoutGroup]
> {code}
> This is a regression in that there were multiple (6 ms) delays before the 
> flush can finish, waiting for the metrics table to be registered.
> When first region of the table is opened on region server, we can proactively 
> register table metrics.
> This would avoid the penalty on first flushes for the table.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-21150) Avoid delay in first flushes due to overheads in table metrics registration

2018-09-05 Thread Duo Zhang (JIRA)



[ 
https://issues.apache.org/jira/browse/HBASE-21150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16604426#comment-16604426
 ] 

Duo Zhang commented on HBASE-21150:
---

OK, I give up. You can write anything you want, and I will vote a -1. No other 
reason, I just do not like the ugly code.

> Avoid delay in first flushes due to overheads in table metrics registration
> ---
>
> Key: HBASE-21150
> URL: https://issues.apache.org/jira/browse/HBASE-21150
> Project: HBase
>  Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Minor
> Fix For: 3.0.0, 2.2.0
>
> Attachments: 21150.v1.txt, 21150.v2.txt, 21150.v3.txt, 21150.v4.txt, 
> 21150.v4.txt, 21150.v5.txt, 21150.v6.txt
>
>
> After HBASE-15728 is integrated, the lazy table metrics registration results 
> in penalty for the first flushes.
> Excerpt from log shows delay (note the same timestamp 08:18:23,234) :
> {code:java}
> 2018-09-02 08:18:23,232 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(124): Creating new  
> MetricsTableSourceImpl for table 'testtb-1535901500805'
> 2018-09-02 08:18:23,233 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(137): registering metrics for testtb-   
> 1535901500805
> 2018-09-02 08:18:23,234 INFO  
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.HRegion(2822): Finished flush of dataSize ~2.29 KB/2343,   
> heapSize ~5.16 KB/5280, currentSize=0 B/0 for 
> fa403f6a4fb8dbc1a1c389744fce2d58 in 280ms, sequenceid=5, compaction 
> requested=false
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 0 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 5 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2,5,FailOnTimeoutGroup]
> {code}
> This is a regression in that there were multiple (6 ms) delays before the 
> flush can finish, waiting for the metrics table to be registered.
> When first region of the table is opened on region server, we can proactively 
> register table metrics.
> This would avoid the penalty on first flushes for the table.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-21150) Avoid delay in first flushes due to overheads in table metrics registration

2018-09-05 Thread Ted Yu (JIRA)



[ 
https://issues.apache.org/jira/browse/HBASE-21150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16604409#comment-16604409
 ] 

Ted Yu commented on HBASE-21150:


I read the javadoc for {{submit}}:
{code}
 * Submits a value-returning task for execution and returns a
 * Future representing the pending results of the task. The
 * Future's {@code get} method will return the task's result upon
 * successful completion.
{code}
Can you tell me how you get the conclusion {{submit}} has same issue ?

> Avoid delay in first flushes due to overheads in table metrics registration
> ---
>
> Key: HBASE-21150
> URL: https://issues.apache.org/jira/browse/HBASE-21150
> Project: HBase
>  Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Minor
> Fix For: 3.0.0, 2.2.0
>
> Attachments: 21150.v1.txt, 21150.v2.txt, 21150.v3.txt, 21150.v4.txt, 
> 21150.v4.txt, 21150.v5.txt, 21150.v6.txt
>
>
> After HBASE-15728 is integrated, the lazy table metrics registration results 
> in penalty for the first flushes.
> Excerpt from log shows delay (note the same timestamp 08:18:23,234) :
> {code:java}
> 2018-09-02 08:18:23,232 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(124): Creating new  
> MetricsTableSourceImpl for table 'testtb-1535901500805'
> 2018-09-02 08:18:23,233 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(137): registering metrics for testtb-   
> 1535901500805
> 2018-09-02 08:18:23,234 INFO  
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.HRegion(2822): Finished flush of dataSize ~2.29 KB/2343,   
> heapSize ~5.16 KB/5280, currentSize=0 B/0 for 
> fa403f6a4fb8dbc1a1c389744fce2d58 in 280ms, sequenceid=5, compaction 
> requested=false
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 0 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 5 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2,5,FailOnTimeoutGroup]
> {code}
> This is a regression in that there were multiple (6 ms) delays before the 
> flush can finish, waiting for the metrics table to be registered.
> When first region of the table is opened on region server, we can proactively 
> register table metrics.
> This would avoid the penalty on first flushes for the table.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-21150) Avoid delay in first flushes due to overheads in table metrics registration

2018-09-05 Thread Duo Zhang (JIRA)



[ 
https://issues.apache.org/jira/browse/HBASE-21150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16604401#comment-16604401
 ] 

Duo Zhang commented on HBASE-21150:
---

The submit method has the same problem.

> Avoid delay in first flushes due to overheads in table metrics registration
> ---
>
> Key: HBASE-21150
> URL: https://issues.apache.org/jira/browse/HBASE-21150
> Project: HBase
>  Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Minor
> Fix For: 3.0.0, 2.2.0
>
> Attachments: 21150.v1.txt, 21150.v2.txt, 21150.v3.txt, 21150.v4.txt, 
> 21150.v4.txt, 21150.v5.txt, 21150.v6.txt
>
>
> After HBASE-15728 is integrated, the lazy table metrics registration results 
> in penalty for the first flushes.
> Excerpt from log shows delay (note the same timestamp 08:18:23,234) :
> {code:java}
> 2018-09-02 08:18:23,232 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(124): Creating new  
> MetricsTableSourceImpl for table 'testtb-1535901500805'
> 2018-09-02 08:18:23,233 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(137): registering metrics for testtb-   
> 1535901500805
> 2018-09-02 08:18:23,234 INFO  
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.HRegion(2822): Finished flush of dataSize ~2.29 KB/2343,   
> heapSize ~5.16 KB/5280, currentSize=0 B/0 for 
> fa403f6a4fb8dbc1a1c389744fce2d58 in 280ms, sequenceid=5, compaction 
> requested=false
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 0 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 5 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2,5,FailOnTimeoutGroup]
> {code}
> This is a regression in that there were multiple (6 ms) delays before the 
> flush can finish, waiting for the metrics table to be registered.
> When first region of the table is opened on region server, we can proactively 
> register table metrics.
> This would avoid the penalty on first flushes for the table.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-21150) Avoid delay in first flushes due to overheads in table metrics registration

2018-09-05 Thread Ted Yu (JIRA)



[ 
https://issues.apache.org/jira/browse/HBASE-21150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16604395#comment-16604395
 ] 

Ted Yu commented on HBASE-21150:


Looking at the javadoc for {{execute}}:
{code}
 * Executes the given command at some time in the future.  The command
 * may execute in a new thread, in a pooled thread, or in the calling
 * thread, at the discretion of the {@code Executor} implementation.
{code}
The last part, in the calling thread, is not what we want. The registration is 
intended to be asynchronous.

I added javadoc in HRegionServer before the registration.

> Avoid delay in first flushes due to overheads in table metrics registration
> ---
>
> Key: HBASE-21150
> URL: https://issues.apache.org/jira/browse/HBASE-21150
> Project: HBase
>  Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Minor
> Fix For: 3.0.0, 2.2.0
>
> Attachments: 21150.v1.txt, 21150.v2.txt, 21150.v3.txt, 21150.v4.txt, 
> 21150.v4.txt, 21150.v5.txt, 21150.v6.txt
>
>
> After HBASE-15728 is integrated, the lazy table metrics registration results 
> in penalty for the first flushes.
> Excerpt from log shows delay (note the same timestamp 08:18:23,234) :
> {code:java}
> 2018-09-02 08:18:23,232 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(124): Creating new  
> MetricsTableSourceImpl for table 'testtb-1535901500805'
> 2018-09-02 08:18:23,233 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(137): registering metrics for testtb-   
> 1535901500805
> 2018-09-02 08:18:23,234 INFO  
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.HRegion(2822): Finished flush of dataSize ~2.29 KB/2343,   
> heapSize ~5.16 KB/5280, currentSize=0 B/0 for 
> fa403f6a4fb8dbc1a1c389744fce2d58 in 280ms, sequenceid=5, compaction 
> requested=false
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 0 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 5 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2,5,FailOnTimeoutGroup]
> {code}
> This is a regression in that there were multiple (6 ms) delays before the 
> flush can finish, waiting for the metrics table to be registered.
> When first region of the table is opened on region server, we can proactively 
> register table metrics.
> This would avoid the penalty on first flushes for the table.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-21150) Avoid delay in first flushes due to overheads in table metrics registration

2018-09-05 Thread Duo Zhang (JIRA)



[ 
https://issues.apache.org/jira/browse/HBASE-21150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16604383#comment-16604383
 ] 

Duo Zhang commented on HBASE-21150:
---

{code}
executorForMetrics.submit(new Callable() {
{code}

Just use execute method here as the register method in MetricsRegionServer does 
not throw any exceptions? This could also eliminate the 
FutureReturnValueIgnored warning?

And please add a note before this line that this is just an optimization, not a 
critical one so we execute it in a thread pool, and do not care whether it is 
successful or not. System can work fine even if we do not call the register 
method.

> Avoid delay in first flushes due to overheads in table metrics registration
> ---
>
> Key: HBASE-21150
> URL: https://issues.apache.org/jira/browse/HBASE-21150
> Project: HBase
>  Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Minor
> Fix For: 3.0.0, 2.2.0
>
> Attachments: 21150.v1.txt, 21150.v2.txt, 21150.v3.txt, 21150.v4.txt, 
> 21150.v4.txt, 21150.v5.txt
>
>
> After HBASE-15728 is integrated, the lazy table metrics registration results 
> in penalty for the first flushes.
> Excerpt from log shows delay (note the same timestamp 08:18:23,234) :
> {code:java}
> 2018-09-02 08:18:23,232 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(124): Creating new  
> MetricsTableSourceImpl for table 'testtb-1535901500805'
> 2018-09-02 08:18:23,233 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(137): registering metrics for testtb-   
> 1535901500805
> 2018-09-02 08:18:23,234 INFO  
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.HRegion(2822): Finished flush of dataSize ~2.29 KB/2343,   
> heapSize ~5.16 KB/5280, currentSize=0 B/0 for 
> fa403f6a4fb8dbc1a1c389744fce2d58 in 280ms, sequenceid=5, compaction 
> requested=false
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 0 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 5 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2,5,FailOnTimeoutGroup]
> {code}
> This is a regression in that there were multiple (6 ms) delays before the 
> flush can finish, waiting for the metrics table to be registered.
> When first region of the table is opened on region server, we can proactively 
> register table metrics.
> This would avoid the penalty on first flushes for the table.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-21150) Avoid delay in first flushes due to overheads in table metrics registration

2018-09-05 Thread Ted Yu (JIRA)



[ 
https://issues.apache.org/jira/browse/HBASE-21150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16604373#comment-16604373
 ] 

Ted Yu commented on HBASE-21150:


In patch v5, I dropped the register() method from MetricsTableAggregateSource 
since the existing method can be used.
Added javadoc in MetricsRegionServer and MetricsTable.

> Avoid delay in first flushes due to overheads in table metrics registration
> ---
>
> Key: HBASE-21150
> URL: https://issues.apache.org/jira/browse/HBASE-21150
> Project: HBase
>  Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Minor
> Fix For: 3.0.0, 2.2.0
>
> Attachments: 21150.v1.txt, 21150.v2.txt, 21150.v3.txt, 21150.v4.txt, 
> 21150.v4.txt, 21150.v5.txt
>
>
> After HBASE-15728 is integrated, the lazy table metrics registration results 
> in penalty for the first flushes.
> Excerpt from log shows delay (note the same timestamp 08:18:23,234) :
> {code:java}
> 2018-09-02 08:18:23,232 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(124): Creating new  
> MetricsTableSourceImpl for table 'testtb-1535901500805'
> 2018-09-02 08:18:23,233 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(137): registering metrics for testtb-   
> 1535901500805
> 2018-09-02 08:18:23,234 INFO  
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.HRegion(2822): Finished flush of dataSize ~2.29 KB/2343,   
> heapSize ~5.16 KB/5280, currentSize=0 B/0 for 
> fa403f6a4fb8dbc1a1c389744fce2d58 in 280ms, sequenceid=5, compaction 
> requested=false
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 0 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 5 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2,5,FailOnTimeoutGroup]
> {code}
> This is a regression in that there were multiple (6 ms) delays before the 
> flush can finish, waiting for the metrics table to be registered.
> When first region of the table is opened on region server, we can proactively 
> register table metrics.
> This would avoid the penalty on first flushes for the table.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-21150) Avoid delay in first flushes due to overheads in table metrics registration

2018-09-05 Thread Duo Zhang (JIRA)



[ 
https://issues.apache.org/jira/browse/HBASE-21150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16604347#comment-16604347
 ] 

Duo Zhang commented on HBASE-21150:
---

Anyway, why do you call a containsKey in register method as well as that we 
will do a get in registerHelper? And the same pattern is in the method 
getOrCreateTableSource, we do a get, and then a get again in the registerHelper 
method. What do you think?

Anyway, for me, you have a register method, and also a getOrCreate method, 
first the name is not aligned, one is register and the other is create, and 
second, you have a register method but another method could also do 
registration?

> Avoid delay in first flushes due to overheads in table metrics registration
> ---
>
> Key: HBASE-21150
> URL: https://issues.apache.org/jira/browse/HBASE-21150
> Project: HBase
>  Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Minor
> Fix For: 3.0.0, 2.2.0
>
> Attachments: 21150.v1.txt, 21150.v2.txt, 21150.v3.txt, 21150.v4.txt, 
> 21150.v4.txt
>
>
> After HBASE-15728 is integrated, the lazy table metrics registration results 
> in penalty for the first flushes.
> Excerpt from log shows delay (note the same timestamp 08:18:23,234) :
> {code:java}
> 2018-09-02 08:18:23,232 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(124): Creating new  
> MetricsTableSourceImpl for table 'testtb-1535901500805'
> 2018-09-02 08:18:23,233 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(137): registering metrics for testtb-   
> 1535901500805
> 2018-09-02 08:18:23,234 INFO  
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.HRegion(2822): Finished flush of dataSize ~2.29 KB/2343,   
> heapSize ~5.16 KB/5280, currentSize=0 B/0 for 
> fa403f6a4fb8dbc1a1c389744fce2d58 in 280ms, sequenceid=5, compaction 
> requested=false
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 0 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 5 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2,5,FailOnTimeoutGroup]
> {code}
> This is a regression in that there were multiple (6 ms) delays before the 
> flush can finish, waiting for the metrics table to be registered.
> When first region of the table is opened on region server, we can proactively 
> register table metrics.
> This would avoid the penalty on first flushes for the table.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-21150) Avoid delay in first flushes due to overheads in table metrics registration

2018-09-05 Thread Ted Yu (JIRA)



[ 
https://issues.apache.org/jira/browse/HBASE-21150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16604339#comment-16604339
 ] 

Ted Yu commented on HBASE-21150:


MetricsTableAggregateSource is marked Private.
Adding the register() method makes the change in HRegionServer easy to 
understand.

Underneath their operation is similar.

> Avoid delay in first flushes due to overheads in table metrics registration
> ---
>
> Key: HBASE-21150
> URL: https://issues.apache.org/jira/browse/HBASE-21150
> Project: HBase
>  Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Minor
> Fix For: 3.0.0, 2.2.0
>
> Attachments: 21150.v1.txt, 21150.v2.txt, 21150.v3.txt, 21150.v4.txt, 
> 21150.v4.txt
>
>
> After HBASE-15728 is integrated, the lazy table metrics registration results 
> in penalty for the first flushes.
> Excerpt from log shows delay (note the same timestamp 08:18:23,234) :
> {code:java}
> 2018-09-02 08:18:23,232 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(124): Creating new  
> MetricsTableSourceImpl for table 'testtb-1535901500805'
> 2018-09-02 08:18:23,233 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(137): registering metrics for testtb-   
> 1535901500805
> 2018-09-02 08:18:23,234 INFO  
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.HRegion(2822): Finished flush of dataSize ~2.29 KB/2343,   
> heapSize ~5.16 KB/5280, currentSize=0 B/0 for 
> fa403f6a4fb8dbc1a1c389744fce2d58 in 280ms, sequenceid=5, compaction 
> requested=false
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 0 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 5 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2,5,FailOnTimeoutGroup]
> {code}
> This is a regression in that there were multiple (6 ms) delays before the 
> flush can finish, waiting for the metrics table to be registered.
> When first region of the table is opened on region server, we can proactively 
> register table metrics.
> This would avoid the penalty on first flushes for the table.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-21150) Avoid delay in first flushes due to overheads in table metrics registration

2018-09-05 Thread Duo Zhang (JIRA)



[ 
https://issues.apache.org/jira/browse/HBASE-21150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16604182#comment-16604182
 ] 

Duo Zhang commented on HBASE-21150:
---

What is the difference between the old getOrCreateTableSource and your new 
registerHelper method? Do we really need to add a new register method here?

> Avoid delay in first flushes due to overheads in table metrics registration
> ---
>
> Key: HBASE-21150
> URL: https://issues.apache.org/jira/browse/HBASE-21150
> Project: HBase
>  Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Minor
> Fix For: 3.0.0, 2.2.0
>
> Attachments: 21150.v1.txt, 21150.v2.txt, 21150.v3.txt, 21150.v4.txt, 
> 21150.v4.txt
>
>
> After HBASE-15728 is integrated, the lazy table metrics registration results 
> in penalty for the first flushes.
> Excerpt from log shows delay (note the same timestamp 08:18:23,234) :
> {code:java}
> 2018-09-02 08:18:23,232 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(124): Creating new  
> MetricsTableSourceImpl for table 'testtb-1535901500805'
> 2018-09-02 08:18:23,233 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(137): registering metrics for testtb-   
> 1535901500805
> 2018-09-02 08:18:23,234 INFO  
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.HRegion(2822): Finished flush of dataSize ~2.29 KB/2343,   
> heapSize ~5.16 KB/5280, currentSize=0 B/0 for 
> fa403f6a4fb8dbc1a1c389744fce2d58 in 280ms, sequenceid=5, compaction 
> requested=false
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 0 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 5 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2,5,FailOnTimeoutGroup]
> {code}
> This is a regression in that there were multiple (6 ms) delays before the 
> flush can finish, waiting for the metrics table to be registered.
> When first region of the table is opened on region server, we can proactively 
> register table metrics.
> This would avoid the penalty on first flushes for the table.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-21150) Avoid delay in first flushes due to overheads in table metrics registration

2018-09-05 Thread Ted Yu (JIRA)



[ 
https://issues.apache.org/jira/browse/HBASE-21150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16604116#comment-16604116
 ] 

Ted Yu commented on HBASE-21150:


I ran TestFromClientSide locally 3 times, TestFromClientSideWithCoprocessor 
locally 2 times, TestReplicationKillSlaveRSWithSeparateOldWALs 1 time with 
patch v4.
All passed.

> Avoid delay in first flushes due to overheads in table metrics registration
> ---
>
> Key: HBASE-21150
> URL: https://issues.apache.org/jira/browse/HBASE-21150
> Project: HBase
>  Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Minor
> Fix For: 3.0.0, 2.2.0
>
> Attachments: 21150.v1.txt, 21150.v2.txt, 21150.v3.txt, 21150.v4.txt, 
> 21150.v4.txt
>
>
> After HBASE-15728 is integrated, the lazy table metrics registration results 
> in penalty for the first flushes.
> Excerpt from log shows delay (note the same timestamp 08:18:23,234) :
> {code:java}
> 2018-09-02 08:18:23,232 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(124): Creating new  
> MetricsTableSourceImpl for table 'testtb-1535901500805'
> 2018-09-02 08:18:23,233 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(137): registering metrics for testtb-   
> 1535901500805
> 2018-09-02 08:18:23,234 INFO  
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.HRegion(2822): Finished flush of dataSize ~2.29 KB/2343,   
> heapSize ~5.16 KB/5280, currentSize=0 B/0 for 
> fa403f6a4fb8dbc1a1c389744fce2d58 in 280ms, sequenceid=5, compaction 
> requested=false
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 0 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 5 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2,5,FailOnTimeoutGroup]
> {code}
> This is a regression in that there were multiple (6 ms) delays before the 
> flush can finish, waiting for the metrics table to be registered.
> When first region of the table is opened on region server, we can proactively 
> register table metrics.
> This would avoid the penalty on first flushes for the table.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-21150) Avoid delay in first flushes due to overheads in table metrics registration

2018-09-05 Thread Ted Yu (JIRA)



[ 
https://issues.apache.org/jira/browse/HBASE-21150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16604100#comment-16604100
 ] 

Ted Yu commented on HBASE-21150:


>From 
>https://builds.apache.org/job/PreCommit-HBASE-Build/14308/artifact/patchprocess/patch-unit-hbase-server.txt
> :
{code}
[ERROR] org.apache.hadoop.hbase.client.TestFromClientSideWithCoprocessor  Time 
elapsed: 10.671 s  <<< ERROR!
java.lang.IllegalStateException: A mini-cluster is already running
at 
org.apache.hadoop.hbase.client.TestFromClientSideWithCoprocessor.setUpBeforeClass(TestFromClientSideWithCoprocessor.java:42)
...
[ERROR] org.apache.hadoop.hbase.client.TestFromClientSide  Time elapsed: 7.358 
s  <<< ERROR!
java.lang.IllegalStateException: A mini-cluster is already running
at 
org.apache.hadoop.hbase.client.TestFromClientSide.initialize(TestFromClientSide.java:155)
at 
org.apache.hadoop.hbase.client.TestFromClientSide.setUpBeforeClass(TestFromClientSide.java:160)
{code}

> Avoid delay in first flushes due to overheads in table metrics registration
> ---
>
> Key: HBASE-21150
> URL: https://issues.apache.org/jira/browse/HBASE-21150
> Project: HBase
>  Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Minor
> Fix For: 3.0.0, 2.2.0
>
> Attachments: 21150.v1.txt, 21150.v2.txt, 21150.v3.txt, 21150.v4.txt
>
>
> After HBASE-15728 is integrated, the lazy table metrics registration results 
> in penalty for the first flushes.
> Excerpt from log shows delay (note the same timestamp 08:18:23,234) :
> {code:java}
> 2018-09-02 08:18:23,232 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(124): Creating new  
> MetricsTableSourceImpl for table 'testtb-1535901500805'
> 2018-09-02 08:18:23,233 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(137): registering metrics for testtb-   
> 1535901500805
> 2018-09-02 08:18:23,234 INFO  
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.HRegion(2822): Finished flush of dataSize ~2.29 KB/2343,   
> heapSize ~5.16 KB/5280, currentSize=0 B/0 for 
> fa403f6a4fb8dbc1a1c389744fce2d58 in 280ms, sequenceid=5, compaction 
> requested=false
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 0 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 5 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2,5,FailOnTimeoutGroup]
> {code}
> This is a regression in that there were multiple (6 ms) delays before the 
> flush can finish, waiting for the metrics table to be registered.
> When first region of the table is opened on region server, we can proactively 
> register table metrics.
> This would avoid the penalty on first flushes for the table.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-21150) Avoid delay in first flushes due to overheads in table metrics registration

2018-09-05 Thread Hadoop QA (JIRA)



[ 
https://issues.apache.org/jira/browse/HBASE-21150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16604021#comment-16604021
 ] 

Hadoop QA commented on HBASE-21150:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
23s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue}  0m  
2s{color} | {color:blue} The patch file was not named according to hbase's 
naming conventions. Please see 
https://yetus.apache.org/documentation/0.7.0/precommit-patchnames for 
instructions. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}  
0m  0s{color} | {color:orange} The patch doesn't appear to include any new or 
modified tests. Please justify why no new tests are needed for this patch. Also 
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
30s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
 2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  2m 
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  4m 
 8s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
23s{color} | {color:blue} hbase-hadoop2-compat in master has 18 extant Findbugs 
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
52s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  3m 
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  2m 
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  2m 
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  4m 
 8s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}  
7m 53s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4 
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  3m  
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
54s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  0m 
26s{color} | {color:green} hbase-hadoop-compat in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  0m 
32s{color} | {color:green} hbase-hadoop2-compat in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}240m 24s{color} 
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  1m 
 3s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}283m 23s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.client.TestFromClientSide |
|   | hadoop.hbase.client.TestFromClientSideWithCoprocessor |
|   |

[jira] [Commented] (HBASE-21150) Avoid delay in first flushes due to overheads in table metrics registration

2018-09-04 Thread Hadoop QA (JIRA)



[ 
https://issues.apache.org/jira/browse/HBASE-21150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16603862#comment-16603862
 ] 

Hadoop QA commented on HBASE-21150:
---

| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
10s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue}  0m  
1s{color} | {color:blue} The patch file was not named according to hbase's 
naming conventions. Please see 
https://yetus.apache.org/documentation/0.7.0/precommit-patchnames for 
instructions. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}  
0m  0s{color} | {color:orange} The patch doesn't appear to include any new or 
modified tests. Please justify why no new tests are needed for this patch. Also 
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
24s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  3m 
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  2m 
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
38s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  4m 
41s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
25s{color} | {color:blue} hbase-hadoop2-compat in master has 18 extant Findbugs 
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
58s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  3m 
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  2m 
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  2m 
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  4m 
30s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}  
8m 37s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4 
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  3m 
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m  
0s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  0m 
29s{color} | {color:green} hbase-hadoop-compat in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  0m 
32s{color} | {color:green} hbase-hadoop2-compat in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}123m  
5s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
59s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}168m 56s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21150 |
| JIRA Patch URL |

[jira] [Commented] (HBASE-21150) Avoid delay in first flushes due to overheads in table metrics registration

2018-09-04 Thread Duo Zhang (JIRA)



[ 
https://issues.apache.org/jira/browse/HBASE-21150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16603852#comment-16603852
 ] 

Duo Zhang commented on HBASE-21150:
---

Please stop changing the issue type [~yuzhih...@gmail.com], this is not a bug.

> Avoid delay in first flushes due to overheads in table metrics registration
> ---
>
> Key: HBASE-21150
> URL: https://issues.apache.org/jira/browse/HBASE-21150
> Project: HBase
>  Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Minor
> Fix For: 3.0.0, 2.2.0
>
> Attachments: 21150.v1.txt, 21150.v2.txt, 21150.v3.txt, 21150.v4.txt
>
>
> After HBASE-15728 is integrated, the lazy table metrics registration results 
> in penalty for the first flushes.
> Excerpt from log shows delay (note the same timestamp 08:18:23,234) :
> {code:java}
> 2018-09-02 08:18:23,232 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(124): Creating new  
> MetricsTableSourceImpl for table 'testtb-1535901500805'
> 2018-09-02 08:18:23,233 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(137): registering metrics for testtb-   
> 1535901500805
> 2018-09-02 08:18:23,234 INFO  
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.HRegion(2822): Finished flush of dataSize ~2.29 KB/2343,   
> heapSize ~5.16 KB/5280, currentSize=0 B/0 for 
> fa403f6a4fb8dbc1a1c389744fce2d58 in 280ms, sequenceid=5, compaction 
> requested=false
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 0 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 5 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2,5,FailOnTimeoutGroup]
> {code}
> This is a regression in that there were multiple (6 ms) delays before the 
> flush can finish, waiting for the metrics table to be registered.
> When first region of the table is opened on region server, we can proactively 
> register table metrics.
> This would avoid the penalty on first flushes for the table.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-21150) Avoid delay in first flushes due to overheads in table metrics registration

2018-09-04 Thread Duo Zhang (JIRA)



[ 
https://issues.apache.org/jira/browse/HBASE-21150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16603812#comment-16603812
 ] 

Duo Zhang commented on HBASE-21150:
---

OK I see the description. Then here the patch will be simple, just call a 
getOrCreateTableSource method after region opening. The modification in 
MetricsTableAggregateSourceImpl is unnecessary. We can use IdLock or 
StripeLock(from guava) to lock a specific table, instead of a big lock that 
locks everything.

> Avoid delay in first flushes due to overheads in table metrics registration
> ---
>
> Key: HBASE-21150
> URL: https://issues.apache.org/jira/browse/HBASE-21150
> Project: HBase
>  Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 21150.v1.txt, 21150.v2.txt, 21150.v3.txt
>
>
> After HBASE-15728 is integrated, the lazy table metrics registration results 
> in penalty for the first flushes.
> Excerpt from log shows delay (note the same timestamp 08:18:23,234) :
> {code:java}
> 2018-09-02 08:18:23,232 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(124): Creating new  
> MetricsTableSourceImpl for table 'testtb-1535901500805'
> 2018-09-02 08:18:23,233 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(137): registering metrics for testtb-   
> 1535901500805
> 2018-09-02 08:18:23,234 INFO  
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.HRegion(2822): Finished flush of dataSize ~2.29 KB/2343,   
> heapSize ~5.16 KB/5280, currentSize=0 B/0 for 
> fa403f6a4fb8dbc1a1c389744fce2d58 in 280ms, sequenceid=5, compaction 
> requested=false
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 0 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 5 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2,5,FailOnTimeoutGroup]
> {code}
> This is a regression in that there were multiple (6 ms) delays before the 
> flush can finish, waiting for the metrics table to be registered.
> When first region of the table is opened on region server, we can proactively 
> register table metrics.
> This would avoid the penalty on first flushes for the table.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-21150) Avoid delay in first flushes due to overheads in table metrics registration

2018-09-04 Thread Duo Zhang (JIRA)



[ 
https://issues.apache.org/jira/browse/HBASE-21150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16603791#comment-16603791
 ] 

Duo Zhang commented on HBASE-21150:
---

What’s your point Ted? What is the delay? How long?

> Avoid delay in first flushes due to overheads in table metrics registration
> ---
>
> Key: HBASE-21150
> URL: https://issues.apache.org/jira/browse/HBASE-21150
> Project: HBase
>  Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 21150.v1.txt, 21150.v2.txt, 21150.v3.txt
>
>
> After HBASE-15728 is integrated, the lazy table metrics registration results 
> in penalty for the first flushes.
> Excerpt from log shows delay (note the same timestamp 08:18:23,234) :
> {code:java}
> 2018-09-02 08:18:23,232 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(124): Creating new  
> MetricsTableSourceImpl for table 'testtb-1535901500805'
> 2018-09-02 08:18:23,233 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(137): registering metrics for testtb-   
> 1535901500805
> 2018-09-02 08:18:23,234 INFO  
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.HRegion(2822): Finished flush of dataSize ~2.29 KB/2343,   
> heapSize ~5.16 KB/5280, currentSize=0 B/0 for 
> fa403f6a4fb8dbc1a1c389744fce2d58 in 280ms, sequenceid=5, compaction 
> requested=false
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 0 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 5 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2,5,FailOnTimeoutGroup]
> {code}
> This is a regression.
> When first region of the table is opened on region server, we can proactively 
> register table metrics.
> This would avoid the penalty on first flushes for the table.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-21150) Avoid delay in first flushes due to overheads in table metrics registration

2018-09-04 Thread Ted Yu (JIRA)



[ 
https://issues.apache.org/jira/browse/HBASE-21150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16603790#comment-16603790
 ] 

Ted Yu commented on HBASE-21150:


TestReplicationKillSlaveRSWithSeparateOldWALs passed locally with patch.

> Avoid delay in first flushes due to overheads in table metrics registration
> ---
>
> Key: HBASE-21150
> URL: https://issues.apache.org/jira/browse/HBASE-21150
> Project: HBase
>  Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 21150.v1.txt, 21150.v2.txt, 21150.v3.txt
>
>
> After HBASE-15728 is integrated, the lazy table metrics registration results 
> in penalty for the first flushes.
> Excerpt from log shows delay (note the same timestamp 08:18:23,234) :
> {code:java}
> 2018-09-02 08:18:23,232 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(124): Creating new  
> MetricsTableSourceImpl for table 'testtb-1535901500805'
> 2018-09-02 08:18:23,233 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(137): registering metrics for testtb-   
> 1535901500805
> 2018-09-02 08:18:23,234 INFO  
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.HRegion(2822): Finished flush of dataSize ~2.29 KB/2343,   
> heapSize ~5.16 KB/5280, currentSize=0 B/0 for 
> fa403f6a4fb8dbc1a1c389744fce2d58 in 280ms, sequenceid=5, compaction 
> requested=false
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 0 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 5 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2,5,FailOnTimeoutGroup]
> {code}
> This is a regression.
> When first region of the table is opened on region server, we can proactively 
> register table metrics.
> This would avoid the penalty on first flushes for the table.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-21150) Avoid delay in first flushes due to overheads in table metrics registration

2018-09-04 Thread Hadoop QA (JIRA)



[ 
https://issues.apache.org/jira/browse/HBASE-21150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16603784#comment-16603784
 ] 

Hadoop QA commented on HBASE-21150:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
17s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue}  0m  
2s{color} | {color:blue} The patch file was not named according to hbase's 
naming conventions. Please see 
https://yetus.apache.org/documentation/0.7.0/precommit-patchnames for 
instructions. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}  
0m  0s{color} | {color:orange} The patch doesn't appear to include any new or 
modified tests. Please justify why no new tests are needed for this patch. Also 
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  1m 
57s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  6m 
27s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  2m 
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  3m 
56s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
20s{color} | {color:blue} hbase-hadoop2-compat in master has 18 extant Findbugs 
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
54s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  3m 
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  2m 
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  2m 
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  4m 
28s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}  
7m 16s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4 
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  2m 
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
48s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  0m 
24s{color} | {color:green} hbase-hadoop-compat in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  0m 
27s{color} | {color:green} hbase-hadoop2-compat in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}231m  9s{color} 
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
40s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}276m 16s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21150 |
| JIRA Patch URL |

[jira] [Commented] (HBASE-21150) Avoid delay in first flushes due to overheads in table metrics registration

2018-09-04 Thread Andrew Purtell (JIRA)



[ 
https://issues.apache.org/jira/browse/HBASE-21150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16603783#comment-16603783
 ] 

Andrew Purtell commented on HBASE-21150:


Yes, my apologies for picking back the patch prematurely. This issue is no 
longer relevant. I'm going to resolve this as Not A Problem (after my reverts). 
Please feel free to reopen if you like but as Xu points out HBASE-21140 is 
already open to do a backport, so we can follow up there if someone wants to 
backport this change, and fix it up for Java 7.

> Avoid delay in first flushes due to overheads in table metrics registration
> ---
>
> Key: HBASE-21150
> URL: https://issues.apache.org/jira/browse/HBASE-21150
> Project: HBase
>  Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 21150.v1.txt, 21150.v2.txt, 21150.v3.txt
>
>
> After HBASE-15728 is integrated, the lazy table metrics registration results 
> in penalty for the first flushes.
> Excerpt from log shows delay (note the same timestamp 08:18:23,234) :
> {code:java}
> 2018-09-02 08:18:23,232 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(124): Creating new  
> MetricsTableSourceImpl for table 'testtb-1535901500805'
> 2018-09-02 08:18:23,233 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(137): registering metrics for testtb-   
> 1535901500805
> 2018-09-02 08:18:23,234 INFO  
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.HRegion(2822): Finished flush of dataSize ~2.29 KB/2343,   
> heapSize ~5.16 KB/5280, currentSize=0 B/0 for 
> fa403f6a4fb8dbc1a1c389744fce2d58 in 280ms, sequenceid=5, compaction 
> requested=false
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 0 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 5 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2,5,FailOnTimeoutGroup]
> {code}
> This is a regression.
> When first region of the table is opened on region server, we can proactively 
> register table metrics.
> This would avoid the penalty on first flushes for the table.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-21150) Avoid delay in first flushes due to overheads in table metrics registration

2018-09-04 Thread Ted Yu (JIRA)



[ 
https://issues.apache.org/jira/browse/HBASE-21150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16603782#comment-16603782
 ] 

Ted Yu commented on HBASE-21150:


bq. you use a thread pool to execute the registration

The pool is used to avoid adding latency to region open.

bq. will still do registration when getting

As I mentioned in the first comment of the JIRA, additional logging at the 
above place didn't show up in the test output.
Meaning, by the time flushing needs to perform metrics operation, the 
registration is already complete.
The call to helper above is a safe net - not to be executed under normal 
situation.

Also, the helper doesn't rely on computeIfAbsent - making it suitable to be 
backported (as Xu has done).

> Avoid delay in first flushes due to overheads in table metrics registration
> ---
>
> Key: HBASE-21150
> URL: https://issues.apache.org/jira/browse/HBASE-21150
> Project: HBase
>  Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 21150.v1.txt, 21150.v2.txt, 21150.v3.txt
>
>
> After HBASE-15728 is integrated, the lazy table metrics registration results 
> in penalty for the first flushes.
> Excerpt from log shows delay (note the same timestamp 08:18:23,234) :
> {code:java}
> 2018-09-02 08:18:23,232 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(124): Creating new  
> MetricsTableSourceImpl for table 'testtb-1535901500805'
> 2018-09-02 08:18:23,233 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(137): registering metrics for testtb-   
> 1535901500805
> 2018-09-02 08:18:23,234 INFO  
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.HRegion(2822): Finished flush of dataSize ~2.29 KB/2343,   
> heapSize ~5.16 KB/5280, currentSize=0 B/0 for 
> fa403f6a4fb8dbc1a1c389744fce2d58 in 280ms, sequenceid=5, compaction 
> requested=false
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 0 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 5 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2,5,FailOnTimeoutGroup]
> {code}
> This is a regression.
> When first region of the table is opened on region server, we can proactively 
> register table metrics.
> This would avoid the penalty on first flushes for the table.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-21150) Avoid delay in first flushes due to overheads in table metrics registration

2018-09-04 Thread Duo Zhang (JIRA)



[ 
https://issues.apache.org/jira/browse/HBASE-21150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16603772#comment-16603772
 ] 

Duo Zhang commented on HBASE-21150:
---

Oh you will still do registration when getting, then what is the real 
difference between the patch here and HBASE-21136? Just adding more complexity 
by adding more steps in region opening and more methods in an interface?

> Avoid delay in first flushes due to overheads in table metrics registration
> ---
>
> Key: HBASE-21150
> URL: https://issues.apache.org/jira/browse/HBASE-21150
> Project: HBase
>  Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 21150.v1.txt, 21150.v2.txt, 21150.v3.txt
>
>
> After HBASE-15728 is integrated, the lazy table metrics registration results 
> in penalty for the first flushes.
> Excerpt from log shows delay (note the same timestamp 08:18:23,234) :
> {code:java}
> 2018-09-02 08:18:23,232 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(124): Creating new  
> MetricsTableSourceImpl for table 'testtb-1535901500805'
> 2018-09-02 08:18:23,233 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(137): registering metrics for testtb-   
> 1535901500805
> 2018-09-02 08:18:23,234 INFO  
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.HRegion(2822): Finished flush of dataSize ~2.29 KB/2343,   
> heapSize ~5.16 KB/5280, currentSize=0 B/0 for 
> fa403f6a4fb8dbc1a1c389744fce2d58 in 280ms, sequenceid=5, compaction 
> requested=false
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 0 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 5 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2,5,FailOnTimeoutGroup]
> {code}
> This is a regression.
> When first region of the table is opened on region server, we can proactively 
> register table metrics.
> This would avoid the penalty on first flushes for the table.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-21150) Avoid delay in first flushes due to overheads in table metrics registration

2018-09-04 Thread Duo Zhang (JIRA)



[ 
https://issues.apache.org/jira/browse/HBASE-21150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16603769#comment-16603769
 ] 

Duo Zhang commented on HBASE-21150:
---

First, please use ConcurrentMap instead of Map MetricsTableAggregateSourceImpl, 
since you removed the computeIfAbsent, it is not necessary to be a 
ConcurrentHashMap but it still need to be a ConcurrentMap, as for different 
tables, we may do get and put at the same time. HashMap is not thread safe.

Second, you use a thread pool to execute the registration, then it could still 
happen that we want to get a metrics but it is not registered yet, and cause 
another NPE...

> Avoid delay in first flushes due to overheads in table metrics registration
> ---
>
> Key: HBASE-21150
> URL: https://issues.apache.org/jira/browse/HBASE-21150
> Project: HBase
>  Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 21150.v1.txt, 21150.v2.txt, 21150.v3.txt
>
>
> After HBASE-15728 is integrated, the lazy table metrics registration results 
> in penalty for the first flushes.
> Excerpt from log shows delay (note the same timestamp 08:18:23,234) :
> {code:java}
> 2018-09-02 08:18:23,232 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(124): Creating new  
> MetricsTableSourceImpl for table 'testtb-1535901500805'
> 2018-09-02 08:18:23,233 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(137): registering metrics for testtb-   
> 1535901500805
> 2018-09-02 08:18:23,234 INFO  
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.HRegion(2822): Finished flush of dataSize ~2.29 KB/2343,   
> heapSize ~5.16 KB/5280, currentSize=0 B/0 for 
> fa403f6a4fb8dbc1a1c389744fce2d58 in 280ms, sequenceid=5, compaction 
> requested=false
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 0 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 5 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2,5,FailOnTimeoutGroup]
> {code}
> This is a regression.
> When first region of the table is opened on region server, we can proactively 
> register table metrics.
> This would avoid the penalty on first flushes for the table.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-21150) Avoid delay in first flushes due to overheads in table metrics registration

2018-09-04 Thread Xu Cang (JIRA)



[ 
https://issues.apache.org/jira/browse/HBASE-21150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16603756#comment-16603756
 ] 

Xu Cang commented on HBASE-21150:
-

I put a patch for branch-1 here: 
https://issues.apache.org/jira/browse/HBASE-21140 

([~apurtell] Since I see you reverted both the fix and original backporting 
code. So my patch contains 3 parts including original metrics code. Please let 
me know if you would like to have a patch just for the fix itself.)

> Avoid delay in first flushes due to overheads in table metrics registration
> ---
>
> Key: HBASE-21150
> URL: https://issues.apache.org/jira/browse/HBASE-21150
> Project: HBase
>  Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 21150.v1.txt, 21150.v2.txt, 21150.v3.txt
>
>
> After HBASE-15728 is integrated, the lazy table metrics registration results 
> in penalty for the first flushes.
> Excerpt from log shows delay (note the same timestamp 08:18:23,234) :
> {code:java}
> 2018-09-02 08:18:23,232 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(124): Creating new  
> MetricsTableSourceImpl for table 'testtb-1535901500805'
> 2018-09-02 08:18:23,233 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(137): registering metrics for testtb-   
> 1535901500805
> 2018-09-02 08:18:23,234 INFO  
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.HRegion(2822): Finished flush of dataSize ~2.29 KB/2343,   
> heapSize ~5.16 KB/5280, currentSize=0 B/0 for 
> fa403f6a4fb8dbc1a1c389744fce2d58 in 280ms, sequenceid=5, compaction 
> requested=false
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 0 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 5 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2,5,FailOnTimeoutGroup]
> {code}
> This is a regression.
> When first region of the table is opened on region server, we can proactively 
> register table metrics.
> This would avoid the penalty on first flushes for the table.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-21150) Avoid delay in first flushes due to overheads in table metrics registration

2018-09-04 Thread Duo Zhang (JIRA)



[ 
https://issues.apache.org/jira/browse/HBASE-21150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16603714#comment-16603714
 ] 

Duo Zhang commented on HBASE-21150:
---

Please do not commit yet. There are obvious concurrent issues in this patch. 
Anyway I like the approach on moving the registration to region open so we can 
just do get when updating metrics. On a mobile device now, will be back later.

> Avoid delay in first flushes due to overheads in table metrics registration
> ---
>
> Key: HBASE-21150
> URL: https://issues.apache.org/jira/browse/HBASE-21150
> Project: HBase
>  Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 21150.v1.txt, 21150.v2.txt
>
>
> After HBASE-15728 is integrated, the lazy table metrics registration results 
> in penalty for the first flushes.
> Excerpt from log shows delay (note the same timestamp 08:18:23,234) :
> {code:java}
> 2018-09-02 08:18:23,232 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(124): Creating new  
> MetricsTableSourceImpl for table 'testtb-1535901500805'
> 2018-09-02 08:18:23,233 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(137): registering metrics for testtb-   
> 1535901500805
> 2018-09-02 08:18:23,234 INFO  
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.HRegion(2822): Finished flush of dataSize ~2.29 KB/2343,   
> heapSize ~5.16 KB/5280, currentSize=0 B/0 for 
> fa403f6a4fb8dbc1a1c389744fce2d58 in 280ms, sequenceid=5, compaction 
> requested=false
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 0 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 5 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2,5,FailOnTimeoutGroup]
> {code}
> This is a regression.
> When first region of the table is opened on region server, we can proactively 
> register table metrics.
> This would avoid the penalty on first flushes for the table.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-21150) Avoid delay in first flushes due to overheads in table metrics registration

2018-09-04 Thread Xu Cang (JIRA)



[ 
https://issues.apache.org/jira/browse/HBASE-21150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16603584#comment-16603584
 ] 

Xu Cang commented on HBASE-21150:
-

[~apurtell]

Actually, Ted's fix in this Jira removed using 'computeIfAbsent'. So, if we 
port Ted's patch here, branch-1 should be okay. I can do it tonight and run 
some tests. (If you can wait until tonight)

> Avoid delay in first flushes due to overheads in table metrics registration
> ---
>
> Key: HBASE-21150
> URL: https://issues.apache.org/jira/browse/HBASE-21150
> Project: HBase
>  Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 21150.v1.txt, 21150.v2.txt
>
>
> After HBASE-15728 is integrated, the lazy table metrics registration results 
> in penalty for the first flushes.
> Excerpt from log shows delay (note the same timestamp 08:18:23,234) :
> {code:java}
> 2018-09-02 08:18:23,232 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(124): Creating new  
> MetricsTableSourceImpl for table 'testtb-1535901500805'
> 2018-09-02 08:18:23,233 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(137): registering metrics for testtb-   
> 1535901500805
> 2018-09-02 08:18:23,234 INFO  
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.HRegion(2822): Finished flush of dataSize ~2.29 KB/2343,   
> heapSize ~5.16 KB/5280, currentSize=0 B/0 for 
> fa403f6a4fb8dbc1a1c389744fce2d58 in 280ms, sequenceid=5, compaction 
> requested=false
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 0 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 5 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2,5,FailOnTimeoutGroup]
> {code}
> This is a regression.
> When first region of the table is opened on region server, we can proactively 
> register table metrics.
> This would avoid the penalty on first flushes for the table.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-21150) Avoid delay in first flushes due to overheads in table metrics registration

2018-09-04 Thread Andrew Purtell (JIRA)



[ 
https://issues.apache.org/jira/browse/HBASE-21150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16603523#comment-16603523
 ] 

Andrew Purtell commented on HBASE-21150:


Oops, ok missed that and the consideration mentioned on that issue. will revert 
what I picked to branch-1

> Avoid delay in first flushes due to overheads in table metrics registration
> ---
>
> Key: HBASE-21150
> URL: https://issues.apache.org/jira/browse/HBASE-21150
> Project: HBase
>  Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 21150.v1.txt, 21150.v2.txt
>
>
> After HBASE-15728 is integrated, the lazy table metrics registration results 
> in penalty for the first flushes.
> Excerpt from log shows delay (note the same timestamp 08:18:23,234) :
> {code:java}
> 2018-09-02 08:18:23,232 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(124): Creating new  
> MetricsTableSourceImpl for table 'testtb-1535901500805'
> 2018-09-02 08:18:23,233 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(137): registering metrics for testtb-   
> 1535901500805
> 2018-09-02 08:18:23,234 INFO  
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.HRegion(2822): Finished flush of dataSize ~2.29 KB/2343,   
> heapSize ~5.16 KB/5280, currentSize=0 B/0 for 
> fa403f6a4fb8dbc1a1c389744fce2d58 in 280ms, sequenceid=5, compaction 
> requested=false
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 0 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 5 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2,5,FailOnTimeoutGroup]
> {code}
> This is a regression.
> When first region of the table is opened on region server, we can proactively 
> register table metrics.
> This would avoid the penalty on first flushes for the table.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-21150) Avoid delay in first flushes due to overheads in table metrics registration

2018-09-04 Thread Xu Cang (JIRA)



[ 
https://issues.apache.org/jira/browse/HBASE-21150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16603509#comment-16603509
 ] 

Xu Cang commented on HBASE-21150:
-

[~apurtell] this Jira https://issues.apache.org/jira/browse/HBASE-21140 is 
tracking the branch-1 work. 

> Avoid delay in first flushes due to overheads in table metrics registration
> ---
>
> Key: HBASE-21150
> URL: https://issues.apache.org/jira/browse/HBASE-21150
> Project: HBase
>  Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 21150.v1.txt, 21150.v2.txt
>
>
> After HBASE-15728 is integrated, the lazy table metrics registration results 
> in penalty for the first flushes.
> Excerpt from log shows delay (note the same timestamp 08:18:23,234) :
> {code:java}
> 2018-09-02 08:18:23,232 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(124): Creating new  
> MetricsTableSourceImpl for table 'testtb-1535901500805'
> 2018-09-02 08:18:23,233 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(137): registering metrics for testtb-   
> 1535901500805
> 2018-09-02 08:18:23,234 INFO  
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.HRegion(2822): Finished flush of dataSize ~2.29 KB/2343,   
> heapSize ~5.16 KB/5280, currentSize=0 B/0 for 
> fa403f6a4fb8dbc1a1c389744fce2d58 in 280ms, sequenceid=5, compaction 
> requested=false
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 0 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 5 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2,5,FailOnTimeoutGroup]
> {code}
> This is a regression.
> When first region of the table is opened on region server, we can proactively 
> register table metrics.
> This would avoid the penalty on first flushes for the table.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-21150) Avoid delay in first flushes due to overheads in table metrics registration

2018-09-04 Thread Xu Cang (JIRA)



[ 
https://issues.apache.org/jira/browse/HBASE-21150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16603505#comment-16603505
 ] 

Xu Cang commented on HBASE-21150:
-

Now I see. Thanks. [~yuzhih...@gmail.com] +1.

> Avoid delay in first flushes due to overheads in table metrics registration
> ---
>
> Key: HBASE-21150
> URL: https://issues.apache.org/jira/browse/HBASE-21150
> Project: HBase
>  Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 21150.v1.txt, 21150.v2.txt
>
>
> After HBASE-15728 is integrated, the lazy table metrics registration results 
> in penalty for the first flushes.
> Excerpt from log shows delay (note the same timestamp 08:18:23,234) :
> {code:java}
> 2018-09-02 08:18:23,232 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(124): Creating new  
> MetricsTableSourceImpl for table 'testtb-1535901500805'
> 2018-09-02 08:18:23,233 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(137): registering metrics for testtb-   
> 1535901500805
> 2018-09-02 08:18:23,234 INFO  
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.HRegion(2822): Finished flush of dataSize ~2.29 KB/2343,   
> heapSize ~5.16 KB/5280, currentSize=0 B/0 for 
> fa403f6a4fb8dbc1a1c389744fce2d58 in 280ms, sequenceid=5, compaction 
> requested=false
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 0 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 5 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2,5,FailOnTimeoutGroup]
> {code}
> This is a regression.
> When first region of the table is opened on region server, we can proactively 
> register table metrics.
> This would avoid the penalty on first flushes for the table.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-21150) Avoid delay in first flushes due to overheads in table metrics registration

2018-09-04 Thread Andrew Purtell (JIRA)



[ 
https://issues.apache.org/jira/browse/HBASE-21150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16603501#comment-16603501
 ] 

Andrew Purtell commented on HBASE-21150:


FYI I'm committing an addendum to the cherry pick of HBASE-15728 to branch-1 to 
fix a NPE surfaced by TestRegionLocationFinder

> Avoid delay in first flushes due to overheads in table metrics registration
> ---
>
> Key: HBASE-21150
> URL: https://issues.apache.org/jira/browse/HBASE-21150
> Project: HBase
>  Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 21150.v1.txt, 21150.v2.txt
>
>
> After HBASE-15728 is integrated, the lazy table metrics registration results 
> in penalty for the first flushes.
> Excerpt from log shows delay (note the same timestamp 08:18:23,234) :
> {code:java}
> 2018-09-02 08:18:23,232 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(124): Creating new  
> MetricsTableSourceImpl for table 'testtb-1535901500805'
> 2018-09-02 08:18:23,233 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(137): registering metrics for testtb-   
> 1535901500805
> 2018-09-02 08:18:23,234 INFO  
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.HRegion(2822): Finished flush of dataSize ~2.29 KB/2343,   
> heapSize ~5.16 KB/5280, currentSize=0 B/0 for 
> fa403f6a4fb8dbc1a1c389744fce2d58 in 280ms, sequenceid=5, compaction 
> requested=false
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 0 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 5 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2,5,FailOnTimeoutGroup]
> {code}
> This is a regression.
> When first region of the table is opened on region server, we can proactively 
> register table metrics.
> This would avoid the penalty on first flushes for the table.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-21150) Avoid delay in first flushes due to overheads in table metrics registration

2018-09-04 Thread Ted Yu (JIRA)



[ 
https://issues.apache.org/jira/browse/HBASE-21150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16603490#comment-16603490
 ] 

Ted Yu commented on HBASE-21150:


computeIfAbsent is not needed.
See how my patch achieves synchronization.

> Avoid delay in first flushes due to overheads in table metrics registration
> ---
>
> Key: HBASE-21150
> URL: https://issues.apache.org/jira/browse/HBASE-21150
> Project: HBase
>  Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 21150.v1.txt
>
>
> After HBASE-15728 is integrated, the lazy table metrics registration results 
> in penalty for the first flushes.
> Excerpt from log shows delay (note the same timestamp 08:18:23,234) :
> {code:java}
> 2018-09-02 08:18:23,232 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(124): Creating new  
> MetricsTableSourceImpl for table 'testtb-1535901500805'
> 2018-09-02 08:18:23,233 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(137): registering metrics for testtb-   
> 1535901500805
> 2018-09-02 08:18:23,234 INFO  
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.HRegion(2822): Finished flush of dataSize ~2.29 KB/2343,   
> heapSize ~5.16 KB/5280, currentSize=0 B/0 for 
> fa403f6a4fb8dbc1a1c389744fce2d58 in 280ms, sequenceid=5, compaction 
> requested=false
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 0 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 5 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2,5,FailOnTimeoutGroup]
> {code}
> This is a regression.
> When first region of the table is opened on region server, we can proactively 
> register table metrics.
> This would avoid the penalty on first flushes for the table.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-21150) Avoid delay in first flushes due to overheads in table metrics registration

2018-09-04 Thread Hadoop QA (JIRA)



[ 
https://issues.apache.org/jira/browse/HBASE-21150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16603486#comment-16603486
 ] 

Hadoop QA commented on HBASE-21150:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
12s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue}  0m  
2s{color} | {color:blue} The patch file was not named according to hbase's 
naming conventions. Please see 
https://yetus.apache.org/documentation/0.7.0/precommit-patchnames for 
instructions. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}  
0m  0s{color} | {color:orange} The patch doesn't appear to include any new or 
modified tests. Please justify why no new tests are needed for this patch. Also 
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
32s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
 4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  2m 
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  4m 
16s{color} | {color:green} branch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
26s{color} | {color:blue} hbase-hadoop2-compat in master has 18 extant Findbugs 
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m  
4s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
 4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  2m 
25s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red}  0m 20s{color} 
| {color:red} hbase-hadoop2-compat generated 1 new + 39 unchanged - 0 fixed = 
40 total (was 39) {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red}  1m 48s{color} 
| {color:red} hbase-server generated 1 new + 187 unchanged - 1 fixed = 188 
total (was 188) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
11s{color} | {color:red} hbase-hadoop-compat: The patch generated 2 new + 0 
unchanged - 0 fixed = 2 total (was 0) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
13s{color} | {color:red} hbase-hadoop2-compat: The patch generated 3 new + 0 
unchanged - 0 fixed = 3 total (was 0) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green}  4m 
11s{color} | {color:green} patch has no errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}  
8m  0s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4 
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  3m  
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
55s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  0m 
26s{color} | {color:green} hbase-hadoop-compat in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  0m 
30s{color} | {color:green} hbase-hadoop2-compat in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}151m 
14s{color} | {color:green} hbase-server

[jira] [Commented] (HBASE-21150) Avoid delay in first flushes due to overheads in table metrics registration

2018-09-04 Thread Xu Cang (JIRA)



[ 
https://issues.apache.org/jira/browse/HBASE-21150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16603470#comment-16603470
 ] 

Xu Cang commented on HBASE-21150:
-

I think we rely on ConcurrentHashMap's "computeIfAbsent" 's atomicity from this 
patch https://issues.apache.org/jira/browse/HBASE-21136 ? 

> Avoid delay in first flushes due to overheads in table metrics registration
> ---
>
> Key: HBASE-21150
> URL: https://issues.apache.org/jira/browse/HBASE-21150
> Project: HBase
>  Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 21150.v1.txt
>
>
> After HBASE-15728 is integrated, the lazy table metrics registration results 
> in penalty for the first flushes.
> Excerpt from log shows delay (note the same timestamp 08:18:23,234) :
> {code:java}
> 2018-09-02 08:18:23,232 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(124): Creating new  
> MetricsTableSourceImpl for table 'testtb-1535901500805'
> 2018-09-02 08:18:23,233 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(137): registering metrics for testtb-   
> 1535901500805
> 2018-09-02 08:18:23,234 INFO  
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.HRegion(2822): Finished flush of dataSize ~2.29 KB/2343,   
> heapSize ~5.16 KB/5280, currentSize=0 B/0 for 
> fa403f6a4fb8dbc1a1c389744fce2d58 in 280ms, sequenceid=5, compaction 
> requested=false
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 0 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 5 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2,5,FailOnTimeoutGroup]
> {code}
> This is a regression.
> When first region of the table is opened on region server, we can proactively 
> register table metrics.
> This would avoid the penalty on first flushes for the table.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-21150) Avoid delay in first flushes due to overheads in table metrics registration

2018-09-04 Thread Ted Yu (JIRA)



[ 
https://issues.apache.org/jira/browse/HBASE-21150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16603403#comment-16603403
 ] 

Ted Yu commented on HBASE-21150:


ConcurrentHashMap doesn't give us benefit in this case: the key is per table, 
instead of region.

> Avoid delay in first flushes due to overheads in table metrics registration
> ---
>
> Key: HBASE-21150
> URL: https://issues.apache.org/jira/browse/HBASE-21150
> Project: HBase
>  Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 21150.v1.txt
>
>
> After HBASE-15728 is integrated, the lazy table metrics registration results 
> in penalty for the first flushes.
> Excerpt from log shows delay (note the same timestamp 08:18:23,234) :
> {code}
> 2018-09-02 08:18:23,232 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(124): Creating new  
> MetricsTableSourceImpl for table 'testtb-1535901500805'
> 2018-09-02 08:18:23,233 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(137): registering metrics for testtb-   
> 1535901500805
> 2018-09-02 08:18:23,234 INFO  
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.HRegion(2822): Finished flush of dataSize ~2.29 KB/2343,   
> heapSize ~5.16 KB/5280, currentSize=0 B/0 for 
> fa403f6a4fb8dbc1a1c389744fce2d58 in 280ms, sequenceid=5, compaction 
> requested=false
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 0 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 5 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2,5,FailOnTimeoutGroup]
> {code}
> This is a regression.
> When first region of the table is opened on region server, we can proactively 
> register table metrics.
> This would avoid the penalty on first flushes for the table.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HBASE-21150) Avoid delay in first flushes due to overheads in table metrics registration

2018-09-04 Thread Xu Cang (JIRA)



[ 
https://issues.apache.org/jira/browse/HBASE-21150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16603382#comment-16603382
 ] 

Xu Cang commented on HBASE-21150:
-

Small question, why using 
{code:java}
synchronized (tableSources) {{code}
is better than

ConcurrentHashMap?

[~yuzhih...@gmail.com] thanks!

> Avoid delay in first flushes due to overheads in table metrics registration
> ---
>
> Key: HBASE-21150
> URL: https://issues.apache.org/jira/browse/HBASE-21150
> Project: HBase
>  Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Major
> Attachments: 21150.v1.txt
>
>
> After HBASE-15728 is integrated, the lazy table metrics registration results 
> in penalty for the first flushes.
> Excerpt from log shows delay (note the same timestamp 08:18:23,234) :
> {code}
> 2018-09-02 08:18:23,232 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(124): Creating new  
> MetricsTableSourceImpl for table 'testtb-1535901500805'
> 2018-09-02 08:18:23,233 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableSourceImpl(137): registering metrics for testtb-   
> 1535901500805
> 2018-09-02 08:18:23,234 INFO  
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.HRegion(2822): Finished flush of dataSize ~2.29 KB/2343,   
> heapSize ~5.16 KB/5280, currentSize=0 B/0 for 
> fa403f6a4fb8dbc1a1c389744fce2d58 in 280ms, sequenceid=5, compaction 
> requested=false
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 0 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-1,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register   
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52762,1535901497314)-snapshot-pool9-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 5 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52758,1535901497238)-snapshot-pool11-thread-2,5,FailOnTimeoutGroup]
> 2018-09-02 08:18:23,234 DEBUG 
> [rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2] 
> regionserver.MetricsTableAggregateSourceImpl(84): it took 6 ms to register  
> testtb-1535901500805 
> Thread[rs(hw13463.attlocal.net,52760,1535901497280)-snapshot-pool10-thread-2,5,FailOnTimeoutGroup]
> {code}
> This is a regression.
> When first region of the table is opened on region server, we can proactively 
> register table metrics.
> This would avoid the penalty on first flushes for the table.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

41 matches

Mail list logo