[jira] [Updated] (HBASE-16285) Drop RPC requests if it must be considered as timeout at client
[ https://issues.apache.org/jira/browse/HBASE-16285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phil Yang updated HBASE-16285: -- Attachment: HBASE-16285-branch-1-v2.patch For branch-1 (and also can be applied to branch-1.3) > Drop RPC requests if it must be considered as timeout at client > --- > > Key: HBASE-16285 > URL: https://issues.apache.org/jira/browse/HBASE-16285 > Project: HBase > Issue Type: Improvement >Reporter: Phil Yang >Assignee: Phil Yang > Attachments: HBASE-16285-branch-1-v1.patch, > HBASE-16285-branch-1-v2.patch, HBASE-16285-v1.patch, HBASE-16285-v2.patch, > HBASE-16285-v3.patch > > > After HBASE-15593, we have a timeout param in header of RPC requests. We can > use it in more scenes. > A straightforward scene is to drop requests if it has waited so long in RPC > queue and has been dropped by client. Even if we handle this request and send > the response back, it will not be used any more. And client may have sent a > retry. In an extreme case, if the server is slow, all requests may be timeout > or queue-full-exception because we should handle previous requests which have > been dropped by client and many resources at server are wasted. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-9899) for idempotent operation dups, return the result instead of throwing conflict exception
[ https://issues.apache.org/jira/browse/HBASE-9899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-9899: -- Attachment: HBASE-9899-v3.patch > for idempotent operation dups, return the result instead of throwing conflict > exception > --- > > Key: HBASE-9899 > URL: https://issues.apache.org/jira/browse/HBASE-9899 > Project: HBase > Issue Type: Improvement >Reporter: Sergey Shelukhin >Assignee: Guanghao Zhang > Attachments: HBASE-9899-v1.patch, HBASE-9899-v2.patch, > HBASE-9899-v3.patch, HBASE-9899-v3.patch > > > After HBASE-3787, we could store mvcc in operation context, and use it to > convert the modification request into read on dups instead of throwing > OperationConflictException. > MVCC tracking will have to be aware of such MVCC numbers present. Given that > scanners are usually relatively short-lived, that would prevent low watermark > from advancing for quite a bit more time -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-9899) for idempotent operation dups, return the result instead of throwing conflict exception
[ https://issues.apache.org/jira/browse/HBASE-9899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15401588#comment-15401588 ] Guanghao Zhang commented on HBASE-9899: --- It seems not ralated. This patch just read mvcc number from WriteEntry and it didn't change the read/write point directly. This failed ut passed in my local machine and I can't reproduce this failure. Let the hadoop QA run again to see whether it failed. > for idempotent operation dups, return the result instead of throwing conflict > exception > --- > > Key: HBASE-9899 > URL: https://issues.apache.org/jira/browse/HBASE-9899 > Project: HBase > Issue Type: Improvement >Reporter: Sergey Shelukhin >Assignee: Guanghao Zhang > Attachments: HBASE-9899-v1.patch, HBASE-9899-v2.patch, > HBASE-9899-v3.patch > > > After HBASE-3787, we could store mvcc in operation context, and use it to > convert the modification request into read on dups instead of throwing > OperationConflictException. > MVCC tracking will have to be aware of such MVCC numbers present. Given that > scanners are usually relatively short-lived, that would prevent low watermark > from advancing for quite a bit more time -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16285) Drop RPC requests if it must be considered as timeout at client
[ https://issues.apache.org/jira/browse/HBASE-16285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phil Yang updated HBASE-16285: -- Attachment: HBASE-16285-v3.patch Use call.startTime > call.deadline rather than call.startTime >= call.deadline > Drop RPC requests if it must be considered as timeout at client > --- > > Key: HBASE-16285 > URL: https://issues.apache.org/jira/browse/HBASE-16285 > Project: HBase > Issue Type: Improvement >Reporter: Phil Yang >Assignee: Phil Yang > Attachments: HBASE-16285-branch-1-v1.patch, HBASE-16285-v1.patch, > HBASE-16285-v2.patch, HBASE-16285-v3.patch > > > After HBASE-15593, we have a timeout param in header of RPC requests. We can > use it in more scenes. > A straightforward scene is to drop requests if it has waited so long in RPC > queue and has been dropped by client. Even if we handle this request and send > the response back, it will not be used any more. And client may have sent a > retry. In an extreme case, if the server is slow, all requests may be timeout > or queue-full-exception because we should handle previous requests which have > been dropped by client and many resources at server are wasted. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16285) Drop RPC requests if it must be considered as timeout at client
[
https://issues.apache.org/jira/browse/HBASE-16285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15401580#comment-15401580
]
Hadoop QA commented on HBASE-16285:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
24s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 34s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 5s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
25s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
39s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
32s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 2s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 59s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
28s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 30s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 30s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 9s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 9s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
23s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
41s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
53m 10s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
51s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 6s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 3s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 41m 56s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 124m 55s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Timed out junit tests | org.apache.hadoop.hbase.security.TestSecureIPC |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12821260/HBASE-16285-v2.patch |
| JIRA Issue | HBASE-16285 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux pomona.apache.org 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT
Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 7e6f5627 |
| Default Java | 1.7.0_80 |
| Multi-JDK versions | /usr/local/jenkins/java/jdk1.8.0:1.8.0
/usr/local/asfpackages/java/jdk1.7.0_80:1.7.0_80 |
| findbugs | v3.0.0 |
| unit |
https
[jira] [Commented] (HBASE-16209) Provide an ExponentialBackOffPolicy sleep between failed region open requests
[ https://issues.apache.org/jira/browse/HBASE-16209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15401570#comment-15401570 ] Hudson commented on HBASE-16209: FAILURE: Integrated in HBase-1.4 #317 (See [https://builds.apache.org/job/HBase-1.4/317/]) HBASE-16209 addendum fix region move delay (zhangduo: rev c46f56b382ca1023c097326acda275e615b6c4e0) * hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/ClosedRegionHandler.java > Provide an ExponentialBackOffPolicy sleep between failed region open requests > - > > Key: HBASE-16209 > URL: https://issues.apache.org/jira/browse/HBASE-16209 > Project: HBase > Issue Type: Bug >Reporter: Joseph >Assignee: Joseph > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-16209-addendum.patch, > HBASE-16209-branch-1-addendum.patch, HBASE-16209-branch-1.patch, > HBASE-16209-v2.patch, HBASE-16209.patch > > > Related to HBASE-16138. As of now we currently have no pause between retrying > failed region open requests. And with a low maximumAttempt default, we can > quickly use up all our regionOpen retries if the server is in a bad state. I > added in a ExponentialBackOffPolicy so that we spread out the timing of our > open region retries in AssignmentManager. Review board at > https://reviews.apache.org/r/50011/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16257) Move staging dir to be under hbase root dir
[
https://issues.apache.org/jira/browse/HBASE-16257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15401540#comment-15401540
]
Hadoop QA commented on HBASE-16257:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 5 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
6s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 36s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 19s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
15s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
50s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
20s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 21s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 20s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
38s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 32s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 32s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 29s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 29s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
54s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 0s
{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
35m 14s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m 6s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 21s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 15s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 59s
{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 3s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 17m 13s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
27s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 87m 55s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12821259/HBASE-16257-v2.patch |
| JIRA Issue |
[jira] [Commented] (HBASE-16296) Reverse scan performance degrades when scanner cache size matches page filter size
[
https://issues.apache.org/jira/browse/HBASE-16296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15401538#comment-15401538
]
ramkrishna.s.vasudevan commented on HBASE-16296:
{code}
if (isFilterDoneInternal()) {
4311 return false;
4312}
{code}
Should work for filterList case. But for single filter this will not work.
It will go ahead with the the previous existing way. (so no problem).
So in order to avoid even that and for changing the behaviour of FilterList I
think its better we return filterRowKey based on filterAllRemaining.
In the sense if filterAllRemaining is true then filterRowKey should also be
true.
> Reverse scan performance degrades when scanner cache size matches page filter
> size
> --
>
> Key: HBASE-16296
> URL: https://issues.apache.org/jira/browse/HBASE-16296
> Project: HBase
> Issue Type: Bug
>Reporter: James Taylor
> Attachments: 16296-MAYBE-v2.txt, 16296-MAYBE.txt,
> generatedata-snippet.java, repro-snippet.java
>
>
> When a reverse scan is done, the server seems to not know it's done when the
> scanner cache size matches the number of rows in a PageFilter. See
> PHOENIX-3121 for how this manifests itself. We have a standalone, pure HBase
> API reproducer too that I'll attach (courtesy of [~churromorales] and
> [~mujtabachohan]).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16225) Refactor ScanQueryMatcher
[
https://issues.apache.org/jira/browse/HBASE-16225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15401519#comment-15401519
]
Hadoop QA commented on HBASE-16225:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 18 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 17s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
19s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 13s
{color} | {color:green} branch-1 passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 3s
{color} | {color:green} branch-1 passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
51s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
38s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
14s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 51s
{color} | {color:green} branch-1 passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 53s
{color} | {color:green} branch-1 passed with JDK v1.7.0_80 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
12s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 13s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 13s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 59s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 59s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
48s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
33s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
19m 25s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
49s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 8s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 53s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 7s
{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 18m 32s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
19s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 63m 12s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12821258/HBASE-16225-branch-1-v1.patch
|
| JIRA Issue | HBASE-16225 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/L
[jira] [Commented] (HBASE-9899) for idempotent operation dups, return the result instead of throwing conflict exception
[ https://issues.apache.org/jira/browse/HBASE-9899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15401518#comment-15401518 ] stack commented on HBASE-9899: -- Is the failure related? The hadoop build failures can't be related. I can take a look at those [~zghaobac] > for idempotent operation dups, return the result instead of throwing conflict > exception > --- > > Key: HBASE-9899 > URL: https://issues.apache.org/jira/browse/HBASE-9899 > Project: HBase > Issue Type: Improvement >Reporter: Sergey Shelukhin >Assignee: Guanghao Zhang > Attachments: HBASE-9899-v1.patch, HBASE-9899-v2.patch, > HBASE-9899-v3.patch > > > After HBASE-3787, we could store mvcc in operation context, and use it to > convert the modification request into read on dups instead of throwing > OperationConflictException. > MVCC tracking will have to be aware of such MVCC numbers present. Given that > scanners are usually relatively short-lived, that would prevent low watermark > from advancing for quite a bit more time -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15756) Pluggable RpcServer
[ https://issues.apache.org/jira/browse/HBASE-15756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15401512#comment-15401512 ] stack commented on HBASE-15756: --- Thanks [~aoxiang] Will try it in morning boss. > Pluggable RpcServer > --- > > Key: HBASE-15756 > URL: https://issues.apache.org/jira/browse/HBASE-15756 > Project: HBase > Issue Type: Improvement > Components: Performance, rpc >Reporter: binlijin >Assignee: binlijin >Priority: Critical > Attachments: Netty4RpcServer_forperf.patch, NettyRpcServer.patch, > NettyRpcServer_forperf.patch, gets.png, idle.png, queue.png > > > Current we use a simple RpcServer, and can not configure and use other > implementation.This issue is to make the RpcServer pluggable, so we can make > other implementation for example netty rpc server. Patch will upload laterly -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-9899) for idempotent operation dups, return the result instead of throwing conflict exception
[
https://issues.apache.org/jira/browse/HBASE-9899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15401510#comment-15401510
]
Hadoop QA commented on HBASE-9899:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 17s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
27s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 5s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 54s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
33s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
57s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 51s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 54s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
7s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 57s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 57s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 55s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 55s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
51s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
33s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 19m 57s
{color} | {color:red} Patch causes 11 errors with Hadoop v2.6.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 22m 48s
{color} | {color:red} Patch causes 11 errors with Hadoop v2.6.2. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 25m 38s
{color} | {color:red} Patch causes 11 errors with Hadoop v2.6.3. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 28m 29s
{color} | {color:red} Patch causes 11 errors with Hadoop v2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
23s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 44s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 52s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 59s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 92m 6s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
31s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 144m 4s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
[jira] [Updated] (HBASE-16285) Drop RPC requests if it must be considered as timeout at client
[ https://issues.apache.org/jira/browse/HBASE-16285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phil Yang updated HBASE-16285: -- Attachment: HBASE-16285-v2.patch Add Call.startTime to save one System.current Remove dropTimeoutRequestDelay to drop timeout requests immediately > Drop RPC requests if it must be considered as timeout at client > --- > > Key: HBASE-16285 > URL: https://issues.apache.org/jira/browse/HBASE-16285 > Project: HBase > Issue Type: Improvement >Reporter: Phil Yang >Assignee: Phil Yang > Attachments: HBASE-16285-branch-1-v1.patch, HBASE-16285-v1.patch, > HBASE-16285-v2.patch > > > After HBASE-15593, we have a timeout param in header of RPC requests. We can > use it in more scenes. > A straightforward scene is to drop requests if it has waited so long in RPC > queue and has been dropped by client. Even if we handle this request and send > the response back, it will not be used any more. And client may have sent a > retry. In an extreme case, if the server is slow, all requests may be timeout > or queue-full-exception because we should handle previous requests which have > been dropped by client and many resources at server are wasted. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14921) Memory optimizations
[ https://issues.apache.org/jira/browse/HBASE-14921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15401503#comment-15401503 ] ramkrishna.s.vasudevan commented on HBASE-14921: [~anastas] My UT was very simple. I selected the testCompaction1Bucket or any such test case. I just added some keyvalues but ensured that there are 50 cols in each row. I called flattening manually and then checked the count of cells in that flattened segment. I think you will get that issue pretty directly. I don't have that UT now as i reverted my changes. Let me know if you still need them I can paste them here. Thank you. > Memory optimizations > > > Key: HBASE-14921 > URL: https://issues.apache.org/jira/browse/HBASE-14921 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: Eshcar Hillel >Assignee: Anastasia Braginsky > Attachments: CellBlocksSegmentInMemStore.pdf, > CellBlocksSegmentinthecontextofMemStore(1).pdf, HBASE-14921-V01.patch, > HBASE-14921-V02.patch, HBASE-14921-V03.patch, HBASE-14921-V04-CA-V02.patch, > HBASE-14921-V04-CA.patch, HBASE-14921-V05-CAO.patch, > HBASE-14921-V06-CAO.patch, InitialCellArrayMapEvaluation.pdf, > IntroductiontoNewFlatandCompactMemStore.pdf, NewCompactingMemStoreFlow.pptx > > > Memory optimizations including compressed format representation and offheap > allocations -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16287) LruBlockCache size should not exceed acceptableSize too many
[
https://issues.apache.org/jira/browse/HBASE-16287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15401498#comment-15401498
]
Hadoop QA commented on HBASE-16287:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
5s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 42s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 39s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
57s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
24s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
14s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 29s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 35s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
51s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 45s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 45s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 45s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 45s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
4s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
25s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 2m 4s
{color} | {color:red} Patch causes 11 errors with Hadoop v2.4.0. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 9m 21s
{color} | {color:red} Patch causes 10 errors with Hadoop v2.5.0. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 11m 26s
{color} | {color:red} Patch causes 11 errors with Hadoop v2.5.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 13m 34s
{color} | {color:red} Patch causes 11 errors with Hadoop v2.5.2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
30s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 29s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 35s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 18m 30s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 65m 27s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12821249/HBASE-16287-v4.patch |
| JIRA Issue | HBASE-16287 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenki
[jira] [Updated] (HBASE-16257) Move staging dir to be under hbase root dir
[
https://issues.apache.org/jira/browse/HBASE-16257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jerry He updated HBASE-16257:
-
Attachment: HBASE-16257-v2.patch
> Move staging dir to be under hbase root dir
> ---
>
> Key: HBASE-16257
> URL: https://issues.apache.org/jira/browse/HBASE-16257
> Project: HBase
> Issue Type: Sub-task
>Reporter: Jerry He
>Assignee: Jerry He
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-16257-v1.patch, HBASE-16257-v2.patch
>
>
> The hbase.bulkload.staging.dir defaults to hbase.fs.tmp.dir which then
> defaults to
> {code}
> public static final String DEFAULT_TEMPORARY_HDFS_DIRECTORY = "/user/"
> + System.getProperty("user.name") + "/hbase-staging";
> {code}
> This default would have problem on local file system standalone case.
> We can move the staging dir to be under hbase.rootdir. We are bringing
> secure bulkload to the core. It makes sense to bring it under core control as
> well, instead of an optional property.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16225) Refactor ScanQueryMatcher
[
https://issues.apache.org/jira/browse/HBASE-16225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-16225:
--
Attachment: HBASE-16225-branch-1-v1.patch
Retry after fixing the problem introduced by HBASE-16209.
> Refactor ScanQueryMatcher
> -
>
> Key: HBASE-16225
> URL: https://issues.apache.org/jira/browse/HBASE-16225
> Project: HBase
> Issue Type: Improvement
>Reporter: Duo Zhang
>Assignee: Duo Zhang
> Attachments: HBASE-16225-branch-1-v1.patch,
> HBASE-16225-branch-1.patch, HBASE-16225-v1.patch, HBASE-16225-v2.patch,
> HBASE-16225-v3.patch, HBASE-16225-v4.patch, HBASE-16225-v5.patch,
> HBASE-16225-v6.patch, HBASE-16225.patch
>
>
> As said in HBASE-16223, the code of {{ScanQueryMatcher}} is too complicated.
> I suggest that we can abstract an interface and implement several sub classes
> which separate different logic into different implementations. For example,
> the requirements of compaction and user scan are different, now we also need
> to consider the logic of user scan even if we only want to add a logic for
> compaction. And at least, the raw scan does not need a query matcher... we
> can implement a dummy query matcher for it.
> Suggestions are welcomed. Thanks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16225) Refactor ScanQueryMatcher
[
https://issues.apache.org/jira/browse/HBASE-16225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-16225:
--
Attachment: (was: HBASE-16225-branch-1-v1.patch)
> Refactor ScanQueryMatcher
> -
>
> Key: HBASE-16225
> URL: https://issues.apache.org/jira/browse/HBASE-16225
> Project: HBase
> Issue Type: Improvement
>Reporter: Duo Zhang
>Assignee: Duo Zhang
> Attachments: HBASE-16225-branch-1.patch, HBASE-16225-v1.patch,
> HBASE-16225-v2.patch, HBASE-16225-v3.patch, HBASE-16225-v4.patch,
> HBASE-16225-v5.patch, HBASE-16225-v6.patch, HBASE-16225.patch
>
>
> As said in HBASE-16223, the code of {{ScanQueryMatcher}} is too complicated.
> I suggest that we can abstract an interface and implement several sub classes
> which separate different logic into different implementations. For example,
> the requirements of compaction and user scan are different, now we also need
> to consider the logic of user scan even if we only want to add a logic for
> compaction. And at least, the raw scan does not need a query matcher... we
> can implement a dummy query matcher for it.
> Suggestions are welcomed. Thanks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HBASE-16209) Provide an ExponentialBackOffPolicy sleep between failed region open requests
[ https://issues.apache.org/jira/browse/HBASE-16209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-16209. --- Resolution: Fixed Hadoop Flags: Reviewed Pushed to branch-1. > Provide an ExponentialBackOffPolicy sleep between failed region open requests > - > > Key: HBASE-16209 > URL: https://issues.apache.org/jira/browse/HBASE-16209 > Project: HBase > Issue Type: Bug >Reporter: Joseph >Assignee: Joseph > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-16209-addendum.patch, > HBASE-16209-branch-1-addendum.patch, HBASE-16209-branch-1.patch, > HBASE-16209-v2.patch, HBASE-16209.patch > > > Related to HBASE-16138. As of now we currently have no pause between retrying > failed region open requests. And with a low maximumAttempt default, we can > quickly use up all our regionOpen retries if the server is in a bad state. I > added in a ExponentialBackOffPolicy so that we spread out the timing of our > open region retries in AssignmentManager. Review board at > https://reviews.apache.org/r/50011/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16296) Reverse scan performance degrades when scanner cache size matches page filter size
[ https://issues.apache.org/jira/browse/HBASE-16296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15401483#comment-15401483 ] ramkrishna.s.vasudevan commented on HBASE-16296: bq. is the scenario you outline in TestFilterList? ( Haven't checked that. But some of the filter tests does not have @Test tag - so even if the scenario is there may be they don't run. I have raised a JIRA to add @Test tag for all the tests. I will check the patch and see if there could be a possible bug as far as I can see. +1 to fix FilterList behaviour consistency in 1.3+. > Reverse scan performance degrades when scanner cache size matches page filter > size > -- > > Key: HBASE-16296 > URL: https://issues.apache.org/jira/browse/HBASE-16296 > Project: HBase > Issue Type: Bug >Reporter: James Taylor > Attachments: 16296-MAYBE-v2.txt, 16296-MAYBE.txt, > generatedata-snippet.java, repro-snippet.java > > > When a reverse scan is done, the server seems to not know it's done when the > scanner cache size matches the number of rows in a PageFilter. See > PHOENIX-3121 for how this manifests itself. We have a standalone, pure HBase > API reproducer too that I'll attach (courtesy of [~churromorales] and > [~mujtabachohan]). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16209) Provide an ExponentialBackOffPolicy sleep between failed region open requests
[ https://issues.apache.org/jira/browse/HBASE-16209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-16209: -- Attachment: HBASE-16209-addendum.patch Addendum fix the region move problem. Will push it to branch-1 to solve the problem right now. You guys can open new issues if you have better solutions. Thanks. > Provide an ExponentialBackOffPolicy sleep between failed region open requests > - > > Key: HBASE-16209 > URL: https://issues.apache.org/jira/browse/HBASE-16209 > Project: HBase > Issue Type: Bug >Reporter: Joseph >Assignee: Joseph > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-16209-addendum.patch, > HBASE-16209-branch-1-addendum.patch, HBASE-16209-branch-1.patch, > HBASE-16209-v2.patch, HBASE-16209.patch > > > Related to HBASE-16138. As of now we currently have no pause between retrying > failed region open requests. And with a low maximumAttempt default, we can > quickly use up all our regionOpen retries if the server is in a bad state. I > added in a ExponentialBackOffPolicy so that we spread out the timing of our > open region retries in AssignmentManager. Review board at > https://reviews.apache.org/r/50011/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15756) Pluggable RpcServer
[ https://issues.apache.org/jira/browse/HBASE-15756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] binlijin updated HBASE-15756: - Attachment: Netty4RpcServer_forperf.patch > Pluggable RpcServer > --- > > Key: HBASE-15756 > URL: https://issues.apache.org/jira/browse/HBASE-15756 > Project: HBase > Issue Type: Improvement > Components: Performance, rpc >Reporter: binlijin >Assignee: binlijin >Priority: Critical > Attachments: Netty4RpcServer_forperf.patch, NettyRpcServer.patch, > NettyRpcServer_forperf.patch, gets.png, idle.png, queue.png > > > Current we use a simple RpcServer, and can not configure and use other > implementation.This issue is to make the RpcServer pluggable, so we can make > other implementation for example netty rpc server. Patch will upload laterly -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15756) Pluggable RpcServer
[ https://issues.apache.org/jira/browse/HBASE-15756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15401477#comment-15401477 ] binlijin commented on HBASE-15756: -- [~stack] Upload Netty4RpcServer_forperf.patch which base on Netty4. > Pluggable RpcServer > --- > > Key: HBASE-15756 > URL: https://issues.apache.org/jira/browse/HBASE-15756 > Project: HBase > Issue Type: Improvement > Components: Performance, rpc >Reporter: binlijin >Assignee: binlijin >Priority: Critical > Attachments: Netty4RpcServer_forperf.patch, NettyRpcServer.patch, > NettyRpcServer_forperf.patch, gets.png, idle.png, queue.png > > > Current we use a simple RpcServer, and can not configure and use other > implementation.This issue is to make the RpcServer pluggable, so we can make > other implementation for example netty rpc server. Patch will upload laterly -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16234) Expect and handle nulls when assigning replicas
[
https://issues.apache.org/jira/browse/HBASE-16234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Heng Chen updated HBASE-16234:
--
Resolution: Fixed
Hadoop Flags: Reviewed
Fix Version/s: 2.0.0
Status: Resolved (was: Patch Available)
push to master.
> Expect and handle nulls when assigning replicas
> ---
>
> Key: HBASE-16234
> URL: https://issues.apache.org/jira/browse/HBASE-16234
> Project: HBase
> Issue Type: Bug
> Components: Region Assignment
>Affects Versions: 2.0.0
>Reporter: Harsh J
>Assignee: Yi Liang
> Fix For: 2.0.0
>
> Attachments: HBASE-16234-V1.patch, HBASE-16234-V2.patch,
> HBASE-16234-V3.patch, HBASE-16234-V4.patch, HBASE-16234-V5.patch
>
>
> Observed this on a cluster:
> {code}
> FATAL org.apache.hadoop.hbase.master.HMaster: Unhandled exception. Starting
> shutdown.
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.master.AssignmentManager.replicaRegionsNotRecordedInMeta(AssignmentManager.java:2799)
>
> at
> org.apache.hadoop.hbase.master.AssignmentManager.assignAllUserRegions(AssignmentManager.java:2778)
>
> at
> org.apache.hadoop.hbase.master.AssignmentManager.processDeadServersAndRegionsInTransition(AssignmentManager.java:638)
>
> at
> org.apache.hadoop.hbase.master.AssignmentManager.joinCluster(AssignmentManager.java:485)
>
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:723)
>
> at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:169)
> at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1481)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> It looks like {{FSTableDescriptors#get(…)}} can be expected to return null in
> some cases, but {{AssignmentManager.replicaRegionsNotRecordedInMeta(…)}} does
> not currently have any handling for such a possibility.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16234) Expect and handle nulls when assigning replicas
[
https://issues.apache.org/jira/browse/HBASE-16234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15401458#comment-15401458
]
Heng Chen commented on HBASE-16234:
---
hadoop check error has no relates with this patch. patch v5 lgtm, will commit
it later if no other concerns.
{code}
[ERROR] Error invoking method 'get(java.lang.Integer)' in java.util.ArrayList
at META-INF/LICENSE.vm[line 1627, column 22]
java.lang.reflect.InvocationTargetException
at sun.reflect.GeneratedMethodAccessor138.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.velocity.util.introspection.UberspectImpl$VelMethodImpl.doInvoke(UberspectImpl.java:395)
at
org.apache.velocity.util.introspection.UberspectImpl$VelMethodImpl.invoke(UberspectImpl.java:384)
at
org.apache.velocity.runtime.parser.node.ASTIndex.execute(ASTIndex.java:149)
at
org.apache.velocity.runtime.parser.node.ASTReference.execute(ASTReference.java:280)
at
org.apache.velocity.runtime.parser.node.ASTReference.evaluate(ASTReference.java:530)
at
org.apache.velocity.runtime.parser.node.ASTExpression.evaluate(ASTExpression.java:62)
at
org.apache.velocity.runtime.parser.node.ASTNotNode.evaluate(ASTNotNode.java:63)
at
org.apache.velocity.runtime.parser.node.ASTExpression.evaluate(ASTExpression.java:62)
at
org.apache.velocity.runtime.parser.node.ASTIfStatement.render(ASTIfStatement.java:85)
at
org.apache.velocity.runtime.parser.node.ASTBlock.render(ASTBlock.java:72)
at
org.apache.velocity.runtime.directive.Foreach.render(Foreach.java:420)
at
org.apache.velocity.runtime.parser.node.ASTDirective.render(ASTDirective.java:207)
at
org.apache.velocity.runtime.parser.node.ASTBlock.render(ASTBlock.java:72)
at
org.apache.velocity.runtime.parser.node.ASTIfStatement.render(ASTIfStatement.java:87)
at
org.apache.velocity.runtime.parser.node.SimpleNode.render(SimpleNode.java:342)
at org.apache.velocity.Template.merge(Template.java:356)
at org.apache.velocity.Template.merge(Template.java:260)
at
org.apache.velocity.app.VelocityEngine.mergeTemplate(VelocityEngine.java:354)
at
org.apache.maven.plugin.resources.remote.ProcessRemoteResourcesMojo.processResourceBundles(ProcessRemoteResourcesMojo.java:1164)
at
org.apache.maven.plugin.resources.remote.ProcessRemoteResourcesMojo.execute(ProcessRemoteResourcesMojo.java:520)
at
org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
at
org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
at
org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
at
org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
at
org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
at
org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
at
org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
at
org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
at java.util.ArrayList.rangeCheck(ArrayList.java:635)
at java.util.ArrayList.get(ArrayList.java:411)
... 45 more
{code}
> Expect and handle nulls when assigning replicas
> ---
>
> Key: HBASE-16234
>
[jira] [Updated] (HBASE-16287) LruBlockCache size should not exceed acceptableSize too many
[
https://issues.apache.org/jira/browse/HBASE-16287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yu Sun updated HBASE-16287:
---
Attachment: HBASE-16287-v4.patch
the faileure ut seems has no relateion with the changes I made and i also cant
reproduce the failure ut in my local machine. so resubmit the patch for
further check.
> LruBlockCache size should not exceed acceptableSize too many
>
>
> Key: HBASE-16287
> URL: https://issues.apache.org/jira/browse/HBASE-16287
> Project: HBase
> Issue Type: Improvement
> Components: BlockCache
>Reporter: Yu Sun
>Assignee: Yu Sun
> Attachments: HBASE-16287-v1.patch, HBASE-16287-v2.patch,
> HBASE-16287-v3.patch, HBASE-16287-v4.patch
>
>
> Our regionserver has a configuation as bellow：
> -Xmn4g -Xms32g -Xmx32g -XX:SurvriorRatio=2 -XX:+UseConcMarkSweepGC
> also we only use blockcache,and set hfile.block.cache.size = 0.3 in
> hbase_site.xml,so under this configuration, the lru block cache size will
> be(32g-1g)*0.3=9.3g. but in some scenarios，some of the rs will occur
> continuous FullGC for hours and most importantly, after FullGC most of the
> object in old will not be GCed. so we dump the heap and analyse with MAT and
> we observed a obvious memory leak in LruBlockCache, which occpy about 16g
> memory, then we set set class LruBlockCache log level to TRACE and observed
> this in log:
> {quote}
> 2016-07-22 12:17:58,158 INFO [LruBlockCacheStatsExecutor]
> hfile.LruBlockCache: totalSize=15.29 GB, freeSize=-5.99 GB, max=9.30 GB,
> blockCount=628182, accesses=101799469125, hits=93517800259, hitRatio=91.86%,
> , cachingAccesses=99462650031, cachingHits=93468334621,
> cachingHitsRatio=93.97%, evictions=238199, evicted=4776350518,
> evictedPerRun=20051.93359375{quote}
> we can see blockcache size has exceeded acceptableSize too many, which will
> cause the FullGC more seriously.
> Afterfter some investigations, I found in this function:
> {code:borderStyle=solid}
> public void cacheBlock(BlockCacheKey cacheKey, Cacheable buf, boolean
> inMemory,
> final boolean cacheDataInL1) {
> {code}
> No matter the blockcache size has been used, just put the block into it. but
> if the evict thread is not fast enough, blockcache size will increament
> significantly.
> So here I think we should have a check, for example, if the blockcache size >
> 1.2 * acceptableSize(), just return and dont put into it until the blockcache
> size if under watrmark. if this is reasonable, I can make a small patch for
> this.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Reopened] (HBASE-16209) Provide an ExponentialBackOffPolicy sleep between failed region open requests
[ https://issues.apache.org/jira/browse/HBASE-16209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang reopened HBASE-16209: --- This causes a large set of UTs to fail on branch-1. See here https://builds.apache.org/job/HBase-1.4/314/ Maybe that we missed something when backporting? > Provide an ExponentialBackOffPolicy sleep between failed region open requests > - > > Key: HBASE-16209 > URL: https://issues.apache.org/jira/browse/HBASE-16209 > Project: HBase > Issue Type: Bug >Reporter: Joseph >Assignee: Joseph > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-16209-branch-1-addendum.patch, > HBASE-16209-branch-1.patch, HBASE-16209-v2.patch, HBASE-16209.patch > > > Related to HBASE-16138. As of now we currently have no pause between retrying > failed region open requests. And with a low maximumAttempt default, we can > quickly use up all our regionOpen retries if the server is in a bad state. I > added in a ExponentialBackOffPolicy so that we spread out the timing of our > open region retries in AssignmentManager. Review board at > https://reviews.apache.org/r/50011/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16308) Contain protobuf references
[ https://issues.apache.org/jira/browse/HBASE-16308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15401445#comment-15401445 ] stack commented on HBASE-16308: --- Uploaded a partially done patch. Patch includes refactor of RegionServerCallable that allows me clean up a bunch of boilerplate in HTable ([~mbertozzi] helped me come up w/ the pattern subsequently applied in HTable). Patch is not finished. Need to do same elsewhere. > Contain protobuf references > --- > > Key: HBASE-16308 > URL: https://issues.apache.org/jira/browse/HBASE-16308 > Project: HBase > Issue Type: Sub-task > Components: Protobufs >Reporter: stack >Assignee: stack > Attachments: HBASE-16308.master.001.patch > > > Clean up our protobuf references so contained to just a few classes rather > than being spread about the codebase. Doing this work will make it easier > landing the parent issue and will make it more clear where the division > between shaded protobuf and unshaded protobuf lies (we need to continue with > unshaded protobuf for HDFS references by AsyncWAL and probably EndPoint > Coprocessors) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16308) Contain protobuf references
[ https://issues.apache.org/jira/browse/HBASE-16308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-16308: -- Attachment: HBASE-16308.master.001.patch > Contain protobuf references > --- > > Key: HBASE-16308 > URL: https://issues.apache.org/jira/browse/HBASE-16308 > Project: HBase > Issue Type: Sub-task > Components: Protobufs >Reporter: stack >Assignee: stack > Attachments: HBASE-16308.master.001.patch > > > Clean up our protobuf references so contained to just a few classes rather > than being spread about the codebase. Doing this work will make it easier > landing the parent issue and will make it more clear where the division > between shaded protobuf and unshaded protobuf lies (we need to continue with > unshaded protobuf for HDFS references by AsyncWAL and probably EndPoint > Coprocessors) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-16308) Contain protobuf references
stack created HBASE-16308: - Summary: Contain protobuf references Key: HBASE-16308 URL: https://issues.apache.org/jira/browse/HBASE-16308 Project: HBase Issue Type: Sub-task Components: Protobufs Reporter: stack Assignee: stack Clean up our protobuf references so contained to just a few classes rather than being spread about the codebase. Doing this work will make it easier landing the parent issue and will make it more clear where the division between shaded protobuf and unshaded protobuf lies (we need to continue with unshaded protobuf for HDFS references by AsyncWAL and probably EndPoint Coprocessors) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16225) Refactor ScanQueryMatcher
[
https://issues.apache.org/jira/browse/HBASE-16225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15401438#comment-15401438
]
Duo Zhang commented on HBASE-16225:
---
I test some of the UTs locally, some of them can not pass, but not sure if it
relates to the patch. Let me dig.
> Refactor ScanQueryMatcher
> -
>
> Key: HBASE-16225
> URL: https://issues.apache.org/jira/browse/HBASE-16225
> Project: HBase
> Issue Type: Improvement
>Reporter: Duo Zhang
>Assignee: Duo Zhang
> Attachments: HBASE-16225-branch-1-v1.patch,
> HBASE-16225-branch-1.patch, HBASE-16225-v1.patch, HBASE-16225-v2.patch,
> HBASE-16225-v3.patch, HBASE-16225-v4.patch, HBASE-16225-v5.patch,
> HBASE-16225-v6.patch, HBASE-16225.patch
>
>
> As said in HBASE-16223, the code of {{ScanQueryMatcher}} is too complicated.
> I suggest that we can abstract an interface and implement several sub classes
> which separate different logic into different implementations. For example,
> the requirements of compaction and user scan are different, now we also need
> to consider the logic of user scan even if we only want to add a logic for
> compaction. And at least, the raw scan does not need a query matcher... we
> can implement a dummy query matcher for it.
> Suggestions are welcomed. Thanks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-9899) for idempotent operation dups, return the result instead of throwing conflict exception
[ https://issues.apache.org/jira/browse/HBASE-9899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-9899: -- Attachment: HBASE-9899-v3.patch Attach v3 again and trigger hadoop QA run again. > for idempotent operation dups, return the result instead of throwing conflict > exception > --- > > Key: HBASE-9899 > URL: https://issues.apache.org/jira/browse/HBASE-9899 > Project: HBase > Issue Type: Improvement >Reporter: Sergey Shelukhin >Assignee: Guanghao Zhang > Attachments: HBASE-9899-v1.patch, HBASE-9899-v2.patch, > HBASE-9899-v3.patch > > > After HBASE-3787, we could store mvcc in operation context, and use it to > convert the modification request into read on dups instead of throwing > OperationConflictException. > MVCC tracking will have to be aware of such MVCC numbers present. Given that > scanners are usually relatively short-lived, that would prevent low watermark > from advancing for quite a bit more time -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-9899) for idempotent operation dups, return the result instead of throwing conflict exception
[ https://issues.apache.org/jira/browse/HBASE-9899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-9899: -- Attachment: (was: HBASE-9899-v3.patch) > for idempotent operation dups, return the result instead of throwing conflict > exception > --- > > Key: HBASE-9899 > URL: https://issues.apache.org/jira/browse/HBASE-9899 > Project: HBase > Issue Type: Improvement >Reporter: Sergey Shelukhin >Assignee: Guanghao Zhang > Attachments: HBASE-9899-v1.patch, HBASE-9899-v2.patch > > > After HBASE-3787, we could store mvcc in operation context, and use it to > convert the modification request into read on dups instead of throwing > OperationConflictException. > MVCC tracking will have to be aware of such MVCC numbers present. Given that > scanners are usually relatively short-lived, that would prevent low watermark > from advancing for quite a bit more time -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16296) Reverse scan performance degrades when scanner cache size matches page filter size
[ https://issues.apache.org/jira/browse/HBASE-16296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15401427#comment-15401427 ] Lars Hofhansl commented on HBASE-16296: --- I will still maintain that the FilterList behavior is inconsistent and should be fixed as well. Maybe for 1.3++ > Reverse scan performance degrades when scanner cache size matches page filter > size > -- > > Key: HBASE-16296 > URL: https://issues.apache.org/jira/browse/HBASE-16296 > Project: HBase > Issue Type: Bug >Reporter: James Taylor > Attachments: 16296-MAYBE-v2.txt, 16296-MAYBE.txt, > generatedata-snippet.java, repro-snippet.java > > > When a reverse scan is done, the server seems to not know it's done when the > scanner cache size matches the number of rows in a PageFilter. See > PHOENIX-3121 for how this manifests itself. We have a standalone, pure HBase > API reproducer too that I'll attach (courtesy of [~churromorales] and > [~mujtabachohan]). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16296) Reverse scan performance degrades when scanner cache size matches page filter size
[ https://issues.apache.org/jira/browse/HBASE-16296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-16296: -- Attachment: 16296-MAYBE-v2.txt What about -v2? Simply breaks early in RegionScannerImpl if the Filter would filterAllRemaining anyway. Is that correct in all cases? Any scenarios where this would not work? I confirmed that that fixes the Phoenix performance issue we've seen there. > Reverse scan performance degrades when scanner cache size matches page filter > size > -- > > Key: HBASE-16296 > URL: https://issues.apache.org/jira/browse/HBASE-16296 > Project: HBase > Issue Type: Bug >Reporter: James Taylor > Attachments: 16296-MAYBE-v2.txt, 16296-MAYBE.txt, > generatedata-snippet.java, repro-snippet.java > > > When a reverse scan is done, the server seems to not know it's done when the > scanner cache size matches the number of rows in a PageFilter. See > PHOENIX-3121 for how this manifests itself. We have a standalone, pure HBase > API reproducer too that I'll attach (courtesy of [~churromorales] and > [~mujtabachohan]). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16296) Reverse scan performance degrades when scanner cache size matches page filter size
[ https://issues.apache.org/jira/browse/HBASE-16296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15401420#comment-15401420 ] Lars Hofhansl commented on HBASE-16296: --- One would have to analyze the code to see if there'd be a end-to-end different behavior possible. In these scenarios the Cells would be filtered later in SQM anyway, so while FilterList might return different values, the actual scanning behavior might not change actually change. > Reverse scan performance degrades when scanner cache size matches page filter > size > -- > > Key: HBASE-16296 > URL: https://issues.apache.org/jira/browse/HBASE-16296 > Project: HBase > Issue Type: Bug >Reporter: James Taylor > Attachments: 16296-MAYBE.txt, generatedata-snippet.java, > repro-snippet.java > > > When a reverse scan is done, the server seems to not know it's done when the > scanner cache size matches the number of rows in a PageFilter. See > PHOENIX-3121 for how this manifests itself. We have a standalone, pure HBase > API reproducer too that I'll attach (courtesy of [~churromorales] and > [~mujtabachohan]). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16296) Reverse scan performance degrades when scanner cache size matches page filter size
[ https://issues.apache.org/jira/browse/HBASE-16296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15401409#comment-15401409 ] Ted Yu commented on HBASE-16296: As Lars acknowledged, since his patch did change the behavior of FilterList, I feel we can adopt change which doesn't change the behavior. The filterAllRemaining() call is an optimization. If it is removed, there would be regression in performance. > Reverse scan performance degrades when scanner cache size matches page filter > size > -- > > Key: HBASE-16296 > URL: https://issues.apache.org/jira/browse/HBASE-16296 > Project: HBase > Issue Type: Bug >Reporter: James Taylor > Attachments: 16296-MAYBE.txt, generatedata-snippet.java, > repro-snippet.java > > > When a reverse scan is done, the server seems to not know it's done when the > scanner cache size matches the number of rows in a PageFilter. See > PHOENIX-3121 for how this manifests itself. We have a standalone, pure HBase > API reproducer too that I'll attach (courtesy of [~churromorales] and > [~mujtabachohan]). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16296) Reverse scan performance degrades when scanner cache size matches page filter size
[ https://issues.apache.org/jira/browse/HBASE-16296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15401251#comment-15401251 ] Andrew Purtell commented on HBASE-16296: I will read the relevant code when back in front of a computer but my read of this issue suggests this would be an allowable bug fix change in the next 0.98 minor. > Reverse scan performance degrades when scanner cache size matches page filter > size > -- > > Key: HBASE-16296 > URL: https://issues.apache.org/jira/browse/HBASE-16296 > Project: HBase > Issue Type: Bug >Reporter: James Taylor > Attachments: 16296-MAYBE.txt, generatedata-snippet.java, > repro-snippet.java > > > When a reverse scan is done, the server seems to not know it's done when the > scanner cache size matches the number of rows in a PageFilter. See > PHOENIX-3121 for how this manifests itself. We have a standalone, pure HBase > API reproducer too that I'll attach (courtesy of [~churromorales] and > [~mujtabachohan]). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14345) Consolidate printUsage in IntegrationTestLoadAndVerify
[
https://issues.apache.org/jira/browse/HBASE-14345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15401194#comment-15401194
]
Reid Chan commented on HBASE-14345:
---
can't understand those contents from QA's reports, they looks unrelated to the
patch?
i'm not sure...
> Consolidate printUsage in IntegrationTestLoadAndVerify
> --
>
> Key: HBASE-14345
> URL: https://issues.apache.org/jira/browse/HBASE-14345
> Project: HBase
> Issue Type: Improvement
> Components: integration tests
>Reporter: Nick Dimiduk
>Assignee: Reid Chan
>Priority: Trivial
> Labels: beginner
> Attachments: HBASE-14345.002.patch, HBASE-14345.003.patch,
> HBASE-14345.patch, itlav-2016-07-07.png, itlv.png
>
>
> Investigating the use of {{itlav}} is a little screwy. Subclasses are not
> overriding the {{printUsage()}} methods correctly, so you have to pass
> {{--help}} to get some info and no arguments to get the rest.
> {noformat}
> [hbase@ndimiduk-112rc2-7 ~]$ hbase
> org.apache.hadoop.hbase.test.IntegrationTestLoadAndVerify --help
> usage: bin/hbase org.apache.hadoop.hbase.test.IntegrationTestLoadAndVerify
>
> Options:
> -h,--help Show usage
> -m,--monkey Which chaos monkey to run
> -monkeyProps The properties file for specifying chaos monkey
> properties.
> -ncc,--noClusterCleanUp Don't clean up the cluster at the end
> [hbase@ndimiduk-112rc2-7 ~]$ hbase
> org.apache.hadoop.hbase.test.IntegrationTestLoadAndVerify
> IntegrationTestLoadAndVerify [-Doptions]
> Loads a table with row dependencies and verifies the dependency chains
> Options
> -Dloadmapper.table=Table to write/verify (default autogen)
> -Dloadmapper.backrefs=Number of backreferences per row (default
> 50)
> -Dloadmapper.num_to_write=Number of rows per mapper (default 100,000
> per mapper)
> -Dloadmapper.deleteAfter= Delete after a successful verify (default
> true)
> -Dloadmapper.numPresplits=Number of presplit regions to start with
> (default 40)
> -Dloadmapper.map.tasks= Number of map tasks for load (default 200)
> -Dverify.reduce.tasks=Number of reduce tasks for verify (default
> 35)
> -Dverify.scannercaching= Number hbase scanner caching rows to read
> (default 50)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14345) Consolidate printUsage in IntegrationTestLoadAndVerify
[
https://issues.apache.org/jira/browse/HBASE-14345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15401176#comment-15401176
]
Hadoop QA commented on HBASE-14345:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 2m 41s
{color} | {color:red} root in master failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m 3s
{color} | {color:red} hbase-it in master failed with JDK vVM. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 21s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
22s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 0s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 18s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 9s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
23s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 26s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 26s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 22s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 22s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
24s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch has 38 line(s) that end in whitespace. Use git
apply --whitespace=fix. {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 1s
{color} | {color:red} The patch has 384 line(s) with tabs. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 3m 44s
{color} | {color:red} Patch causes 17 errors with Hadoop v2.4.0. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 5m 51s
{color} | {color:red} Patch causes 11 errors with Hadoop v2.4.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 0s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 17s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 9s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 20s
{color} | {color:green} hbase-it in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
7s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 38m 58s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12821214/HBASE-14345.003.patch
|
| JIRA Issue | HBASE-14345 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 69d1
[jira] [Commented] (HBASE-16225) Refactor ScanQueryMatcher
[
https://issues.apache.org/jira/browse/HBASE-16225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15401173#comment-15401173
]
Hadoop QA commented on HBASE-16225:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 18 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 14s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
0s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 5s
{color} | {color:green} branch-1 passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 52s
{color} | {color:green} branch-1 passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
46s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
32s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
38s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 50s
{color} | {color:green} branch-1 passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 49s
{color} | {color:green} branch-1 passed with JDK v1.7.0_80 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 9s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
4s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 4s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 4s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 52s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 52s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
32s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 10m 46s
{color} | {color:red} Patch causes 11 errors with Hadoop v2.6.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 12m 10s
{color} | {color:red} Patch causes 11 errors with Hadoop v2.6.2. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 13m 34s
{color} | {color:red} Patch causes 11 errors with Hadoop v2.6.3. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 14m 59s
{color} | {color:red} Patch causes 11 errors with Hadoop v2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 3s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 44s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 50s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 53s
{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 136m 57s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
29s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 173m 53s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Fa
[jira] [Commented] (HBASE-16296) Reverse scan performance degrades when scanner cache size matches page filter size
[
https://issues.apache.org/jira/browse/HBASE-16296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15401162#comment-15401162
]
Lars Hofhansl commented on HBASE-16296:
---
Actually {{WhileMatchFilter}} is used in {{TestFilterList}} but {{SkipFilter}}
is not. But since {{SkipFilter}} does not implement {{filterAllRemaining}} (in
0.98) we should be good.
> Reverse scan performance degrades when scanner cache size matches page filter
> size
> --
>
> Key: HBASE-16296
> URL: https://issues.apache.org/jira/browse/HBASE-16296
> Project: HBase
> Issue Type: Bug
>Reporter: James Taylor
> Attachments: 16296-MAYBE.txt, generatedata-snippet.java,
> repro-snippet.java
>
>
> When a reverse scan is done, the server seems to not know it's done when the
> scanner cache size matches the number of rows in a PageFilter. See
> PHOENIX-3121 for how this manifests itself. We have a standalone, pure HBase
> API reproducer too that I'll attach (courtesy of [~churromorales] and
> [~mujtabachohan]).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14345) Consolidate printUsage in IntegrationTestLoadAndVerify
[
https://issues.apache.org/jira/browse/HBASE-14345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reid Chan updated HBASE-14345:
--
Attachment: (was: HBASE-14345.003.patch)
> Consolidate printUsage in IntegrationTestLoadAndVerify
> --
>
> Key: HBASE-14345
> URL: https://issues.apache.org/jira/browse/HBASE-14345
> Project: HBase
> Issue Type: Improvement
> Components: integration tests
>Reporter: Nick Dimiduk
>Assignee: Reid Chan
>Priority: Trivial
> Labels: beginner
> Attachments: HBASE-14345.002.patch, HBASE-14345.003.patch,
> HBASE-14345.patch, itlav-2016-07-07.png, itlv.png
>
>
> Investigating the use of {{itlav}} is a little screwy. Subclasses are not
> overriding the {{printUsage()}} methods correctly, so you have to pass
> {{--help}} to get some info and no arguments to get the rest.
> {noformat}
> [hbase@ndimiduk-112rc2-7 ~]$ hbase
> org.apache.hadoop.hbase.test.IntegrationTestLoadAndVerify --help
> usage: bin/hbase org.apache.hadoop.hbase.test.IntegrationTestLoadAndVerify
>
> Options:
> -h,--help Show usage
> -m,--monkey Which chaos monkey to run
> -monkeyProps The properties file for specifying chaos monkey
> properties.
> -ncc,--noClusterCleanUp Don't clean up the cluster at the end
> [hbase@ndimiduk-112rc2-7 ~]$ hbase
> org.apache.hadoop.hbase.test.IntegrationTestLoadAndVerify
> IntegrationTestLoadAndVerify [-Doptions]
> Loads a table with row dependencies and verifies the dependency chains
> Options
> -Dloadmapper.table=Table to write/verify (default autogen)
> -Dloadmapper.backrefs=Number of backreferences per row (default
> 50)
> -Dloadmapper.num_to_write=Number of rows per mapper (default 100,000
> per mapper)
> -Dloadmapper.deleteAfter= Delete after a successful verify (default
> true)
> -Dloadmapper.numPresplits=Number of presplit regions to start with
> (default 40)
> -Dloadmapper.map.tasks= Number of map tasks for load (default 200)
> -Dverify.reduce.tasks=Number of reduce tasks for verify (default
> 35)
> -Dverify.scannercaching= Number hbase scanner caching rows to read
> (default 50)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14345) Consolidate printUsage in IntegrationTestLoadAndVerify
[
https://issues.apache.org/jira/browse/HBASE-14345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reid Chan updated HBASE-14345:
--
Attachment: HBASE-14345.003.patch
> Consolidate printUsage in IntegrationTestLoadAndVerify
> --
>
> Key: HBASE-14345
> URL: https://issues.apache.org/jira/browse/HBASE-14345
> Project: HBase
> Issue Type: Improvement
> Components: integration tests
>Reporter: Nick Dimiduk
>Assignee: Reid Chan
>Priority: Trivial
> Labels: beginner
> Attachments: HBASE-14345.002.patch, HBASE-14345.003.patch,
> HBASE-14345.patch, itlav-2016-07-07.png, itlv.png
>
>
> Investigating the use of {{itlav}} is a little screwy. Subclasses are not
> overriding the {{printUsage()}} methods correctly, so you have to pass
> {{--help}} to get some info and no arguments to get the rest.
> {noformat}
> [hbase@ndimiduk-112rc2-7 ~]$ hbase
> org.apache.hadoop.hbase.test.IntegrationTestLoadAndVerify --help
> usage: bin/hbase org.apache.hadoop.hbase.test.IntegrationTestLoadAndVerify
>
> Options:
> -h,--help Show usage
> -m,--monkey Which chaos monkey to run
> -monkeyProps The properties file for specifying chaos monkey
> properties.
> -ncc,--noClusterCleanUp Don't clean up the cluster at the end
> [hbase@ndimiduk-112rc2-7 ~]$ hbase
> org.apache.hadoop.hbase.test.IntegrationTestLoadAndVerify
> IntegrationTestLoadAndVerify [-Doptions]
> Loads a table with row dependencies and verifies the dependency chains
> Options
> -Dloadmapper.table=Table to write/verify (default autogen)
> -Dloadmapper.backrefs=Number of backreferences per row (default
> 50)
> -Dloadmapper.num_to_write=Number of rows per mapper (default 100,000
> per mapper)
> -Dloadmapper.deleteAfter= Delete after a successful verify (default
> true)
> -Dloadmapper.numPresplits=Number of presplit regions to start with
> (default 40)
> -Dloadmapper.map.tasks= Number of map tasks for load (default 200)
> -Dverify.reduce.tasks=Number of reduce tasks for verify (default
> 35)
> -Dverify.scannercaching= Number hbase scanner caching rows to read
> (default 50)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16296) Reverse scan performance degrades when scanner cache size matches page filter size
[
https://issues.apache.org/jira/browse/HBASE-16296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15401160#comment-15401160
]
Lars Hofhansl commented on HBASE-16296:
---
{{SkipFilter}} does not implement {{filterAllRemaining}}, but
{{WhileMatchFilter}} does. And {{WhileMatchFilter}} does not add this logic to
{{filterRowKey(...)}}
> Reverse scan performance degrades when scanner cache size matches page filter
> size
> --
>
> Key: HBASE-16296
> URL: https://issues.apache.org/jira/browse/HBASE-16296
> Project: HBase
> Issue Type: Bug
>Reporter: James Taylor
> Attachments: 16296-MAYBE.txt, generatedata-snippet.java,
> repro-snippet.java
>
>
> When a reverse scan is done, the server seems to not know it's done when the
> scanner cache size matches the number of rows in a PageFilter. See
> PHOENIX-3121 for how this manifests itself. We have a standalone, pure HBase
> API reproducer too that I'll attach (courtesy of [~churromorales] and
> [~mujtabachohan]).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (HBASE-16296) Reverse scan performance degrades when scanner cache size matches page filter size
[
https://issues.apache.org/jira/browse/HBASE-16296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15401158#comment-15401158
]
Lars Hofhansl edited comment on HBASE-16296 at 7/31/16 2:17 PM:
Yeah... That's what I wanted to start the discussion on. :)
I think FilterList shouldn't add any logic that should either be in the wrapped
filter(s) itself or in HRegion. But it *is* a change. Not for any of the
Filters we have (I think) but folks could conceivably have written their own
filters and relied on this.
[~ram_krish] is the scenario you outline in TestFilterList? (sorry can't check
right now)
[~apurtell] any opinions on 0.98?
What about the other releases [~mantonov], [~busbey], [~enis], [~ndimiduk]?
{{FilterList}} is {{public}} and {{stable}}.
was (Author: lhofhansl):
Yeah... That's what I wanted to start the discussion on. :)
I think FilterList shouldn't add any logic that should either be in the wrapped
filter(s) itself or in HRegion. But it *is* a change. Not for any of the
Filters we have (I think) but folks could conceivably have written their own
filters and relied on this.
[~ram_krish] is the scenario you outline in TestFilterList? (sorry can't check
right now)
[~apurtell] any opinions on 0.98.
What about the other releases [~mantonov], [~busbey], [~enis], [~ndimiduk].
{{FilterList}} is {{public}} and {{stable}}.
> Reverse scan performance degrades when scanner cache size matches page filter
> size
> --
>
> Key: HBASE-16296
> URL: https://issues.apache.org/jira/browse/HBASE-16296
> Project: HBase
> Issue Type: Bug
>Reporter: James Taylor
> Attachments: 16296-MAYBE.txt, generatedata-snippet.java,
> repro-snippet.java
>
>
> When a reverse scan is done, the server seems to not know it's done when the
> scanner cache size matches the number of rows in a PageFilter. See
> PHOENIX-3121 for how this manifests itself. We have a standalone, pure HBase
> API reproducer too that I'll attach (courtesy of [~churromorales] and
> [~mujtabachohan]).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16296) Reverse scan performance degrades when scanner cache size matches page filter size
[
https://issues.apache.org/jira/browse/HBASE-16296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15401158#comment-15401158
]
Lars Hofhansl commented on HBASE-16296:
---
Yeah... That's what I wanted to start the discussion on. :)
I think FilterList shouldn't add any logic that should either be in the wrapped
filter(s) itself or in HRegion. But it *is* a change. Not for any of the
Filters we have (I think) but folks could conceivably have written their own
filters and relied on this.
[~ram_krish] is the scenario you outline in TestFilterList? (sorry can't check
right now)
[~apurtell] any opinions on 0.98.
What about the other releases [~mantonov], [~busbey], [~enis], [~ndimiduk].
{{FilterList}} is {{public}} and {{stable}}.
> Reverse scan performance degrades when scanner cache size matches page filter
> size
> --
>
> Key: HBASE-16296
> URL: https://issues.apache.org/jira/browse/HBASE-16296
> Project: HBase
> Issue Type: Bug
>Reporter: James Taylor
> Attachments: 16296-MAYBE.txt, generatedata-snippet.java,
> repro-snippet.java
>
>
> When a reverse scan is done, the server seems to not know it's done when the
> scanner cache size matches the number of rows in a PageFilter. See
> PHOENIX-3121 for how this manifests itself. We have a standalone, pure HBase
> API reproducer too that I'll attach (courtesy of [~churromorales] and
> [~mujtabachohan]).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14345) Consolidate printUsage in IntegrationTestLoadAndVerify
[
https://issues.apache.org/jira/browse/HBASE-14345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reid Chan updated HBASE-14345:
--
Attachment: (was: 2016-07-30-Without System_exit.png)
> Consolidate printUsage in IntegrationTestLoadAndVerify
> --
>
> Key: HBASE-14345
> URL: https://issues.apache.org/jira/browse/HBASE-14345
> Project: HBase
> Issue Type: Improvement
> Components: integration tests
>Reporter: Nick Dimiduk
>Assignee: Reid Chan
>Priority: Trivial
> Labels: beginner
> Attachments: HBASE-14345.002.patch, HBASE-14345.003.patch,
> HBASE-14345.patch, itlav-2016-07-07.png, itlv.png
>
>
> Investigating the use of {{itlav}} is a little screwy. Subclasses are not
> overriding the {{printUsage()}} methods correctly, so you have to pass
> {{--help}} to get some info and no arguments to get the rest.
> {noformat}
> [hbase@ndimiduk-112rc2-7 ~]$ hbase
> org.apache.hadoop.hbase.test.IntegrationTestLoadAndVerify --help
> usage: bin/hbase org.apache.hadoop.hbase.test.IntegrationTestLoadAndVerify
>
> Options:
> -h,--help Show usage
> -m,--monkey Which chaos monkey to run
> -monkeyProps The properties file for specifying chaos monkey
> properties.
> -ncc,--noClusterCleanUp Don't clean up the cluster at the end
> [hbase@ndimiduk-112rc2-7 ~]$ hbase
> org.apache.hadoop.hbase.test.IntegrationTestLoadAndVerify
> IntegrationTestLoadAndVerify [-Doptions]
> Loads a table with row dependencies and verifies the dependency chains
> Options
> -Dloadmapper.table=Table to write/verify (default autogen)
> -Dloadmapper.backrefs=Number of backreferences per row (default
> 50)
> -Dloadmapper.num_to_write=Number of rows per mapper (default 100,000
> per mapper)
> -Dloadmapper.deleteAfter= Delete after a successful verify (default
> true)
> -Dloadmapper.numPresplits=Number of presplit regions to start with
> (default 40)
> -Dloadmapper.map.tasks= Number of map tasks for load (default 200)
> -Dverify.reduce.tasks=Number of reduce tasks for verify (default
> 35)
> -Dverify.scannercaching= Number hbase scanner caching rows to read
> (default 50)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-9899) for idempotent operation dups, return the result instead of throwing conflict exception
[
https://issues.apache.org/jira/browse/HBASE-9899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15401130#comment-15401130
]
Hadoop QA commented on HBASE-9899:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
8s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 16s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 3s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
57s {color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} mvneclipse {color} | {color:red} 0m 11s
{color} | {color:red} hbase-client in master failed. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 2s
{color} | {color:red} hbase-client in master failed. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 3s
{color} | {color:red} hbase-server in master failed. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 2s
{color} | {color:red} hbase-client in master failed with JDK v#. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 2s
{color} | {color:red} hbase-server in master failed with JDK v#. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 2s
{color} | {color:red} hbase-client in master failed with JDK v#. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 3s
{color} | {color:red} hbase-server in master failed with JDK v#. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 2s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 2s
{color} | {color:red} hbase-client in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 2s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m 8s
{color} | {color:red} hbase-client in the patch failed with JDK v#. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m 3s
{color} | {color:red} hbase-server in the patch failed with JDK v#. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 8s {color}
| {color:red} hbase-client in the patch failed with JDK v#. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 3s {color}
| {color:red} hbase-server in the patch failed with JDK v#. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m 2s
{color} | {color:red} hbase-client in the patch failed with JDK v#. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m 2s
{color} | {color:red} hbase-server in the patch failed with JDK v#. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 2s {color}
| {color:red} hbase-client in the patch failed with JDK v#. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 2s {color}
| {color:red} hbase-server in the patch failed with JDK v#. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
5s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} mvneclipse {color} | {color:red} 0m 3s
{color} | {color:red} hbase-client in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvneclipse {color} | {color:red} 0m 2s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch has 640 line(s) that end in whitespace. Use git
apply --whitespace=fix. {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 17s
{color} | {color:red} The patch has 6912 line(s) with tabs. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} |
[jira] [Updated] (HBASE-9899) for idempotent operation dups, return the result instead of throwing conflict exception
[ https://issues.apache.org/jira/browse/HBASE-9899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-9899: -- Attachment: HBASE-9899-v3.patch Attach a v3 patch. Fix failed ut TestScannerHeartbeatMessages and TestMultiParallel. > for idempotent operation dups, return the result instead of throwing conflict > exception > --- > > Key: HBASE-9899 > URL: https://issues.apache.org/jira/browse/HBASE-9899 > Project: HBase > Issue Type: Improvement >Reporter: Sergey Shelukhin >Assignee: Guanghao Zhang > Attachments: HBASE-9899-v1.patch, HBASE-9899-v2.patch, > HBASE-9899-v3.patch > > > After HBASE-3787, we could store mvcc in operation context, and use it to > convert the modification request into read on dups instead of throwing > OperationConflictException. > MVCC tracking will have to be aware of such MVCC numbers present. Given that > scanners are usually relatively short-lived, that would prevent low watermark > from advancing for quite a bit more time -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16225) Refactor ScanQueryMatcher
[
https://issues.apache.org/jira/browse/HBASE-16225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-16225:
--
Attachment: HBASE-16225-branch-1-v1.patch
Fix the findbugs and javadoc warnings.
> Refactor ScanQueryMatcher
> -
>
> Key: HBASE-16225
> URL: https://issues.apache.org/jira/browse/HBASE-16225
> Project: HBase
> Issue Type: Improvement
>Reporter: Duo Zhang
>Assignee: Duo Zhang
> Attachments: HBASE-16225-branch-1-v1.patch,
> HBASE-16225-branch-1.patch, HBASE-16225-v1.patch, HBASE-16225-v2.patch,
> HBASE-16225-v3.patch, HBASE-16225-v4.patch, HBASE-16225-v5.patch,
> HBASE-16225-v6.patch, HBASE-16225.patch
>
>
> As said in HBASE-16223, the code of {{ScanQueryMatcher}} is too complicated.
> I suggest that we can abstract an interface and implement several sub classes
> which separate different logic into different implementations. For example,
> the requirements of compaction and user scan are different, now we also need
> to consider the logic of user scan even if we only want to add a logic for
> compaction. And at least, the raw scan does not need a query matcher... we
> can implement a dummy query matcher for it.
> Suggestions are welcomed. Thanks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16224) Reduce the number of RPCs for the large PUTs
[
https://issues.apache.org/jira/browse/HBASE-16224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15401119#comment-15401119
]
Hadoop QA commented on HBASE-16224:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
28s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 51s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 36s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
47s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
22s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
43s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 49s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 37s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
39s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 48s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 48s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 34s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 34s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
46s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
23s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 0s
{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 19m 58s
{color} | {color:red} Patch causes 11 errors with Hadoop v2.6.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 22m 50s
{color} | {color:red} Patch causes 11 errors with Hadoop v2.6.2. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 25m 41s
{color} | {color:red} Patch causes 11 errors with Hadoop v2.6.3. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 28m 32s
{color} | {color:red} Patch causes 11 errors with Hadoop v2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 9s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 46s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 37s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 45s
{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 58s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s {color} | {color:green} Patch does not generate ASF License warnings.
{color
[jira] [Commented] (HBASE-14921) Memory optimizations
[ https://issues.apache.org/jira/browse/HBASE-14921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15401099#comment-15401099 ] Anoop Sam John commented on HBASE-14921: 1. Ours is a write only workload. So BC size not that relevant any way.. Ya it was way lower than 0.38. Less than 10% I believe 2. writeToWAL = false > Memory optimizations > > > Key: HBASE-14921 > URL: https://issues.apache.org/jira/browse/HBASE-14921 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: Eshcar Hillel >Assignee: Anastasia Braginsky > Attachments: CellBlocksSegmentInMemStore.pdf, > CellBlocksSegmentinthecontextofMemStore(1).pdf, HBASE-14921-V01.patch, > HBASE-14921-V02.patch, HBASE-14921-V03.patch, HBASE-14921-V04-CA-V02.patch, > HBASE-14921-V04-CA.patch, HBASE-14921-V05-CAO.patch, > HBASE-14921-V06-CAO.patch, InitialCellArrayMapEvaluation.pdf, > IntroductiontoNewFlatandCompactMemStore.pdf, NewCompactingMemStoreFlow.pptx > > > Memory optimizations including compressed format representation and offheap > allocations -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16224) Reduce the number of RPCs for the large PUTs
[ https://issues.apache.org/jira/browse/HBASE-16224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ChiaPing Tsai updated HBASE-16224: -- Status: Patch Available (was: Open) > Reduce the number of RPCs for the large PUTs > > > Key: HBASE-16224 > URL: https://issues.apache.org/jira/browse/HBASE-16224 > Project: HBase > Issue Type: Improvement >Reporter: ChiaPing Tsai >Assignee: ChiaPing Tsai >Priority: Minor > Attachments: HBASE-16224-v1.patch, HBASE-16224-v2.patch, > HBASE-16224-v3.patch, HBASE-16224-v4.patch, HBASE-16224-v5.patch, > experiment.xlsx > > > This patch is proposed to reduce the number of RPC for the large PUTs > The number and data size of write thread(SingleServerRequestRunnable) is a > result of three main factors： > 1) The flush size taken by BufferedMutatorImpl#backgroundFlushCommits > 2) The limit of task number > 3) ClientBackoffPolicy > A lot of threads created with less MUTATIONs is a result of two reason: 1) > many regions of target table are in different server. 2) flush size in step > one is summed by “all” server rather than “individual” server > This patch removes the limit of flush size in step one and add maximum size > to submit for each server in the AsyncProcess -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16224) Reduce the number of RPCs for the large PUTs
[ https://issues.apache.org/jira/browse/HBASE-16224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ChiaPing Tsai updated HBASE-16224: -- Attachment: HBASE-16224-v5.patch The compiling against various Hadoop versions is passed locally > Reduce the number of RPCs for the large PUTs > > > Key: HBASE-16224 > URL: https://issues.apache.org/jira/browse/HBASE-16224 > Project: HBase > Issue Type: Improvement >Reporter: ChiaPing Tsai >Assignee: ChiaPing Tsai >Priority: Minor > Attachments: HBASE-16224-v1.patch, HBASE-16224-v2.patch, > HBASE-16224-v3.patch, HBASE-16224-v4.patch, HBASE-16224-v5.patch, > experiment.xlsx > > > This patch is proposed to reduce the number of RPC for the large PUTs > The number and data size of write thread(SingleServerRequestRunnable) is a > result of three main factors： > 1) The flush size taken by BufferedMutatorImpl#backgroundFlushCommits > 2) The limit of task number > 3) ClientBackoffPolicy > A lot of threads created with less MUTATIONs is a result of two reason: 1) > many regions of target table are in different server. 2) flush size in step > one is summed by “all” server rather than “individual” server > This patch removes the limit of flush size in step one and add maximum size > to submit for each server in the AsyncProcess -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16224) Reduce the number of RPCs for the large PUTs
[ https://issues.apache.org/jira/browse/HBASE-16224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ChiaPing Tsai updated HBASE-16224: -- Status: Open (was: Patch Available) > Reduce the number of RPCs for the large PUTs > > > Key: HBASE-16224 > URL: https://issues.apache.org/jira/browse/HBASE-16224 > Project: HBase > Issue Type: Improvement >Reporter: ChiaPing Tsai >Assignee: ChiaPing Tsai >Priority: Minor > Attachments: HBASE-16224-v1.patch, HBASE-16224-v2.patch, > HBASE-16224-v3.patch, HBASE-16224-v4.patch, experiment.xlsx > > > This patch is proposed to reduce the number of RPC for the large PUTs > The number and data size of write thread(SingleServerRequestRunnable) is a > result of three main factors： > 1) The flush size taken by BufferedMutatorImpl#backgroundFlushCommits > 2) The limit of task number > 3) ClientBackoffPolicy > A lot of threads created with less MUTATIONs is a result of two reason: 1) > many regions of target table are in different server. 2) flush size in step > one is summed by “all” server rather than “individual” server > This patch removes the limit of flush size in step one and add maximum size > to submit for each server in the AsyncProcess -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16224) Reduce the number of RPCs for the large PUTs
[
https://issues.apache.org/jira/browse/HBASE-16224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15401088#comment-15401088
]
Hadoop QA commented on HBASE-16224:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 9s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
28s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 49s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 35s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
46s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
23s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
45s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 40s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 37s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
40s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 39s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 39s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 35s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 35s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
23s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 0s
{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 19m 59s
{color} | {color:red} Patch causes 11 errors with Hadoop v2.6.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 22m 51s
{color} | {color:red} Patch causes 11 errors with Hadoop v2.6.2. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 25m 41s
{color} | {color:red} Patch causes 11 errors with Hadoop v2.6.3. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 28m 32s
{color} | {color:red} Patch causes 11 errors with Hadoop v2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 8s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 47s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 36s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 46s
{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 58s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s {color} | {color:green} Patch does not generate ASF License warnings.
{color}
[jira] [Commented] (HBASE-14881) Provide a Put API that uses the provided row without coping
[
https://issues.apache.org/jira/browse/HBASE-14881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15401084#comment-15401084
]
Hadoop QA commented on HBASE-14881:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
30s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 31s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 21s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
31s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
13s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 25s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 22s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
23s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 22s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 22s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 20s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 20s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
28s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 6m 16s
{color} | {color:red} Patch causes 11 errors with Hadoop v2.4.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 6m 46s
{color} | {color:red} Patch causes 12 errors with Hadoop v2.5.0. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 8m 49s
{color} | {color:red} Patch causes 11 errors with Hadoop v2.5.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 10m 57s
{color} | {color:red} Patch causes 11 errors with Hadoop v2.5.2. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 13m 23s
{color} | {color:red} Patch causes 8 errors with Hadoop v2.6.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 13m 54s
{color} | {color:red} Patch causes 12 errors with Hadoop v2.6.2. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 16m 9s
{color} | {color:red} Patch causes 7 errors with Hadoop v2.6.3. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 16m 40s
{color} | {color:red} Patch causes 12 errors with Hadoop v2.7.1. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 14s
{color} | {color:red} hbase-client in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 17s
{color} | {color:red} hbase-client in the patch failed with JDK vVM. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 8s
{color} | {color:red} hbase-client in the patch failed with JDK v1.7.0_80.
{color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 3s {color} |
{color:red} hbase-client in the patch failed. {color} |
| {color:blue}0{color} | {color:blue} asflicense {color} | {color:blue} 0m 2s
{color} | {color:blue} ASF License ch
[jira] [Commented] (HBASE-14921) Memory optimizations
[ https://issues.apache.org/jira/browse/HBASE-14921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15401076#comment-15401076 ] Eshcar Hillel commented on HBASE-14921: --- Questions on PE tests: 1. you set hbase.regionserver.global.memstore.size to 0.42. Did you also set hfile.block.cache.size to 0.38? or even lower? 2. which setting did you use for WAL writeToWAL=true/false? > Memory optimizations > > > Key: HBASE-14921 > URL: https://issues.apache.org/jira/browse/HBASE-14921 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: Eshcar Hillel >Assignee: Anastasia Braginsky > Attachments: CellBlocksSegmentInMemStore.pdf, > CellBlocksSegmentinthecontextofMemStore(1).pdf, HBASE-14921-V01.patch, > HBASE-14921-V02.patch, HBASE-14921-V03.patch, HBASE-14921-V04-CA-V02.patch, > HBASE-14921-V04-CA.patch, HBASE-14921-V05-CAO.patch, > HBASE-14921-V06-CAO.patch, InitialCellArrayMapEvaluation.pdf, > IntroductiontoNewFlatandCompactMemStore.pdf, NewCompactingMemStoreFlow.pptx > > > Memory optimizations including compressed format representation and offheap > allocations -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14881) Provide a Put API that uses the provided row without coping
[ https://issues.apache.org/jira/browse/HBASE-14881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiang Li updated HBASE-14881: - Status: Open (was: Patch Available) > Provide a Put API that uses the provided row without coping > --- > > Key: HBASE-14881 > URL: https://issues.apache.org/jira/browse/HBASE-14881 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.2.0 >Reporter: Jerry He >Assignee: Xiang Li > Fix For: 2.0.0 > > Attachments: HBASE-14881-master-v0.patch, > HBASE-14881.master.001.patch, HBASE-14881.master.002.patch, > HBASE-14881.master.003.patch > > > The current available Put API always makes a copy of the rowkey. > Let's provide an API that accepts an immutable byte array as rowkey without > making a copy. > There are cases where the caller of Put has created the immutable byte array > (e.g from a serializer) and will not change it for the Put duration. We can > avoid making a copy again. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14881) Provide a Put API that uses the provided row without coping
[ https://issues.apache.org/jira/browse/HBASE-14881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiang Li updated HBASE-14881: - Attachment: HBASE-14881.master.003.patch > Provide a Put API that uses the provided row without coping > --- > > Key: HBASE-14881 > URL: https://issues.apache.org/jira/browse/HBASE-14881 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.2.0 >Reporter: Jerry He >Assignee: Xiang Li > Fix For: 2.0.0 > > Attachments: HBASE-14881-master-v0.patch, > HBASE-14881.master.001.patch, HBASE-14881.master.002.patch, > HBASE-14881.master.003.patch > > > The current available Put API always makes a copy of the rowkey. > Let's provide an API that accepts an immutable byte array as rowkey without > making a copy. > There are cases where the caller of Put has created the immutable byte array > (e.g from a serializer) and will not change it for the Put duration. We can > avoid making a copy again. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14881) Provide a Put API that uses the provided row without coping
[ https://issues.apache.org/jira/browse/HBASE-14881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiang Li updated HBASE-14881: - Status: Patch Available (was: Open) > Provide a Put API that uses the provided row without coping > --- > > Key: HBASE-14881 > URL: https://issues.apache.org/jira/browse/HBASE-14881 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.2.0 >Reporter: Jerry He >Assignee: Xiang Li > Fix For: 2.0.0 > > Attachments: HBASE-14881-master-v0.patch, > HBASE-14881.master.001.patch, HBASE-14881.master.002.patch, > HBASE-14881.master.003.patch > > > The current available Put API always makes a copy of the rowkey. > Let's provide an API that accepts an immutable byte array as rowkey without > making a copy. > There are cases where the caller of Put has created the immutable byte array > (e.g from a serializer) and will not change it for the Put duration. We can > avoid making a copy again. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14881) Provide a Put API that uses the provided row without coping
[ https://issues.apache.org/jira/browse/HBASE-14881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15401075#comment-15401075 ] Xiang Li commented on HBASE-14881: -- Hi Anoop, thanks for the review again! I removed the line of 155(checkRow() in else) and also correct an error in one comment. Patch 003 is uploaded. > Provide a Put API that uses the provided row without coping > --- > > Key: HBASE-14881 > URL: https://issues.apache.org/jira/browse/HBASE-14881 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.2.0 >Reporter: Jerry He >Assignee: Xiang Li > Fix For: 2.0.0 > > Attachments: HBASE-14881-master-v0.patch, > HBASE-14881.master.001.patch, HBASE-14881.master.002.patch > > > The current available Put API always makes a copy of the rowkey. > Let's provide an API that accepts an immutable byte array as rowkey without > making a copy. > There are cases where the caller of Put has created the immutable byte array > (e.g from a serializer) and will not change it for the Put duration. We can > avoid making a copy again. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16224) Reduce the number of RPCs for the large PUTs
[ https://issues.apache.org/jira/browse/HBASE-16224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ChiaPing Tsai updated HBASE-16224: -- Status: Patch Available (was: Open) > Reduce the number of RPCs for the large PUTs > > > Key: HBASE-16224 > URL: https://issues.apache.org/jira/browse/HBASE-16224 > Project: HBase > Issue Type: Improvement >Reporter: ChiaPing Tsai >Assignee: ChiaPing Tsai >Priority: Minor > Attachments: HBASE-16224-v1.patch, HBASE-16224-v2.patch, > HBASE-16224-v3.patch, HBASE-16224-v4.patch, experiment.xlsx > > > This patch is proposed to reduce the number of RPC for the large PUTs > The number and data size of write thread(SingleServerRequestRunnable) is a > result of three main factors： > 1) The flush size taken by BufferedMutatorImpl#backgroundFlushCommits > 2) The limit of task number > 3) ClientBackoffPolicy > A lot of threads created with less MUTATIONs is a result of two reason: 1) > many regions of target table are in different server. 2) flush size in step > one is summed by “all” server rather than “individual” server > This patch removes the limit of flush size in step one and add maximum size > to submit for each server in the AsyncProcess -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16224) Reduce the number of RPCs for the large PUTs
[ https://issues.apache.org/jira/browse/HBASE-16224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ChiaPing Tsai updated HBASE-16224: -- Attachment: HBASE-16224-v4.patch resubmit > Reduce the number of RPCs for the large PUTs > > > Key: HBASE-16224 > URL: https://issues.apache.org/jira/browse/HBASE-16224 > Project: HBase > Issue Type: Improvement >Reporter: ChiaPing Tsai >Assignee: ChiaPing Tsai >Priority: Minor > Attachments: HBASE-16224-v1.patch, HBASE-16224-v2.patch, > HBASE-16224-v3.patch, HBASE-16224-v4.patch, experiment.xlsx > > > This patch is proposed to reduce the number of RPC for the large PUTs > The number and data size of write thread(SingleServerRequestRunnable) is a > result of three main factors： > 1) The flush size taken by BufferedMutatorImpl#backgroundFlushCommits > 2) The limit of task number > 3) ClientBackoffPolicy > A lot of threads created with less MUTATIONs is a result of two reason: 1) > many regions of target table are in different server. 2) flush size in step > one is summed by “all” server rather than “individual” server > This patch removes the limit of flush size in step one and add maximum size > to submit for each server in the AsyncProcess -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16224) Reduce the number of RPCs for the large PUTs
[ https://issues.apache.org/jira/browse/HBASE-16224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ChiaPing Tsai updated HBASE-16224: -- Attachment: (was: HBASE-16224-v4.patch) > Reduce the number of RPCs for the large PUTs > > > Key: HBASE-16224 > URL: https://issues.apache.org/jira/browse/HBASE-16224 > Project: HBase > Issue Type: Improvement >Reporter: ChiaPing Tsai >Assignee: ChiaPing Tsai >Priority: Minor > Attachments: HBASE-16224-v1.patch, HBASE-16224-v2.patch, > HBASE-16224-v3.patch, experiment.xlsx > > > This patch is proposed to reduce the number of RPC for the large PUTs > The number and data size of write thread(SingleServerRequestRunnable) is a > result of three main factors： > 1) The flush size taken by BufferedMutatorImpl#backgroundFlushCommits > 2) The limit of task number > 3) ClientBackoffPolicy > A lot of threads created with less MUTATIONs is a result of two reason: 1) > many regions of target table are in different server. 2) flush size in step > one is summed by “all” server rather than “individual” server > This patch removes the limit of flush size in step one and add maximum size > to submit for each server in the AsyncProcess -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16224) Reduce the number of RPCs for the large PUTs
[ https://issues.apache.org/jira/browse/HBASE-16224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ChiaPing Tsai updated HBASE-16224: -- Status: Open (was: Patch Available) > Reduce the number of RPCs for the large PUTs > > > Key: HBASE-16224 > URL: https://issues.apache.org/jira/browse/HBASE-16224 > Project: HBase > Issue Type: Improvement >Reporter: ChiaPing Tsai >Assignee: ChiaPing Tsai >Priority: Minor > Attachments: HBASE-16224-v1.patch, HBASE-16224-v2.patch, > HBASE-16224-v3.patch, experiment.xlsx > > > This patch is proposed to reduce the number of RPC for the large PUTs > The number and data size of write thread(SingleServerRequestRunnable) is a > result of three main factors： > 1) The flush size taken by BufferedMutatorImpl#backgroundFlushCommits > 2) The limit of task number > 3) ClientBackoffPolicy > A lot of threads created with less MUTATIONs is a result of two reason: 1) > many regions of target table are in different server. 2) flush size in step > one is summed by “all” server rather than “individual” server > This patch removes the limit of flush size in step one and add maximum size > to submit for each server in the AsyncProcess -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14345) Consolidate printUsage in IntegrationTestLoadAndVerify
[
https://issues.apache.org/jira/browse/HBASE-14345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reid Chan updated HBASE-14345:
--
Status: Open (was: Patch Available)
> Consolidate printUsage in IntegrationTestLoadAndVerify
> --
>
> Key: HBASE-14345
> URL: https://issues.apache.org/jira/browse/HBASE-14345
> Project: HBase
> Issue Type: Improvement
> Components: integration tests
>Reporter: Nick Dimiduk
>Assignee: Reid Chan
>Priority: Trivial
> Labels: beginner
> Attachments: 2016-07-30-Without System_exit.png,
> HBASE-14345.002.patch, HBASE-14345.003.patch, HBASE-14345.patch,
> itlav-2016-07-07.png, itlv.png
>
>
> Investigating the use of {{itlav}} is a little screwy. Subclasses are not
> overriding the {{printUsage()}} methods correctly, so you have to pass
> {{--help}} to get some info and no arguments to get the rest.
> {noformat}
> [hbase@ndimiduk-112rc2-7 ~]$ hbase
> org.apache.hadoop.hbase.test.IntegrationTestLoadAndVerify --help
> usage: bin/hbase org.apache.hadoop.hbase.test.IntegrationTestLoadAndVerify
>
> Options:
> -h,--help Show usage
> -m,--monkey Which chaos monkey to run
> -monkeyProps The properties file for specifying chaos monkey
> properties.
> -ncc,--noClusterCleanUp Don't clean up the cluster at the end
> [hbase@ndimiduk-112rc2-7 ~]$ hbase
> org.apache.hadoop.hbase.test.IntegrationTestLoadAndVerify
> IntegrationTestLoadAndVerify [-Doptions]
> Loads a table with row dependencies and verifies the dependency chains
> Options
> -Dloadmapper.table=Table to write/verify (default autogen)
> -Dloadmapper.backrefs=Number of backreferences per row (default
> 50)
> -Dloadmapper.num_to_write=Number of rows per mapper (default 100,000
> per mapper)
> -Dloadmapper.deleteAfter= Delete after a successful verify (default
> true)
> -Dloadmapper.numPresplits=Number of presplit regions to start with
> (default 40)
> -Dloadmapper.map.tasks= Number of map tasks for load (default 200)
> -Dverify.reduce.tasks=Number of reduce tasks for verify (default
> 35)
> -Dverify.scannercaching= Number hbase scanner caching rows to read
> (default 50)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14345) Consolidate printUsage in IntegrationTestLoadAndVerify
[
https://issues.apache.org/jira/browse/HBASE-14345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reid Chan updated HBASE-14345:
--
Status: Patch Available (was: Open)
> Consolidate printUsage in IntegrationTestLoadAndVerify
> --
>
> Key: HBASE-14345
> URL: https://issues.apache.org/jira/browse/HBASE-14345
> Project: HBase
> Issue Type: Improvement
> Components: integration tests
>Reporter: Nick Dimiduk
>Assignee: Reid Chan
>Priority: Trivial
> Labels: beginner
> Attachments: 2016-07-30-Without System_exit.png,
> HBASE-14345.002.patch, HBASE-14345.003.patch, HBASE-14345.patch,
> itlav-2016-07-07.png, itlv.png
>
>
> Investigating the use of {{itlav}} is a little screwy. Subclasses are not
> overriding the {{printUsage()}} methods correctly, so you have to pass
> {{--help}} to get some info and no arguments to get the rest.
> {noformat}
> [hbase@ndimiduk-112rc2-7 ~]$ hbase
> org.apache.hadoop.hbase.test.IntegrationTestLoadAndVerify --help
> usage: bin/hbase org.apache.hadoop.hbase.test.IntegrationTestLoadAndVerify
>
> Options:
> -h,--help Show usage
> -m,--monkey Which chaos monkey to run
> -monkeyProps The properties file for specifying chaos monkey
> properties.
> -ncc,--noClusterCleanUp Don't clean up the cluster at the end
> [hbase@ndimiduk-112rc2-7 ~]$ hbase
> org.apache.hadoop.hbase.test.IntegrationTestLoadAndVerify
> IntegrationTestLoadAndVerify [-Doptions]
> Loads a table with row dependencies and verifies the dependency chains
> Options
> -Dloadmapper.table=Table to write/verify (default autogen)
> -Dloadmapper.backrefs=Number of backreferences per row (default
> 50)
> -Dloadmapper.num_to_write=Number of rows per mapper (default 100,000
> per mapper)
> -Dloadmapper.deleteAfter= Delete after a successful verify (default
> true)
> -Dloadmapper.numPresplits=Number of presplit regions to start with
> (default 40)
> -Dloadmapper.map.tasks= Number of map tasks for load (default 200)
> -Dverify.reduce.tasks=Number of reduce tasks for verify (default
> 35)
> -Dverify.scannercaching= Number hbase scanner caching rows to read
> (default 50)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-9899) for idempotent operation dups, return the result instead of throwing conflict exception
[
https://issues.apache.org/jira/browse/HBASE-9899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15401046#comment-15401046
]
Hadoop QA commented on HBASE-9899:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 9s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
27s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 4s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 55s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
51s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
33s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
58s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 45s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 53s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
7s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 3s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 3s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 55s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 55s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
34s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 20m 0s
{color} | {color:red} Patch causes 11 errors with Hadoop v2.6.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 22m 50s
{color} | {color:red} Patch causes 11 errors with Hadoop v2.6.2. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 25m 41s
{color} | {color:red} Patch causes 11 errors with Hadoop v2.6.3. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 28m 34s
{color} | {color:red} Patch causes 11 errors with Hadoop v2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
23s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 52s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 54s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 58s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 89m 53s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
28s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 141m 58s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | h
[jira] [Commented] (HBASE-16225) Refactor ScanQueryMatcher
[
https://issues.apache.org/jira/browse/HBASE-16225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15401031#comment-15401031
]
Hadoop QA commented on HBASE-16225:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 18 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 27s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
13s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 10s
{color} | {color:green} branch-1 passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 57s
{color} | {color:green} branch-1 passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
35s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
53s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 52s
{color} | {color:green} branch-1 passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 52s
{color} | {color:green} branch-1 passed with JDK v1.7.0_80 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
7s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 7s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 7s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 57s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 57s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
34s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
18m 40s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m 25s
{color} | {color:red} hbase-server generated 3 new + 0 unchanged - 0 fixed = 3
total (was 0) {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 3m 15s
{color} | {color:red} hbase-server-jdk1.8.0 with JDK v1.8.0 generated 1 new + 3
unchanged - 0 fixed = 4 total (was 3) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 50s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 4m 6s
{color} | {color:red} hbase-server-jdk1.7.0_80 with JDK v1.7.0_80 generated 1
new + 3 unchanged - 0 fixed = 4 total (was 3) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 50s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 1s
{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 49m 29s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 91m 51s {color
[jira] [Commented] (HBASE-14921) Memory optimizations
[ https://issues.apache.org/jira/browse/HBASE-14921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15401013#comment-15401013 ] Anastasia Braginsky commented on HBASE-14921: - Hey [~ram_krish], thank you for your comment explaining the problem with MemStoreCompactorIterator! You said that you have the UT exposing the bug. I fixed the problem (at least I think I do), but in order to be sure I would very much like to run your UT. Can you please share the UT with me anyhow? I am OK with even copy-pasting the method here. > Memory optimizations > > > Key: HBASE-14921 > URL: https://issues.apache.org/jira/browse/HBASE-14921 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: Eshcar Hillel >Assignee: Anastasia Braginsky > Attachments: CellBlocksSegmentInMemStore.pdf, > CellBlocksSegmentinthecontextofMemStore(1).pdf, HBASE-14921-V01.patch, > HBASE-14921-V02.patch, HBASE-14921-V03.patch, HBASE-14921-V04-CA-V02.patch, > HBASE-14921-V04-CA.patch, HBASE-14921-V05-CAO.patch, > HBASE-14921-V06-CAO.patch, InitialCellArrayMapEvaluation.pdf, > IntroductiontoNewFlatandCompactMemStore.pdf, NewCompactingMemStoreFlow.pptx > > > Memory optimizations including compressed format representation and offheap > allocations -- This message was sent by Atlassian JIRA (v6.3.4#6332)
