[jira] [Updated] (HBASE-18078) [C++] Harden RPC by handling various communication abnormalities
[ https://issues.apache.org/jira/browse/HBASE-18078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HBASE-18078: -- Attachment: HBASE-18078.002.patch v2 fixed connection pool test failure. > [C++] Harden RPC by handling various communication abnormalities > > > Key: HBASE-18078 > URL: https://issues.apache.org/jira/browse/HBASE-18078 > Project: HBase > Issue Type: Sub-task >Reporter: Xiaobing Zhou >Assignee: Xiaobing Zhou > Attachments: HBASE-18078.000.patch, HBASE-18078.001.patch, > HBASE-18078.002.patch > > > RPC layer should handle various communication abnormalities (e.g. connection > timeout, server aborted connection, and so on). Ideally, the corresponding > exceptions should be raised and propagated through handlers of pipeline in > client. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18130) Refactor ReplicationSource
[ https://issues.apache.org/jira/browse/HBASE-18130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028751#comment-16028751 ] Chia-Ping Tsai commented on HBASE-18130: LGTM > Refactor ReplicationSource > -- > > Key: HBASE-18130 > URL: https://issues.apache.org/jira/browse/HBASE-18130 > Project: HBase > Issue Type: Improvement >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang > Attachments: HBASE-18130.master.001.patch, > HBASE-18130.master.002.patch, HBASE-18130.master.003.patch, > HBASE-18130.master.004.patch, HBASE-18130.master.004.patch > > > One basic idea is move the code about recovered queue to a new subclass > RecoveredReplicationSource. Then ReplicationSource will don't need call > isQueueRecovered many times. This will make the code more clearly. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18129) truncate_preserve fails when the truncate method doesn't exists on the master
[
https://issues.apache.org/jira/browse/HBASE-18129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028742#comment-16028742
]
Hadoop QA commented on HBASE-18129:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 30s
{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} rubocop {color} | {color:blue} 0m 0s
{color} | {color:blue} rubocop was not available. {color} |
| {color:blue}0{color} | {color:blue} ruby-lint {color} | {color:blue} 0m 0s
{color} | {color:blue} Ruby-lint was not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
20s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
24s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 13s
{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
23s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
24s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
30m 9s {color} | {color:green} The patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
17s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 13s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 7m 29s
{color} | {color:green} hbase-shell in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 43m 59s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.03.0-ce Server=17.03.0-ce Image:yetus/hbase:58c504e |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12870190/HBASE-18129-branch-1.patch
|
| JIRA Issue | HBASE-18129 |
| Optional Tests | asflicense javac javadoc unit rubocop ruby_lint |
| uname | Linux 662d72a8512e 4.8.3-std-1 #1 SMP Fri Oct 21 11:15:43 UTC 2016
x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/hbase.sh |
| git revision | branch-1 / 1a37f3b |
| Default Java | 1.8.0_131 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/6994/testReport/ |
| modules | C: hbase-shell U: hbase-shell |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/6994/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> truncate_preserve fails when the truncate method doesn't exists on the master
> -
>
> Key: HBASE-18129
> URL: https://issues.apache.org/jira/browse/HBASE-18129
> Project: HBase
> Issue Type: Bug
> Components: shell
>Affects Versions: 2.0.0, 1.2.5
>Reporter: Guangxu Cheng
>Assignee: Guangxu Cheng
> Attachments: HBASE-18129-branch-1.patch
>
>
> Recently, I runs a rolling upgrade from HBase 0.98.x to HBase 1.2.5. During
> the master hasn't been upgraded yet, I truncate a table by the command
> truncate_preserve of 1.2.5, but failed.
> {code}
> hbase(main):001:0> truncate_preserve 'cf_logs'
> Truncating 'cf_logs' table (it may take a while):
> - Disabling table...
> - Truncating table...
> - Dropping table...
> - Creating table with region boundaries...
> ERROR: no method 'createTable' for arguments
> (org.apache.hadoop.hbase.HTableDescriptor,org.jruby.java.proxies.ArrayJavaProxy)
> on
[jira] [Updated] (HBASE-18129) truncate_preserve fails when the truncate method doesn't exists on the master
[
https://issues.apache.org/jira/browse/HBASE-18129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-18129:
---
Status: Patch Available (was: Open)
> truncate_preserve fails when the truncate method doesn't exists on the master
> -
>
> Key: HBASE-18129
> URL: https://issues.apache.org/jira/browse/HBASE-18129
> Project: HBase
> Issue Type: Bug
> Components: shell
>Affects Versions: 1.2.5, 2.0.0
>Reporter: Guangxu Cheng
>Assignee: Guangxu Cheng
> Attachments: HBASE-18129-branch-1.patch
>
>
> Recently, I runs a rolling upgrade from HBase 0.98.x to HBase 1.2.5. During
> the master hasn't been upgraded yet, I truncate a table by the command
> truncate_preserve of 1.2.5, but failed.
> {code}
> hbase(main):001:0> truncate_preserve 'cf_logs'
> Truncating 'cf_logs' table (it may take a while):
> - Disabling table...
> - Truncating table...
> - Dropping table...
> - Creating table with region boundaries...
> ERROR: no method 'createTable' for arguments
> (org.apache.hadoop.hbase.HTableDescriptor,org.jruby.java.proxies.ArrayJavaProxy)
> on Java::OrgApacheHadoopHbaseClient::HBaseAdmin
> {code}
> After checking code and commit history, I found it's HBASE-12833 which causes
> this bug.so we should fix it.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-18129) truncate_preserve fails when the truncate method doesn't exists on the master
[
https://issues.apache.org/jira/browse/HBASE-18129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028732#comment-16028732
]
Ted Yu commented on HBASE-18129:
Can you attach patch for master branch ?
> truncate_preserve fails when the truncate method doesn't exists on the master
> -
>
> Key: HBASE-18129
> URL: https://issues.apache.org/jira/browse/HBASE-18129
> Project: HBase
> Issue Type: Bug
> Components: shell
>Affects Versions: 2.0.0, 1.2.5
>Reporter: Guangxu Cheng
>Assignee: Guangxu Cheng
> Attachments: HBASE-18129-branch-1.patch
>
>
> Recently, I runs a rolling upgrade from HBase 0.98.x to HBase 1.2.5. During
> the master hasn't been upgraded yet, I truncate a table by the command
> truncate_preserve of 1.2.5, but failed.
> {code}
> hbase(main):001:0> truncate_preserve 'cf_logs'
> Truncating 'cf_logs' table (it may take a while):
> - Disabling table...
> - Truncating table...
> - Dropping table...
> - Creating table with region boundaries...
> ERROR: no method 'createTable' for arguments
> (org.apache.hadoop.hbase.HTableDescriptor,org.jruby.java.proxies.ArrayJavaProxy)
> on Java::OrgApacheHadoopHbaseClient::HBaseAdmin
> {code}
> After checking code and commit history, I found it's HBASE-12833 which causes
> this bug.so we should fix it.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-18129) truncate_preserve fails when the truncate method doesn't exists on the master
[
https://issues.apache.org/jira/browse/HBASE-18129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028728#comment-16028728
]
Ted Yu commented on HBASE-18129:
It is Okay to include the test in this JIRA so that we know the fix is
effective.
> truncate_preserve fails when the truncate method doesn't exists on the master
> -
>
> Key: HBASE-18129
> URL: https://issues.apache.org/jira/browse/HBASE-18129
> Project: HBase
> Issue Type: Bug
> Components: shell
>Affects Versions: 2.0.0, 1.2.5
>Reporter: Guangxu Cheng
>Assignee: Guangxu Cheng
> Attachments: HBASE-18129-branch-1.patch
>
>
> Recently, I runs a rolling upgrade from HBase 0.98.x to HBase 1.2.5. During
> the master hasn't been upgraded yet, I truncate a table by the command
> truncate_preserve of 1.2.5, but failed.
> {code}
> hbase(main):001:0> truncate_preserve 'cf_logs'
> Truncating 'cf_logs' table (it may take a while):
> - Disabling table...
> - Truncating table...
> - Dropping table...
> - Creating table with region boundaries...
> ERROR: no method 'createTable' for arguments
> (org.apache.hadoop.hbase.HTableDescriptor,org.jruby.java.proxies.ArrayJavaProxy)
> on Java::OrgApacheHadoopHbaseClient::HBaseAdmin
> {code}
> After checking code and commit history, I found it's HBASE-12833 which causes
> this bug.so we should fix it.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-18129) truncate_preserve fails when the truncate method doesn't exists on the master
[
https://issues.apache.org/jira/browse/HBASE-18129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028725#comment-16028725
]
Guangxu Cheng commented on HBASE-18129:
---
[~yuzhih...@gmail.com], mind taking a look at it ? thanks
> truncate_preserve fails when the truncate method doesn't exists on the master
> -
>
> Key: HBASE-18129
> URL: https://issues.apache.org/jira/browse/HBASE-18129
> Project: HBase
> Issue Type: Bug
> Components: shell
>Affects Versions: 2.0.0, 1.2.5
>Reporter: Guangxu Cheng
>Assignee: Guangxu Cheng
> Attachments: HBASE-18129-branch-1.patch
>
>
> Recently, I runs a rolling upgrade from HBase 0.98.x to HBase 1.2.5. During
> the master hasn't been upgraded yet, I truncate a table by the command
> truncate_preserve of 1.2.5, but failed.
> {code}
> hbase(main):001:0> truncate_preserve 'cf_logs'
> Truncating 'cf_logs' table (it may take a while):
> - Disabling table...
> - Truncating table...
> - Dropping table...
> - Creating table with region boundaries...
> ERROR: no method 'createTable' for arguments
> (org.apache.hadoop.hbase.HTableDescriptor,org.jruby.java.proxies.ArrayJavaProxy)
> on Java::OrgApacheHadoopHbaseClient::HBaseAdmin
> {code}
> After checking code and commit history, I found it's HBASE-12833 which causes
> this bug.so we should fix it.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-18130) Refactor ReplicationSource
[
https://issues.apache.org/jira/browse/HBASE-18130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028713#comment-16028713
]
Hadoop QA commented on HBASE-18130:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 48s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
16s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 48s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
58s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
10s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
54s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 47s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 47s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
4s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
19s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
34m 53s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
21s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 32s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 123m 44s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
30s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 175m 22s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:757bf37 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12870337/HBASE-18130.master.004.patch
|
| JIRA Issue | HBASE-18130 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 884a810a77dc 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2
15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 6846b03 |
| Default Java | 1.8.0_131 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/6993/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/6993/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Refactor ReplicationSource
> --
>
> Key: HBASE-18130
> URL: https://issues.apache.org/jira/browse/HBASE-18130
> Project: HBase
> Issue Type: Improvement
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
> Attachments: HBASE-18130.master.001.patch,
> HBASE-18130.master.002.patch, HBASE-18130.master.003.patch,
> HBASE-18130.master.004.patch,
[jira] [Commented] (HBASE-14614) Procedure v2: Core Assignment Manager
[
https://issues.apache.org/jira/browse/HBASE-14614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028699#comment-16028699
]
Hadoop QA commented on HBASE-14614:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 31s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 100 new or modified
test files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 35s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
18s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m 5s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 5m
16s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m
37s {color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m 52s
{color} | {color:red} hbase-protocol-shaded in master has 24 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m 23s
{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 19s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
26s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m 22s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 5m 22s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 5m 22s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 5m
23s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m
46s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 1s
{color} | {color:red} The patch has 655 line(s) that end in whitespace. Use git
apply --whitespace=fix. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
57m 36s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 3m
50s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 17m
36s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m 28s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m 8s
{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 1s
{color} | {color:green} hbase-protocol-shaded in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 4m 1s
{color} | {color:green} hbase-procedure in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 28s
{color} | {color:green} hbase-hadoop-compat in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 40s
{color} | {color:green} hbase-hadoop2-compat in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m 25s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 195m 31s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 4s
{color} | {color:green} hbase-rsgroup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 40s
{color} | {color:green} hbase-it in the patch
[jira] [Updated] (HBASE-18130) Refactor ReplicationSource
[ https://issues.apache.org/jira/browse/HBASE-18130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-18130: --- Attachment: HBASE-18130.master.004.patch > Refactor ReplicationSource > -- > > Key: HBASE-18130 > URL: https://issues.apache.org/jira/browse/HBASE-18130 > Project: HBase > Issue Type: Improvement >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang > Attachments: HBASE-18130.master.001.patch, > HBASE-18130.master.002.patch, HBASE-18130.master.003.patch, > HBASE-18130.master.004.patch, HBASE-18130.master.004.patch > > > One basic idea is move the code about recovered queue to a new subclass > RecoveredReplicationSource. Then ReplicationSource will don't need call > isQueueRecovered many times. This will make the code more clearly. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18097) Client can save 1 RPC call for CloseScannerRequest
[
https://issues.apache.org/jira/browse/HBASE-18097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028663#comment-16028663
]
Karan Mehta commented on HBASE-18097:
-
The first part for saving 1 RPC request is already implemented as a part of
HBASE-17508, where the scannerId is set to -1 whenever results are not left in
region.
For the second part related to ScanResponse proto, we can save some data in RPC
call.
[~Apache9] Do you feel it is reasonable to do so from the next version?
> Client can save 1 RPC call for CloseScannerRequest
> --
>
> Key: HBASE-18097
> URL: https://issues.apache.org/jira/browse/HBASE-18097
> Project: HBase
> Issue Type: Improvement
>Reporter: Karan Mehta
>
> Starting version 1.3, HBase automatically closes scanner on server side
> whenever the results are exhausted and corresponding bits are set in the
> {{ScanResponse}} proto returned to the client. We can use that info to
> eliminate the closeScanRequest RPC call, thereby saving 1 RPC per region per
> scan. This can be particularly useful for tables with more regions.
> Also, currently the {{ScanResponse}} proto sends out 1 bit per {{Result}}
> that it has embeds inside the {{CellScanner}} to indicate if it is partial or
> not.
> {code}
> // In every RPC response there should be at most a single partial result.
> Furthermore, if
> // there is a partial result, it is guaranteed to be in the last position
> of the array.
> {code}
> According to client, only the last result can be partial, thus this repeated
> bool can be converted to a bool, thus reducing overhead of serialization and
> deserialization of the array. This will break wire compatibility therefore
> this is something to look for in upcoming versions.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-15903) Delete Object
[ https://issues.apache.org/jira/browse/HBASE-15903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028645#comment-16028645 ] Ted Yu commented on HBASE-15903: For AddColumns() and AddColumn(), the timestamp parameter is used to construct Cell - not to change the timestamp of Delete object. This would be corrected in the next patch. > Delete Object > - > > Key: HBASE-15903 > URL: https://issues.apache.org/jira/browse/HBASE-15903 > Project: HBase > Issue Type: Sub-task >Reporter: Sudeep Sunthankar >Assignee: Ted Yu > Attachments: 15903.v2.txt, 15903.v4.txt, > HBASE-15903.HBASE-14850.v1.patch > > > Patch for creating Delete objects. These Delete objects are used by the Table > implementation to delete rowkey from a table. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-14614) Procedure v2: Core Assignment Manager
[ https://issues.apache.org/jira/browse/HBASE-14614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028595#comment-16028595 ] stack commented on HBASE-14614: --- Retrying. These failing tests run a long time but pass for me locally... > Procedure v2: Core Assignment Manager > - > > Key: HBASE-14614 > URL: https://issues.apache.org/jira/browse/HBASE-14614 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Affects Versions: 2.0.0 >Reporter: Stephen Yuan Jiang >Assignee: Matteo Bertozzi > Fix For: 2.0.0 > > Attachments: HBASE-14614.master.003.patch, > HBASE-14614.master.004.patch, HBASE-14614.master.005.patch, > HBASE-14614.master.006.patch, HBASE-14614.master.007.patch, > HBASE-14614.master.008.patch, HBASE-14614.master.009.patch, > HBASE-14614.master.010.patch, HBASE-14614.master.012.patch, > HBASE-14614.master.013.patch, HBASE-14614.master.014.patch, > HBASE-14614.master.015.patch, HBASE-14614.master.017.patch, > HBASE-14614.master.018.patch, HBASE-14614.master.019.patch, > HBASE-14614.master.020.patch, HBASE-14614.master.022.patch, > HBASE-14614.master.023.patch, HBASE-14614.master.024.patch, > HBASE-14614.master.025.patch, HBASE-14614.master.026.patch, > HBASE-14614.master.027.patch, HBASE-14614.master.028.patch, > HBASE-14614.master.029.patch, HBASE-14614.master.030.patch, > HBASE-14614.master.033.patch, HBASE-14614.master.038.patch, > HBASE-14614.master.039.patch, HBASE-14614.master.040.patch, > HBASE-14614.master.041.patch, HBASE-14614.master.042.patch, > HBASE-14614.master.043.patch, HBASE-14614.master.044.patch, > HBASE-14614.master.045.patch, HBASE-14614.master.045.patch, > HBASE-14614.master.046.patch, HBASE-14614.master.047.patch > > > New AssignmentManager implemented using proc-v2. > - AssignProcedure handle assignment operation > - UnassignProcedure handle unassign operation > - MoveRegionProcedure handle move/balance operation > Concurrent Assign operations are batched together and sent to the balancer > Concurrent Assign and Unassign operation ready to be sent to the RS are > batched together > This patch is an intermediate state where we add the new AM as > AssignmentManager2() to the master, to be reached by tests. but the new AM > will not be integrated with the rest of the system. Only new am unit-tests > will exercise the new assigment manager. The integration with the master code > is part of HBASE-14616 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-14614) Procedure v2: Core Assignment Manager
[ https://issues.apache.org/jira/browse/HBASE-14614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-14614: -- Attachment: HBASE-14614.master.047.patch > Procedure v2: Core Assignment Manager > - > > Key: HBASE-14614 > URL: https://issues.apache.org/jira/browse/HBASE-14614 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Affects Versions: 2.0.0 >Reporter: Stephen Yuan Jiang >Assignee: Matteo Bertozzi > Fix For: 2.0.0 > > Attachments: HBASE-14614.master.003.patch, > HBASE-14614.master.004.patch, HBASE-14614.master.005.patch, > HBASE-14614.master.006.patch, HBASE-14614.master.007.patch, > HBASE-14614.master.008.patch, HBASE-14614.master.009.patch, > HBASE-14614.master.010.patch, HBASE-14614.master.012.patch, > HBASE-14614.master.013.patch, HBASE-14614.master.014.patch, > HBASE-14614.master.015.patch, HBASE-14614.master.017.patch, > HBASE-14614.master.018.patch, HBASE-14614.master.019.patch, > HBASE-14614.master.020.patch, HBASE-14614.master.022.patch, > HBASE-14614.master.023.patch, HBASE-14614.master.024.patch, > HBASE-14614.master.025.patch, HBASE-14614.master.026.patch, > HBASE-14614.master.027.patch, HBASE-14614.master.028.patch, > HBASE-14614.master.029.patch, HBASE-14614.master.030.patch, > HBASE-14614.master.033.patch, HBASE-14614.master.038.patch, > HBASE-14614.master.039.patch, HBASE-14614.master.040.patch, > HBASE-14614.master.041.patch, HBASE-14614.master.042.patch, > HBASE-14614.master.043.patch, HBASE-14614.master.044.patch, > HBASE-14614.master.045.patch, HBASE-14614.master.045.patch, > HBASE-14614.master.046.patch, HBASE-14614.master.047.patch > > > New AssignmentManager implemented using proc-v2. > - AssignProcedure handle assignment operation > - UnassignProcedure handle unassign operation > - MoveRegionProcedure handle move/balance operation > Concurrent Assign operations are batched together and sent to the balancer > Concurrent Assign and Unassign operation ready to be sent to the RS are > batched together > This patch is an intermediate state where we add the new AM as > AssignmentManager2() to the master, to be reached by tests. but the new AM > will not be integrated with the rest of the system. Only new am unit-tests > will exercise the new assigment manager. The integration with the master code > is part of HBASE-14616 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18010) Connect CellChunkMap to be used for flattening in CompactingMemStore
[ https://issues.apache.org/jira/browse/HBASE-18010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028567#comment-16028567 ] stack commented on HBASE-18010: --- What is going on in this issue is great. > Connect CellChunkMap to be used for flattening in CompactingMemStore > > > Key: HBASE-18010 > URL: https://issues.apache.org/jira/browse/HBASE-18010 > Project: HBase > Issue Type: Sub-task >Reporter: Anastasia Braginsky > > The CellChunkMap helps to create a new type of ImmutableSegment, where the > index (CellSet's delegatee) is going to be CellChunkMap. No big cells or > upserted cells are going to be supported here. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18010) Connect CellChunkMap to be used for flattening in CompactingMemStore
[
https://issues.apache.org/jira/browse/HBASE-18010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028508#comment-16028508
]
ramkrishna.s.vasudevan commented on HBASE-18010:
bq. 3. C was reused from pool and returned to the user once again
At this point of time we again add it back to the mapping. See
ChunkCreator#getChunk()
{code}
// put this chunk into the chunkIdMap
this.chunkIdMap.put(chunk.getId(), new SoftReference<>(chunk));
// now we need to actually do the expensive memory allocation step in case
of a new chunk,
// else only the offset is set to the beginning of the chunk to accept
allocations
chunk.init();
{code}
> Connect CellChunkMap to be used for flattening in CompactingMemStore
>
>
> Key: HBASE-18010
> URL: https://issues.apache.org/jira/browse/HBASE-18010
> Project: HBase
> Issue Type: Sub-task
>Reporter: Anastasia Braginsky
>
> The CellChunkMap helps to create a new type of ImmutableSegment, where the
> index (CellSet's delegatee) is going to be CellChunkMap. No big cells or
> upserted cells are going to be supported here.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-18130) Refactor ReplicationSource
[
https://issues.apache.org/jira/browse/HBASE-18130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028494#comment-16028494
]
Hadoop QA commented on HBASE-18130:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 36s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
20s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 48s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
58s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
21s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 8s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
49s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 42s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 42s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
57s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
20s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
31m 3s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
11s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 32s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 45m 4s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 92m 5s {color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Timed out junit tests | org.apache.hadoop.hbase.ipc.TestSimpleRpcScheduler |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.10.1 Server=1.10.1 Image:yetus/hbase:757bf37 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12870311/HBASE-18130.master.004.patch
|
| JIRA Issue | HBASE-18130 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux f00422803f87 3.19.0-25-generic #26~14.04.1-Ubuntu SMP Fri Jul
24 21:16:20 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 6846b03 |
| Default Java | 1.8.0_131 |
| findbugs | v3.0.0 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/6991/artifact/patchprocess/patch-unit-hbase-server.txt
|
| unit test logs |
https://builds.apache.org/job/PreCommit-HBASE-Build/6991/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/6991/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/6991/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Refactor ReplicationSource
> --
>
> Key: HBASE-18130
> URL:
[jira] [Commented] (HBASE-18130) Refactor ReplicationSource
[
https://issues.apache.org/jira/browse/HBASE-18130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028439#comment-16028439
]
Hadoop QA commented on HBASE-18130:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 31s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
39s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 38s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
53s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 29s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
43s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 38s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 38s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
47s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
31m 35s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
19s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 32s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 97m 37s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
28s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 143m 47s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hbase.replication.multiwal.TestReplicationSyncUpToolWithMultipleWAL |
| |
hadoop.hbase.replication.multiwal.TestReplicationSyncUpToolWithMultipleAsyncWAL
|
| | hadoop.hbase.replication.TestReplicationSyncUpTool |
| Timed out junit tests | org.apache.hadoop.hbase.client.TestMetaWithReplicas |
| | org.apache.hadoop.hbase.client.TestFromClientSideWithCoprocessor |
| | org.apache.hadoop.hbase.client.TestTableSnapshotScanner |
| | org.apache.hadoop.hbase.replication.TestSerialReplication |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:757bf37 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12870298/HBASE-18130.master.003.patch
|
| JIRA Issue | HBASE-18130 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 5b084bd273f8 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6
15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 6846b03 |
| Default Java | 1.8.0_131 |
| findbugs | v3.0.0 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/6990/artifact/patchprocess/patch-unit-hbase-server.txt
|
| unit test logs |
https://builds.apache.org/job/PreCommit-HBASE-Build/6990/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test
[jira] [Updated] (HBASE-18130) Refactor ReplicationSource
[ https://issues.apache.org/jira/browse/HBASE-18130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-18130: --- Attachment: HBASE-18130.master.004.patch Fix ut. > Refactor ReplicationSource > -- > > Key: HBASE-18130 > URL: https://issues.apache.org/jira/browse/HBASE-18130 > Project: HBase > Issue Type: Improvement >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang > Attachments: HBASE-18130.master.001.patch, > HBASE-18130.master.002.patch, HBASE-18130.master.003.patch, > HBASE-18130.master.004.patch > > > One basic idea is move the code about recovered queue to a new subclass > RecoveredReplicationSource. Then ReplicationSource will don't need call > isQueueRecovered many times. This will make the code more clearly. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18130) Refactor ReplicationSource
[
https://issues.apache.org/jira/browse/HBASE-18130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028317#comment-16028317
]
Hadoop QA commented on HBASE-18130:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 26s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
59s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 47s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
53s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
14s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
50s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 41s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 41s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
47s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
30m 50s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
55s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 27s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 111m 26s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 157m 10s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hbase.replication.multiwal.TestReplicationSyncUpToolWithMultipleWAL |
| |
hadoop.hbase.replication.multiwal.TestReplicationSyncUpToolWithMultipleAsyncWAL
|
| | hadoop.hbase.util.TestHBaseFsckReplicas |
| | hadoop.hbase.replication.TestReplicationSyncUpTool |
| Timed out junit tests |
org.apache.hadoop.hbase.security.access.TestAccessController2 |
| | org.apache.hadoop.hbase.security.TestSecureIPC |
| | org.apache.hadoop.hbase.security.access.TestAccessController |
| | org.apache.hadoop.hbase.replication.TestSerialReplication |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.03.0-ce Server=17.03.0-ce Image:yetus/hbase:757bf37 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12870288/HBASE-18130.master.002.patch
|
| JIRA Issue | HBASE-18130 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 05387f77d3a5 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 6846b03 |
| Default Java | 1.8.0_131 |
| findbugs | v3.0.0 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/6989/artifact/patchprocess/patch-unit-hbase-server.txt
|
| unit test logs |
[jira] [Assigned] (HBASE-18056) Change CompactingMemStore in BASIC mode to merge multiple segments in pipeline
[ https://issues.apache.org/jira/browse/HBASE-18056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Bortnikov reassigned HBASE-18056: Assignee: Anastasia Braginsky > Change CompactingMemStore in BASIC mode to merge multiple segments in pipeline > -- > > Key: HBASE-18056 > URL: https://issues.apache.org/jira/browse/HBASE-18056 > Project: HBase > Issue Type: Sub-task >Reporter: Anastasia Braginsky >Assignee: Anastasia Braginsky > Attachments: HBASE-18056-V01.patch > > > Under HBASE-16417 it was decided that CompactingMemStore in BASIC mode should > merge multiple ImmutableSegments in CompactionPipeline. Basic+Merge actually > demonstrated reduction in GC, alongside improvement in other metrics. > However, the limit on the number of segments in pipeline is still set to 30. > Under this JIRA it should be changed to 1, as it was tested under HBASE-16417. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-18130) Refactor ReplicationSource
[ https://issues.apache.org/jira/browse/HBASE-18130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-18130: --- Attachment: HBASE-18130.master.003.patch > Refactor ReplicationSource > -- > > Key: HBASE-18130 > URL: https://issues.apache.org/jira/browse/HBASE-18130 > Project: HBase > Issue Type: Improvement >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang > Attachments: HBASE-18130.master.001.patch, > HBASE-18130.master.002.patch, HBASE-18130.master.003.patch > > > One basic idea is move the code about recovered queue to a new subclass > RecoveredReplicationSource. Then ReplicationSource will don't need call > isQueueRecovered many times. This will make the code more clearly. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18010) Connect CellChunkMap to be used for flattening in CompactingMemStore
[ https://issues.apache.org/jira/browse/HBASE-18010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028264#comment-16028264 ] Anastasia Braginsky commented on HBASE-18010: - May be the previous scenario was not detailed enough, here is the more detailed one: 1. Chunk C was allocated and got ID 5 1.1 After a while, C and all its cells are not in any use any longer 2. C was reclaimed and removed from mapping, but was returned to pool and is still holding ID 5 3. C was reused from pool and returned to the user once again 3.1 Cell A was allocated on C (new cell) 4. Cell residing on C is asked what is its chunk id and 5 is the answer 5. ChunkCreator is asked to translate chunk ID 5, but returns null, because C's mapping was removed at Step 2 bq. when the cell is in use for any reason then we should not be returning that back to pool right? In time when chunk was reclaimed all its cells were not in use. Chunk was returned to ChunkCreator legitimately. The problem is that chunk was reused and its mapping was deleted. > Connect CellChunkMap to be used for flattening in CompactingMemStore > > > Key: HBASE-18010 > URL: https://issues.apache.org/jira/browse/HBASE-18010 > Project: HBase > Issue Type: Sub-task >Reporter: Anastasia Braginsky > > The CellChunkMap helps to create a new type of ImmutableSegment, where the > index (CellSet's delegatee) is going to be CellChunkMap. No big cells or > upserted cells are going to be supported here. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18010) Connect CellChunkMap to be used for flattening in CompactingMemStore
[ https://issues.apache.org/jira/browse/HBASE-18010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028251#comment-16028251 ] ramkrishna.s.vasudevan commented on HBASE-18010: bq.5. ChunkCreator is asked to translate chunk ID 5, but returns null... when the cell is in use for any reason then we should not be returning that back to pool right? so chunkCreator should be knowing the chunkId. At step 3 the chunkId Map in chunkCreator should be having the id for that chunk. Why do you think chunkCreator will return null? > Connect CellChunkMap to be used for flattening in CompactingMemStore > > > Key: HBASE-18010 > URL: https://issues.apache.org/jira/browse/HBASE-18010 > Project: HBase > Issue Type: Sub-task >Reporter: Anastasia Braginsky > > The CellChunkMap helps to create a new type of ImmutableSegment, where the > index (CellSet's delegatee) is going to be CellChunkMap. No big cells or > upserted cells are going to be supported here. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-12794) Guidelines for filing JIRA issues
[ https://issues.apache.org/jira/browse/HBASE-12794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028247#comment-16028247 ] Chia-Ping Tsai commented on HBASE-12794: Good guidelines never comes too late. bq. want me to take the Google Doc and turn it into something in the book? +1 > Guidelines for filing JIRA issues > - > > Key: HBASE-12794 > URL: https://issues.apache.org/jira/browse/HBASE-12794 > Project: HBase > Issue Type: Task > Components: documentation >Reporter: stack >Assignee: stack > > Following on from Andrew's JIRA year-end cleaning spree, lets get some > guidelines on filing issues e.g. fill out all pertinent fields, add context > and provenance, add value (i.e. triage), don't file issues that are nought > but repeat of info available elsewhere (build box or mailing list), be > reluctant filing issues that don't have a resource behind them, don't file > issues on behalf of others, don't split fixes across multiple issues (because > there are poor folks coming behind us trying to backport our mess and > piecemeal makes their jobs harder), and so on. > The guidelines are not meant to put a chill on the opening of issues when > problems are found, especially not for new contributors. They are more meant > for quoting to veteran contributors who continue to file issues in violation > of what was thought a common understanding; rather than explain each time why > an issue has been marked invalid, it would be better if we can quote chapter > and verse from the refguide. > Dump any suggestion in here and I'll wind them up as a patch that I'll run by > dev mailing list to get consensus before committing. > Here is a running google doc if you'd like to add comment: > https://docs.google.com/document/d/1p3ArVLcnQnifk6ZsF635qWBhMmfTUJsISyK15DXnam0/edit?usp=sharing -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Comment Edited] (HBASE-18098) Region data ended up in a file of an another region
[ https://issues.apache.org/jira/browse/HBASE-18098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028222#comment-16028222 ] Pavel Salimov edited comment on HBASE-18098 at 5/29/17 10:51 AM: - Unfortunately no, I can't reproduce :( But I have found another region in the same state. The both problematic regions were the last ones BTW. was (Author: chcat): Unfortunately no :( But I have wound another region in the same state. The both problematic regions were the last ones BTW. > Region data ended up in a file of an another region > --- > > Key: HBASE-18098 > URL: https://issues.apache.org/jira/browse/HBASE-18098 > Project: HBase > Issue Type: Bug >Affects Versions: 1.2.0 >Reporter: Pavel Salimov > > Somehow it happened that a region, say AB, was split onto A and B, but some > portion of B's data were missing in B's H-files, still presenting in A's > files. I am not completely sure, was it a result of just a failure during > split, or of applying hbck repair after that. > Anyway, data of some rows belonging to B were missing accessing normally. I > was able to access them making a scan type query regarding these rows to > region A (despite the rows are after its end). Hbck repairs, splitting A onto > A0, A1 and (full) compaction did not change the situation: the data were > still missing in B migrated to A1. > Copying the file from A1 dir to B finally made the data accessible. > > I am reporting the issue in hope to get an advice on how to detect such an > inconsistency as well as hoping to clarify what leaded to such a state (and > fixed if it is a bug). -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18098) Region data ended up in a file of an another region
[ https://issues.apache.org/jira/browse/HBASE-18098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028222#comment-16028222 ] Pavel Salimov commented on HBASE-18098: --- Unfortunately no :( But I have wound another region in the same state. The both problematic regions were the last ones BTW. > Region data ended up in a file of an another region > --- > > Key: HBASE-18098 > URL: https://issues.apache.org/jira/browse/HBASE-18098 > Project: HBase > Issue Type: Bug >Affects Versions: 1.2.0 >Reporter: Pavel Salimov > > Somehow it happened that a region, say AB, was split onto A and B, but some > portion of B's data were missing in B's H-files, still presenting in A's > files. I am not completely sure, was it a result of just a failure during > split, or of applying hbck repair after that. > Anyway, data of some rows belonging to B were missing accessing normally. I > was able to access them making a scan type query regarding these rows to > region A (despite the rows are after its end). Hbck repairs, splitting A onto > A0, A1 and (full) compaction did not change the situation: the data were > still missing in B migrated to A1. > Copying the file from A1 dir to B finally made the data accessible. > > I am reporting the issue in hope to get an advice on how to detect such an > inconsistency as well as hoping to clarify what leaded to such a state (and > fixed if it is a bug). -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-18130) Refactor ReplicationSource
[ https://issues.apache.org/jira/browse/HBASE-18130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-18130: --- Attachment: HBASE-18130.master.002.patch > Refactor ReplicationSource > -- > > Key: HBASE-18130 > URL: https://issues.apache.org/jira/browse/HBASE-18130 > Project: HBase > Issue Type: Improvement >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang > Attachments: HBASE-18130.master.001.patch, > HBASE-18130.master.002.patch > > > One basic idea is move the code about recovered queue to a new subclass > RecoveredReplicationSource. Then ReplicationSource will don't need call > isQueueRecovered many times. This will make the code more clearly. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18010) Connect CellChunkMap to be used for flattening in CompactingMemStore
[ https://issues.apache.org/jira/browse/HBASE-18010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028200#comment-16028200 ] Anastasia Braginsky commented on HBASE-18010: - Thanks [~ramkrishna], I understand that you would like to clean mapping of the chunks no longer in use. But what do you say about the following scenario: 1. Chunk C was allocated and got ID 5 2. C was reclaimed and removed from mapping, but was returned to pool still holding ID 5 3. C was reused from pool and returned to the user once again 4. Cell residing on C is asked what is its chunk id and 5 is the answer 5. ChunkCreator is asked to translate chunk ID 5, but returns null... Am I wrong? > Connect CellChunkMap to be used for flattening in CompactingMemStore > > > Key: HBASE-18010 > URL: https://issues.apache.org/jira/browse/HBASE-18010 > Project: HBase > Issue Type: Sub-task >Reporter: Anastasia Braginsky > > The CellChunkMap helps to create a new type of ImmutableSegment, where the > index (CellSet's delegatee) is going to be CellChunkMap. No big cells or > upserted cells are going to be supported here. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18122) Scanner id should include ServerName of region server
[
https://issues.apache.org/jira/browse/HBASE-18122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028178#comment-16028178
]
Chia-Ping Tsai commented on HBASE-18122:
Sorry for missing the comments in the patch.
{code}
+ * We have 64 bits to use.
+ * The first 32 bits are MurmurHash32 of ServerName string "host:port,ts".
+ * The ServerName contains both host, port, and start timestamp so it can
prevent collision.
{code}
The formation of the returned string from ServerName#toString is "host,port,ts".
> Scanner id should include ServerName of region server
> -
>
> Key: HBASE-18122
> URL: https://issues.apache.org/jira/browse/HBASE-18122
> Project: HBase
> Issue Type: Bug
>Reporter: Phil Yang
>Assignee: Phil Yang
> Attachments: HBASE-18122.v01.patch, HBASE-18122.v02.patch,
> HBASE-18122.v03.patch
>
>
> Now the scanner id is a long number from 1 to max in a region server. Each
> new scanner will have a scanner id.
> If a client has a scanner whose id is x, when the RS restart and the scanner
> id is also incremented to x or a little larger, there will be a scanner id
> collision.
> So the scanner id should now be same during each time the RS restart. We can
> add the start timestamp as the highest several bits in scanner id uint64.
> And because HBASE-18121 is not easy to fix and there are many clients with
> old version. We can also encode server host:port into the scanner id.
> So we can use ServerName.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-18122) Scanner id should include ServerName of region server
[
https://issues.apache.org/jira/browse/HBASE-18122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028172#comment-16028172
]
Chia-Ping Tsai commented on HBASE-18122:
bq. We can also encode server host:port into the scanner id.
The ServerName#toString includes the start code, so maybe the
ServerName#toShortString is more suitable.
{code}
+public class ScannerIdGenerator {
+
+ private long serverNameHash;
+ private AtomicInteger scannerIdGen = new AtomicInteger(0);
{code}
Would you make these fields 'final'?
> Scanner id should include ServerName of region server
> -
>
> Key: HBASE-18122
> URL: https://issues.apache.org/jira/browse/HBASE-18122
> Project: HBase
> Issue Type: Bug
>Reporter: Phil Yang
>Assignee: Phil Yang
> Attachments: HBASE-18122.v01.patch, HBASE-18122.v02.patch,
> HBASE-18122.v03.patch
>
>
> Now the scanner id is a long number from 1 to max in a region server. Each
> new scanner will have a scanner id.
> If a client has a scanner whose id is x, when the RS restart and the scanner
> id is also incremented to x or a little larger, there will be a scanner id
> collision.
> So the scanner id should now be same during each time the RS restart. We can
> add the start timestamp as the highest several bits in scanner id uint64.
> And because HBASE-18121 is not easy to fix and there are many clients with
> old version. We can also encode server host:port into the scanner id.
> So we can use ServerName.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-14178) regionserver blocks because of waiting for offsetLock
[
https://issues.apache.org/jira/browse/HBASE-14178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028169#comment-16028169
]
Anoop Sam John commented on HBASE-14178:
First we try to get the block from cache with out getting any lock. In case of
miss, it may try again with cache with acquired offset lock if the reading
blocks as to be cached. (This is as per configs at RS level as well as config
set in the read req). So then ya, the reading handlers in RS might get blocked
for acquiring the locks. But this will avoid the need for every readers to go
to HDFS and fetch the block.
> regionserver blocks because of waiting for offsetLock
> -
>
> Key: HBASE-14178
> URL: https://issues.apache.org/jira/browse/HBASE-14178
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 0.98.6
>Reporter: Heng Chen
>Assignee: Heng Chen
> Fix For: 2.0.0, 0.98.14, 1.0.2, 1.2.0, 1.1.2
>
> Attachments: HBASE-14178-0.98.patch, HBASE-14178-0.98_v8.patch,
> HBASE-14178-branch_1_v8.patch, HBASE-14178.patch, HBASE-14178_v1.patch,
> HBASE-14178_v2.patch, HBASE-14178_v3.patch, HBASE-14178_v4.patch,
> HBASE-14178_v5.patch, HBASE-14178_v6.patch, HBASE-14178_v7.patch,
> HBASE-14178_v8.patch, jstack
>
>
> My regionserver blocks, and all client rpc timeout.
> I print the regionserver's jstack, it seems a lot of threads were blocked
> for waiting offsetLock, detail infomation belows:
> PS: my table's block cache is off
> {code}
> "B.DefaultRpcServer.handler=2,queue=2,port=60020" #82 daemon prio=5 os_prio=0
> tid=0x01827000 nid=0x2cdc in Object.wait() [0x7f3831b72000]
>java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at java.lang.Object.wait(Object.java:502)
> at org.apache.hadoop.hbase.util.IdLock.getLockEntry(IdLock.java:79)
> - locked <0x000773af7c18> (a
> org.apache.hadoop.hbase.util.IdLock$Entry)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:352)
> at
> org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.loadDataBlockWithScanInfo(HFileBlockIndex.java:253)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.seekTo(HFileReaderV2.java:524)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.reseekTo(HFileReaderV2.java:572)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseekAtOrAfter(StoreFileScanner.java:257)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseek(StoreFileScanner.java:173)
> at
> org.apache.hadoop.hbase.regionserver.NonLazyKeyValueScanner.doRealSeek(NonLazyKeyValueScanner.java:55)
> at
> org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:313)
> at
> org.apache.hadoop.hbase.regionserver.KeyValueHeap.requestSeek(KeyValueHeap.java:269)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:695)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.seekAsDirection(StoreScanner.java:683)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:533)
> at
> org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:140)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.populateResult(HRegion.java:3889)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:3969)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:3847)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:3820)
> - locked <0x0005e5c55ad0> (a
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:3807)
> at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:4779)
> at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:4753)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:2916)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:29583)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2027)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:108)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:114)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:94)
> at java.lang.Thread.run(Thread.java:745)
>Locked ownable
[jira] [Commented] (HBASE-18010) Connect CellChunkMap to be used for flattening in CompactingMemStore
[ https://issues.apache.org/jira/browse/HBASE-18010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028167#comment-16028167 ] ramkrishna.s.vasudevan commented on HBASE-18010: bq.Why do we remove the mapping of the chunks later added to the pool back again? Those chunks can be used once again and their chunk ID maybe required... This has been done to avoid having the map with chunkIds in them for a long time. Now if you getChunk() there every time we get a chunk we put the id into the map. So every time we putback the chunk we need to remove them. Anyway the chunk has already got the id associated with it. So it is not the new id every time > Connect CellChunkMap to be used for flattening in CompactingMemStore > > > Key: HBASE-18010 > URL: https://issues.apache.org/jira/browse/HBASE-18010 > Project: HBase > Issue Type: Sub-task >Reporter: Anastasia Braginsky > > The CellChunkMap helps to create a new type of ImmutableSegment, where the > index (CellSet's delegatee) is going to be CellChunkMap. No big cells or > upserted cells are going to be supported here. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18010) Connect CellChunkMap to be used for flattening in CompactingMemStore
[
https://issues.apache.org/jira/browse/HBASE-18010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028155#comment-16028155
]
Anastasia Braginsky commented on HBASE-18010:
-
I have one more question regarding the ChunkCreator, when the chunks are
reclaimed we run the following code:
{code}
/**
* Add the chunks to the pool, when the pool achieves the max size, it will
skip the remaining
* chunks
* @param chunks
*/
private void putbackChunks(Set chunks) {
int toAdd = Math.min(chunks.size(), this.maxCount -
reclaimedChunks.size());
Iterator iterator = chunks.iterator();
while (iterator.hasNext()) {
Integer chunkId = iterator.next();
// remove the chunks every time though they are from the pool or not
Chunk chunk = ChunkCreator.this.removeChunk(chunkId);
if (chunk != null) {
if (chunk.isFromPool() && toAdd > 0) {
reclaimedChunks.add(chunk);
}
toAdd--;
}
}
}
{code}
Why do we remove the mapping of the chunks later added to the pool back again?
Those chunks can be used once again and their chunk ID maybe required...
> Connect CellChunkMap to be used for flattening in CompactingMemStore
>
>
> Key: HBASE-18010
> URL: https://issues.apache.org/jira/browse/HBASE-18010
> Project: HBase
> Issue Type: Sub-task
>Reporter: Anastasia Braginsky
>
> The CellChunkMap helps to create a new type of ImmutableSegment, where the
> index (CellSet's delegatee) is going to be CellChunkMap. No big cells or
> upserted cells are going to be supported here.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17876) Release 1.2.6
[ https://issues.apache.org/jira/browse/HBASE-17876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028148#comment-16028148 ] Hudson commented on HBASE-17876: SUCCESS: Integrated in Jenkins build HBase-1.2-JDK8 #138 (See [https://builds.apache.org/job/HBase-1.2-JDK8/138/]) HBASE-17876 update version to 1.2.6 for RC0. (busbey: rev f6ecbf972e15da81718e44ba44a0c2a780a45c1e) * (edit) hbase-annotations/pom.xml * (edit) hbase-client/pom.xml * (edit) hbase-it/pom.xml * (edit) hbase-rest/pom.xml * (edit) hbase-shaded/hbase-shaded-client/pom.xml * (edit) hbase-external-blockcache/pom.xml * (edit) hbase-procedure/pom.xml * (edit) hbase-hadoop-compat/pom.xml * (edit) hbase-server/pom.xml * (edit) pom.xml * (edit) hbase-testing-util/pom.xml * (edit) hbase-thrift/pom.xml * (edit) hbase-protocol/pom.xml * (edit) hbase-assembly/pom.xml * (edit) hbase-common/pom.xml * (edit) hbase-hadoop2-compat/pom.xml * (edit) hbase-prefix-tree/pom.xml * (edit) hbase-resource-bundle/pom.xml * (edit) hbase-shaded/pom.xml * (edit) hbase-shell/pom.xml * (edit) hbase-shaded/hbase-shaded-server/pom.xml * (edit) hbase-checkstyle/pom.xml * (edit) hbase-examples/pom.xml HBASE-17876 update CHANGES.txt for 1.2.6 RC0. (busbey: rev 2f9b9e17d0522e36063bf52ecc58576243d20b3f) * (edit) CHANGES.txt > Release 1.2.6 > - > > Key: HBASE-17876 > URL: https://issues.apache.org/jira/browse/HBASE-17876 > Project: HBase > Issue Type: Task > Components: community >Reporter: Sean Busbey >Assignee: Sean Busbey > Fix For: 1.2.6 > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-17876) Release 1.2.6
[ https://issues.apache.org/jira/browse/HBASE-17876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028136#comment-16028136 ] Hudson commented on HBASE-17876: SUCCESS: Integrated in Jenkins build HBase-1.2-JDK7 #143 (See [https://builds.apache.org/job/HBase-1.2-JDK7/143/]) HBASE-17876 update version to 1.2.6 for RC0. (busbey: rev f6ecbf972e15da81718e44ba44a0c2a780a45c1e) * (edit) hbase-client/pom.xml * (edit) hbase-procedure/pom.xml * (edit) hbase-examples/pom.xml * (edit) hbase-common/pom.xml * (edit) hbase-assembly/pom.xml * (edit) hbase-protocol/pom.xml * (edit) hbase-hadoop-compat/pom.xml * (edit) hbase-prefix-tree/pom.xml * (edit) hbase-server/pom.xml * (edit) hbase-rest/pom.xml * (edit) hbase-shell/pom.xml * (edit) hbase-annotations/pom.xml * (edit) hbase-shaded/hbase-shaded-client/pom.xml * (edit) hbase-it/pom.xml * (edit) hbase-testing-util/pom.xml * (edit) pom.xml * (edit) hbase-external-blockcache/pom.xml * (edit) hbase-shaded/hbase-shaded-server/pom.xml * (edit) hbase-checkstyle/pom.xml * (edit) hbase-thrift/pom.xml * (edit) hbase-hadoop2-compat/pom.xml * (edit) hbase-resource-bundle/pom.xml * (edit) hbase-shaded/pom.xml HBASE-17876 update CHANGES.txt for 1.2.6 RC0. (busbey: rev 2f9b9e17d0522e36063bf52ecc58576243d20b3f) * (edit) CHANGES.txt > Release 1.2.6 > - > > Key: HBASE-17876 > URL: https://issues.apache.org/jira/browse/HBASE-17876 > Project: HBase > Issue Type: Task > Components: community >Reporter: Sean Busbey >Assignee: Sean Busbey > Fix For: 1.2.6 > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-15302) Reenable the other tests disabled by HBASE-14678
[
https://issues.apache.org/jira/browse/HBASE-15302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028119#comment-16028119
]
Hadoop QA commented on HBASE-15302:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 45s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
37s {color} | {color:green} branch-1.3 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 3s
{color} | {color:green} branch-1.3 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
31s {color} | {color:green} branch-1.3 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
22s {color} | {color:green} branch-1.3 passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m 47s
{color} | {color:red} hbase-server in branch-1.3 has 1 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 47s
{color} | {color:green} branch-1.3 passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
22s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 4s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 4s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
32s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
23s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
32m 31s {color} | {color:green} The patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
20s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
52s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 48s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 133m 49s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
24s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 188m 39s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.03.0-ce Server=17.03.0-ce Image:yetus/hbase:e1e11ad |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12834142/HBASE-15302-branch-1.3-append-v1.patch
|
| JIRA Issue | HBASE-15302 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux 754966b6a783 4.8.3-std-1 #1 SMP Fri Oct 21 11:15:43 UTC 2016
x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/hbase.sh |
| git revision | branch-1.3 / 2277c2b |
| Default Java | 1.8.0_131 |
| findbugs | v3.0.0 |
| findbugs |
https://builds.apache.org/job/PreCommit-HBASE-Build/6988/artifact/patchprocess/branch-findbugs-hbase-server-warnings.html
|
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/6988/artifact/patchprocess/patch-unit-hbase-server.txt
|
| unit test logs |
https://builds.apache.org/job/PreCommit-HBASE-Build/6988/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/6988/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/6988/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
[jira] [Commented] (HBASE-18128) compaction marker could be skipped
[
https://issues.apache.org/jira/browse/HBASE-18128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028083#comment-16028083
]
Anoop Sam John commented on HBASE-18128:
Interesting. Though functionally there is no issue, the purpose of the marker
not getting fulfilled. Are you proposing a fix here?
> compaction marker could be skipped
> ---
>
> Key: HBASE-18128
> URL: https://issues.apache.org/jira/browse/HBASE-18128
> Project: HBase
> Issue Type: Improvement
> Components: Compaction, regionserver
>Reporter: Jingyun Tian
>
> The sequence for a compaction are as follows:
> 1. Compaction writes new files under region/.tmp directory (compaction output)
> 2. Compaction atomically moves the temporary file under region directory
> 3. Compaction appends a WAL edit containing the compaction input and output
> files. Forces sync on WAL.
> 4. Compaction deletes the input files from the region directory.
> But if a flush happened between 3 and 4, then the regionserver crushed. The
> compaction marker will be skipped when splitting log because the sequence id
> of compaction marker is smaller than lastFlushedSequenceId.
> {code}
> if (lastFlushedSequenceId >= entry.getKey().getLogSeqNum()) {
> editsSkipped++;
> continue;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HBASE-17876) Release 1.2.6
[ https://issues.apache.org/jira/browse/HBASE-17876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028076#comment-16028076 ] Hudson commented on HBASE-17876: SUCCESS: Integrated in Jenkins build HBase-1.2-IT #876 (See [https://builds.apache.org/job/HBase-1.2-IT/876/]) HBASE-17876 update version to 1.2.6 for RC0. (busbey: rev f6ecbf972e15da81718e44ba44a0c2a780a45c1e) * (edit) hbase-prefix-tree/pom.xml * (edit) hbase-external-blockcache/pom.xml * (edit) hbase-it/pom.xml * (edit) hbase-server/pom.xml * (edit) hbase-common/pom.xml * (edit) hbase-shaded/hbase-shaded-server/pom.xml * (edit) hbase-thrift/pom.xml * (edit) hbase-resource-bundle/pom.xml * (edit) hbase-annotations/pom.xml * (edit) hbase-assembly/pom.xml * (edit) hbase-examples/pom.xml * (edit) hbase-rest/pom.xml * (edit) hbase-shaded/hbase-shaded-client/pom.xml * (edit) pom.xml * (edit) hbase-checkstyle/pom.xml * (edit) hbase-procedure/pom.xml * (edit) hbase-hadoop2-compat/pom.xml * (edit) hbase-client/pom.xml * (edit) hbase-testing-util/pom.xml * (edit) hbase-shaded/pom.xml * (edit) hbase-shell/pom.xml * (edit) hbase-hadoop-compat/pom.xml * (edit) hbase-protocol/pom.xml HBASE-17876 update CHANGES.txt for 1.2.6 RC0. (busbey: rev 2f9b9e17d0522e36063bf52ecc58576243d20b3f) * (edit) CHANGES.txt > Release 1.2.6 > - > > Key: HBASE-17876 > URL: https://issues.apache.org/jira/browse/HBASE-17876 > Project: HBase > Issue Type: Task > Components: community >Reporter: Sean Busbey >Assignee: Sean Busbey > Fix For: 1.2.6 > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
