[jira] [Commented] (HBASE-20940) HStore.cansplit should not allow split to happen if it has references
[
https://issues.apache.org/jira/browse/HBASE-20940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16598285#comment-16598285
]
Lars Hofhansl commented on HBASE-20940:
---
For now I'm just doing this in Phoenix in said class:
{code:java}
- if (getRegion().getStore(family).hasReferences()) {
+ if (StoreUtils.hasReferences((getRegion().getStore(family).getStorefiles(
{{code}
That restores the old perf behavior.
> HStore.cansplit should not allow split to happen if it has references
> -
>
> Key: HBASE-20940
> URL: https://issues.apache.org/jira/browse/HBASE-20940
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.2
>Reporter: Vishal Khandelwal
>Assignee: Vishal Khandelwal
>Priority: Major
> Fix For: 3.0.0, 1.5.0, 1.3.3, 2.2.0, 2.1.1, 1.4.7, 2.0.3
>
> Attachments: HBASE-20940-branch-1-addendum.patch,
> HBASE-20940.branch-1.3.v1.patch, HBASE-20940.branch-1.3.v2.patch,
> HBASE-20940.branch-1.v1.patch, HBASE-20940.branch-1.v2.patch,
> HBASE-20940.branch-1.v3.patch, HBASE-20940.branch-1.v5.patch,
> HBASE-20940.v1.patch, HBASE-20940.v2.patch, HBASE-20940.v3.patch,
> HBASE-20940.v4.patch, result_HBASE-20940.branch-1.v2.log
>
>
> When split happens and immediately another split happens, it may result into
> a split of a region who still has references to its parent. More details
> about scenario can be found here HBASE-20933
> HStore.hasReferences should check from fs.storefile rather than in memory
> objects.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-21104) client.TestRestoreSnapshotFromClientWithRegionReplicas failing on branch-1.3, branch-1.2
[ https://issues.apache.org/jira/browse/HBASE-21104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16598280#comment-16598280 ] Toshihiro Suzuki edited comment on HBASE-21104 at 8/31/18 5:55 AM: --- I don't think the test failures in the last QA are related to the patch. The tests were successful locally. Could you please review the patch? [~busbey] If you are okay with this patch, I will commit this path to branch-1.2 and branch-1.3. Thanks. was (Author: brfrn169): I don't think the test failures in the last QA is related to the patch. The tests were successful locally. Could you please review the patch? [~busbey] If you are okay with this patch, I will commit this path to branch-1.2 and branch-1.3. Thanks. > client.TestRestoreSnapshotFromClientWithRegionReplicas failing on branch-1.3, > branch-1.2 > > > Key: HBASE-21104 > URL: https://issues.apache.org/jira/browse/HBASE-21104 > Project: HBase > Issue Type: Bug > Components: read replicas, snapshots >Affects Versions: 1.3.3, 1.2.7 >Reporter: Sean Busbey >Assignee: Toshihiro Suzuki >Priority: Major > Attachments: HBASE-21104.branch-1.2.001.patch, > HBASE-21104.branch-1.2.001.patch > > > client.TestRestoreSnapshotFromClientWithRegionReplicas is in both branch-1.3 > and branch-1.2's flaky list as failing every run. > need to triage and fix. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21104) client.TestRestoreSnapshotFromClientWithRegionReplicas failing on branch-1.3, branch-1.2
[ https://issues.apache.org/jira/browse/HBASE-21104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16598280#comment-16598280 ] Toshihiro Suzuki commented on HBASE-21104: -- I don't think the test failures in the last QA is related to the patch. The tests were successful locally. Could you please review the patch? [~busbey] If you are okay with this patch, I will commit this path to branch-1.2 and branch-1.3. Thanks. > client.TestRestoreSnapshotFromClientWithRegionReplicas failing on branch-1.3, > branch-1.2 > > > Key: HBASE-21104 > URL: https://issues.apache.org/jira/browse/HBASE-21104 > Project: HBase > Issue Type: Bug > Components: read replicas, snapshots >Affects Versions: 1.3.3, 1.2.7 >Reporter: Sean Busbey >Assignee: Toshihiro Suzuki >Priority: Major > Attachments: HBASE-21104.branch-1.2.001.patch, > HBASE-21104.branch-1.2.001.patch > > > client.TestRestoreSnapshotFromClientWithRegionReplicas is in both branch-1.3 > and branch-1.2's flaky list as failing every run. > need to triage and fix. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21104) client.TestRestoreSnapshotFromClientWithRegionReplicas failing on branch-1.3, branch-1.2
[
https://issues.apache.org/jira/browse/HBASE-21104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16598272#comment-16598272
]
Hadoop QA commented on HBASE-21104:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
21s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} branch-1.2 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 12m
15s{color} | {color:green} branch-1.2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
33s{color} | {color:green} branch-1.2 passed with JDK v1.8.0_181 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
37s{color} | {color:green} branch-1.2 passed with JDK v1.7.0_191 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
14s{color} | {color:green} branch-1.2 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
11s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} branch-1.2 passed with JDK v1.8.0_181 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
31s{color} | {color:green} branch-1.2 passed with JDK v1.7.0_191 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
30s{color} | {color:green} the patch passed with JDK v1.8.0_181 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
33s{color} | {color:green} the patch passed with JDK v1.7.0_191 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
9s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
7m 57s{color} | {color:green} Patch does not cause any errors with Hadoop 2.4.1
2.5.2 2.6.5 2.7.4. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green} the patch passed with JDK v1.8.0_181 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
32s{color} | {color:green} the patch passed with JDK v1.7.0_191 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 98m 25s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
23s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}132m 0s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hbase.master.normalizer.TestSimpleRegionNormalizerOnCluster |
| | hadoop.hbase.trace.TestHTraceHooks |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:34a9b27 |
| JIRA Issue | HBASE-21104 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12937849/HBASE-21104.branch-1.2.001.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle
[jira] [Commented] (HBASE-21098) Improve Snapshot Performance with Temporary Snapshot Directory when rootDir on S3
[
https://issues.apache.org/jira/browse/HBASE-21098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16598271#comment-16598271
]
Hadoop QA commented on HBASE-21098:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
11s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 8 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
31s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
15s{color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 1m
20s{color} | {color:red} hbase-server in master failed. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
17s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
4s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
36s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
1s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 2m
30s{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 2m 30s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
8s{color} | {color:green} hbase-server: The patch generated 0 new + 283
unchanged - 1 fixed = 283 total (was 284) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
47s{color} | {color:green} The patch hbase-mapreduce passed checkstyle {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 6m
56s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
14m 25s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
47s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}117m 32s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 18m 13s{color}
| {color:red} hbase-mapreduce in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
50s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}202m 10s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.master.procedure.TestTruncateTableProcedure
|
| | hadoop.hbase.master.procedure.TestRestoreSnapshotProcedure |
| | hadoop.hbase.mapreduce.TestSyncTable |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21098 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12937694/HBASE-21098.master.008.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti
[jira] [Commented] (HBASE-21132) return wrong result in rest multiget
[
https://issues.apache.org/jira/browse/HBASE-21132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16598256#comment-16598256
]
Hadoop QA commented on HBASE-21132:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
12s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
53s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
35s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
27s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
15s{color} | {color:red} hbase-rest: The patch generated 1 new + 4 unchanged -
0 fixed = 5 total (was 4) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
22s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
7m 57s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
26s{color} | {color:green} hbase-rest in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
9s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 32m 31s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21132 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12937856/HBASE-21132.master.001.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux cb2482226b6d 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / 7c1fad4992 |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC3 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HBASE-Build/14263/artifact/patchprocess/diff-checkstyle-hbase-rest.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/14263/testReport/ |
| Max. process+thread count | 2129 (vs. ulimit of 1) |
| modules | C: hbase-rest U: hbase-rest |
| Console output |
[jira] [Updated] (HBASE-21132) return wrong result in rest multiget
[
https://issues.apache.org/jira/browse/HBASE-21132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guangxu Cheng updated HBASE-21132:
--
Attachment: HBASE-21132.master.001.patch
> return wrong result in rest multiget
>
>
> Key: HBASE-21132
> URL: https://issues.apache.org/jira/browse/HBASE-21132
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0, 2.0.0
>Reporter: Guangxu Cheng
>Assignee: Guangxu Cheng
>Priority: Major
> Attachments: HBASE-21132.master.001.patch,
> HBASE-21132.master.001.patch, HBASE-21132.master.001.patch
>
>
> There are two ways to specify columns in multi-gets feature。
> 1、Specify columns in PathParam as described in HBASE-15870. Examples:
> {code:sh}GET /t1/multiget/cf1:c1,cf2?row=r1{code}
> 2、Specify columns in QueryParam. Examples:
> {code:sh} GET /t1/multiget?row=r1/cf1:c1,cf2=r2/cf2 {code}
> However, when we specify columns in QueryParam, the result is wrong , the
> rowkey contains the columns.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21129) Clean up duplicate codes in #equals and #hashCode methods of Filter
[ https://issues.apache.org/jira/browse/HBASE-21129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-21129: -- Attachment: HBASE-21129.master.003.patch > Clean up duplicate codes in #equals and #hashCode methods of Filter > --- > > Key: HBASE-21129 > URL: https://issues.apache.org/jira/browse/HBASE-21129 > Project: HBase > Issue Type: Improvement > Components: Filters >Affects Versions: 3.0.0, 2.2.0 >Reporter: Reid Chan >Assignee: Reid Chan >Priority: Minor > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-21129.master.001.patch, > HBASE-21129.master.002.patch, HBASE-21129.master.003.patch > > > It is a follow-up of HBASE-19008, aiming to clean up duplicate codes in > #equals and #hashCode methods. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20940) HStore.cansplit should not allow split to happen if it has references
[
https://issues.apache.org/jira/browse/HBASE-20940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16598242#comment-16598242
]
Lars Hofhansl commented on HBASE-20940:
---
Hmm... I do not see a patch that actually removes {{private boolean
checkForReferenceFiles()}} in Phoenix' {{RegionScannerFactory.getWrappedScanner
-> anonymous RegionScanner}}.
Doing this in the HalfstoreFileReaders is the right thing, I just don't see the
actual expensive check being removed in any of the proposed patches.
For compactions this is totally the right thing to do.
> HStore.cansplit should not allow split to happen if it has references
> -
>
> Key: HBASE-20940
> URL: https://issues.apache.org/jira/browse/HBASE-20940
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.2
>Reporter: Vishal Khandelwal
>Assignee: Vishal Khandelwal
>Priority: Major
> Fix For: 3.0.0, 1.5.0, 1.3.3, 2.2.0, 2.1.1, 1.4.7, 2.0.3
>
> Attachments: HBASE-20940-branch-1-addendum.patch,
> HBASE-20940.branch-1.3.v1.patch, HBASE-20940.branch-1.3.v2.patch,
> HBASE-20940.branch-1.v1.patch, HBASE-20940.branch-1.v2.patch,
> HBASE-20940.branch-1.v3.patch, HBASE-20940.branch-1.v5.patch,
> HBASE-20940.v1.patch, HBASE-20940.v2.patch, HBASE-20940.v3.patch,
> HBASE-20940.v4.patch, result_HBASE-20940.branch-1.v2.log
>
>
> When split happens and immediately another split happens, it may result into
> a split of a region who still has references to its parent. More details
> about scenario can be found here HBASE-20933
> HStore.hasReferences should check from fs.storefile rather than in memory
> objects.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21127) TableRecordReader need to handle cursor result too
[
https://issues.apache.org/jira/browse/HBASE-21127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16598232#comment-16598232
]
Hadoop QA commented on HBASE-21127:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
12s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
20s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
38s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
15s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
17s{color} | {color:red} hbase-mapreduce: The patch generated 2 new + 12
unchanged - 0 fixed = 14 total (was 12) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
9s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
7m 27s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 18m 43s{color}
| {color:red} hbase-mapreduce in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 46m 40s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.mapred.TestMultiTableSnapshotInputFormat |
| | hadoop.hbase.mapreduce.TestSyncTable |
| | hadoop.hbase.mapreduce.TestMultiTableSnapshotInputFormat |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21127 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12937850/HBASE-21127.master.001.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux d8743b0c252e 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / 7c1fad4992 |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC3 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HBASE-Build/14260/artifact/patchprocess/diff-checkstyle-hbase-mapreduce.txt
|
| unit |
[jira] [Commented] (HBASE-20940) HStore.cansplit should not allow split to happen if it has references
[ https://issues.apache.org/jira/browse/HBASE-20940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16598221#comment-16598221 ] Vishal Khandelwal commented on HBASE-20940: --- [~lhofhansl] I think changes done in [^PHOENIX-4839.patch] should mitigate the perf risk. preStoreScannerOpen method which makes the call to hasReference has been remove. We may still have this call during preCompact but IMO that should be fine. Also, we were coming to this situation mostly because of HBASE-21047 and once it is fixed I hope chances of hitting this would reduce. > HStore.cansplit should not allow split to happen if it has references > - > > Key: HBASE-20940 > URL: https://issues.apache.org/jira/browse/HBASE-20940 > Project: HBase > Issue Type: Bug >Affects Versions: 1.3.2 >Reporter: Vishal Khandelwal >Assignee: Vishal Khandelwal >Priority: Major > Fix For: 3.0.0, 1.5.0, 1.3.3, 2.2.0, 2.1.1, 1.4.7, 2.0.3 > > Attachments: HBASE-20940-branch-1-addendum.patch, > HBASE-20940.branch-1.3.v1.patch, HBASE-20940.branch-1.3.v2.patch, > HBASE-20940.branch-1.v1.patch, HBASE-20940.branch-1.v2.patch, > HBASE-20940.branch-1.v3.patch, HBASE-20940.branch-1.v5.patch, > HBASE-20940.v1.patch, HBASE-20940.v2.patch, HBASE-20940.v3.patch, > HBASE-20940.v4.patch, result_HBASE-20940.branch-1.v2.log > > > When split happens and immediately another split happens, it may result into > a split of a region who still has references to its parent. More details > about scenario can be found here HBASE-20933 > HStore.hasReferences should check from fs.storefile rather than in memory > objects. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20940) HStore.cansplit should not allow split to happen if it has references
[ https://issues.apache.org/jira/browse/HBASE-20940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16598217#comment-16598217 ] Lars Hofhansl commented on HBASE-20940: --- No no, I can make a change in Phoenix to make it fast again. > HStore.cansplit should not allow split to happen if it has references > - > > Key: HBASE-20940 > URL: https://issues.apache.org/jira/browse/HBASE-20940 > Project: HBase > Issue Type: Bug >Affects Versions: 1.3.2 >Reporter: Vishal Khandelwal >Assignee: Vishal Khandelwal >Priority: Major > Fix For: 3.0.0, 1.5.0, 1.3.3, 2.2.0, 2.1.1, 1.4.7, 2.0.3 > > Attachments: HBASE-20940-branch-1-addendum.patch, > HBASE-20940.branch-1.3.v1.patch, HBASE-20940.branch-1.3.v2.patch, > HBASE-20940.branch-1.v1.patch, HBASE-20940.branch-1.v2.patch, > HBASE-20940.branch-1.v3.patch, HBASE-20940.branch-1.v5.patch, > HBASE-20940.v1.patch, HBASE-20940.v2.patch, HBASE-20940.v3.patch, > HBASE-20940.v4.patch, result_HBASE-20940.branch-1.v2.log > > > When split happens and immediately another split happens, it may result into > a split of a region who still has references to its parent. More details > about scenario can be found here HBASE-20933 > HStore.hasReferences should check from fs.storefile rather than in memory > objects. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20940) HStore.cansplit should not allow split to happen if it has references
[ https://issues.apache.org/jira/browse/HBASE-20940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16598198#comment-16598198 ] Andrew Purtell commented on HBASE-20940: Since we made this change for Phoenix if it is not useful let's revert it. I can pull 1.4.7 RC0 before this gets released. > HStore.cansplit should not allow split to happen if it has references > - > > Key: HBASE-20940 > URL: https://issues.apache.org/jira/browse/HBASE-20940 > Project: HBase > Issue Type: Bug >Affects Versions: 1.3.2 >Reporter: Vishal Khandelwal >Assignee: Vishal Khandelwal >Priority: Major > Fix For: 3.0.0, 1.5.0, 1.3.3, 2.2.0, 2.1.1, 1.4.7, 2.0.3 > > Attachments: HBASE-20940-branch-1-addendum.patch, > HBASE-20940.branch-1.3.v1.patch, HBASE-20940.branch-1.3.v2.patch, > HBASE-20940.branch-1.v1.patch, HBASE-20940.branch-1.v2.patch, > HBASE-20940.branch-1.v3.patch, HBASE-20940.branch-1.v5.patch, > HBASE-20940.v1.patch, HBASE-20940.v2.patch, HBASE-20940.v3.patch, > HBASE-20940.v4.patch, result_HBASE-20940.branch-1.v2.log > > > When split happens and immediately another split happens, it may result into > a split of a region who still has references to its parent. More details > about scenario can be found here HBASE-20933 > HStore.hasReferences should check from fs.storefile rather than in memory > objects. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21129) Clean up duplicate codes in #equals and #hashCode methods of Filter
[
https://issues.apache.org/jira/browse/HBASE-21129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16598196#comment-16598196
]
Hadoop QA commented on HBASE-21129:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
24s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
41s{color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 1m
18s{color} | {color:red} hbase-server in master failed. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
22s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
4s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 1m
34s{color} | {color:red} root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
24s{color} | {color:red} hbase-client in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
21s{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 24s{color}
| {color:red} hbase-client in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 21s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
39s{color} | {color:red} hbase-client: The patch generated 5 new + 421
unchanged - 0 fixed = 426 total (was 421) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
37s{color} | {color:red} hbase-server: The patch generated 1 new + 0 unchanged
- 36 fixed = 1 total (was 36) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} shadedjars {color} | {color:red} 3m
0s{color} | {color:red} patch has 14 errors when building our shaded downstream
artifacts. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 1m
27s{color} | {color:red} The patch causes 15 errors with Hadoop v2.7.4. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 2m
58s{color} | {color:red} The patch causes 15 errors with Hadoop v3.0.0. {color}
|
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
5s{color} | {color:red} hbase-client generated 28 new + 0 unchanged - 0 fixed =
28 total (was 0) {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
20s{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
14s{color} | {color:red} hbase-server in the patch failed. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
4s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 20s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 4m
38s{color} | {color:green} hbase-spark in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} |
[jira] [Updated] (HBASE-21127) TableRecordReader need to handle cursor result too
[ https://issues.apache.org/jira/browse/HBASE-21127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-21127: --- Affects Version/s: 1.5.0 1.4.7 > TableRecordReader need to handle cursor result too > -- > > Key: HBASE-21127 > URL: https://issues.apache.org/jira/browse/HBASE-21127 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0, 2.1.0, 1.5.0, 2.0.1, 2.2.0, 1.4.7 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Attachments: HBASE-21127.master.001.patch > > > *strong text*TableRecordReaderImpl need to handle cursor result too. If not, > nextKeyValue may return false and miss some data when get a cursor result. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HBASE-21127) TableRecordReader need to handle cursor result too
[ https://issues.apache.org/jira/browse/HBASE-21127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang reassigned HBASE-21127: -- Assignee: Guanghao Zhang > TableRecordReader need to handle cursor result too > -- > > Key: HBASE-21127 > URL: https://issues.apache.org/jira/browse/HBASE-21127 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0, 2.1.0, 2.0.1, 2.2.0 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Attachments: HBASE-21127.master.001.patch > > > *strong text*TableRecordReaderImpl need to handle cursor result too. If not, > nextKeyValue may return false and miss some data when get a cursor result. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21127) TableRecordReader need to handle cursor result too
[ https://issues.apache.org/jira/browse/HBASE-21127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-21127: --- Affects Version/s: 2.2.0 3.0.0 2.1.0 2.0.1 > TableRecordReader need to handle cursor result too > -- > > Key: HBASE-21127 > URL: https://issues.apache.org/jira/browse/HBASE-21127 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0, 2.1.0, 2.0.1, 2.2.0 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Attachments: HBASE-21127.master.001.patch > > > *strong text*TableRecordReaderImpl need to handle cursor result too. If not, > nextKeyValue may return false and miss some data when get a cursor result. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21127) TableRecordReader need to handle cursor result too
[ https://issues.apache.org/jira/browse/HBASE-21127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-21127: --- Status: Patch Available (was: Open) > TableRecordReader need to handle cursor result too > -- > > Key: HBASE-21127 > URL: https://issues.apache.org/jira/browse/HBASE-21127 > Project: HBase > Issue Type: Bug >Reporter: Guanghao Zhang >Priority: Major > Attachments: HBASE-21127.master.001.patch > > > *strong text*TableRecordReaderImpl need to handle cursor result too. If not, > nextKeyValue may return false and miss some data when get a cursor result. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21127) TableRecordReader need to handle cursor result too
[ https://issues.apache.org/jira/browse/HBASE-21127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-21127: --- Attachment: HBASE-21127.master.001.patch > TableRecordReader need to handle cursor result too > -- > > Key: HBASE-21127 > URL: https://issues.apache.org/jira/browse/HBASE-21127 > Project: HBase > Issue Type: Bug >Reporter: Guanghao Zhang >Priority: Major > Attachments: HBASE-21127.master.001.patch > > > *strong text*TableRecordReaderImpl need to handle cursor result too. If not, > nextKeyValue may return false and miss some data when get a cursor result. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21104) client.TestRestoreSnapshotFromClientWithRegionReplicas failing on branch-1.3, branch-1.2
[ https://issues.apache.org/jira/browse/HBASE-21104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16598180#comment-16598180 ] Toshihiro Suzuki commented on HBASE-21104: -- I just re-attatched the patch to run QA. > client.TestRestoreSnapshotFromClientWithRegionReplicas failing on branch-1.3, > branch-1.2 > > > Key: HBASE-21104 > URL: https://issues.apache.org/jira/browse/HBASE-21104 > Project: HBase > Issue Type: Bug > Components: read replicas, snapshots >Affects Versions: 1.3.3, 1.2.7 >Reporter: Sean Busbey >Assignee: Toshihiro Suzuki >Priority: Major > Attachments: HBASE-21104.branch-1.2.001.patch, > HBASE-21104.branch-1.2.001.patch > > > client.TestRestoreSnapshotFromClientWithRegionReplicas is in both branch-1.3 > and branch-1.2's flaky list as failing every run. > need to triage and fix. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21104) client.TestRestoreSnapshotFromClientWithRegionReplicas failing on branch-1.3, branch-1.2
[ https://issues.apache.org/jira/browse/HBASE-21104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki updated HBASE-21104: - Attachment: HBASE-21104.branch-1.2.001.patch > client.TestRestoreSnapshotFromClientWithRegionReplicas failing on branch-1.3, > branch-1.2 > > > Key: HBASE-21104 > URL: https://issues.apache.org/jira/browse/HBASE-21104 > Project: HBase > Issue Type: Bug > Components: read replicas, snapshots >Affects Versions: 1.3.3, 1.2.7 >Reporter: Sean Busbey >Assignee: Toshihiro Suzuki >Priority: Major > Attachments: HBASE-21104.branch-1.2.001.patch, > HBASE-21104.branch-1.2.001.patch > > > client.TestRestoreSnapshotFromClientWithRegionReplicas is in both branch-1.3 > and branch-1.2's flaky list as failing every run. > need to triage and fix. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21129) Clean up duplicate codes in #equals and #hashCode methods of Filter
[ https://issues.apache.org/jira/browse/HBASE-21129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16598155#comment-16598155 ] Reid Chan commented on HBASE-21129: --- Thanks Ted. > Clean up duplicate codes in #equals and #hashCode methods of Filter > --- > > Key: HBASE-21129 > URL: https://issues.apache.org/jira/browse/HBASE-21129 > Project: HBase > Issue Type: Improvement > Components: Filters >Affects Versions: 3.0.0, 2.2.0 >Reporter: Reid Chan >Assignee: Reid Chan >Priority: Minor > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-21129.master.001.patch, > HBASE-21129.master.002.patch > > > It is a follow-up of HBASE-19008, aiming to clean up duplicate codes in > #equals and #hashCode methods. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21127) TableRecordReader need to handle cursor result too
[ https://issues.apache.org/jira/browse/HBASE-21127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guangxu Cheng updated HBASE-21127: -- Description: *strong text*TableRecordReaderImpl need to handle cursor result too. If not, nextKeyValue may return false and miss some data when get a cursor result. (was: TableRecordReaderImpl need to handle cursor result too. If not, nextKeyValue may return false and miss some data when get a cursor result.) > TableRecordReader need to handle cursor result too > -- > > Key: HBASE-21127 > URL: https://issues.apache.org/jira/browse/HBASE-21127 > Project: HBase > Issue Type: Bug >Reporter: Guanghao Zhang >Priority: Major > > *strong text*TableRecordReaderImpl need to handle cursor result too. If not, > nextKeyValue may return false and miss some data when get a cursor result. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21129) Clean up duplicate codes in #equals and #hashCode methods of Filter
[ https://issues.apache.org/jira/browse/HBASE-21129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16598152#comment-16598152 ] Ted Yu commented on HBASE-21129: QA is back. Triggered a QA run. > Clean up duplicate codes in #equals and #hashCode methods of Filter > --- > > Key: HBASE-21129 > URL: https://issues.apache.org/jira/browse/HBASE-21129 > Project: HBase > Issue Type: Improvement > Components: Filters >Affects Versions: 3.0.0, 2.2.0 >Reporter: Reid Chan >Assignee: Reid Chan >Priority: Minor > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-21129.master.001.patch, > HBASE-21129.master.002.patch > > > It is a follow-up of HBASE-19008, aiming to clean up duplicate codes in > #equals and #hashCode methods. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21133) '/version' in rest should return hbase rest client version
[
https://issues.apache.org/jira/browse/HBASE-21133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16598135#comment-16598135
]
Guangxu Cheng commented on HBASE-21133:
---
Thanks [~busbey] for reviewing.
There is already an interface '/version/cluster' to return the version of the
HBase cluster.It is okay to add a new field to record the HBase client version.
However, the REST client is part of HBase, is it necessary to keep a version
number inside the REST? If it is needed to retain, then when will the version
number be changed? As you said, there is no standard. As far as I know, after
HBASE-10952, the version number of REST has not changed.
> '/version' in rest should return hbase rest client version
> --
>
> Key: HBASE-21133
> URL: https://issues.apache.org/jira/browse/HBASE-21133
> Project: HBase
> Issue Type: Bug
> Components: REST
>Affects Versions: 3.0.0, 2.0.1, 2.2.0
>Reporter: Guangxu Cheng
>Assignee: Guangxu Cheng
>Priority: Major
> Attachments: HBASE-21133.master.001.patch
>
>
> restVersion is a constant, which is meaningless.
> {code:java}
> public VersionModel(ServletContext context) {
> restVersion = RESTServlet.VERSION_STRING;
> {code}
> {code:java}
> String VERSION_STRING = "0.0.3";
> {code}
> the result of '/version'
> {code:json}
> {"Server":"jetty/6.1.26","Jersey":"1.9","REST":"0.0.3","JVM":"Oracle
> Corporation 1.8.0_102-25.102-b14","OS":"Linux
> 2.6.32.57-tlinux-1.1.2-container amd64"}
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21070) SnapshotFileCache won't update for snapshots stored in S3
[
https://issues.apache.org/jira/browse/HBASE-21070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16598122#comment-16598122
]
Mingliang Liu commented on HBASE-21070:
---
Thanks [~zyork]. If all access methods are synchronized, the {{cache}} and
{{snapshots}} will be thread safe. No stale reference will be seen. Currently
only one method {{SnapshotFileCache::SnapshotFileCache}} is not synchronized
which I think should be.
I understand the difficulty of testing this. I'll post any idea I get.
> SnapshotFileCache won't update for snapshots stored in S3
> -
>
> Key: HBASE-21070
> URL: https://issues.apache.org/jira/browse/HBASE-21070
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 3.0.0, 2.1.1, 1.4.7
>Reporter: Zach York
>Assignee: Zach York
>Priority: Critical
> Labels: FSRedo
> Attachments: HBASE-21070.master.001.patch,
> HBASE-21070.master.002.patch
>
>
> The SnapshotFileCache depends on last modified time to determine whether to
> update the Snapshot HFile cache. However, in S3, real 'folders' don't exist.
> S3 filesystems create a dummy file in place of a folder, but the dummy file
> last modified time is not updated when files are changed 'under' it. This
> means that the SnapshotFileCache doesn't pick up new snapshot HFiles and
> these files aren't removed from the HFileCleaner and can be eligible for
> deletion.
>
> My patch removes the lastmodified assumption.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21132) return wrong result in rest multiget
[
https://issues.apache.org/jira/browse/HBASE-21132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16598121#comment-16598121
]
Guangxu Cheng commented on HBASE-21132:
---
Not sure why Hadoop QA was not triggered. Re-submit the patch in order to
trigger Hadoop QA.Thanks
> return wrong result in rest multiget
>
>
> Key: HBASE-21132
> URL: https://issues.apache.org/jira/browse/HBASE-21132
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0, 2.0.0
>Reporter: Guangxu Cheng
>Assignee: Guangxu Cheng
>Priority: Major
> Attachments: HBASE-21132.master.001.patch,
> HBASE-21132.master.001.patch
>
>
> There are two ways to specify columns in multi-gets feature。
> 1、Specify columns in PathParam as described in HBASE-15870. Examples:
> {code:sh}GET /t1/multiget/cf1:c1,cf2?row=r1{code}
> 2、Specify columns in QueryParam. Examples:
> {code:sh} GET /t1/multiget?row=r1/cf1:c1,cf2=r2/cf2 {code}
> However, when we specify columns in QueryParam, the result is wrong , the
> rowkey contains the columns.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21132) return wrong result in rest multiget
[
https://issues.apache.org/jira/browse/HBASE-21132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guangxu Cheng updated HBASE-21132:
--
Attachment: HBASE-21132.master.001.patch
> return wrong result in rest multiget
>
>
> Key: HBASE-21132
> URL: https://issues.apache.org/jira/browse/HBASE-21132
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0, 2.0.0
>Reporter: Guangxu Cheng
>Assignee: Guangxu Cheng
>Priority: Major
> Attachments: HBASE-21132.master.001.patch,
> HBASE-21132.master.001.patch
>
>
> There are two ways to specify columns in multi-gets feature。
> 1、Specify columns in PathParam as described in HBASE-15870. Examples:
> {code:sh}GET /t1/multiget/cf1:c1,cf2?row=r1{code}
> 2、Specify columns in QueryParam. Examples:
> {code:sh} GET /t1/multiget?row=r1/cf1:c1,cf2=r2/cf2 {code}
> However, when we specify columns in QueryParam, the result is wrong , the
> rowkey contains the columns.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21105) TestHBaseFsck failing in branch-1, branch-1.4, branch-1.3 with NPE
[
https://issues.apache.org/jira/browse/HBASE-21105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16598118#comment-16598118
]
Sean Busbey commented on HBASE-21105:
-
can this be closed now as superseded by the addendum on HBASE-20940?
> TestHBaseFsck failing in branch-1, branch-1.4, branch-1.3 with NPE
> --
>
> Key: HBASE-21105
> URL: https://issues.apache.org/jira/browse/HBASE-21105
> Project: HBase
> Issue Type: Bug
> Components: hbck, test
>Affects Versions: 1.5.0, 1.3.3, 1.4.7
>Reporter: Sean Busbey
>Assignee: Vishal Khandelwal
>Priority: Major
> Attachments: HBASE-21105.branch-1.v1.patch
>
>
> TestHBaseFsck in the mentioned branches has two tests that rely on
> TestEndToEndSplitTransaction for blocking in the same way TestTableResource
> used to before HBASE-21076.
> Both tests appear to specifically be testing that something happens after a
> split, so we'll need a solution that removes the cross-test dependency but
> still allows for "wait until this split has finished"
> example failure from branch-1
> {code}
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.util.TestHBaseFsck.testSplitDaughtersNotInMeta(TestHBaseFsck.java:1985)
> {code}
> {code}
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.util.TestHBaseFsck.testValidLingeringSplitParent(TestHBaseFsck.java:1934)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HBASE-20430) Improve store file management for non-HDFS filesystems
[ https://issues.apache.org/jira/browse/HBASE-20430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu reassigned HBASE-20430: - Assignee: Mingliang Liu > Improve store file management for non-HDFS filesystems > -- > > Key: HBASE-20430 > URL: https://issues.apache.org/jira/browse/HBASE-20430 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Assignee: Mingliang Liu >Priority: Major > > HBase keeps a file open for every active store file so no additional round > trips to the NameNode are needed after the initial open. HDFS internally > multiplexes open files, but the Hadoop S3 filesystem implementations do not, > or, at least, not as well. As the bulk of data under management increases we > observe the required number of concurrently open connections will rise, and > expect it will eventually exhaust a limit somewhere (the client, the OS file > descriptor table or open file limits, or the S3 service). > Initially we can simply introduce an option to close every store file after > the reader has finished, and determine the performance impact. Use cases > backed by non-HDFS filesystems will already have to cope with a different > read performance profile. Based on experiments with the S3 backed Hadoop > filesystems, notably S3A, even with aggressively tuned options simple reads > can be very slow when there are blockcache misses, 15-20 seconds observed for > Get of a single small row, for example. We expect extensive use of the > BucketCache to mitigate in this application already. Could be backed by > offheap storage, but more likely a large number of cache files managed by the > file engine on local SSD storage. If misses are already going to be super > expensive, then the motivation to do more than simply open store files on > demand is largely absent. > Still, we could employ a predictive cache. Where frequent access to a given > store file (or, at least, its store) is predicted, keep a reference to the > store file open. Can keep statistics about read frequency, write it out to > HFiles during compaction, and note these stats when opening the region, > perhaps by reading all meta blocks of region HFiles when opening. Otherwise, > close the file after reading and open again on demand. Need to be careful not > to use ARC or equivalent as cache replacement strategy as it is encumbered. > The size of the cache can be determined at startup after detecting the > underlying filesystem. Eg. setCacheSize(VERY_LARGE_CONSTANT) if (fs > instanceof DistributedFileSystem), so we don't lose much when on HDFS still. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-21098) Improve Snapshot Performance with Temporary Snapshot Directory when rootDir on S3
[
https://issues.apache.org/jira/browse/HBASE-21098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16598105#comment-16598105
]
Mingliang Liu edited comment on HBASE-21098 at 8/31/18 1:31 AM:
Thanks [~mtylr] for updating the patch. Now it looks better to me overall. We
need a clean QA. I expect some checkstyle warnings (e.g. {{getHBaseAdmin()}} ->
{{getAdmin()}}).
Major comments:
# {{TestSnapshotDFSTemporaryDirectory}} is flaky. It attempts to set the
{{SNAPSHOT_WORKING_DIR}} as HDFS directory, however it actually is not testing
with HDFS directory on the mini DFS cluster, as the test:
#- does not set a value that is within the default working directory.
Precondition check in {{TakeSnapshotHandler}} should fail if it's not.
#- sets the temporary value *before* the start of mini DFS cluster, so its
value will be default local directory, instead of DFS temporary directory.
# {{hbase-default.xml}} is the list of configurable properties, serving both as
default values and documentation. This patch adds a new config that, if set,
will change the default working directory for snapshot. So I think we should
put the config (key/description) in that file. As to the default value, it
should be empty.
Minor comments:
# In {{TestExportSnapshotWithTemporaryDirectory.java}},
{{TEST_UTIL.startMiniCluster(1, 3)}} can be {{TEST_UTIL.startMiniCluster(3)}}
per HBASE-21071
# Can {{TestExportSnapshotWithTemporaryDirectory::setUpBaseConf()}} call
{{TestExportSnapshot::setUpBaseConf()}} first? That will make the subclass
setup method two lines of code.
# In {{TestExportSnapshotWithTemporaryDirectory}} the temporary directory
{{TEMP_DIR}} is under current workspace dir and not cleaned after test. Maybe
{{Files.createTempDirectory}} is a better solution.
# {{TakeSnapshotHandler::process()}} has another FileSystem exists
pattern, we can delete the exists as well.
{code:java}
if (workingDirFs.exists(workingDir) && !this.workingDirFs.delete(workingDir,
true)) {
{code}
# nit: in method usually we don't need {{this.}} to refer to a private field.
It makes more sense to constructors or getters/setters.
Discussion:
# {{TakeSnapshotHandler::completeSnapshot()}} tries to {{rename}} if it's the
same file system, otherwise if different file system or rename fails, it falls
back to {{copy}}. Is this the copy after rename fails necessary?
#- The logic might be clearer if we add some comment, and/or split the if
clause. I understand it's currently concise so it's up to you.
#- Meanwhile, it may fallback to {{copy}} even if {{rename}} is possible. The
reason is that it compares the {{FileSystem}} object using {{equals}}, while
{{FileSystem}} does not override the behavior. So by default, it compares the
same instance reference. This is indeterministic as {{FileSystem}} can have
configurable caching (enabled/disabled) behavior in which case
{{FileSystem.get(new URI("myfs://a"), conf)}} _may_ or _may not_ be the same
instance as another {{FileSystem.get(new URI("myfs://a"), conf)}}. I assume
this is not a big problem though as the default fs cache is enabled.
# Just curious, do you have any test system running with this patch where
snapshot is on HDFS and rootdir S3?
was (Author: liuml07):
Thanks [~mtylr] for updating the patch. Now it looks better to me overall. We
need a clean QA. I expect some checkstyle warnings (e.g. {{getHBaseAdmin()}} ->
{{getAdmin()}}).
Major comments:
# {{TestSnapshotDFSTemporaryDirectory}} is flaky. It attempts to set the
{{SNAPSHOT_WORKING_DIR}} as HDFS directory, however it actually is not testing
with HDFS directory on the mini DFS cluster:
#- does not set a value that is within the default working directory
#- the temporary value it sets is *before* the start of mini DFS cluster, so
its value will be default local directory.
# {{hbase-default.xml}} is the list of configurable properties, serving both as
default values and documentation. This patch adds a new config that, if set,
will change the default working directory for snapshot. So I think we should
put the config (key/description) in that file. As to the default value, it
should be empty.
Minor comments:
# In {{TestExportSnapshotWithTemporaryDirectory.java}},
{{TEST_UTIL.startMiniCluster(1, 3)}} can be {{TEST_UTIL.startMiniCluster(3)}}
per HBASE-21071
# Can {{TestExportSnapshotWithTemporaryDirectory::setUpBaseConf()}} call
{{TestExportSnapshot::setUpBaseConf()}} first? That will make the subclass
setup method two lines of code.
# In {{TestExportSnapshotWithTemporaryDirectory}} the temporary directory
{{TEMP_DIR}} is under current workspace dir and not cleaned after test. Maybe
{{Files.createTempDirectory}} is a better solution.
# {{TakeSnapshotHandler::process()}} has another FileSystem exists
pattern, we can delete the exists as well.
{code:java}
if (workingDirFs.exists(workingDir) &&
[jira] [Commented] (HBASE-6721) RegionServer Group based Assignment
[ https://issues.apache.org/jira/browse/HBASE-6721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16598107#comment-16598107 ] stack commented on HBASE-6721: -- bq. Due to a build issue on my local instance, I have not yet completed it. I can run any patch you might have [~daisuke.kobayashi] ... Just say sir. > RegionServer Group based Assignment > --- > > Key: HBASE-6721 > URL: https://issues.apache.org/jira/browse/HBASE-6721 > Project: HBase > Issue Type: New Feature > Components: regionserver >Reporter: Francis Liu >Assignee: Francis Liu >Priority: Major > Labels: hbase-6721 > Fix For: 2.0.0 > > Attachments: 6721-master-webUI.patch, HBASE-6721 > GroupBasedLoadBalancer Sequence Diagram.xml, HBASE-6721-DesigDoc.pdf, > HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, > HBASE-6721_0.98_2.patch, HBASE-6721_10.patch, HBASE-6721_11.patch, > HBASE-6721_12.patch, HBASE-6721_13.patch, HBASE-6721_14.patch, > HBASE-6721_15.patch, HBASE-6721_8.patch, HBASE-6721_9.patch, > HBASE-6721_9.patch, HBASE-6721_94.patch, HBASE-6721_94.patch, > HBASE-6721_94_2.patch, HBASE-6721_94_3.patch, HBASE-6721_94_3.patch, > HBASE-6721_94_4.patch, HBASE-6721_94_5.patch, HBASE-6721_94_6.patch, > HBASE-6721_94_7.patch, HBASE-6721_98_1.patch, HBASE-6721_98_2.patch, > HBASE-6721_hbase-6721_addendum.patch, HBASE-6721_trunk.patch, > HBASE-6721_trunk.patch, HBASE-6721_trunk.patch, HBASE-6721_trunk1.patch, > HBASE-6721_trunk2.patch, balanceCluster Sequence Diagram.svg, > hbase-6721-v15-branch-1.1.patch, hbase-6721-v16.patch, hbase-6721-v17.patch, > hbase-6721-v18.patch, hbase-6721-v19.patch, hbase-6721-v20.patch, > hbase-6721-v21.patch, hbase-6721-v22.patch, hbase-6721-v23.patch, > hbase-6721-v25.patch, hbase-6721-v26.patch, hbase-6721-v26_draft1.patch, > hbase-6721-v27.patch, hbase-6721-v27.patch, hbase-6721-v27.patch.txt, > hbase-6721-v28.patch, hbase-6721-v28.patch, hbase-6721-v29.patch, > immediateAssignments Sequence Diagram.svg, randomAssignment Sequence > Diagram.svg, retainAssignment Sequence Diagram.svg, roundRobinAssignment > Sequence Diagram.svg > > > In multi-tenant deployments of HBase, it is likely that a RegionServer will > be serving out regions from a number of different tables owned by various > client applications. Being able to group a subset of running RegionServers > and assign specific tables to it, provides a client application a level of > isolation and resource allocation. > The proposal essentially is to have an AssignmentManager which is aware of > RegionServer groups and assigns tables to region servers based on groupings. > Load balancing will occur on a per group basis as well. > This is essentially a simplification of the approach taken in HBASE-4120. See > attached document. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21098) Improve Snapshot Performance with Temporary Snapshot Directory when rootDir on S3
[ https://issues.apache.org/jira/browse/HBASE-21098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HBASE-21098: -- Affects Version/s: 1.4.8 > Improve Snapshot Performance with Temporary Snapshot Directory when rootDir > on S3 > - > > Key: HBASE-21098 > URL: https://issues.apache.org/jira/browse/HBASE-21098 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0, 1.4.8, 2.1.1 >Reporter: Tyler Mi >Priority: Major > Attachments: HBASE-21098.master.001.patch, > HBASE-21098.master.002.patch, HBASE-21098.master.003.patch, > HBASE-21098.master.004.patch, HBASE-21098.master.005.patch, > HBASE-21098.master.006.patch, HBASE-21098.master.007.patch, > HBASE-21098.master.008.patch > > > When using Apache HBase, the snapshot feature can be used to make a point in > time recovery. To do this, HBase creates a manifest of all the files in all > of the Regions so that those files can be referenced again when a user > restores a snapshot. With HBase's S3 storage mode, developers can store their > data off-cluster on Amazon S3. However, utilizing S3 as a file system is > inefficient in some operations, namely renames. Most Hadoop ecosystem > applications use an atomic rename as a method of committing data. However, > with S3, a rename is a separate copy and then a delete of every file which is > no longer atomic and, in fact, quite costly. In addition, puts and deletes on > S3 have latency issues that traditional filesystems do not encounter when > manipulating the region snapshots to consolidate into a single manifest. When > HBase on S3 users have a significant amount of regions, puts, deletes, and > renames (the final commit stage of the snapshot) become the bottleneck > causing snapshots to take many minutes or even hours to complete. > The purpose of this patch is to increase the overall performance of snapshots > while utilizing HBase on S3 through the use of a temporary directory for the > snapshots that exists on a traditional filesystem like HDFS to circumvent the > bottlenecks. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21098) Improve Snapshot Performance with Temporary Snapshot Directory when rootDir on S3
[
https://issues.apache.org/jira/browse/HBASE-21098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16598105#comment-16598105
]
Mingliang Liu commented on HBASE-21098:
---
Thanks [~mtylr] for updating the patch. Now it looks better to me overall. We
need a clean QA. I expect some checkstyle warnings (e.g. {{getHBaseAdmin()}} ->
{{getAdmin()}}).
Major comments:
# {{TestSnapshotDFSTemporaryDirectory}} is flaky. It attempts to set the
{{SNAPSHOT_WORKING_DIR}} as HDFS directory, however it actually is not testing
with HDFS directory on the mini DFS cluster:
#- does not set a value that is within the default working directory
#- the temporary value it sets is *before* the start of mini DFS cluster, so
its value will be default local directory.
# {{hbase-default.xml}} is the list of configurable properties, serving both as
default values and documentation. This patch adds a new config that, if set,
will change the default working directory for snapshot. So I think we should
put the config (key/description) in that file. As to the default value, it
should be empty.
Minor comments:
# In {{TestExportSnapshotWithTemporaryDirectory.java}},
{{TEST_UTIL.startMiniCluster(1, 3)}} can be {{TEST_UTIL.startMiniCluster(3)}}
per HBASE-21071
# Can {{TestExportSnapshotWithTemporaryDirectory::setUpBaseConf()}} call
{{TestExportSnapshot::setUpBaseConf()}} first? That will make the subclass
setup method two lines of code.
# In {{TestExportSnapshotWithTemporaryDirectory}} the temporary directory
{{TEMP_DIR}} is under current workspace dir and not cleaned after test. Maybe
{{Files.createTempDirectory}} is a better solution.
# {{TakeSnapshotHandler::process()}} has another FileSystem exists
pattern, we can delete the exists as well.
{code:java}
if (workingDirFs.exists(workingDir) && !this.workingDirFs.delete(workingDir,
true)) {
{code}
# nit: in method usually we don't need {{this.}} to refer to a private field.
It makes more sense to constructors or getters/setters.
Discussion:
# {{TakeSnapshotHandler::completeSnapshot()}} tries to {{rename}} if it's the
same file system, otherwise if different file system or rename fails, it falls
back to {{copy}}. The logic might be clearer if we add some comment, and/or
split the if clause. I understand it's currently concise so it's up to you.
Meanwhile, it may fallback to {{copy}} even if {{rename}} is possible. The
reason is that it compares the {{FileSystem}} object using {{equals}}, while
{{FileSystem}} does not override the behavior. So by default, it compares the
same instance reference. This is indeterministic as {{FileSystem}} can have
configurable caching (enabled/disabled) behavior in which case
{{FileSystem.get(new URI("myfs://a"), conf)}} _may_ or _may not_ be the same
instance as another {{FileSystem.get(new URI("myfs://a"), conf)}}. I assume
this is not a big problem though as the default fs cache is enabled.
# Just curious, do you have any test system running with this patch where
snapshot is on HDFS and rootdir S3?
> Improve Snapshot Performance with Temporary Snapshot Directory when rootDir
> on S3
> -
>
> Key: HBASE-21098
> URL: https://issues.apache.org/jira/browse/HBASE-21098
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0, 2.1.1
>Reporter: Tyler Mi
>Priority: Major
> Attachments: HBASE-21098.master.001.patch,
> HBASE-21098.master.002.patch, HBASE-21098.master.003.patch,
> HBASE-21098.master.004.patch, HBASE-21098.master.005.patch,
> HBASE-21098.master.006.patch, HBASE-21098.master.007.patch,
> HBASE-21098.master.008.patch
>
>
> When using Apache HBase, the snapshot feature can be used to make a point in
> time recovery. To do this, HBase creates a manifest of all the files in all
> of the Regions so that those files can be referenced again when a user
> restores a snapshot. With HBase's S3 storage mode, developers can store their
> data off-cluster on Amazon S3. However, utilizing S3 as a file system is
> inefficient in some operations, namely renames. Most Hadoop ecosystem
> applications use an atomic rename as a method of committing data. However,
> with S3, a rename is a separate copy and then a delete of every file which is
> no longer atomic and, in fact, quite costly. In addition, puts and deletes on
> S3 have latency issues that traditional filesystems do not encounter when
> manipulating the region snapshots to consolidate into a single manifest. When
> HBase on S3 users have a significant amount of regions, puts, deletes, and
> renames (the final commit stage of the snapshot) become the bottleneck
> causing snapshots to take many minutes or even hours to complete.
> The purpose of this patch is to increase the overall performance of snapshots
> while
[jira] [Commented] (HBASE-21061) fix synchronization of org.apache.hadoop.hbase.ipc.RpcServer$Connection.useWrap
[
https://issues.apache.org/jira/browse/HBASE-21061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16598102#comment-16598102
]
Sean Busbey commented on HBASE-21061:
-
the shadedjars failure is because I used a docker image from master, which has
jdk8, which results in an enforcer problem on the hbase-annotations module when
{{-Prelease}} is present (which it is for checking the shaded jars).
I'll rerun that one with jdk7.
> fix synchronization of
> org.apache.hadoop.hbase.ipc.RpcServer$Connection.useWrap
> ---
>
> Key: HBASE-21061
> URL: https://issues.apache.org/jira/browse/HBASE-21061
> Project: HBase
> Issue Type: Sub-task
> Components: rpc
>Affects Versions: 1.5.0, 1.3.3, 1.2.7, 1.4.7
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Critical
> Attachments: HBASE-21061-branch-1.v0.patch
>
>
> fix findbugs highlighted issue
> {code}
> Inconsistent synchronization of
> org.apache.hadoop.hbase.ipc.RpcServer$Connection.useWrap; locked 75% of time
> Unsynchronized access at RpcServer.java:75% of time Unsynchronized access at
> RpcServer.java
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21061) fix synchronization of org.apache.hadoop.hbase.ipc.RpcServer$Connection.useWrap
[
https://issues.apache.org/jira/browse/HBASE-21061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16598099#comment-16598099
]
Sean Busbey commented on HBASE-21061:
-
javac warning:
{code}
$ cat /tmp/yetus-27066.15542/diff-compile-javac-hbase-server.txt
[WARNING]
/testptch/hbase/hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcServer.java:[1713,28]
[InputStreamSlowMultibyteRead] Please also override int read(byte[], int,
int), otherwise multi-byte reads from this input stream are likely to be slow.
{code}
It's unrelated, just not recognized as a dup because of how many lines the
method moved due to the patch. shows up above as one fixed and one new:
{code}
$ diff /tmp/yetus-27066.15542/branch-compile-javac-hbase-server.txt
/tmp/yetus-27066.15542/patch-compile-javac-hbase-server.txt
2c2
< [WARNING]
/testptch/hbase/hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcServer.java:[1702,26]
[InputStreamSlowMultibyteRead] Please also override int read(byte[], int,
int), otherwise multi-byte reads from this input stream are likely to be slow.
---
> [WARNING]
> /testptch/hbase/hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcServer.java:[1713,28]
> [InputStreamSlowMultibyteRead] Please also override int read(byte[], int,
> int), otherwise multi-byte reads from this input stream are likely to be slow.
{code}
> fix synchronization of
> org.apache.hadoop.hbase.ipc.RpcServer$Connection.useWrap
> ---
>
> Key: HBASE-21061
> URL: https://issues.apache.org/jira/browse/HBASE-21061
> Project: HBase
> Issue Type: Sub-task
> Components: rpc
>Affects Versions: 1.5.0, 1.3.3, 1.2.7, 1.4.7
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Critical
> Attachments: HBASE-21061-branch-1.v0.patch
>
>
> fix findbugs highlighted issue
> {code}
> Inconsistent synchronization of
> org.apache.hadoop.hbase.ipc.RpcServer$Connection.useWrap; locked 75% of time
> Unsynchronized access at RpcServer.java:75% of time Unsynchronized access at
> RpcServer.java
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21061) fix synchronization of org.apache.hadoop.hbase.ipc.RpcServer$Connection.useWrap
[
https://issues.apache.org/jira/browse/HBASE-21061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16598088#comment-16598088
]
Sean Busbey commented on HBASE-21061:
-
oh right, no flaky lists while asf jenkins is down. let me see what the logs
say about those failures.
> fix synchronization of
> org.apache.hadoop.hbase.ipc.RpcServer$Connection.useWrap
> ---
>
> Key: HBASE-21061
> URL: https://issues.apache.org/jira/browse/HBASE-21061
> Project: HBase
> Issue Type: Sub-task
> Components: rpc
>Affects Versions: 1.5.0, 1.3.3, 1.2.7, 1.4.7
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Critical
> Attachments: HBASE-21061-branch-1.v0.patch
>
>
> fix findbugs highlighted issue
> {code}
> Inconsistent synchronization of
> org.apache.hadoop.hbase.ipc.RpcServer$Connection.useWrap; locked 75% of time
> Unsynchronized access at RpcServer.java:75% of time Unsynchronized access at
> RpcServer.java
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20940) HStore.cansplit should not allow split to happen if it has references
[ https://issues.apache.org/jira/browse/HBASE-20940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16598086#comment-16598086 ] Lars Hofhansl commented on HBASE-20940: --- Unfortunately this make Phoenix with local indexes very slow, since it is called the same API for each index scan (which I think is not necessary - the same race does not exist there). > HStore.cansplit should not allow split to happen if it has references > - > > Key: HBASE-20940 > URL: https://issues.apache.org/jira/browse/HBASE-20940 > Project: HBase > Issue Type: Bug >Affects Versions: 1.3.2 >Reporter: Vishal Khandelwal >Assignee: Vishal Khandelwal >Priority: Major > Fix For: 3.0.0, 1.5.0, 1.3.3, 2.2.0, 2.1.1, 1.4.7, 2.0.3 > > Attachments: HBASE-20940-branch-1-addendum.patch, > HBASE-20940.branch-1.3.v1.patch, HBASE-20940.branch-1.3.v2.patch, > HBASE-20940.branch-1.v1.patch, HBASE-20940.branch-1.v2.patch, > HBASE-20940.branch-1.v3.patch, HBASE-20940.branch-1.v5.patch, > HBASE-20940.v1.patch, HBASE-20940.v2.patch, HBASE-20940.v3.patch, > HBASE-20940.v4.patch, result_HBASE-20940.branch-1.v2.log > > > When split happens and immediately another split happens, it may result into > a split of a region who still has references to its parent. More details > about scenario can be found here HBASE-20933 > HStore.hasReferences should check from fs.storefile rather than in memory > objects. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21107) add a metrics for netty direct memory
[ https://issues.apache.org/jira/browse/HBASE-21107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huaxiang sun updated HBASE-21107: - Attachment: HBASE-21107-master-v2.patch > add a metrics for netty direct memory > - > > Key: HBASE-21107 > URL: https://issues.apache.org/jira/browse/HBASE-21107 > Project: HBase > Issue Type: Improvement > Components: IPC/RPC >Affects Versions: 2.0.1 >Reporter: huaxiang sun >Assignee: huaxiang sun >Priority: Minor > Attachments: HBASE-21107-master-v1.patch, > HBASE-21107-master-v1.patch, HBASE-21107-master-v2.patch, > HBASE-21107-master-v2.patch, Screen Shot 2018-08-23 at 4.55.01 PM.png > > > This is separated from HBASE-20913 as I realized that netty direct memory > usage needs to show up in metrics instead of the web page. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20307) LoadTestTool prints too much zookeeper logging
[ https://issues.apache.org/jira/browse/HBASE-20307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16597962#comment-16597962 ] Colin Garcia commented on HBASE-20307: -- Bump. Just wondering if you guys had any thoughts about the proposed changes? > LoadTestTool prints too much zookeeper logging > -- > > Key: HBASE-20307 > URL: https://issues.apache.org/jira/browse/HBASE-20307 > Project: HBase > Issue Type: Bug > Components: tooling >Reporter: Mike Drob >Assignee: Colin Garcia >Priority: Major > Labels: beginner > Attachments: HBASE-20307.000.patch > > > When running ltt there is a ton of ZK related cruft that I probably don't > care about. Hide it behind -verbose flag or point people at log4j > configuration but don't print it by default. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21061) fix synchronization of org.apache.hadoop.hbase.ipc.RpcServer$Connection.useWrap
[
https://issues.apache.org/jira/browse/HBASE-21061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16597946#comment-16597946
]
QABot from busbey commented on HBASE-21061:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 2m
5s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} branch-1 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
35s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
20s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
2s{color} | {color:green} branch-1 passed {color} |
| {color:red}-1{color} | {color:red} shadedjars {color} | {color:red} 0m
11s{color} | {color:red} branch has 10 errors when building our shaded
downstream artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
26s{color} | {color:blue} hbase-server in branch-1 has 4 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} branch-1 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 1m 6s{color}
| {color:red} hbase-server generated 1 new + 193 unchanged - 1 fixed = 194
total (was 194) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} shadedjars {color} | {color:red} 0m
7s{color} | {color:red} patch has 10 errors when building our shaded downstream
artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
5m 25s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.1
2.7.2 2.7.3 2.7.4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
32s{color} | {color:green} hbase-server generated 0 new + 1 unchanged - 3 fixed
= 1 total (was 4) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 38m 55s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 63m 12s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.master.balancer.TestRegionLocationFinder |
| | hadoop.hbase.client.TestRollbackFromClient |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=18.06.1-ce Server=18.06.1-ce Image:yetus/hbase:date2018-08-30
|
| JIRA Issue | HBASE-21061 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12937720/HBASE-21061-branch-1.v0.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux ee51145da9f0 3.10.0-693.5.2.el7.x86_64 #1 SMP Fri Oct 20
20:32:50 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | hbase-personality.sh |
| git revision | branch-1 / fb74f215b4 |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| shadedjars | artifact/patchprocess/branch-shadedjars.txt |
| findbugs |
[jira] [Commented] (HBASE-21023) Add completeProcedure/s() API to HbckService
[ https://issues.apache.org/jira/browse/HBASE-21023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16597822#comment-16597822 ] Umesh Agashe commented on HBASE-21023: -- HBASE-21083 adds bypassing procedures to completion. This could be used as an alternative to purging procedures. So subject and description of the Jira is now changed to completeProcedure/s() which will bypass the procedure/s and parents to completion without doing actual work. This will be useful for operators from hbck2 to unstuck procedures. > Add completeProcedure/s() API to HbckService > > > Key: HBASE-21023 > URL: https://issues.apache.org/jira/browse/HBASE-21023 > Project: HBase > Issue Type: Sub-task > Components: hbck2 >Affects Versions: 2.0.1 >Reporter: Umesh Agashe >Assignee: Umesh Agashe >Priority: Major > Fix For: 2.2.0 > > > completeProcedure/s(): some procedures do not support abort at every step. > When these procedures get stuck then they can not be aborted or make further > progress. Corrective action is to bypass these procedures to completion. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21023) Add completeProcedure/s() API to HbckService
[ https://issues.apache.org/jira/browse/HBASE-21023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Umesh Agashe updated HBASE-21023: - Description: completeProcedure/s(): some procedures do not support abort at every step. When these procedures get stuck then they can not be aborted or make further progress. Corrective action is to bypass these procedures to completion. (was: purgeProcedure/s(): some procedures do not support abort at every step. When these procedures get stuck then they can not be aborted or make further progress. Corrective action is to purge these procedures from ProcWAL. Provide option to purge sub-procedures as well.) > Add completeProcedure/s() API to HbckService > > > Key: HBASE-21023 > URL: https://issues.apache.org/jira/browse/HBASE-21023 > Project: HBase > Issue Type: Sub-task > Components: hbck2 >Affects Versions: 2.0.1 >Reporter: Umesh Agashe >Assignee: Umesh Agashe >Priority: Major > Fix For: 2.2.0 > > > completeProcedure/s(): some procedures do not support abort at every step. > When these procedures get stuck then they can not be aborted or make further > progress. Corrective action is to bypass these procedures to completion. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21023) Add completeProcedure/s() API to HbckService
[ https://issues.apache.org/jira/browse/HBASE-21023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Umesh Agashe updated HBASE-21023: - Summary: Add completeProcedure/s() API to HbckService (was: Add purgeProcedure/s() API to HbckService) > Add completeProcedure/s() API to HbckService > > > Key: HBASE-21023 > URL: https://issues.apache.org/jira/browse/HBASE-21023 > Project: HBase > Issue Type: Sub-task > Components: hbck2 >Affects Versions: 2.0.1 >Reporter: Umesh Agashe >Assignee: Umesh Agashe >Priority: Major > Fix For: 2.2.0 > > > purgeProcedure/s(): some procedures do not support abort at every step. When > these procedures get stuck then they can not be aborted or make further > progress. Corrective action is to purge these procedures from ProcWAL. > Provide option to purge sub-procedures as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HBASE-21134) Add guardrails to cell tags in order to avoid the tags length to overflow
[
https://issues.apache.org/jira/browse/HBASE-21134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Esteban Gutierrez reassigned HBASE-21134:
-
Assignee: Esteban Gutierrez
> Add guardrails to cell tags in order to avoid the tags length to overflow
> --
>
> Key: HBASE-21134
> URL: https://issues.apache.org/jira/browse/HBASE-21134
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.5.0
>Reporter: Esteban Gutierrez
>Assignee: Esteban Gutierrez
>Priority: Critical
>
> We found that per cell tags can easily overflow and and cause failures while
> reading HFiles. If a mutation has more than 32KB for the byte array with the
> tags we should reject the operation on the client side (proactively) and the
> server side as we deserialize the request.
> {code}
> 2018-08-21 11:08:45,387 ERROR
> org.apache.hadoop.hbase.regionserver.CompactSplitThread: Compaction failed
> Request = regionName=table1,,1534870486680.9112ca53504084152da5e28116f40ec2.,
> storeName=c1, fileCount=4, fileSize=254.2 K (138.0 K, 33.5 K, 34.0 K, 48.7
> K), priority=1, time=8555785624243
> java.lang.IllegalStateException: Invalid currTagsLen -20658. Block offset: 0,
> block length: 44912, position: 0 (without header).
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV3$ScannerV3.checkTagsLen(HFileReaderV3.java:226)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV3$ScannerV3.readKeyValueLen(HFileReaderV3.java:251)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.updateCurrBlock(HFileReaderV2.java:956)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.seekTo(HFileReaderV2.java:919)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekAtOrAfter(StoreFileScanner.java:304)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.seek(StoreFileScanner.java:200)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.seekScanners(StoreScanner.java:350)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:269)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:231)
> at
> org.apache.hadoop.hbase.regionserver.compactions.Compactor.createScanner(Compactor.java:414)
> at
> org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:91)
> at
> org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:125)
> at org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1247)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:1915)
> at
> org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.doCompaction(CompactSplitThread.java:529)
> at
> org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.run(CompactSplitThread.java:566)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21083) Introduce a mechanism to bypass the execution of a stuck procedure
[ https://issues.apache.org/jira/browse/HBASE-21083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16597818#comment-16597818 ] stack commented on HBASE-21083: --- [~uagashe] My fault. Was local only. I had not pushed it. Done. > Introduce a mechanism to bypass the execution of a stuck procedure > -- > > Key: HBASE-21083 > URL: https://issues.apache.org/jira/browse/HBASE-21083 > Project: HBase > Issue Type: Sub-task > Components: amv2 >Affects Versions: 2.1.0, 2.0.1 >Reporter: Allan Yang >Assignee: Allan Yang >Priority: Major > Fix For: 3.0.0, 2.1.1, 2.0.2 > > Attachments: HBASE-21083.branch-2.0.001.patch, > HBASE-21083.branch-2.0.002.patch, HBASE-21083.branch-2.0.003.patch, > HBASE-21083.branch-2.0.003.patch, HBASE-21083.branch-2.0.003.patch, > HBASE-21083.branch-2.1.001.patch > > > Offline discussed with [~stack] and [~Apache9]. We all agreed that we need to > introduce a mechanism to 'force complete' a stuck procedure, so the AMv2 can > continue running. > we still have some unrevealed bugs hiding in our AMv2 and procedureV2 > system, we need something to interfere with stuck procedures before HBCK2 can > work. This is very crucial for a production ready system. > For now, we have little ways to interfere with running procedures. Aborting > them is not a good choice, since some procedures are not abort-able. And some > procedure may have overridden the abort() method, which will ignore the abort > request. > So, here, I will introduce a mechanism to bypass the execution of a stuck > procedure. > Basically, I added a field called 'bypass' to Procedure class. If we set this > field to true, all the logic in execute/rollback will be skipped, letting > this procedure and its ancestors complete normally and releasing the lock > resources at last. > Notice that bypassing a procedure may leave the cluster in a middle state, > e.g. the region not assigned, or some hdfs files left behind. > The Operators need know the side effect of bypassing and recover the > inconsistent state of the cluster themselves, like issuing new procedures to > assign the regions. > A patch will be uploaded and review board will be open. For now, only APIs in > ProcedureExecutor are provided. If anything is fine, I will add it to master > service and add a shell command to bypass a procedure. Or, maybe we can use > dynamically compiled JSPs to execute those APIs as mentioned in HBASE-20679. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-21083) Introduce a mechanism to bypass the execution of a stuck procedure
[ https://issues.apache.org/jira/browse/HBASE-21083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16597814#comment-16597814 ] Umesh Agashe edited comment on HBASE-21083 at 8/30/18 7:20 PM: --- [~stack], can this be committed to master as well? was (Author: uagashe): @stack, can this be committed to master as well? > Introduce a mechanism to bypass the execution of a stuck procedure > -- > > Key: HBASE-21083 > URL: https://issues.apache.org/jira/browse/HBASE-21083 > Project: HBase > Issue Type: Sub-task > Components: amv2 >Affects Versions: 2.1.0, 2.0.1 >Reporter: Allan Yang >Assignee: Allan Yang >Priority: Major > Fix For: 3.0.0, 2.1.1, 2.0.2 > > Attachments: HBASE-21083.branch-2.0.001.patch, > HBASE-21083.branch-2.0.002.patch, HBASE-21083.branch-2.0.003.patch, > HBASE-21083.branch-2.0.003.patch, HBASE-21083.branch-2.0.003.patch, > HBASE-21083.branch-2.1.001.patch > > > Offline discussed with [~stack] and [~Apache9]. We all agreed that we need to > introduce a mechanism to 'force complete' a stuck procedure, so the AMv2 can > continue running. > we still have some unrevealed bugs hiding in our AMv2 and procedureV2 > system, we need something to interfere with stuck procedures before HBCK2 can > work. This is very crucial for a production ready system. > For now, we have little ways to interfere with running procedures. Aborting > them is not a good choice, since some procedures are not abort-able. And some > procedure may have overridden the abort() method, which will ignore the abort > request. > So, here, I will introduce a mechanism to bypass the execution of a stuck > procedure. > Basically, I added a field called 'bypass' to Procedure class. If we set this > field to true, all the logic in execute/rollback will be skipped, letting > this procedure and its ancestors complete normally and releasing the lock > resources at last. > Notice that bypassing a procedure may leave the cluster in a middle state, > e.g. the region not assigned, or some hdfs files left behind. > The Operators need know the side effect of bypassing and recover the > inconsistent state of the cluster themselves, like issuing new procedures to > assign the regions. > A patch will be uploaded and review board will be open. For now, only APIs in > ProcedureExecutor are provided. If anything is fine, I will add it to master > service and add a shell command to bypass a procedure. Or, maybe we can use > dynamically compiled JSPs to execute those APIs as mentioned in HBASE-20679. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21083) Introduce a mechanism to bypass the execution of a stuck procedure
[ https://issues.apache.org/jira/browse/HBASE-21083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16597814#comment-16597814 ] Umesh Agashe commented on HBASE-21083: -- @stack, can this be committed to master as well? > Introduce a mechanism to bypass the execution of a stuck procedure > -- > > Key: HBASE-21083 > URL: https://issues.apache.org/jira/browse/HBASE-21083 > Project: HBase > Issue Type: Sub-task > Components: amv2 >Affects Versions: 2.1.0, 2.0.1 >Reporter: Allan Yang >Assignee: Allan Yang >Priority: Major > Fix For: 3.0.0, 2.1.1, 2.0.2 > > Attachments: HBASE-21083.branch-2.0.001.patch, > HBASE-21083.branch-2.0.002.patch, HBASE-21083.branch-2.0.003.patch, > HBASE-21083.branch-2.0.003.patch, HBASE-21083.branch-2.0.003.patch, > HBASE-21083.branch-2.1.001.patch > > > Offline discussed with [~stack] and [~Apache9]. We all agreed that we need to > introduce a mechanism to 'force complete' a stuck procedure, so the AMv2 can > continue running. > we still have some unrevealed bugs hiding in our AMv2 and procedureV2 > system, we need something to interfere with stuck procedures before HBCK2 can > work. This is very crucial for a production ready system. > For now, we have little ways to interfere with running procedures. Aborting > them is not a good choice, since some procedures are not abort-able. And some > procedure may have overridden the abort() method, which will ignore the abort > request. > So, here, I will introduce a mechanism to bypass the execution of a stuck > procedure. > Basically, I added a field called 'bypass' to Procedure class. If we set this > field to true, all the logic in execute/rollback will be skipped, letting > this procedure and its ancestors complete normally and releasing the lock > resources at last. > Notice that bypassing a procedure may leave the cluster in a middle state, > e.g. the region not assigned, or some hdfs files left behind. > The Operators need know the side effect of bypassing and recover the > inconsistent state of the cluster themselves, like issuing new procedures to > assign the regions. > A patch will be uploaded and review board will be open. For now, only APIs in > ProcedureExecutor are provided. If anything is fine, I will add it to master > service and add a shell command to bypass a procedure. Or, maybe we can use > dynamically compiled JSPs to execute those APIs as mentioned in HBASE-20679. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-21134) Add guardrails to cell tags in order to avoid the tags length to overflow
Esteban Gutierrez created HBASE-21134:
-
Summary: Add guardrails to cell tags in order to avoid the tags
length to overflow
Key: HBASE-21134
URL: https://issues.apache.org/jira/browse/HBASE-21134
Project: HBase
Issue Type: Bug
Affects Versions: 1.5.0
Reporter: Esteban Gutierrez
We found that per cell tags can easily overflow and and cause failures while
reading HFiles. If a mutation has more than 32KB for the byte array with the
tags we should reject the operation on the client side (proactively) and the
server side as we deserialize the request.
{code}
2018-08-21 11:08:45,387 ERROR
org.apache.hadoop.hbase.regionserver.CompactSplitThread: Compaction failed
Request = regionName=table1,,1534870486680.9112ca53504084152da5e28116f40ec2.,
storeName=c1, fileCount=4, fileSize=254.2 K (138.0 K, 33.5 K, 34.0 K, 48.7 K),
priority=1, time=8555785624243
java.lang.IllegalStateException: Invalid currTagsLen -20658. Block offset: 0,
block length: 44912, position: 0 (without header).
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV3$ScannerV3.checkTagsLen(HFileReaderV3.java:226)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV3$ScannerV3.readKeyValueLen(HFileReaderV3.java:251)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.updateCurrBlock(HFileReaderV2.java:956)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.seekTo(HFileReaderV2.java:919)
at
org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekAtOrAfter(StoreFileScanner.java:304)
at
org.apache.hadoop.hbase.regionserver.StoreFileScanner.seek(StoreFileScanner.java:200)
at
org.apache.hadoop.hbase.regionserver.StoreScanner.seekScanners(StoreScanner.java:350)
at
org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:269)
at
org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:231)
at
org.apache.hadoop.hbase.regionserver.compactions.Compactor.createScanner(Compactor.java:414)
at
org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:91)
at
org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:125)
at org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1247)
at
org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:1915)
at
org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.doCompaction(CompactSplitThread.java:529)
at
org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.run(CompactSplitThread.java:566)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20704) Sometimes some compacted storefiles are not archived on region close
[ https://issues.apache.org/jira/browse/HBASE-20704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16597696#comment-16597696 ] Francis Liu commented on HBASE-20704: - Sounds good the 1.x patch will be much simpler will post one for that as soon as this passes buildbot > Sometimes some compacted storefiles are not archived on region close > > > Key: HBASE-20704 > URL: https://issues.apache.org/jira/browse/HBASE-20704 > Project: HBase > Issue Type: Bug > Components: Compaction >Affects Versions: 3.0.0, 1.3.0, 1.4.0, 1.5.0, 2.0.0 >Reporter: Francis Liu >Assignee: Francis Liu >Priority: Critical > Fix For: 3.0.0, 2.2.0, 2.1.1, 2.0.3 > > Attachments: HBASE-20704.001.patch, HBASE-20704.002.patch, > HBASE-20704.003.patch, HBASE-20704.004.draft.patch, HBASE-20704.005.patch, > HBASE-20704.006.patch > > > During region close compacted files which have not yet been archived by the > discharger are archived as part of the region closing process. It is > important that these files are wholly archived to insure data consistency. ie > a storefile containing delete tombstones can be archived while older > storefiles containing cells that were supposed to be deleted are left > unarchived thereby undeleting those cells. > On region close a compacted storefile is skipped from archiving if it has > read references (ie open scanners). This behavior is correct for when the > discharger chore runs but on region close consistency is of course more > important so we should add a special case to ignore any references on the > storefile and go ahead and archive it. > Attached patch contains a unit test that reproduces the problem and the > proposed fix. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20704) Sometimes some compacted storefiles are not archived on region close
[ https://issues.apache.org/jira/browse/HBASE-20704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francis Liu updated HBASE-20704: Status: Patch Available (was: Open) resubmitting to kick buildbot > Sometimes some compacted storefiles are not archived on region close > > > Key: HBASE-20704 > URL: https://issues.apache.org/jira/browse/HBASE-20704 > Project: HBase > Issue Type: Bug > Components: Compaction >Affects Versions: 2.0.0, 1.4.0, 1.3.0, 3.0.0, 1.5.0 >Reporter: Francis Liu >Assignee: Francis Liu >Priority: Critical > Fix For: 3.0.0, 2.2.0, 2.1.1, 2.0.3 > > Attachments: HBASE-20704.001.patch, HBASE-20704.002.patch, > HBASE-20704.003.patch, HBASE-20704.004.draft.patch, HBASE-20704.005.patch, > HBASE-20704.006.patch > > > During region close compacted files which have not yet been archived by the > discharger are archived as part of the region closing process. It is > important that these files are wholly archived to insure data consistency. ie > a storefile containing delete tombstones can be archived while older > storefiles containing cells that were supposed to be deleted are left > unarchived thereby undeleting those cells. > On region close a compacted storefile is skipped from archiving if it has > read references (ie open scanners). This behavior is correct for when the > discharger chore runs but on region close consistency is of course more > important so we should add a special case to ignore any references on the > storefile and go ahead and archive it. > Attached patch contains a unit test that reproduces the problem and the > proposed fix. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20704) Sometimes some compacted storefiles are not archived on region close
[ https://issues.apache.org/jira/browse/HBASE-20704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francis Liu updated HBASE-20704: Status: Open (was: Patch Available) > Sometimes some compacted storefiles are not archived on region close > > > Key: HBASE-20704 > URL: https://issues.apache.org/jira/browse/HBASE-20704 > Project: HBase > Issue Type: Bug > Components: Compaction >Affects Versions: 2.0.0, 1.4.0, 1.3.0, 3.0.0, 1.5.0 >Reporter: Francis Liu >Assignee: Francis Liu >Priority: Critical > Fix For: 3.0.0, 2.2.0, 2.1.1, 2.0.3 > > Attachments: HBASE-20704.001.patch, HBASE-20704.002.patch, > HBASE-20704.003.patch, HBASE-20704.004.draft.patch, HBASE-20704.005.patch, > HBASE-20704.006.patch > > > During region close compacted files which have not yet been archived by the > discharger are archived as part of the region closing process. It is > important that these files are wholly archived to insure data consistency. ie > a storefile containing delete tombstones can be archived while older > storefiles containing cells that were supposed to be deleted are left > unarchived thereby undeleting those cells. > On region close a compacted storefile is skipped from archiving if it has > read references (ie open scanners). This behavior is correct for when the > discharger chore runs but on region close consistency is of course more > important so we should add a special case to ignore any references on the > storefile and go ahead and archive it. > Attached patch contains a unit test that reproduces the problem and the > proposed fix. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21104) client.TestRestoreSnapshotFromClientWithRegionReplicas failing on branch-1.3, branch-1.2
[ https://issues.apache.org/jira/browse/HBASE-21104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki updated HBASE-21104: - Status: Patch Available (was: Open) > client.TestRestoreSnapshotFromClientWithRegionReplicas failing on branch-1.3, > branch-1.2 > > > Key: HBASE-21104 > URL: https://issues.apache.org/jira/browse/HBASE-21104 > Project: HBase > Issue Type: Bug > Components: read replicas, snapshots >Affects Versions: 1.3.3, 1.2.7 >Reporter: Sean Busbey >Assignee: Toshihiro Suzuki >Priority: Major > Attachments: HBASE-21104.branch-1.2.001.patch > > > client.TestRestoreSnapshotFromClientWithRegionReplicas is in both branch-1.3 > and branch-1.2's flaky list as failing every run. > need to triage and fix. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21104) client.TestRestoreSnapshotFromClientWithRegionReplicas failing on branch-1.3, branch-1.2
[ https://issues.apache.org/jira/browse/HBASE-21104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16597611#comment-16597611 ] Toshihiro Suzuki commented on HBASE-21104: -- It seems like we need to backport HBASE-19934 to fix this issue. I just attached a backport patch for branch-1.2. > client.TestRestoreSnapshotFromClientWithRegionReplicas failing on branch-1.3, > branch-1.2 > > > Key: HBASE-21104 > URL: https://issues.apache.org/jira/browse/HBASE-21104 > Project: HBase > Issue Type: Bug > Components: read replicas, snapshots >Affects Versions: 1.3.3, 1.2.7 >Reporter: Sean Busbey >Assignee: Toshihiro Suzuki >Priority: Major > Attachments: HBASE-21104.branch-1.2.001.patch > > > client.TestRestoreSnapshotFromClientWithRegionReplicas is in both branch-1.3 > and branch-1.2's flaky list as failing every run. > need to triage and fix. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21104) client.TestRestoreSnapshotFromClientWithRegionReplicas failing on branch-1.3, branch-1.2
[ https://issues.apache.org/jira/browse/HBASE-21104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki updated HBASE-21104: - Attachment: HBASE-21104.branch-1.2.001.patch > client.TestRestoreSnapshotFromClientWithRegionReplicas failing on branch-1.3, > branch-1.2 > > > Key: HBASE-21104 > URL: https://issues.apache.org/jira/browse/HBASE-21104 > Project: HBase > Issue Type: Bug > Components: read replicas, snapshots >Affects Versions: 1.3.3, 1.2.7 >Reporter: Sean Busbey >Assignee: Toshihiro Suzuki >Priority: Major > Attachments: HBASE-21104.branch-1.2.001.patch > > > client.TestRestoreSnapshotFromClientWithRegionReplicas is in both branch-1.3 > and branch-1.2's flaky list as failing every run. > need to triage and fix. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21133) '/version' in rest should return hbase rest client version
[
https://issues.apache.org/jira/browse/HBASE-21133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16597588#comment-16597588
]
Sean Busbey commented on HBASE-21133:
-
This also point out a gap since I don't see any obvious docs (in the code or
otherwise) guiding devs on when we ought to be incrementing the REST version
number.
> '/version' in rest should return hbase rest client version
> --
>
> Key: HBASE-21133
> URL: https://issues.apache.org/jira/browse/HBASE-21133
> Project: HBase
> Issue Type: Bug
> Components: REST
>Affects Versions: 3.0.0, 2.0.1, 2.2.0
>Reporter: Guangxu Cheng
>Assignee: Guangxu Cheng
>Priority: Major
> Attachments: HBASE-21133.master.001.patch
>
>
> restVersion is a constant, which is meaningless.
> {code:java}
> public VersionModel(ServletContext context) {
> restVersion = RESTServlet.VERSION_STRING;
> {code}
> {code:java}
> String VERSION_STRING = "0.0.3";
> {code}
> the result of '/version'
> {code:json}
> {"Server":"jetty/6.1.26","Jersey":"1.9","REST":"0.0.3","JVM":"Oracle
> Corporation 1.8.0_102-25.102-b14","OS":"Linux
> 2.6.32.57-tlinux-1.1.2-container amd64"}
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21133) '/version' in rest should return hbase rest client version
[
https://issues.apache.org/jira/browse/HBASE-21133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16597583#comment-16597583
]
Sean Busbey commented on HBASE-21133:
-
I believe that version string is of the REST API provided by HBase. It might
make more sense to add another field to the version string, e.g. "HBase :
2.2.0".
> '/version' in rest should return hbase rest client version
> --
>
> Key: HBASE-21133
> URL: https://issues.apache.org/jira/browse/HBASE-21133
> Project: HBase
> Issue Type: Bug
> Components: REST
>Affects Versions: 3.0.0, 2.0.1, 2.2.0
>Reporter: Guangxu Cheng
>Assignee: Guangxu Cheng
>Priority: Major
> Attachments: HBASE-21133.master.001.patch
>
>
> restVersion is a constant, which is meaningless.
> {code:java}
> public VersionModel(ServletContext context) {
> restVersion = RESTServlet.VERSION_STRING;
> {code}
> {code:java}
> String VERSION_STRING = "0.0.3";
> {code}
> the result of '/version'
> {code:json}
> {"Server":"jetty/6.1.26","Jersey":"1.9","REST":"0.0.3","JVM":"Oracle
> Corporation 1.8.0_102-25.102-b14","OS":"Linux
> 2.6.32.57-tlinux-1.1.2-container amd64"}
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21133) '/version' in rest should return hbase rest client version
[
https://issues.apache.org/jira/browse/HBASE-21133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-21133:
Status: Patch Available (was: Open)
> '/version' in rest should return hbase rest client version
> --
>
> Key: HBASE-21133
> URL: https://issues.apache.org/jira/browse/HBASE-21133
> Project: HBase
> Issue Type: Bug
> Components: REST
>Affects Versions: 2.0.1, 3.0.0, 2.2.0
>Reporter: Guangxu Cheng
>Assignee: Guangxu Cheng
>Priority: Major
> Attachments: HBASE-21133.master.001.patch
>
>
> restVersion is a constant, which is meaningless.
> {code:java}
> public VersionModel(ServletContext context) {
> restVersion = RESTServlet.VERSION_STRING;
> {code}
> {code:java}
> String VERSION_STRING = "0.0.3";
> {code}
> the result of '/version'
> {code:json}
> {"Server":"jetty/6.1.26","Jersey":"1.9","REST":"0.0.3","JVM":"Oracle
> Corporation 1.8.0_102-25.102-b14","OS":"Linux
> 2.6.32.57-tlinux-1.1.2-container amd64"}
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21132) return wrong result in rest multiget
[
https://issues.apache.org/jira/browse/HBASE-21132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16597464#comment-16597464
]
Ted Yu commented on HBASE-21132:
lgtm
> return wrong result in rest multiget
>
>
> Key: HBASE-21132
> URL: https://issues.apache.org/jira/browse/HBASE-21132
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0, 2.0.0
>Reporter: Guangxu Cheng
>Assignee: Guangxu Cheng
>Priority: Major
> Attachments: HBASE-21132.master.001.patch
>
>
> There are two ways to specify columns in multi-gets feature。
> 1、Specify columns in PathParam as described in HBASE-15870. Examples:
> {code:sh}GET /t1/multiget/cf1:c1,cf2?row=r1{code}
> 2、Specify columns in QueryParam. Examples:
> {code:sh} GET /t1/multiget?row=r1/cf1:c1,cf2=r2/cf2 {code}
> However, when we specify columns in QueryParam, the result is wrong , the
> rowkey contains the columns.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21133) '/version' in rest should return hbase rest client version
[
https://issues.apache.org/jira/browse/HBASE-21133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guangxu Cheng updated HBASE-21133:
--
Attachment: HBASE-21133.master.001.patch
> '/version' in rest should return hbase rest client version
> --
>
> Key: HBASE-21133
> URL: https://issues.apache.org/jira/browse/HBASE-21133
> Project: HBase
> Issue Type: Bug
> Components: REST
>Affects Versions: 3.0.0, 2.0.1, 2.2.0
>Reporter: Guangxu Cheng
>Assignee: Guangxu Cheng
>Priority: Major
> Attachments: HBASE-21133.master.001.patch
>
>
> restVersion is a constant, which is meaningless.
> {code:java}
> public VersionModel(ServletContext context) {
> restVersion = RESTServlet.VERSION_STRING;
> {code}
> {code:java}
> String VERSION_STRING = "0.0.3";
> {code}
> the result of '/version'
> {code:json}
> {"Server":"jetty/6.1.26","Jersey":"1.9","REST":"0.0.3","JVM":"Oracle
> Corporation 1.8.0_102-25.102-b14","OS":"Linux
> 2.6.32.57-tlinux-1.1.2-container amd64"}
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21133) '/version' in rest should return hbase rest client version
[
https://issues.apache.org/jira/browse/HBASE-21133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16597460#comment-16597460
]
Guangxu Cheng commented on HBASE-21133:
---
Upload the first patch for review.
> '/version' in rest should return hbase rest client version
> --
>
> Key: HBASE-21133
> URL: https://issues.apache.org/jira/browse/HBASE-21133
> Project: HBase
> Issue Type: Bug
> Components: REST
>Affects Versions: 3.0.0, 2.0.1, 2.2.0
>Reporter: Guangxu Cheng
>Assignee: Guangxu Cheng
>Priority: Major
> Attachments: HBASE-21133.master.001.patch
>
>
> restVersion is a constant, which is meaningless.
> {code:java}
> public VersionModel(ServletContext context) {
> restVersion = RESTServlet.VERSION_STRING;
> {code}
> {code:java}
> String VERSION_STRING = "0.0.3";
> {code}
> the result of '/version'
> {code:json}
> {"Server":"jetty/6.1.26","Jersey":"1.9","REST":"0.0.3","JVM":"Oracle
> Corporation 1.8.0_102-25.102-b14","OS":"Linux
> 2.6.32.57-tlinux-1.1.2-container amd64"}
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-21133) '/version' in rest should return hbase rest client version
Guangxu Cheng created HBASE-21133:
-
Summary: '/version' in rest should return hbase rest client version
Key: HBASE-21133
URL: https://issues.apache.org/jira/browse/HBASE-21133
Project: HBase
Issue Type: Bug
Components: REST
Affects Versions: 2.0.1, 3.0.0, 2.2.0
Reporter: Guangxu Cheng
Assignee: Guangxu Cheng
restVersion is a constant, which is meaningless.
{code:java}
public VersionModel(ServletContext context) {
restVersion = RESTServlet.VERSION_STRING;
{code}
{code:java}
String VERSION_STRING = "0.0.3";
{code}
the result of '/version'
{code:json}
{"Server":"jetty/6.1.26","Jersey":"1.9","REST":"0.0.3","JVM":"Oracle
Corporation 1.8.0_102-25.102-b14","OS":"Linux 2.6.32.57-tlinux-1.1.2-container
amd64"}
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21132) return wrong result in rest multiget
[
https://issues.apache.org/jira/browse/HBASE-21132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guangxu Cheng updated HBASE-21132:
--
Status: Patch Available (was: Open)
> return wrong result in rest multiget
>
>
> Key: HBASE-21132
> URL: https://issues.apache.org/jira/browse/HBASE-21132
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0, 3.0.0
>Reporter: Guangxu Cheng
>Assignee: Guangxu Cheng
>Priority: Major
> Attachments: HBASE-21132.master.001.patch
>
>
> There are two ways to specify columns in multi-gets feature。
> 1、Specify columns in PathParam as described in HBASE-15870. Examples:
> {code:sh}GET /t1/multiget/cf1:c1,cf2?row=r1{code}
> 2、Specify columns in QueryParam. Examples:
> {code:sh} GET /t1/multiget?row=r1/cf1:c1,cf2=r2/cf2 {code}
> However, when we specify columns in QueryParam, the result is wrong , the
> rowkey contains the columns.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21132) return wrong result in rest multiget
[
https://issues.apache.org/jira/browse/HBASE-21132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16597426#comment-16597426
]
Guangxu Cheng commented on HBASE-21132:
---
Simple patch.
> return wrong result in rest multiget
>
>
> Key: HBASE-21132
> URL: https://issues.apache.org/jira/browse/HBASE-21132
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0, 2.0.0
>Reporter: Guangxu Cheng
>Assignee: Guangxu Cheng
>Priority: Major
> Attachments: HBASE-21132.master.001.patch
>
>
> There are two ways to specify columns in multi-gets feature。
> 1、Specify columns in PathParam as described in HBASE-15870. Examples:
> {code:sh}GET /t1/multiget/cf1:c1,cf2?row=r1{code}
> 2、Specify columns in QueryParam. Examples:
> {code:sh} GET /t1/multiget?row=r1/cf1:c1,cf2=r2/cf2 {code}
> However, when we specify columns in QueryParam, the result is wrong , the
> rowkey contains the columns.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21132) return wrong result in rest multiget
[
https://issues.apache.org/jira/browse/HBASE-21132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guangxu Cheng updated HBASE-21132:
--
Attachment: HBASE-21132.master.001.patch
> return wrong result in rest multiget
>
>
> Key: HBASE-21132
> URL: https://issues.apache.org/jira/browse/HBASE-21132
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0, 2.0.0
>Reporter: Guangxu Cheng
>Assignee: Guangxu Cheng
>Priority: Major
> Attachments: HBASE-21132.master.001.patch
>
>
> There are two ways to specify columns in multi-gets feature。
> 1、Specify columns in PathParam as described in HBASE-15870. Examples:
> {code:sh}GET /t1/multiget/cf1:c1,cf2?row=r1{code}
> 2、Specify columns in QueryParam. Examples:
> {code:sh} GET /t1/multiget?row=r1/cf1:c1,cf2=r2/cf2 {code}
> However, when we specify columns in QueryParam, the result is wrong , the
> rowkey contains the columns.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-21132) return wrong result in rest multiget
Guangxu Cheng created HBASE-21132:
-
Summary: return wrong result in rest multiget
Key: HBASE-21132
URL: https://issues.apache.org/jira/browse/HBASE-21132
Project: HBase
Issue Type: Bug
Affects Versions: 2.0.0, 3.0.0
Reporter: Guangxu Cheng
Assignee: Guangxu Cheng
There are two ways to specify columns in multi-gets feature。
1、Specify columns in PathParam as described in HBASE-15870. Examples:
{code:sh}GET /t1/multiget/cf1:c1,cf2?row=r1{code}
2、Specify columns in QueryParam. Examples:
{code:sh} GET /t1/multiget?row=r1/cf1:c1,cf2=r2/cf2 {code}
However, when we specify columns in QueryParam, the result is wrong , the
rowkey contains the columns.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21102) ServerCrashProcedure should select target server where no other replicas exist for the current region
[ https://issues.apache.org/jira/browse/HBASE-21102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-21102: --- Attachment: HBASE-21102_1.patch > ServerCrashProcedure should select target server where no other replicas > exist for the current region > - > > Key: HBASE-21102 > URL: https://issues.apache.org/jira/browse/HBASE-21102 > Project: HBase > Issue Type: Bug > Components: Region Assignment >Affects Versions: 3.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Major > Attachments: HBASE-21102_1.patch, HBASE-21102_initial.patch > > > Currently when a server with region replica crashes, when the target server > is created for the replica region assignment there is no guarentee that a > server is selected where there is no other replica for the current region > getting assigned. It so happens that currently we do an assignment randomly > and later the LB comes and identifies these cases and again does MOVE for > such regions. It will be better if we can identify target servers at least > minimally ensuring that replicas are not colocated. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21102) ServerCrashProcedure should select target server where no other replicas exist for the current region
[ https://issues.apache.org/jira/browse/HBASE-21102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16597369#comment-16597369 ] ramkrishna.s.vasudevan commented on HBASE-21102: A more formal patch. Running the test cases to see if there are any missing cases. By the time, any reviews/feedback appreciated. > ServerCrashProcedure should select target server where no other replicas > exist for the current region > - > > Key: HBASE-21102 > URL: https://issues.apache.org/jira/browse/HBASE-21102 > Project: HBase > Issue Type: Bug > Components: Region Assignment >Affects Versions: 3.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Major > Attachments: HBASE-21102_1.patch, HBASE-21102_initial.patch > > > Currently when a server with region replica crashes, when the target server > is created for the replica region assignment there is no guarentee that a > server is selected where there is no other replica for the current region > getting assigned. It so happens that currently we do an assignment randomly > and later the LB comes and identifies these cases and again does MOVE for > such regions. It will be better if we can identify target servers at least > minimally ensuring that replicas are not colocated. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21102) ServerCrashProcedure should select target server where no other replicas exist for the current region
[ https://issues.apache.org/jira/browse/HBASE-21102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-21102: --- Status: Patch Available (was: Open) > ServerCrashProcedure should select target server where no other replicas > exist for the current region > - > > Key: HBASE-21102 > URL: https://issues.apache.org/jira/browse/HBASE-21102 > Project: HBase > Issue Type: Bug > Components: Region Assignment >Affects Versions: 3.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Major > Attachments: HBASE-21102_1.patch, HBASE-21102_initial.patch > > > Currently when a server with region replica crashes, when the target server > is created for the replica region assignment there is no guarentee that a > server is selected where there is no other replica for the current region > getting assigned. It so happens that currently we do an assignment randomly > and later the LB comes and identifies these cases and again does MOVE for > such regions. It will be better if we can identify target servers at least > minimally ensuring that replicas are not colocated. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21129) Clean up duplicate codes in #equals and #hashCode methods of Filter
[ https://issues.apache.org/jira/browse/HBASE-21129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-21129: -- Status: Patch Available (was: Open) Trigger QA again. > Clean up duplicate codes in #equals and #hashCode methods of Filter > --- > > Key: HBASE-21129 > URL: https://issues.apache.org/jira/browse/HBASE-21129 > Project: HBase > Issue Type: Improvement > Components: Filters >Affects Versions: 3.0.0, 2.2.0 >Reporter: Reid Chan >Assignee: Reid Chan >Priority: Minor > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-21129.master.001.patch, > HBASE-21129.master.002.patch > > > It is a follow-up of HBASE-19008, aiming to clean up duplicate codes in > #equals and #hashCode methods. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21129) Clean up duplicate codes in #equals and #hashCode methods of Filter
[ https://issues.apache.org/jira/browse/HBASE-21129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-21129: -- Attachment: HBASE-21129.master.002.patch > Clean up duplicate codes in #equals and #hashCode methods of Filter > --- > > Key: HBASE-21129 > URL: https://issues.apache.org/jira/browse/HBASE-21129 > Project: HBase > Issue Type: Improvement > Components: Filters >Affects Versions: 3.0.0, 2.2.0 >Reporter: Reid Chan >Assignee: Reid Chan >Priority: Minor > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-21129.master.001.patch, > HBASE-21129.master.002.patch > > > It is a follow-up of HBASE-19008, aiming to clean up duplicate codes in > #equals and #hashCode methods. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21129) Clean up duplicate codes in #equals and #hashCode methods of Filter
[ https://issues.apache.org/jira/browse/HBASE-21129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-21129: -- Attachment: (was: HBASE-21129.master.002.patch) > Clean up duplicate codes in #equals and #hashCode methods of Filter > --- > > Key: HBASE-21129 > URL: https://issues.apache.org/jira/browse/HBASE-21129 > Project: HBase > Issue Type: Improvement > Components: Filters >Affects Versions: 3.0.0, 2.2.0 >Reporter: Reid Chan >Assignee: Reid Chan >Priority: Minor > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-21129.master.001.patch, > HBASE-21129.master.002.patch > > > It is a follow-up of HBASE-19008, aiming to clean up duplicate codes in > #equals and #hashCode methods. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21129) Clean up duplicate codes in #equals and #hashCode methods of Filter
[ https://issues.apache.org/jira/browse/HBASE-21129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-21129: -- Status: Open (was: Patch Available) > Clean up duplicate codes in #equals and #hashCode methods of Filter > --- > > Key: HBASE-21129 > URL: https://issues.apache.org/jira/browse/HBASE-21129 > Project: HBase > Issue Type: Improvement > Components: Filters >Affects Versions: 3.0.0, 2.2.0 >Reporter: Reid Chan >Assignee: Reid Chan >Priority: Minor > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-21129.master.001.patch, > HBASE-21129.master.002.patch > > > It is a follow-up of HBASE-19008, aiming to clean up duplicate codes in > #equals and #hashCode methods. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21129) Clean up duplicate codes in #equals and #hashCode methods of Filter
[ https://issues.apache.org/jira/browse/HBASE-21129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16597317#comment-16597317 ] Reid Chan commented on HBASE-21129: --- QA doesn't work...? > Clean up duplicate codes in #equals and #hashCode methods of Filter > --- > > Key: HBASE-21129 > URL: https://issues.apache.org/jira/browse/HBASE-21129 > Project: HBase > Issue Type: Improvement > Components: Filters >Affects Versions: 3.0.0, 2.2.0 >Reporter: Reid Chan >Assignee: Reid Chan >Priority: Minor > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-21129.master.001.patch, > HBASE-21129.master.002.patch > > > It is a follow-up of HBASE-19008, aiming to clean up duplicate codes in > #equals and #hashCode methods. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
