[jira] [Commented] (HBASE-19184) clean up nightly source artifact test to match expectations from switch to git-archive
[
https://issues.apache.org/jira/browse/HBASE-19184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245312#comment-16245312
]
Hadoop QA commented on HBASE-19184:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
9s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} shelldocs {color} | {color:blue} 0m
4s{color} | {color:blue} Shelldocs was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
0s{color} | {color:green} There were no new shellcheck issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
10s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 0m 29s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19184 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12896815/HBASE-19184.1.patch |
| Optional Tests | asflicense shellcheck shelldocs |
| uname | Linux 8f9934006e7e 3.13.0-133-generic #182-Ubuntu SMP Tue Sep 19
15:49:21 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / cffbe4bef7 |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| shellcheck | v0.4.4 |
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/9706/console |
| Powered by | Apache Yetus 0.6.0 http://yetus.apache.org |
This message was automatically generated.
> clean up nightly source artifact test to match expectations from switch to
> git-archive
> --
>
> Key: HBASE-19184
> URL: https://issues.apache.org/jira/browse/HBASE-19184
> Project: HBase
> Issue Type: Bug
> Components: build
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Critical
> Attachments: HBASE-19184.0.patch, HBASE-19184.1.patch
>
>
> HBASE-19152 changed recommended way of building RC source artifacts to rely
> on git archive instead of the maven assembly plugin.
> update nightly test to match those expectations..
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-19184) clean up nightly source artifact test to match expectations from switch to git-archive
[ https://issues.apache.org/jira/browse/HBASE-19184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-19184: Attachment: HBASE-19184.1.patch -1 - clarify what steps we're talking about in comments. How's this look [~appy]? > clean up nightly source artifact test to match expectations from switch to > git-archive > -- > > Key: HBASE-19184 > URL: https://issues.apache.org/jira/browse/HBASE-19184 > Project: HBase > Issue Type: Bug > Components: build >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > Attachments: HBASE-19184.0.patch, HBASE-19184.1.patch > > > HBASE-19152 changed recommended way of building RC source artifacts to rely > on git archive instead of the maven assembly plugin. > update nightly test to match those expectations.. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19184) clean up nightly source artifact test to match expectations from switch to git-archive
[ https://issues.apache.org/jira/browse/HBASE-19184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245307#comment-16245307 ] Sean Busbey commented on HBASE-19184: - building the source jar was labeled "step 3" in the ref guide prior to HBASE-19152. afterwards it became "step 6" although it is not properly labeled as such due to a rendering error. > clean up nightly source artifact test to match expectations from switch to > git-archive > -- > > Key: HBASE-19184 > URL: https://issues.apache.org/jira/browse/HBASE-19184 > Project: HBase > Issue Type: Bug > Components: build >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > Attachments: HBASE-19184.0.patch > > > HBASE-19152 changed recommended way of building RC source artifacts to rely > on git archive instead of the maven assembly plugin. > update nightly test to match those expectations.. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19165) TODO Handle stuck in transition: rit=OPENING, location=ve0538....
[
https://issues.apache.org/jira/browse/HBASE-19165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245306#comment-16245306
]
Hadoop QA commented on HBASE-19165:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
11s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
24s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 6m
22s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
48s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
7s{color} | {color:red} hbase-server: The patch generated 2 new + 173 unchanged
- 1 fixed = 175 total (was 174) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
43s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
55m 5s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 2.7.4 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
57s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}107m 58s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
35s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}189m 11s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19165 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12896800/HBASE-19165.master.003.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 24723fc67410 3.13.0-133-generic #182-Ubuntu SMP Tue Sep 19
15:49:21 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / cffbe4bef7 |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| checkstyle |
[jira] [Updated] (HBASE-18661) [HLC] Move clocks to a separate o.a.h.h.clocks package
[ https://issues.apache.org/jira/browse/HBASE-18661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-18661: - Status: Open (was: Patch Available) > [HLC] Move clocks to a separate o.a.h.h.clocks package > -- > > Key: HBASE-18661 > URL: https://issues.apache.org/jira/browse/HBASE-18661 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Amit Patel > Attachments: HBASE-18661.HBASE-14070.HLC.001.patch, > HBASE-18661.HBASE-14070.HLC.002.patch > > > --I think we can move all clocks related code out of h-common and h-server > into a new module which only needs to depend on h-shaded-protocol. If it's > possible, I think it'll be much cleaner code layout and dependency > structure.-- > Edit: let's just move to a separate package in hbase-common for now. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19184) clean up nightly source artifact test to match expectations from switch to git-archive
[ https://issues.apache.org/jira/browse/HBASE-19184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245298#comment-16245298 ] Appy commented on HBASE-19184: -- +1 What was "step 3"? Why is it "step 6" now? I don't see any other step markers. Seems confusing. > clean up nightly source artifact test to match expectations from switch to > git-archive > -- > > Key: HBASE-19184 > URL: https://issues.apache.org/jira/browse/HBASE-19184 > Project: HBase > Issue Type: Bug > Components: build >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > Attachments: HBASE-19184.0.patch > > > HBASE-19152 changed recommended way of building RC source artifacts to rely > on git archive instead of the maven assembly plugin. > update nightly test to match those expectations.. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19222) update jruby to 9.1.14.0+
Sean Busbey created HBASE-19222: --- Summary: update jruby to 9.1.14.0+ Key: HBASE-19222 URL: https://issues.apache.org/jira/browse/HBASE-19222 Project: HBase Issue Type: Sub-task Components: shell Reporter: Sean Busbey JRuby's 9.1.14.0 release is described as "things generally work in jdk9" https://twitter.com/headius/status/928405094407827457 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19200) Remove Registry and use AsyncRegistry directly
[ https://issues.apache.org/jira/browse/HBASE-19200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245282#comment-16245282 ] stack commented on HBASE-19200: --- What about MasterAddressTracker,MetaTableAccessor, ZkAclReset, ZKLeaderManager, etc., that still use ZooKeeperWatcher? Won't these depend on any hbase-zookeeper module we might make? > Remove Registry and use AsyncRegistry directly > -- > > Key: HBASE-19200 > URL: https://issues.apache.org/jira/browse/HBASE-19200 > Project: HBase > Issue Type: Task > Components: Client, Zookeeper >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19200-v1.patch, HBASE-19200-v2.patch, > HBASE-19200.patch > > > So that we can move most of the zookeeper related code out of hbase-client > module. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19200) Remove Registry and use AsyncRegistry directly
[
https://issues.apache.org/jira/browse/HBASE-19200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245281#comment-16245281
]
Hadoop QA commented on HBASE-19200:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
8s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 5 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
24s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 6m
7s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
46s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
12s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
29s{color} | {color:red} hbase-client: The patch generated 1 new + 192
unchanged - 17 fixed = 193 total (was 209) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
32s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
50m 45s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 2.7.4 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
48s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}118m
37s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
37s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}194m 9s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19200 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12896796/HBASE-19200-v2.patch |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 86773b1c831a 3.13.0-133-generic #182-Ubuntu SMP Tue Sep 19
15:49:21 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / cffbe4bef7 |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| checkstyle |
[jira] [Commented] (HBASE-19127) Set State.SPLITTING, MERGING, MERGING_NEW, SPLITTING_NEW properly in RegionStatesNode
[
https://issues.apache.org/jira/browse/HBASE-19127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245276#comment-16245276
]
stack commented on HBASE-19127:
---
[~easyliangjob] This looks ready to commit. Would you agree?
> Set State.SPLITTING, MERGING, MERGING_NEW, SPLITTING_NEW properly in
> RegionStatesNode
> -
>
> Key: HBASE-19127
> URL: https://issues.apache.org/jira/browse/HBASE-19127
> Project: HBase
> Issue Type: Sub-task
>Reporter: Yi Liang
>Assignee: Yi Liang
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19126-v1-master.patch, region_state.patch,
> state.patch
>
>
> In current code, we did not set above states to a region node at all, but we
> still have statements like below to check if node have above states.
> {code}
> else if (!regionNode.isInState(State.CLOSING, State.SPLITTING)) {
>
> }
> {code}
> We need to set above states in a correct place.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-19127) Set State.SPLITTING, MERGING, MERGING_NEW, SPLITTING_NEW properly in RegionStatesNode
[
https://issues.apache.org/jira/browse/HBASE-19127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-19127:
--
Fix Version/s: 2.0.0-beta-1
> Set State.SPLITTING, MERGING, MERGING_NEW, SPLITTING_NEW properly in
> RegionStatesNode
> -
>
> Key: HBASE-19127
> URL: https://issues.apache.org/jira/browse/HBASE-19127
> Project: HBase
> Issue Type: Sub-task
>Reporter: Yi Liang
>Assignee: Yi Liang
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19126-v1-master.patch, region_state.patch,
> state.patch
>
>
> In current code, we did not set above states to a region node at all, but we
> still have statements like below to check if node have above states.
> {code}
> else if (!regionNode.isInState(State.CLOSING, State.SPLITTING)) {
>
> }
> {code}
> We need to set above states in a correct place.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19204) branch-1.2 times out and is taking 6-7 hours to complete
[ https://issues.apache.org/jira/browse/HBASE-19204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245273#comment-16245273 ] stack commented on HBASE-19204: --- New run took 8.43h. Using Yu Li config, it reports the test run successful but secondPart of hbase-server is spew of timeouts and killed forks. The *dump* files talk of Killing self fork JVM.. after getting SHUTDOWN from Maven shutdown hook... > branch-1.2 times out and is taking 6-7 hours to complete > > > Key: HBASE-19204 > URL: https://issues.apache.org/jira/browse/HBASE-19204 > Project: HBase > Issue Type: Umbrella > Components: test >Reporter: stack > > Sean has been looking at tooling and infra. This Umbrellas is about looking > at actual tests. For example, running locally on dedicated machine I picked a > random test, TestPerColumnFamilyFlush. In my test run, it wrote 16M lines. It > seems to be having zk issues but it is catching interrupts and ignoring them > ([~carp84] fixed this in later versions over in HBASE-18441). > Let me try and do some fixup under this umbrella so we can get a 1.2.7 out > the door. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19165) TODO Handle stuck in transition: rit=OPENING, location=ve0538....
[ https://issues.apache.org/jira/browse/HBASE-19165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245270#comment-16245270 ] ramkrishna.s.vasudevan commented on HBASE-19165: Bad eyes. Now got it. Thanks. LGTM. > TODO Handle stuck in transition: rit=OPENING, location=ve0538 > - > > Key: HBASE-19165 > URL: https://issues.apache.org/jira/browse/HBASE-19165 > Project: HBase > Issue Type: Bug > Components: migration >Reporter: stack >Assignee: stack >Priority: Critical > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19165.master.001.patch, > HBASE-19165.master.002.patch, HBASE-19165.master.003.patch, > HBASE-19165.master.003.patch > > > A few of us trying migration from hbase1 to hbase2 have run into various > assignment issues. In my case, I see that Master coming online, reading > hbase:meta, finding an issue in OPENING state and then just not assigning. My > thought is that its expecting a callback to come in. Eventually log is fills > with this sort of stuff: > 2017-11-02 15:02:54,237 WARN [ProcExecTimeout] assignment.AssignmentManager: > TODO Handle stuck in transition: rit=OPENING, > location=ve0528.halxg.cloudera.com,16020,1509657692581, > table=IntegrationTestBigLinkedList, region=919cc6636ffbed17f628f335d3a58726 > Let me get a cleaner run. This current log is polluted by a few restarts. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19147) All branch-2 unit tests pass
[ https://issues.apache.org/jira/browse/HBASE-19147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245267#comment-16245267 ] stack commented on HBASE-19147: --- New run. 7.21h. Tests run: 2665, Failures: 24, Errors: 65, Skipped: 42 .. which is a good few more tests than ran above. Nightly says ' Tests run: 2039, Failures: 1, Errors: 0, Skipped: 24, Flakes: 2' for second part (looking at a few ... they are all just over 2k for second part of hbase-server). > All branch-2 unit tests pass > > > Key: HBASE-19147 > URL: https://issues.apache.org/jira/browse/HBASE-19147 > Project: HBase > Issue Type: Bug > Components: test >Reporter: stack >Priority: Blocker > Fix For: 2.0.0-beta-1 > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19200) Remove Registry and use AsyncRegistry directly
[ https://issues.apache.org/jira/browse/HBASE-19200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245265#comment-16245265 ] Duo Zhang commented on HBASE-19200: --- We still need zk in hbase-client, we need to get meta location and master address. But now client has its own logic to get the information, very simple, so we do not depend the ZooKeeperWatcher any more. The stuffs can then be moved to another module without messing the dependency tree of hbase-client. Thanks. > Remove Registry and use AsyncRegistry directly > -- > > Key: HBASE-19200 > URL: https://issues.apache.org/jira/browse/HBASE-19200 > Project: HBase > Issue Type: Task > Components: Client, Zookeeper >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19200-v1.patch, HBASE-19200-v2.patch, > HBASE-19200.patch > > > So that we can move most of the zookeeper related code out of hbase-client > module. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (HBASE-19089) Fix the list of included moduleSets in src and binary tars
[ https://issues.apache.org/jira/browse/HBASE-19089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245264#comment-16245264 ] Appy edited comment on HBASE-19089 at 11/9/17 6:24 AM: --- Yeah. Waiting for HBASE-19184 to finish, so i have some way to test the patch. Then i'll get back to it. :) was (Author: appy): Waiting for HBASE-19184 to finish, so i can test the patch. :) > Fix the list of included moduleSets in src and binary tars > -- > > Key: HBASE-19089 > URL: https://issues.apache.org/jira/browse/HBASE-19089 > Project: HBase > Issue Type: Bug > Components: build >Reporter: Appy >Assignee: Appy >Priority: Blocker > Fix For: 3.0.0, 2.0.0-beta-1 > > Attachments: HBASE-19089.master.001.patch, > HBASE-19089.master.002.patch > > > List of moduleSets included in src.xml and hadoop-two-compat.xml differ quite > a lot. Particularly, hadoop-two-compat.xml is missing quite a few modules. > The core issue is duplication involved in list. Let me try to get > rid of it by using a shared list and including it using -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19089) Fix the list of included moduleSets in src and binary tars
[ https://issues.apache.org/jira/browse/HBASE-19089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245264#comment-16245264 ] Appy commented on HBASE-19089: -- Waiting for HBASE-19184 to finish, so i can test the patch. :) > Fix the list of included moduleSets in src and binary tars > -- > > Key: HBASE-19089 > URL: https://issues.apache.org/jira/browse/HBASE-19089 > Project: HBase > Issue Type: Bug > Components: build >Reporter: Appy >Assignee: Appy >Priority: Blocker > Fix For: 3.0.0, 2.0.0-beta-1 > > Attachments: HBASE-19089.master.001.patch, > HBASE-19089.master.002.patch > > > List of moduleSets included in src.xml and hadoop-two-compat.xml differ quite > a lot. Particularly, hadoop-two-compat.xml is missing quite a few modules. > The core issue is duplication involved in list. Let me try to get > rid of it by using a shared list and including it using -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19220) Async tests time out talking to zk; 'clusterid came back null'
[ https://issues.apache.org/jira/browse/HBASE-19220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245261#comment-16245261 ] Duo Zhang commented on HBASE-19220: --- Is it a good idea to change a config in normal code path which only aims to make a UT pass? Anyway, I think we can commit this first to make our UTs more stable. And open another issue to tune the retry configs. Thanks. > Async tests time out talking to zk; 'clusterid came back null' > -- > > Key: HBASE-19220 > URL: https://issues.apache.org/jira/browse/HBASE-19220 > Project: HBase > Issue Type: Sub-task > Components: test >Reporter: stack >Assignee: stack > Fix For: 2.0.0-beta-1 > > Attachments: 19220.patch > > > I see this in test runs on a dedicated machine: > [ERROR] Tests run: 6, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: > 652.514 s <<< FAILURE! - in > org.apache.hadoop.hbase.client.TestAsyncAdminBuilder > [ERROR] > testRpcTimeout[0](org.apache.hadoop.hbase.client.TestAsyncAdminBuilder) Time > elapsed: 213.618 s <<< ERROR! > java.util.concurrent.ExecutionException: java.io.IOException: clusterid came > back null > at > org.apache.hadoop.hbase.client.TestAsyncAdminBuilder.testRpcTimeout(TestAsyncAdminBuilder.java:105) > Caused by: java.io.IOException: clusterid came back null > [ERROR] org.apache.hadoop.hbase.client.TestAsyncTableScanMetrics Time > elapsed: 0.007 s <<< ERROR! > java.util.concurrent.ExecutionException: java.io.IOException: clusterid came > back null > at > org.apache.hadoop.hbase.client.TestAsyncTableScanMetrics.setUp(TestAsyncTableScanMetrics.java:97) > Caused by: java.io.IOException: clusterid came back null > [ERROR] org.apache.hadoop.hbase.client.TestRawAsyncScanCursor Time elapsed: > 0.005 s <<< ERROR! > java.util.concurrent.ExecutionException: java.io.IOException: clusterid came > back null > at > org.apache.hadoop.hbase.client.TestRawAsyncScanCursor.setUpBeforeClass(TestRawAsyncScanCursor.java:42) > Caused by: java.io.IOException: clusterid came back null > [ERROR] org.apache.hadoop.hbase.client.TestAsyncNamespaceAdminApi Time > elapsed: 0.005 s <<< ERROR! > java.util.concurrent.ExecutionException: java.io.IOException: clusterid came > back null > at > org.apache.hadoop.hbase.client.TestAsyncNamespaceAdminApi.setUpBeforeClass(TestAsyncNamespaceAdminApi.java:66) > Caused by: java.io.IOException: clusterid came back null > If I up the retries, they go away. > At least on this machine, I notice that zk connections can take a while... > see HBASE-19102 where we add a wait on the Connection to come up before > progressing. > Suggest that I up the retries. No harm in trying more. It is currently set to > 3 retries at a one second interval. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19221) NoClassDefFoundError: org/hamcrest/SelfDescribing while running IT tests in 2.0-alpha4
ramkrishna.s.vasudevan created HBASE-19221:
--
Summary: NoClassDefFoundError: org/hamcrest/SelfDescribing while
running IT tests in 2.0-alpha4
Key: HBASE-19221
URL: https://issues.apache.org/jira/browse/HBASE-19221
Project: HBase
Issue Type: Bug
Affects Versions: 2.0.0-alpha-3
Reporter: ramkrishna.s.vasudevan
Priority: Blocker
Fix For: 2.0.0-beta-1
Copying the mail from the dev@
{code}
I tried running some IT test cases using the alpha-4 RC. I found this issue
Exception in thread "main" java.lang.NoClassDefFoundError:
org/hamcrest/SelfDescribing
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:760)
at
java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
...
at
org.apache.hadoop.hbase.IntegrationTestsDriver.doWork(IntegrationTestsDriver.java:111)
at
org.apache.hadoop.hbase.util.AbstractHBaseTool.run(AbstractHBaseTool.java:154)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at
org.apache.hadoop.hbase.IntegrationTestsDriver.main(IntegrationTestsDriver.java:47)
The same when run against latest master it runs without any issues
{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (HBASE-12091) Optionally ignore edits for dropped tables for replication.
[ https://issues.apache.org/jira/browse/HBASE-12091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245249#comment-16245249 ] Lars Hofhansl edited comment on HBASE-12091 at 11/9/17 6:20 AM: bq. Is there something that prevents this? Not that I am aware of. I generally don't know how we send exceptions back to the clients. Looks like it's going through Protobufs builtin ServiceException, which is thrown internally. Don't like the parsing either. At least now the sink controls exactly how the exception is thrown, so it should be safe. was (Author: lhofhansl): bq. Is there something that prevents this? Not that I am aware of. I generally don't know how we send exceptions back to the clients. Will take a look. Don't like the parsing either. At least now the sink controls exactly how the exception is thrown, so it should be safe. > Optionally ignore edits for dropped tables for replication. > --- > > Key: HBASE-12091 > URL: https://issues.apache.org/jira/browse/HBASE-12091 > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Fix For: 2.0.0, 3.0.0, 1.4.1, 1.5.0 > > Attachments: 12091-branch-2.patch, 12091-v2-branch-1.txt, > 12091-v3-branch-1.txt, 12091-v4-branch-1.txt, 12091-v5-branch-1.txt, > 12091-v6-branch-1.txt, 12091-v7-branch-1.txt, 12091-v7-master.txt, > 12091-v8-master.txt, 12091.txt > > > We just ran into a scenario where we dropped a table from both the source and > the sink, but the source still has outstanding edits that now it could not > get rid of. Now all replication is backed up behind these unreplicatable > edits. > We should have an option to ignore edits for tables dropped at the source. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19200) Remove Registry and use AsyncRegistry directly
[
https://issues.apache.org/jira/browse/HBASE-19200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245259#comment-16245259
]
Appy commented on HBASE-19200:
--
Yeah. And it's nice too!
I don't think it's purging everything though. MetaTableLocator is still using
ZooKeeperWatcher.
But we only use MTL#{getMetaRegions, getMetaRegionsAndLocations} in
hbase-client, and looks like [~Apache9] already has
AsyncRegistry#getMetaRegionLocation to tackle it.
> Remove Registry and use AsyncRegistry directly
> --
>
> Key: HBASE-19200
> URL: https://issues.apache.org/jira/browse/HBASE-19200
> Project: HBase
> Issue Type: Task
> Components: Client, Zookeeper
>Reporter: Duo Zhang
>Assignee: Duo Zhang
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19200-v1.patch, HBASE-19200-v2.patch,
> HBASE-19200.patch
>
>
> So that we can move most of the zookeeper related code out of hbase-client
> module.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19089) Fix the list of included moduleSets in src and binary tars
[ https://issues.apache.org/jira/browse/HBASE-19089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245256#comment-16245256 ] stack commented on HBASE-19089: --- This need a rebase [~appy] (especially after the spark purge)? > Fix the list of included moduleSets in src and binary tars > -- > > Key: HBASE-19089 > URL: https://issues.apache.org/jira/browse/HBASE-19089 > Project: HBase > Issue Type: Bug > Components: build >Reporter: Appy >Assignee: Appy >Priority: Blocker > Fix For: 3.0.0, 2.0.0-beta-1 > > Attachments: HBASE-19089.master.001.patch, > HBASE-19089.master.002.patch > > > List of moduleSets included in src.xml and hadoop-two-compat.xml differ quite > a lot. Particularly, hadoop-two-compat.xml is missing quite a few modules. > The core issue is duplication involved in list. Let me try to get > rid of it by using a shared list and including it using -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (HBASE-13631) Migration from 0.94 to 2.0.0
[ https://issues.apache.org/jira/browse/HBASE-13631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-13631. --- Resolution: Won't Fix Resolving as "won't do" going of discussion up on dev list [1] 1. http://search-hadoop.com/m/HBase/YGbbQPjXy1CZQSf2?subj=DISCUSSION+Minimum+hbase1+version+from+which+you+can+upgrade+to+hbase2+1+2+x+ > Migration from 0.94 to 2.0.0 > > > Key: HBASE-13631 > URL: https://issues.apache.org/jira/browse/HBASE-13631 > Project: HBase > Issue Type: Sub-task > Components: migration >Reporter: Anoop Sam John >Priority: Blocker > Fix For: 2.0.0-beta-1 > > > We have HFile V2 (minor version-2) only in 0.94 and 2.0 needs HFile V3 with > minor version 3 atleast. We can test and document clearly the path of upgrade > from 94.x to 2.0.0 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19165) TODO Handle stuck in transition: rit=OPENING, location=ve0538....
[ https://issues.apache.org/jira/browse/HBASE-19165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245248#comment-16245248 ] stack commented on HBASE-19165: --- The 'fix' is EnableTableProcedure calling getRegionsOfTable passing the 'true' flag so it will also assign regions that are OFFLINE (previous it would only assign regions that were NOT OFFLINE which worked when we defaulted empty region info state as OPENING). Thanks for taking a look [~ram_krish] > TODO Handle stuck in transition: rit=OPENING, location=ve0538 > - > > Key: HBASE-19165 > URL: https://issues.apache.org/jira/browse/HBASE-19165 > Project: HBase > Issue Type: Bug > Components: migration >Reporter: stack >Assignee: stack >Priority: Critical > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19165.master.001.patch, > HBASE-19165.master.002.patch, HBASE-19165.master.003.patch, > HBASE-19165.master.003.patch > > > A few of us trying migration from hbase1 to hbase2 have run into various > assignment issues. In my case, I see that Master coming online, reading > hbase:meta, finding an issue in OPENING state and then just not assigning. My > thought is that its expecting a callback to come in. Eventually log is fills > with this sort of stuff: > 2017-11-02 15:02:54,237 WARN [ProcExecTimeout] assignment.AssignmentManager: > TODO Handle stuck in transition: rit=OPENING, > location=ve0528.halxg.cloudera.com,16020,1509657692581, > table=IntegrationTestBigLinkedList, region=919cc6636ffbed17f628f335d3a58726 > Let me get a cleaner run. This current log is polluted by a few restarts. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-12091) Optionally ignore edits for dropped tables for replication.
[ https://issues.apache.org/jira/browse/HBASE-12091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245249#comment-16245249 ] Lars Hofhansl commented on HBASE-12091: --- bq. Is there something that prevents this? Not that I am aware of. I generally don't know how we send exceptions back to the clients. Will take a look. Don't like the parsing either. At least now the sink controls exactly how the exception is thrown, so it should be safe. > Optionally ignore edits for dropped tables for replication. > --- > > Key: HBASE-12091 > URL: https://issues.apache.org/jira/browse/HBASE-12091 > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Fix For: 2.0.0, 3.0.0, 1.4.1, 1.5.0 > > Attachments: 12091-branch-2.patch, 12091-v2-branch-1.txt, > 12091-v3-branch-1.txt, 12091-v4-branch-1.txt, 12091-v5-branch-1.txt, > 12091-v6-branch-1.txt, 12091-v7-branch-1.txt, 12091-v7-master.txt, > 12091-v8-master.txt, 12091.txt > > > We just ran into a scenario where we dropped a table from both the source and > the sink, but the source still has outstanding edits that now it could not > get rid of. Now all replication is backed up behind these unreplicatable > edits. > We should have an option to ignore edits for tables dropped at the source. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19002) Introduce more examples to show how to intercept normal region operations
[ https://issues.apache.org/jira/browse/HBASE-19002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245243#comment-16245243 ] stack commented on HBASE-19002: --- Really great. +1. Fat release note talking up these fancy, new examples. > Introduce more examples to show how to intercept normal region operations > - > > Key: HBASE-19002 > URL: https://issues.apache.org/jira/browse/HBASE-19002 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Duo Zhang >Assignee: Josh Elser >Priority: Minor > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19002.001.branch-2.patch, > HBASE-19002.002.branch-2.patch, HBASE-19002.003.branch-2.patch, > HBASE-19002.004.branch-2.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-12091) Optionally ignore edits for dropped tables for replication.
[ https://issues.apache.org/jira/browse/HBASE-12091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-12091: -- Attachment: 12091-v8-master.txt -v8 has the test back, and also fixes the comment. > Optionally ignore edits for dropped tables for replication. > --- > > Key: HBASE-12091 > URL: https://issues.apache.org/jira/browse/HBASE-12091 > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Fix For: 2.0.0, 3.0.0, 1.4.1, 1.5.0 > > Attachments: 12091-branch-2.patch, 12091-v2-branch-1.txt, > 12091-v3-branch-1.txt, 12091-v4-branch-1.txt, 12091-v5-branch-1.txt, > 12091-v6-branch-1.txt, 12091-v7-branch-1.txt, 12091-v7-master.txt, > 12091-v8-master.txt, 12091.txt > > > We just ran into a scenario where we dropped a table from both the source and > the sink, but the source still has outstanding edits that now it could not > get rid of. Now all replication is backed up behind these unreplicatable > edits. > We should have an option to ignore edits for tables dropped at the source. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19200) Remove Registry and use AsyncRegistry directly
[ https://issues.apache.org/jira/browse/HBASE-19200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245236#comment-16245236 ] stack commented on HBASE-19200: --- Patch is radical. Is it a purge of all zk from hbase-client? HBASE-19114 could go in after this is committed? > Remove Registry and use AsyncRegistry directly > -- > > Key: HBASE-19200 > URL: https://issues.apache.org/jira/browse/HBASE-19200 > Project: HBase > Issue Type: Task > Components: Client, Zookeeper >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19200-v1.patch, HBASE-19200-v2.patch, > HBASE-19200.patch > > > So that we can move most of the zookeeper related code out of hbase-client > module. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18624) Added support for clearing BlockCache based on table name
[ https://issues.apache.org/jira/browse/HBASE-18624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245235#comment-16245235 ] Anoop Sam John commented on HBASE-18624: [~zyork] Release Notes pls? > Added support for clearing BlockCache based on table name > - > > Key: HBASE-18624 > URL: https://issues.apache.org/jira/browse/HBASE-18624 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0, 1.3.0 >Reporter: Ajay Jadhav >Assignee: Zach York > Fix For: 2.0.0, 3.0.0 > > Attachments: HBASE-18624.branch-1.001.patch, > HBASE-18624.master.001.patch, HBASE-18624.master.002.patch, > HBASE-18624.master.003.patch, HBASE-18624.master.004.patch, > HBASE-18624.master.005.patch, HBASE-18624.master.006.patch, > HBASE-18624.master.007.patch, HBASE-18624.master.008.patch, > HBASE-18624.master.009.patch, HBASE-18624.master.010.patch, > HBASE-18624.master.011.patch, HBASE-18624.master.012.patch, > HBASE-18624.master.013.patch > > > Bulk loading the primary HBase cluster triggers a lot of compactions > resulting in archival/ creation > of multiple HFiles. This process will cause a lot of items to become stale in > replica’s BlockCache. > This patch will help users to clear the block cache for a given table by > either using shell or API. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19194) TestRSGroupsBase has some always false checks
[
https://issues.apache.org/jira/browse/HBASE-19194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245233#comment-16245233
]
Lars Hofhansl commented on HBASE-19194:
---
Belated +1
> TestRSGroupsBase has some always false checks
> -
>
> Key: HBASE-19194
> URL: https://issues.apache.org/jira/browse/HBASE-19194
> Project: HBase
> Issue Type: Bug
> Components: rsgroup, test
>Reporter: Mike Drob
>Assignee: Andrew Purtell
>Priority: Blocker
> Fix For: 2.0.0, 3.0.0, 1.4.0
>
> Attachments: HBASE-19194-branch-1.patch, HBASE-19194.patch
>
>
> Discovered via error-prone analysis:
> {noformat}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.6.1:testCompile
> (default-testCompile) on project hbase-rsgroup: Compilation failure:
> Compilation failure:
> [ERROR]
> /Users/mdrob/IdeaProjects/hbase/hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup/TestRSGroupsBase.java:[421,40]
> [CollectionIncompatibleType] Argument 'region' should not be passed to this
> method; its type RegionInfo is not compatible with its collection's type
> argument String
> [ERROR] (see http://errorprone.info/bugpattern/CollectionIncompatibleType)
> [ERROR]
> /Users/mdrob/IdeaProjects/hbase/hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup/TestRSGroupsBase.java:[531,65]
> [CollectionIncompatibleType] Argument 'targetRegion' should not be passed to
> this method; its type String is not compatible with its collection's type
> argument RegionInfo
> [ERROR] (see http://errorprone.info/bugpattern/CollectionIncompatibleType)
> {noformat}
> Both of these are checks to {{List.contains}} with an element of the wrong
> type so they will always return false. The first is an optimization that we
> can probably live without (or short circuit on the Master, at least) the
> second is an assertion that will never fail, rendering the test ineffective.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19220) Async tests time out talking to zk; 'clusterid came back null'
[
https://issues.apache.org/jira/browse/HBASE-19220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245228#comment-16245228
]
Hadoop QA commented on HBASE-19220:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
7s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
5s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
37s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
49m 35s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 2.7.4 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
37s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
9s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 68m 39s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19220 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12896804/19220.patch |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 33940b80ef46 3.13.0-129-generic #178-Ubuntu SMP Fri Aug 11
12:48:20 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / cffbe4bef7 |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/9704/testReport/ |
| modules | C: hbase-client U: hbase-client |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/9704/console |
| Powered by | Apache Yetus 0.6.0 http://yetus.apache.org |
This message was automatically generated.
> Async tests time out talking to zk; 'clusterid came back null'
>
[jira] [Commented] (HBASE-19114) Split out o.a.h.h.zookeeper from hbase-server and hbase-client
[ https://issues.apache.org/jira/browse/HBASE-19114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245223#comment-16245223 ] stack commented on HBASE-19114: --- H. I like this patch (Not sure about hbase-common depending on thirdparty...). Moving stuff out of hbase-server is an important project. It seems like new zk module would have 7 files as I count it (one of which would be the pom). [~Apache9] would like to avoid hbase-client having to depend on hbase-zookeeper in 2.0.0. I like the plan above for disentangling hbase-client. Move this out to hbase3 [~appy]? (There is good stuff in this patch that could go in apart from the making of the hbase-zookeeper module... do in separate JIRA?) > Split out o.a.h.h.zookeeper from hbase-server and hbase-client > -- > > Key: HBASE-19114 > URL: https://issues.apache.org/jira/browse/HBASE-19114 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Appy > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19114.master.001.patch, > HBASE-19114.master.002.patch, HBASE-19114.master.003.patch, > HBASE-19114.master.004.patch, HBASE-19114.master.005.patch > > > Changes so far: > - Moved DrainingServerTracker and RegionServerTracker to > hbase-server:o.a.h.h.master. > - Move Abortable to hbase-common. Since it's IA.Private and independent of > anything, moving it to hbase-common which is at bottom of the dependency tree > is better. > - Moved RecoveringRegionWatcher to hbase-server:o.a.h.h.regionserver > - Moved SplitOrMergeTracker to oahh.master (because it depends on a PB) > - Moving hbase-client:oahh.zookeeper.* to hbase-zookeeper module. We want to > keep hbase-zookeeper very independent and hence at lowest levels in our > dependency tree. > - ZKUtil is a huge tangle since it's linked to almost everything in > \[hbase-client/]oahh.zookeeper. And pulling it down requires some basic proto > functions (mergeFrom, PBmagic, etc). So what i did was: >** Pulled down common and basic protobuf functions (which only depend on > com.google.protobuf.\*) to hbase-common so other code depending on them can > be pulled down if possible/wanted in future. This will help future dependency > untangling too. These are ProtobufMagic and ProtobufHelpers. > ** Didn't move any hbase-specific PB stuff to hbase-common. We can't pull > things into hbase-common which add dependency between it and > hbase-protobuf/hbase-shaded-protobuf since we very recently untangled them. > - DEFAULT_REPLICA_ID is used in many places in ZK. Declared a new constant in > HConstants (since it's in hbase-common) and using it in hbase-zookeeper. > RegionInfo.DEFAULT_REPLICA_ID also takes its value from it to avoid case > where two values can become different. > - Renamed some classes to use a consistent naming for classes - ZK instead of > mix of ZK, Zk , ZooKeeper. Couldn't rename following public classes: > MiniZooKeeperCluster, ZooKeeperConnectionException. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18090) Improve TableSnapshotInputFormat to allow more multiple mappers per region
[ https://issues.apache.org/jira/browse/HBASE-18090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xinxin fan updated HBASE-18090: --- Attachment: (was: HBASE-18090.branch-1.patch) > Improve TableSnapshotInputFormat to allow more multiple mappers per region > -- > > Key: HBASE-18090 > URL: https://issues.apache.org/jira/browse/HBASE-18090 > Project: HBase > Issue Type: Improvement > Components: mapreduce >Reporter: Mikhail Antonov >Assignee: xinxin fan > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-18090-V3-master.patch, > HBASE-18090-V4-master.patch, HBASE-18090-V5-master.patch, > HBASE-18090-branch-1.3-v1.patch, HBASE-18090-branch-1.3-v2.patch > > > TableSnapshotInputFormat runs one map task per region in the table snapshot. > This places unnecessary restriction that the region layout of the original > table needs to take the processing resources available to MR job into > consideration. Allowing to run multiple mappers per region (assuming > reasonably even key distribution) would be useful. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18090) Improve TableSnapshotInputFormat to allow more multiple mappers per region
[ https://issues.apache.org/jira/browse/HBASE-18090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xinxin fan updated HBASE-18090: --- Attachment: HBASE-18090.branch-1.patch > Improve TableSnapshotInputFormat to allow more multiple mappers per region > -- > > Key: HBASE-18090 > URL: https://issues.apache.org/jira/browse/HBASE-18090 > Project: HBase > Issue Type: Improvement > Components: mapreduce >Reporter: Mikhail Antonov >Assignee: xinxin fan > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-18090-V3-master.patch, > HBASE-18090-V4-master.patch, HBASE-18090-V5-master.patch, > HBASE-18090-branch-1.3-v1.patch, HBASE-18090-branch-1.3-v2.patch, > HBASE-18090.branch-1.patch > > > TableSnapshotInputFormat runs one map task per region in the table snapshot. > This places unnecessary restriction that the region layout of the original > table needs to take the processing resources available to MR job into > consideration. Allowing to run multiple mappers per region (assuming > reasonably even key distribution) would be useful. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18090) Improve TableSnapshotInputFormat to allow more multiple mappers per region
[ https://issues.apache.org/jira/browse/HBASE-18090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xinxin fan updated HBASE-18090: --- Attachment: (was: HBASE-18090.branch-1-V2.patch) > Improve TableSnapshotInputFormat to allow more multiple mappers per region > -- > > Key: HBASE-18090 > URL: https://issues.apache.org/jira/browse/HBASE-18090 > Project: HBase > Issue Type: Improvement > Components: mapreduce >Reporter: Mikhail Antonov >Assignee: xinxin fan > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-18090-V3-master.patch, > HBASE-18090-V4-master.patch, HBASE-18090-V5-master.patch, > HBASE-18090-branch-1.3-v1.patch, HBASE-18090-branch-1.3-v2.patch > > > TableSnapshotInputFormat runs one map task per region in the table snapshot. > This places unnecessary restriction that the region layout of the original > table needs to take the processing resources available to MR job into > consideration. Allowing to run multiple mappers per region (assuming > reasonably even key distribution) would be useful. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19199) RatioBasedCompactionPolicy#shouldPerformMajorCompaction() always return true when only one file needs to compact
[
https://issues.apache.org/jira/browse/HBASE-19199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245222#comment-16245222
]
Hadoop QA commented on HBASE-19199:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 3m
49s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
47s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
0s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
52m 54s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 2.7.4 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}129m 47s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
24s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}207m 0s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19199 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12896786/HBASE-19199.patch |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux c33d6e5d4814 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26
18:04:35 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / cffbe4bef7 |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/9700/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/9700/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/9700/console |
| Powered by | Apache Yetus 0.6.0 http://yetus.apache.org
[jira] [Commented] (HBASE-19211) B: update configuration string in BackupRestoreConstants
[ https://issues.apache.org/jira/browse/HBASE-19211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245212#comment-16245212 ] Hudson commented on HBASE-19211: FAILURE: Integrated in Jenkins build HBase-2.0 #822 (See [https://builds.apache.org/job/HBase-2.0/822/]) HBASE-19211 B: update configuration string in BackupRestoreConstants (tedyu: rev 25c2b7bf73dc5db66c3ba6c057d39cdaddd80ae8) * (edit) hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/BackupRestoreConstants.java HBASE-19211 B: update configuration string in BackupRestoreConstants (tedyu: rev 6d88c495f77c6b7cd17556c87177a223f0c8afd6) * (edit) hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/BackupRestoreConstants.java > B: update configuration string in BackupRestoreConstants > -- > > Key: HBASE-19211 > URL: https://issues.apache.org/jira/browse/HBASE-19211 > Project: HBase > Issue Type: Bug >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Minor > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19211-v1.patch, HBASE-19211-v2.patch > > > To include custom region observer implementation for tracking bulk loading > events. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19211) B: update configuration string in BackupRestoreConstants
[ https://issues.apache.org/jira/browse/HBASE-19211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245211#comment-16245211 ] Hudson commented on HBASE-19211: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4022 (See [https://builds.apache.org/job/HBase-Trunk_matrix/4022/]) HBASE-19211 B: update configuration string in BackupRestoreConstants (tedyu: rev a5a38b79166d83db37491b2740711ee655ed9b63) * (edit) hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/BackupRestoreConstants.java HBASE-19211 B: update configuration string in BackupRestoreConstants (tedyu: rev cffbe4bef719af972af4a954e3f8de3b396c56bf) * (edit) hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/BackupRestoreConstants.java > B: update configuration string in BackupRestoreConstants > -- > > Key: HBASE-19211 > URL: https://issues.apache.org/jira/browse/HBASE-19211 > Project: HBase > Issue Type: Bug >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Minor > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19211-v1.patch, HBASE-19211-v2.patch > > > To include custom region observer implementation for tracking bulk loading > events. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18817) Pull hbase-spark module out of branch-2
[ https://issues.apache.org/jira/browse/HBASE-18817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-18817: Resolution: Fixed Status: Resolved (was: Patch Available) > Pull hbase-spark module out of branch-2 > --- > > Key: HBASE-18817 > URL: https://issues.apache.org/jira/browse/HBASE-18817 > Project: HBase > Issue Type: Task > Components: spark >Affects Versions: 2.0.0-alpha-2 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Blocker > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-18817-branch-2.v0.patch, > HBASE-18817-branch-2.v1.patch > > > see DISCUSS here: > https://s.apache.org/UJAf > Sadly, feature is slipping out of branch-2. We can work out inclusion for > downstream once we have some inertia again. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18911) Unify Admin and AsyncAdmin's methods name
[ https://issues.apache.org/jira/browse/HBASE-18911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245205#comment-16245205 ] stack commented on HBASE-18911: --- What you think of this mighty [~psomogyi] sir? > Unify Admin and AsyncAdmin's methods name > - > > Key: HBASE-18911 > URL: https://issues.apache.org/jira/browse/HBASE-18911 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-18911.master.001.patch > > > Different Methods > || AsyncAdmin || Admin || unified name || > | listTables | listTableDescriptors | listTableDescriptors | > | getOnlineRegions | getRegions | getRegions | > | getTableRegions | getRegions | getRegions | > | getTableDescriptor | getDescriptor | getDescriptor | > | getRegionLoads | getRegionLoad | getRegionLoads | > | execProcedureWithRet | execProcedureWithReturn | execProcedureWithReturn | > | setNormalizerOn | normalizerSwitch | normalizerSwitch | > | isNormalizerOn | isNormalizerEnabled | isNormalizerEnabled | > | setBalancerOn | balancerSwitch | balancerSwitch | > | isBalancerOn | isBalancerEnabled | isBalancerEnabled | > | setCleanerChoreOn | cleanerChoreSwitch | cleanerChoreSwitch | > | isCleanerChoreOn | isCleanerChoreEnabled | isCleanerChoreEnabled | > | setCatalogJanitorOn | catalogJanitorSwitch | catalogJanitorSwitch | > | isCatalogJanitorOn | isCatalogJanitorEnabled | isCatalogJanitorEnabled | > | setSplitOn/setMergeOn | splitOrMergeEnabledSwitch | splitSwitch/mergeSwitch > | > | isSplitOn/isMergeOn| isSplitOrMergeEnabled | isSplitEnabled/isMergeEnabled | > Methods only in AsyncAdmin > || AsyncAdmin || > | majorCompactRegionServer | > | getMaster | > | getBackupMasters | > | getRegionServers | > Methods only in Admin > || Admin || > | listTableDescriptorsByNamespace | > | listTableNamesByNamespace | > | modifyTable | > | getMasterCoprocessors | -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Work stopped] (HBASE-18381) HBase regionserver crashes when reading column qualifier >64MB
[
https://issues.apache.org/jira/browse/HBASE-18381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HBASE-18381 stopped by stack.
-
> HBase regionserver crashes when reading column qualifier >64MB
> --
>
> Key: HBASE-18381
> URL: https://issues.apache.org/jira/browse/HBASE-18381
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 2.0.0-alpha-1
> Environment: HBase 1.2.0-cdh5.10.0
>Reporter: Daniel Jelinski
>Assignee: stack
>Priority: Critical
> Fix For: 2.0.0-beta-1
>
> Attachments: 18381.v1.txt, 18381.v2.txt
>
>
> After putting a cell with 64MB column qualifier to a MOB-enabled table,
> region server crashes when flushing data. Subsequent WAL replay attempts also
> result in region server crashes.
> Gist of code used to create the table:
> private String table = "poisonPill";
> private byte[] familyBytes = Bytes.toBytes("cf");
> private void createTable(Connection conn) throws IOException {
>Admin hbase_admin = conn.getAdmin();
>HTableDescriptor htable = new HTableDescriptor(TableName.valueOf(table));
>HColumnDescriptor hfamily = new HColumnDescriptor(familyBytes);
>hfamily.setMobEnabled(true);
>htable.setConfiguration("hfile.format.version","3");
>htable.addFamily(hfamily);
>hbase_admin.createTable(htable);
> }
> private void killTable(Connection conn) throws IOException {
>Table tbl = conn.getTable(TableName.valueOf(table));
>byte[] data = new byte[1<<26];
>byte[] smalldata = new byte[0];
>Put put = new Put(Bytes.toBytes("1"));
>put.addColumn(familyBytes, data, smalldata);
>tbl.put(put);
> }
> Region server logs (redacted):
> 2017-07-11 09:34:53,747 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Flushing 1/1 column families, memstore=64.00 MB; WAL is null, using passed
> sequenceid=7
> 2017-07-11 09:34:53,757 WARN org.apache.hadoop.hbase.io.hfile.HFileWriterV2:
> A minimum HFile version of 3 is required to support cell attributes/tags.
> Consider setting hfile.format.version accordingly.
> 2017-07-11 09:34:54,504 INFO
> org.apache.hadoop.hbase.mob.DefaultMobStoreFlusher: Flushed, sequenceid=7,
> memsize=67109096, hasBloomFilter=true, into tmp file
> hdfs://sandbox/hbase/data/default/poisonPill/f82e20f32302dfdd95c89ecc3be5a211/.tmp/7858d223eddd4199ad220fc77bb612eb
> 2017-07-11 09:34:54,694 ERROR org.apache.hadoop.hbase.regionserver.HStore:
> Failed to open store file :
> hdfs://sandbox/hbase/data/default/poisonPill/f82e20f32302dfdd95c89ecc3be5a211/.tmp/7858d223eddd4199ad220fc77bb612eb,
> keeping it in tmp location
> org.apache.hadoop.hbase.io.hfile.CorruptHFileException: Problem reading HFile
> Trailer from file
> hdfs://sandbox/hbase/data/default/poisonPill/f82e20f32302dfdd95c89ecc3be5a211/.tmp/7858d223eddd4199ad220fc77bb612eb
> at
> org.apache.hadoop.hbase.io.hfile.HFile.pickReaderVersion(HFile.java:497)
> at org.apache.hadoop.hbase.io.hfile.HFile.createReader(HFile.java:525)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile$Reader.(StoreFile.java:1105)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileInfo.open(StoreFileInfo.java:265)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.open(StoreFile.java:404)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:509)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:499)
> at
> org.apache.hadoop.hbase.regionserver.HStore.createStoreFileAndReader(HStore.java:675)
> at
> org.apache.hadoop.hbase.regionserver.HStore.createStoreFileAndReader(HStore.java:667)
> at
> org.apache.hadoop.hbase.regionserver.HStore.validateStoreFile(HStore.java:1746)
> at
> org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:942)
> at
> org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.flushCache(HStore.java:2299)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2372)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2102)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.replayRecoveredEdits(HRegion.java:4139)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.replayRecoveredEditsIfAny(HRegion.java:3934)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionInternals(HRegion.java:828)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:799)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6480)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6441)
> at
>
[jira] [Assigned] (HBASE-18381) HBase regionserver crashes when reading column qualifier >64MB
[
https://issues.apache.org/jira/browse/HBASE-18381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack reassigned HBASE-18381:
-
Assignee: Ted Yu (was: stack)
> HBase regionserver crashes when reading column qualifier >64MB
> --
>
> Key: HBASE-18381
> URL: https://issues.apache.org/jira/browse/HBASE-18381
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 2.0.0-alpha-1
> Environment: HBase 1.2.0-cdh5.10.0
>Reporter: Daniel Jelinski
>Assignee: Ted Yu
>Priority: Critical
> Fix For: 2.0.0-beta-1
>
> Attachments: 18381.v1.txt, 18381.v2.txt
>
>
> After putting a cell with 64MB column qualifier to a MOB-enabled table,
> region server crashes when flushing data. Subsequent WAL replay attempts also
> result in region server crashes.
> Gist of code used to create the table:
> private String table = "poisonPill";
> private byte[] familyBytes = Bytes.toBytes("cf");
> private void createTable(Connection conn) throws IOException {
>Admin hbase_admin = conn.getAdmin();
>HTableDescriptor htable = new HTableDescriptor(TableName.valueOf(table));
>HColumnDescriptor hfamily = new HColumnDescriptor(familyBytes);
>hfamily.setMobEnabled(true);
>htable.setConfiguration("hfile.format.version","3");
>htable.addFamily(hfamily);
>hbase_admin.createTable(htable);
> }
> private void killTable(Connection conn) throws IOException {
>Table tbl = conn.getTable(TableName.valueOf(table));
>byte[] data = new byte[1<<26];
>byte[] smalldata = new byte[0];
>Put put = new Put(Bytes.toBytes("1"));
>put.addColumn(familyBytes, data, smalldata);
>tbl.put(put);
> }
> Region server logs (redacted):
> 2017-07-11 09:34:53,747 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Flushing 1/1 column families, memstore=64.00 MB; WAL is null, using passed
> sequenceid=7
> 2017-07-11 09:34:53,757 WARN org.apache.hadoop.hbase.io.hfile.HFileWriterV2:
> A minimum HFile version of 3 is required to support cell attributes/tags.
> Consider setting hfile.format.version accordingly.
> 2017-07-11 09:34:54,504 INFO
> org.apache.hadoop.hbase.mob.DefaultMobStoreFlusher: Flushed, sequenceid=7,
> memsize=67109096, hasBloomFilter=true, into tmp file
> hdfs://sandbox/hbase/data/default/poisonPill/f82e20f32302dfdd95c89ecc3be5a211/.tmp/7858d223eddd4199ad220fc77bb612eb
> 2017-07-11 09:34:54,694 ERROR org.apache.hadoop.hbase.regionserver.HStore:
> Failed to open store file :
> hdfs://sandbox/hbase/data/default/poisonPill/f82e20f32302dfdd95c89ecc3be5a211/.tmp/7858d223eddd4199ad220fc77bb612eb,
> keeping it in tmp location
> org.apache.hadoop.hbase.io.hfile.CorruptHFileException: Problem reading HFile
> Trailer from file
> hdfs://sandbox/hbase/data/default/poisonPill/f82e20f32302dfdd95c89ecc3be5a211/.tmp/7858d223eddd4199ad220fc77bb612eb
> at
> org.apache.hadoop.hbase.io.hfile.HFile.pickReaderVersion(HFile.java:497)
> at org.apache.hadoop.hbase.io.hfile.HFile.createReader(HFile.java:525)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile$Reader.(StoreFile.java:1105)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileInfo.open(StoreFileInfo.java:265)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.open(StoreFile.java:404)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:509)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:499)
> at
> org.apache.hadoop.hbase.regionserver.HStore.createStoreFileAndReader(HStore.java:675)
> at
> org.apache.hadoop.hbase.regionserver.HStore.createStoreFileAndReader(HStore.java:667)
> at
> org.apache.hadoop.hbase.regionserver.HStore.validateStoreFile(HStore.java:1746)
> at
> org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:942)
> at
> org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.flushCache(HStore.java:2299)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2372)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2102)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.replayRecoveredEdits(HRegion.java:4139)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.replayRecoveredEditsIfAny(HRegion.java:3934)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionInternals(HRegion.java:828)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:799)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6480)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6441)
> at
>
[jira] [Comment Edited] (HBASE-18381) HBase regionserver crashes when reading column qualifier >64MB
[
https://issues.apache.org/jira/browse/HBASE-18381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245201#comment-16245201
]
stack edited comment on HBASE-18381 at 11/9/17 4:50 AM:
Changed title and resumed progress. The patch fixes in wrong location according
to [~dujin...@gmail.com] comment (no dissent). Moving off 'patch availalble'.
We should disallow qualifiers of 64MB from coming in in the first place.
was (Author: stack):
Changed title and resumed progress. The patch fixes in wrong location according
to [~dujin...@gmail.com] comment (no dissent). Moving off 'patch availalble'.
> HBase regionserver crashes when reading column qualifier >64MB
> --
>
> Key: HBASE-18381
> URL: https://issues.apache.org/jira/browse/HBASE-18381
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 2.0.0-alpha-1
> Environment: HBase 1.2.0-cdh5.10.0
>Reporter: Daniel Jelinski
>Assignee: Ted Yu
>Priority: Critical
> Fix For: 2.0.0-beta-1
>
> Attachments: 18381.v1.txt, 18381.v2.txt
>
>
> After putting a cell with 64MB column qualifier to a MOB-enabled table,
> region server crashes when flushing data. Subsequent WAL replay attempts also
> result in region server crashes.
> Gist of code used to create the table:
> private String table = "poisonPill";
> private byte[] familyBytes = Bytes.toBytes("cf");
> private void createTable(Connection conn) throws IOException {
>Admin hbase_admin = conn.getAdmin();
>HTableDescriptor htable = new HTableDescriptor(TableName.valueOf(table));
>HColumnDescriptor hfamily = new HColumnDescriptor(familyBytes);
>hfamily.setMobEnabled(true);
>htable.setConfiguration("hfile.format.version","3");
>htable.addFamily(hfamily);
>hbase_admin.createTable(htable);
> }
> private void killTable(Connection conn) throws IOException {
>Table tbl = conn.getTable(TableName.valueOf(table));
>byte[] data = new byte[1<<26];
>byte[] smalldata = new byte[0];
>Put put = new Put(Bytes.toBytes("1"));
>put.addColumn(familyBytes, data, smalldata);
>tbl.put(put);
> }
> Region server logs (redacted):
> 2017-07-11 09:34:53,747 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Flushing 1/1 column families, memstore=64.00 MB; WAL is null, using passed
> sequenceid=7
> 2017-07-11 09:34:53,757 WARN org.apache.hadoop.hbase.io.hfile.HFileWriterV2:
> A minimum HFile version of 3 is required to support cell attributes/tags.
> Consider setting hfile.format.version accordingly.
> 2017-07-11 09:34:54,504 INFO
> org.apache.hadoop.hbase.mob.DefaultMobStoreFlusher: Flushed, sequenceid=7,
> memsize=67109096, hasBloomFilter=true, into tmp file
> hdfs://sandbox/hbase/data/default/poisonPill/f82e20f32302dfdd95c89ecc3be5a211/.tmp/7858d223eddd4199ad220fc77bb612eb
> 2017-07-11 09:34:54,694 ERROR org.apache.hadoop.hbase.regionserver.HStore:
> Failed to open store file :
> hdfs://sandbox/hbase/data/default/poisonPill/f82e20f32302dfdd95c89ecc3be5a211/.tmp/7858d223eddd4199ad220fc77bb612eb,
> keeping it in tmp location
> org.apache.hadoop.hbase.io.hfile.CorruptHFileException: Problem reading HFile
> Trailer from file
> hdfs://sandbox/hbase/data/default/poisonPill/f82e20f32302dfdd95c89ecc3be5a211/.tmp/7858d223eddd4199ad220fc77bb612eb
> at
> org.apache.hadoop.hbase.io.hfile.HFile.pickReaderVersion(HFile.java:497)
> at org.apache.hadoop.hbase.io.hfile.HFile.createReader(HFile.java:525)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile$Reader.(StoreFile.java:1105)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileInfo.open(StoreFileInfo.java:265)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.open(StoreFile.java:404)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:509)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:499)
> at
> org.apache.hadoop.hbase.regionserver.HStore.createStoreFileAndReader(HStore.java:675)
> at
> org.apache.hadoop.hbase.regionserver.HStore.createStoreFileAndReader(HStore.java:667)
> at
> org.apache.hadoop.hbase.regionserver.HStore.validateStoreFile(HStore.java:1746)
> at
> org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:942)
> at
> org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.flushCache(HStore.java:2299)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2372)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2102)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.replayRecoveredEdits(HRegion.java:4139)
> at
>
[jira] [Assigned] (HBASE-18381) HBase regionserver crashes when reading column qualifier >64MB
[
https://issues.apache.org/jira/browse/HBASE-18381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack reassigned HBASE-18381:
-
Assignee: stack (was: Ted Yu)
> HBase regionserver crashes when reading column qualifier >64MB
> --
>
> Key: HBASE-18381
> URL: https://issues.apache.org/jira/browse/HBASE-18381
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 2.0.0-alpha-1
> Environment: HBase 1.2.0-cdh5.10.0
>Reporter: Daniel Jelinski
>Assignee: stack
>Priority: Critical
> Fix For: 2.0.0-beta-1
>
> Attachments: 18381.v1.txt, 18381.v2.txt
>
>
> After putting a cell with 64MB column qualifier to a MOB-enabled table,
> region server crashes when flushing data. Subsequent WAL replay attempts also
> result in region server crashes.
> Gist of code used to create the table:
> private String table = "poisonPill";
> private byte[] familyBytes = Bytes.toBytes("cf");
> private void createTable(Connection conn) throws IOException {
>Admin hbase_admin = conn.getAdmin();
>HTableDescriptor htable = new HTableDescriptor(TableName.valueOf(table));
>HColumnDescriptor hfamily = new HColumnDescriptor(familyBytes);
>hfamily.setMobEnabled(true);
>htable.setConfiguration("hfile.format.version","3");
>htable.addFamily(hfamily);
>hbase_admin.createTable(htable);
> }
> private void killTable(Connection conn) throws IOException {
>Table tbl = conn.getTable(TableName.valueOf(table));
>byte[] data = new byte[1<<26];
>byte[] smalldata = new byte[0];
>Put put = new Put(Bytes.toBytes("1"));
>put.addColumn(familyBytes, data, smalldata);
>tbl.put(put);
> }
> Region server logs (redacted):
> 2017-07-11 09:34:53,747 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Flushing 1/1 column families, memstore=64.00 MB; WAL is null, using passed
> sequenceid=7
> 2017-07-11 09:34:53,757 WARN org.apache.hadoop.hbase.io.hfile.HFileWriterV2:
> A minimum HFile version of 3 is required to support cell attributes/tags.
> Consider setting hfile.format.version accordingly.
> 2017-07-11 09:34:54,504 INFO
> org.apache.hadoop.hbase.mob.DefaultMobStoreFlusher: Flushed, sequenceid=7,
> memsize=67109096, hasBloomFilter=true, into tmp file
> hdfs://sandbox/hbase/data/default/poisonPill/f82e20f32302dfdd95c89ecc3be5a211/.tmp/7858d223eddd4199ad220fc77bb612eb
> 2017-07-11 09:34:54,694 ERROR org.apache.hadoop.hbase.regionserver.HStore:
> Failed to open store file :
> hdfs://sandbox/hbase/data/default/poisonPill/f82e20f32302dfdd95c89ecc3be5a211/.tmp/7858d223eddd4199ad220fc77bb612eb,
> keeping it in tmp location
> org.apache.hadoop.hbase.io.hfile.CorruptHFileException: Problem reading HFile
> Trailer from file
> hdfs://sandbox/hbase/data/default/poisonPill/f82e20f32302dfdd95c89ecc3be5a211/.tmp/7858d223eddd4199ad220fc77bb612eb
> at
> org.apache.hadoop.hbase.io.hfile.HFile.pickReaderVersion(HFile.java:497)
> at org.apache.hadoop.hbase.io.hfile.HFile.createReader(HFile.java:525)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile$Reader.(StoreFile.java:1105)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileInfo.open(StoreFileInfo.java:265)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.open(StoreFile.java:404)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:509)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:499)
> at
> org.apache.hadoop.hbase.regionserver.HStore.createStoreFileAndReader(HStore.java:675)
> at
> org.apache.hadoop.hbase.regionserver.HStore.createStoreFileAndReader(HStore.java:667)
> at
> org.apache.hadoop.hbase.regionserver.HStore.validateStoreFile(HStore.java:1746)
> at
> org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:942)
> at
> org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.flushCache(HStore.java:2299)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2372)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2102)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.replayRecoveredEdits(HRegion.java:4139)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.replayRecoveredEditsIfAny(HRegion.java:3934)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionInternals(HRegion.java:828)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:799)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6480)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6441)
> at
>
[jira] [Updated] (HBASE-18381) HBase regionserver crashes when reading column qualifier >64MB
[
https://issues.apache.org/jira/browse/HBASE-18381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-18381:
--
Status: In Progress (was: Patch Available)
Changed title and resumed progress. The patch fixes in wrong location according
to [~dujin...@gmail.com] comment (no dissent). Moving off 'patch availalble'.
> HBase regionserver crashes when reading column qualifier >64MB
> --
>
> Key: HBASE-18381
> URL: https://issues.apache.org/jira/browse/HBASE-18381
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 2.0.0-alpha-1
> Environment: HBase 1.2.0-cdh5.10.0
>Reporter: Daniel Jelinski
>Assignee: Ted Yu
>Priority: Critical
> Fix For: 2.0.0-beta-1
>
> Attachments: 18381.v1.txt, 18381.v2.txt
>
>
> After putting a cell with 64MB column qualifier to a MOB-enabled table,
> region server crashes when flushing data. Subsequent WAL replay attempts also
> result in region server crashes.
> Gist of code used to create the table:
> private String table = "poisonPill";
> private byte[] familyBytes = Bytes.toBytes("cf");
> private void createTable(Connection conn) throws IOException {
>Admin hbase_admin = conn.getAdmin();
>HTableDescriptor htable = new HTableDescriptor(TableName.valueOf(table));
>HColumnDescriptor hfamily = new HColumnDescriptor(familyBytes);
>hfamily.setMobEnabled(true);
>htable.setConfiguration("hfile.format.version","3");
>htable.addFamily(hfamily);
>hbase_admin.createTable(htable);
> }
> private void killTable(Connection conn) throws IOException {
>Table tbl = conn.getTable(TableName.valueOf(table));
>byte[] data = new byte[1<<26];
>byte[] smalldata = new byte[0];
>Put put = new Put(Bytes.toBytes("1"));
>put.addColumn(familyBytes, data, smalldata);
>tbl.put(put);
> }
> Region server logs (redacted):
> 2017-07-11 09:34:53,747 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Flushing 1/1 column families, memstore=64.00 MB; WAL is null, using passed
> sequenceid=7
> 2017-07-11 09:34:53,757 WARN org.apache.hadoop.hbase.io.hfile.HFileWriterV2:
> A minimum HFile version of 3 is required to support cell attributes/tags.
> Consider setting hfile.format.version accordingly.
> 2017-07-11 09:34:54,504 INFO
> org.apache.hadoop.hbase.mob.DefaultMobStoreFlusher: Flushed, sequenceid=7,
> memsize=67109096, hasBloomFilter=true, into tmp file
> hdfs://sandbox/hbase/data/default/poisonPill/f82e20f32302dfdd95c89ecc3be5a211/.tmp/7858d223eddd4199ad220fc77bb612eb
> 2017-07-11 09:34:54,694 ERROR org.apache.hadoop.hbase.regionserver.HStore:
> Failed to open store file :
> hdfs://sandbox/hbase/data/default/poisonPill/f82e20f32302dfdd95c89ecc3be5a211/.tmp/7858d223eddd4199ad220fc77bb612eb,
> keeping it in tmp location
> org.apache.hadoop.hbase.io.hfile.CorruptHFileException: Problem reading HFile
> Trailer from file
> hdfs://sandbox/hbase/data/default/poisonPill/f82e20f32302dfdd95c89ecc3be5a211/.tmp/7858d223eddd4199ad220fc77bb612eb
> at
> org.apache.hadoop.hbase.io.hfile.HFile.pickReaderVersion(HFile.java:497)
> at org.apache.hadoop.hbase.io.hfile.HFile.createReader(HFile.java:525)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile$Reader.(StoreFile.java:1105)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileInfo.open(StoreFileInfo.java:265)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.open(StoreFile.java:404)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:509)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:499)
> at
> org.apache.hadoop.hbase.regionserver.HStore.createStoreFileAndReader(HStore.java:675)
> at
> org.apache.hadoop.hbase.regionserver.HStore.createStoreFileAndReader(HStore.java:667)
> at
> org.apache.hadoop.hbase.regionserver.HStore.validateStoreFile(HStore.java:1746)
> at
> org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:942)
> at
> org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.flushCache(HStore.java:2299)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2372)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2102)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.replayRecoveredEdits(HRegion.java:4139)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.replayRecoveredEditsIfAny(HRegion.java:3934)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionInternals(HRegion.java:828)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:799)
> at
>
[jira] [Updated] (HBASE-18381) HBase regionserver crashes when reading column qualifier >64MB
[
https://issues.apache.org/jira/browse/HBASE-18381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-18381:
--
Summary: HBase regionserver crashes when reading column qualifier >64MB
(was: HBase regionserver crashes when reading MOB file with column qualifier
>64MB)
> HBase regionserver crashes when reading column qualifier >64MB
> --
>
> Key: HBASE-18381
> URL: https://issues.apache.org/jira/browse/HBASE-18381
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 2.0.0-alpha-1
> Environment: HBase 1.2.0-cdh5.10.0
>Reporter: Daniel Jelinski
>Assignee: Ted Yu
>Priority: Critical
> Fix For: 2.0.0-beta-1
>
> Attachments: 18381.v1.txt, 18381.v2.txt
>
>
> After putting a cell with 64MB column qualifier to a MOB-enabled table,
> region server crashes when flushing data. Subsequent WAL replay attempts also
> result in region server crashes.
> Gist of code used to create the table:
> private String table = "poisonPill";
> private byte[] familyBytes = Bytes.toBytes("cf");
> private void createTable(Connection conn) throws IOException {
>Admin hbase_admin = conn.getAdmin();
>HTableDescriptor htable = new HTableDescriptor(TableName.valueOf(table));
>HColumnDescriptor hfamily = new HColumnDescriptor(familyBytes);
>hfamily.setMobEnabled(true);
>htable.setConfiguration("hfile.format.version","3");
>htable.addFamily(hfamily);
>hbase_admin.createTable(htable);
> }
> private void killTable(Connection conn) throws IOException {
>Table tbl = conn.getTable(TableName.valueOf(table));
>byte[] data = new byte[1<<26];
>byte[] smalldata = new byte[0];
>Put put = new Put(Bytes.toBytes("1"));
>put.addColumn(familyBytes, data, smalldata);
>tbl.put(put);
> }
> Region server logs (redacted):
> 2017-07-11 09:34:53,747 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Flushing 1/1 column families, memstore=64.00 MB; WAL is null, using passed
> sequenceid=7
> 2017-07-11 09:34:53,757 WARN org.apache.hadoop.hbase.io.hfile.HFileWriterV2:
> A minimum HFile version of 3 is required to support cell attributes/tags.
> Consider setting hfile.format.version accordingly.
> 2017-07-11 09:34:54,504 INFO
> org.apache.hadoop.hbase.mob.DefaultMobStoreFlusher: Flushed, sequenceid=7,
> memsize=67109096, hasBloomFilter=true, into tmp file
> hdfs://sandbox/hbase/data/default/poisonPill/f82e20f32302dfdd95c89ecc3be5a211/.tmp/7858d223eddd4199ad220fc77bb612eb
> 2017-07-11 09:34:54,694 ERROR org.apache.hadoop.hbase.regionserver.HStore:
> Failed to open store file :
> hdfs://sandbox/hbase/data/default/poisonPill/f82e20f32302dfdd95c89ecc3be5a211/.tmp/7858d223eddd4199ad220fc77bb612eb,
> keeping it in tmp location
> org.apache.hadoop.hbase.io.hfile.CorruptHFileException: Problem reading HFile
> Trailer from file
> hdfs://sandbox/hbase/data/default/poisonPill/f82e20f32302dfdd95c89ecc3be5a211/.tmp/7858d223eddd4199ad220fc77bb612eb
> at
> org.apache.hadoop.hbase.io.hfile.HFile.pickReaderVersion(HFile.java:497)
> at org.apache.hadoop.hbase.io.hfile.HFile.createReader(HFile.java:525)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile$Reader.(StoreFile.java:1105)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileInfo.open(StoreFileInfo.java:265)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.open(StoreFile.java:404)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:509)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:499)
> at
> org.apache.hadoop.hbase.regionserver.HStore.createStoreFileAndReader(HStore.java:675)
> at
> org.apache.hadoop.hbase.regionserver.HStore.createStoreFileAndReader(HStore.java:667)
> at
> org.apache.hadoop.hbase.regionserver.HStore.validateStoreFile(HStore.java:1746)
> at
> org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:942)
> at
> org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.flushCache(HStore.java:2299)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2372)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2102)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.replayRecoveredEdits(HRegion.java:4139)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.replayRecoveredEditsIfAny(HRegion.java:3934)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionInternals(HRegion.java:828)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:799)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6480)
> at
>
[jira] [Commented] (HBASE-18817) Pull hbase-spark module out of branch-2
[ https://issues.apache.org/jira/browse/HBASE-18817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245200#comment-16245200 ] Sean Busbey commented on HBASE-18817: - unit test failures are failures to start surefire launches for some of the hbase-server tests, which can't be related to removing the spark module. > Pull hbase-spark module out of branch-2 > --- > > Key: HBASE-18817 > URL: https://issues.apache.org/jira/browse/HBASE-18817 > Project: HBase > Issue Type: Task > Components: spark >Affects Versions: 2.0.0-alpha-2 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Blocker > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-18817-branch-2.v0.patch, > HBASE-18817-branch-2.v1.patch > > > see DISCUSS here: > https://s.apache.org/UJAf > Sadly, feature is slipping out of branch-2. We can work out inclusion for > downstream once we have some inertia again. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18962) Support atomic BatchOperations through batchMutate()
[
https://issues.apache.org/jira/browse/HBASE-18962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245195#comment-16245195
]
Hadoop QA commented on HBASE-18962:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
52s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 6m
35s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
34s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
17s{color} | {color:green} hbase-server: The patch generated 0 new + 412
unchanged - 6 fixed = 412 total (was 418) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
15s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
51m 7s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 2.7.4 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 98m
11s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}171m 32s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-18962 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12896782/hbase-18962.master.004.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 7bc9ac1f6e35 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7
11:05:26 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / cffbe4bef7 |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/9699/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/9699/console |
| Powered by | Apache Yetus 0.6.0 http://yetus.apache.org |
This message was automatically generated.
> Support atomic BatchOperations through batchMutate()
>
[jira] [Created] (HBASE-19220) Async tests time out talking to zk; 'clusterid came back null'
stack created HBASE-19220: - Summary: Async tests time out talking to zk; 'clusterid came back null' Key: HBASE-19220 URL: https://issues.apache.org/jira/browse/HBASE-19220 Project: HBase Issue Type: Sub-task Reporter: stack Assignee: stack Fix For: 2.0.0-beta-1 I see this in test runs on a dedicated machine: [ERROR] Tests run: 6, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 652.514 s <<< FAILURE! - in org.apache.hadoop.hbase.client.TestAsyncAdminBuilder [ERROR] testRpcTimeout[0](org.apache.hadoop.hbase.client.TestAsyncAdminBuilder) Time elapsed: 213.618 s <<< ERROR! java.util.concurrent.ExecutionException: java.io.IOException: clusterid came back null at org.apache.hadoop.hbase.client.TestAsyncAdminBuilder.testRpcTimeout(TestAsyncAdminBuilder.java:105) Caused by: java.io.IOException: clusterid came back null [ERROR] org.apache.hadoop.hbase.client.TestAsyncTableScanMetrics Time elapsed: 0.007 s <<< ERROR! java.util.concurrent.ExecutionException: java.io.IOException: clusterid came back null at org.apache.hadoop.hbase.client.TestAsyncTableScanMetrics.setUp(TestAsyncTableScanMetrics.java:97) Caused by: java.io.IOException: clusterid came back null [ERROR] org.apache.hadoop.hbase.client.TestRawAsyncScanCursor Time elapsed: 0.005 s <<< ERROR! java.util.concurrent.ExecutionException: java.io.IOException: clusterid came back null at org.apache.hadoop.hbase.client.TestRawAsyncScanCursor.setUpBeforeClass(TestRawAsyncScanCursor.java:42) Caused by: java.io.IOException: clusterid came back null [ERROR] org.apache.hadoop.hbase.client.TestAsyncNamespaceAdminApi Time elapsed: 0.005 s <<< ERROR! java.util.concurrent.ExecutionException: java.io.IOException: clusterid came back null at org.apache.hadoop.hbase.client.TestAsyncNamespaceAdminApi.setUpBeforeClass(TestAsyncNamespaceAdminApi.java:66) Caused by: java.io.IOException: clusterid came back null If I up the retries, they go away. At least on this machine, I notice that zk connections can take a while... see HBASE-19102 where we add a wait on the Connection to come up before progressing. Suggest that I up the retries. No harm in trying more. It is currently set to 3 retries at a one second interval. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18817) Pull hbase-spark module out of branch-2
[ https://issues.apache.org/jira/browse/HBASE-18817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245192#comment-16245192 ] stack commented on HBASE-18817: --- +1 Looks comprehensive (tearing up) > Pull hbase-spark module out of branch-2 > --- > > Key: HBASE-18817 > URL: https://issues.apache.org/jira/browse/HBASE-18817 > Project: HBase > Issue Type: Task > Components: spark >Affects Versions: 2.0.0-alpha-2 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Blocker > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-18817-branch-2.v0.patch, > HBASE-18817-branch-2.v1.patch > > > see DISCUSS here: > https://s.apache.org/UJAf > Sadly, feature is slipping out of branch-2. We can work out inclusion for > downstream once we have some inertia again. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19220) Async tests time out talking to zk; 'clusterid came back null'
[ https://issues.apache.org/jira/browse/HBASE-19220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19220: -- Attachment: 19220.patch FYI [~Apache9] Ok I commit this? > Async tests time out talking to zk; 'clusterid came back null' > -- > > Key: HBASE-19220 > URL: https://issues.apache.org/jira/browse/HBASE-19220 > Project: HBase > Issue Type: Sub-task > Components: test >Reporter: stack >Assignee: stack > Fix For: 2.0.0-beta-1 > > Attachments: 19220.patch > > > I see this in test runs on a dedicated machine: > [ERROR] Tests run: 6, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: > 652.514 s <<< FAILURE! - in > org.apache.hadoop.hbase.client.TestAsyncAdminBuilder > [ERROR] > testRpcTimeout[0](org.apache.hadoop.hbase.client.TestAsyncAdminBuilder) Time > elapsed: 213.618 s <<< ERROR! > java.util.concurrent.ExecutionException: java.io.IOException: clusterid came > back null > at > org.apache.hadoop.hbase.client.TestAsyncAdminBuilder.testRpcTimeout(TestAsyncAdminBuilder.java:105) > Caused by: java.io.IOException: clusterid came back null > [ERROR] org.apache.hadoop.hbase.client.TestAsyncTableScanMetrics Time > elapsed: 0.007 s <<< ERROR! > java.util.concurrent.ExecutionException: java.io.IOException: clusterid came > back null > at > org.apache.hadoop.hbase.client.TestAsyncTableScanMetrics.setUp(TestAsyncTableScanMetrics.java:97) > Caused by: java.io.IOException: clusterid came back null > [ERROR] org.apache.hadoop.hbase.client.TestRawAsyncScanCursor Time elapsed: > 0.005 s <<< ERROR! > java.util.concurrent.ExecutionException: java.io.IOException: clusterid came > back null > at > org.apache.hadoop.hbase.client.TestRawAsyncScanCursor.setUpBeforeClass(TestRawAsyncScanCursor.java:42) > Caused by: java.io.IOException: clusterid came back null > [ERROR] org.apache.hadoop.hbase.client.TestAsyncNamespaceAdminApi Time > elapsed: 0.005 s <<< ERROR! > java.util.concurrent.ExecutionException: java.io.IOException: clusterid came > back null > at > org.apache.hadoop.hbase.client.TestAsyncNamespaceAdminApi.setUpBeforeClass(TestAsyncNamespaceAdminApi.java:66) > Caused by: java.io.IOException: clusterid came back null > If I up the retries, they go away. > At least on this machine, I notice that zk connections can take a while... > see HBASE-19102 where we add a wait on the Connection to come up before > progressing. > Suggest that I up the retries. No harm in trying more. It is currently set to > 3 retries at a one second interval. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19220) Async tests time out talking to zk; 'clusterid came back null'
[ https://issues.apache.org/jira/browse/HBASE-19220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19220: -- Status: Patch Available (was: Open) > Async tests time out talking to zk; 'clusterid came back null' > -- > > Key: HBASE-19220 > URL: https://issues.apache.org/jira/browse/HBASE-19220 > Project: HBase > Issue Type: Sub-task > Components: test >Reporter: stack >Assignee: stack > Fix For: 2.0.0-beta-1 > > Attachments: 19220.patch > > > I see this in test runs on a dedicated machine: > [ERROR] Tests run: 6, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: > 652.514 s <<< FAILURE! - in > org.apache.hadoop.hbase.client.TestAsyncAdminBuilder > [ERROR] > testRpcTimeout[0](org.apache.hadoop.hbase.client.TestAsyncAdminBuilder) Time > elapsed: 213.618 s <<< ERROR! > java.util.concurrent.ExecutionException: java.io.IOException: clusterid came > back null > at > org.apache.hadoop.hbase.client.TestAsyncAdminBuilder.testRpcTimeout(TestAsyncAdminBuilder.java:105) > Caused by: java.io.IOException: clusterid came back null > [ERROR] org.apache.hadoop.hbase.client.TestAsyncTableScanMetrics Time > elapsed: 0.007 s <<< ERROR! > java.util.concurrent.ExecutionException: java.io.IOException: clusterid came > back null > at > org.apache.hadoop.hbase.client.TestAsyncTableScanMetrics.setUp(TestAsyncTableScanMetrics.java:97) > Caused by: java.io.IOException: clusterid came back null > [ERROR] org.apache.hadoop.hbase.client.TestRawAsyncScanCursor Time elapsed: > 0.005 s <<< ERROR! > java.util.concurrent.ExecutionException: java.io.IOException: clusterid came > back null > at > org.apache.hadoop.hbase.client.TestRawAsyncScanCursor.setUpBeforeClass(TestRawAsyncScanCursor.java:42) > Caused by: java.io.IOException: clusterid came back null > [ERROR] org.apache.hadoop.hbase.client.TestAsyncNamespaceAdminApi Time > elapsed: 0.005 s <<< ERROR! > java.util.concurrent.ExecutionException: java.io.IOException: clusterid came > back null > at > org.apache.hadoop.hbase.client.TestAsyncNamespaceAdminApi.setUpBeforeClass(TestAsyncNamespaceAdminApi.java:66) > Caused by: java.io.IOException: clusterid came back null > If I up the retries, they go away. > At least on this machine, I notice that zk connections can take a while... > see HBASE-19102 where we add a wait on the Connection to come up before > progressing. > Suggest that I up the retries. No harm in trying more. It is currently set to > 3 retries at a one second interval. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19165) TODO Handle stuck in transition: rit=OPENING, location=ve0538....
[ https://issues.apache.org/jira/browse/HBASE-19165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245179#comment-16245179 ] ramkrishna.s.vasudevan commented on HBASE-19165: What is the diff between 003 and 001 patch? I can see some additional LOG msg changes and some comment fixes. Probably am missing seeing the difference? > TODO Handle stuck in transition: rit=OPENING, location=ve0538 > - > > Key: HBASE-19165 > URL: https://issues.apache.org/jira/browse/HBASE-19165 > Project: HBase > Issue Type: Bug > Components: migration >Reporter: stack >Assignee: stack >Priority: Critical > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19165.master.001.patch, > HBASE-19165.master.002.patch, HBASE-19165.master.003.patch, > HBASE-19165.master.003.patch > > > A few of us trying migration from hbase1 to hbase2 have run into various > assignment issues. In my case, I see that Master coming online, reading > hbase:meta, finding an issue in OPENING state and then just not assigning. My > thought is that its expecting a callback to come in. Eventually log is fills > with this sort of stuff: > 2017-11-02 15:02:54,237 WARN [ProcExecTimeout] assignment.AssignmentManager: > TODO Handle stuck in transition: rit=OPENING, > location=ve0528.halxg.cloudera.com,16020,1509657692581, > table=IntegrationTestBigLinkedList, region=919cc6636ffbed17f628f335d3a58726 > Let me get a cleaner run. This current log is polluted by a few restarts. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19165) TODO Handle stuck in transition: rit=OPENING, location=ve0538....
[ https://issues.apache.org/jira/browse/HBASE-19165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19165: -- Attachment: HBASE-19165.master.003.patch Retry. Test passes locally. > TODO Handle stuck in transition: rit=OPENING, location=ve0538 > - > > Key: HBASE-19165 > URL: https://issues.apache.org/jira/browse/HBASE-19165 > Project: HBase > Issue Type: Bug > Components: migration >Reporter: stack >Assignee: stack >Priority: Critical > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19165.master.001.patch, > HBASE-19165.master.002.patch, HBASE-19165.master.003.patch, > HBASE-19165.master.003.patch > > > A few of us trying migration from hbase1 to hbase2 have run into various > assignment issues. In my case, I see that Master coming online, reading > hbase:meta, finding an issue in OPENING state and then just not assigning. My > thought is that its expecting a callback to come in. Eventually log is fills > with this sort of stuff: > 2017-11-02 15:02:54,237 WARN [ProcExecTimeout] assignment.AssignmentManager: > TODO Handle stuck in transition: rit=OPENING, > location=ve0528.halxg.cloudera.com,16020,1509657692581, > table=IntegrationTestBigLinkedList, region=919cc6636ffbed17f628f335d3a58726 > Let me get a cleaner run. This current log is polluted by a few restarts. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19009) implement modifyTable and enable/disableTableReplication for AsyncAdmin
[ https://issues.apache.org/jira/browse/HBASE-19009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-19009: --- Summary: implement modifyTable and enable/disableTableReplication for AsyncAdmin (was: implement enable/disableTableReplication for AsyncAdmin) > implement modifyTable and enable/disableTableReplication for AsyncAdmin > --- > > Key: HBASE-19009 > URL: https://issues.apache.org/jira/browse/HBASE-19009 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang > Fix For: 3.0.0, 2.0.0-beta-1 > > Attachments: HBASE-19009.master.001.patch, > HBASE-19009.master.002.patch, HBASE-19009.master.003.patch, > HBASE-19009.master.004.patch, HBASE-19009.master.005.patch, > HBASE-19009.master.006.patch, HBASE-19009.master.007.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (HBASE-19009) implement enable/disableTableReplication for AsyncAdmin
[ https://issues.apache.org/jira/browse/HBASE-19009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245168#comment-16245168 ] Appy edited comment on HBASE-19009 at 11/9/17 3:55 AM: --- We are adding more that just enable/disable replication. Let's update the title. And list the functions being added in description and commit message. Great stuff! was (Author: appy): We are adding more that just enable/disable replication. Let's update the title. And list the functions being added in description and commit message. > implement enable/disableTableReplication for AsyncAdmin > --- > > Key: HBASE-19009 > URL: https://issues.apache.org/jira/browse/HBASE-19009 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang > Fix For: 3.0.0, 2.0.0-beta-1 > > Attachments: HBASE-19009.master.001.patch, > HBASE-19009.master.002.patch, HBASE-19009.master.003.patch, > HBASE-19009.master.004.patch, HBASE-19009.master.005.patch, > HBASE-19009.master.006.patch, HBASE-19009.master.007.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19009) implement enable/disableTableReplication for AsyncAdmin
[ https://issues.apache.org/jira/browse/HBASE-19009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245168#comment-16245168 ] Appy commented on HBASE-19009: -- We are adding more that just enable/disable replication. Let's update the title. And list the functions being added in description and commit message. > implement enable/disableTableReplication for AsyncAdmin > --- > > Key: HBASE-19009 > URL: https://issues.apache.org/jira/browse/HBASE-19009 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang > Fix For: 3.0.0, 2.0.0-beta-1 > > Attachments: HBASE-19009.master.001.patch, > HBASE-19009.master.002.patch, HBASE-19009.master.003.patch, > HBASE-19009.master.004.patch, HBASE-19009.master.005.patch, > HBASE-19009.master.006.patch, HBASE-19009.master.007.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19200) Remove Registry and use AsyncRegistry directly
[ https://issues.apache.org/jira/browse/HBASE-19200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-19200: -- Attachment: HBASE-19200-v2.patch Remove Registry completely. Use AsyncRegistry directly. > Remove Registry and use AsyncRegistry directly > -- > > Key: HBASE-19200 > URL: https://issues.apache.org/jira/browse/HBASE-19200 > Project: HBase > Issue Type: Task > Components: Client, Zookeeper >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19200-v1.patch, HBASE-19200-v2.patch, > HBASE-19200.patch > > > So that we can move most of the zookeeper related code out of hbase-client > module. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19165) TODO Handle stuck in transition: rit=OPENING, location=ve0538....
[
https://issues.apache.org/jira/browse/HBASE-19165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245150#comment-16245150
]
Hadoop QA commented on HBASE-19165:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
35s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 7m
56s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
17s{color} | {color:red} hbase-server: The patch generated 2 new + 173
unchanged - 1 fixed = 175 total (was 174) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 6m
0s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
58m 49s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 2.7.4 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
47s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
36s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}125m 46s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
36s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}216m 28s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.master.procedure.TestTruncateTableProcedure
|
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19165 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12896759/HBASE-19165.master.003.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 5f64cdfe0bc5 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31
14:13:22 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / cffbe4bef7 |
| maven | version: Apache Maven 3.5.2
[jira] [Commented] (HBASE-18943) Cannot start mini dfs cluster using hadoop-3 in test due to NoSuchMethodError in jetty
[
https://issues.apache.org/jira/browse/HBASE-18943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245149#comment-16245149
]
Ted Yu commented on HBASE-18943:
Ran TestReplicationSource with patch locally which passed.
Depending feedback, I can rerun the tests thru QA.
> Cannot start mini dfs cluster using hadoop-3 in test due to NoSuchMethodError
> in jetty
> ---
>
> Key: HBASE-18943
> URL: https://issues.apache.org/jira/browse/HBASE-18943
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Priority: Blocker
> Fix For: 2.0.0-beta-1
>
> Attachments: 18943.v1.txt
>
>
> When starting mini dfs cluster against hadoop-3:
> {code}
> dfsCluster = startMiniDFSCluster(numDataNodes, dataNodeHosts);
> {code}
> The above call would end up with:
> {code}
> java.lang.NoSuchMethodError:
> org.eclipse.jetty.server.session.SessionHandler.getSessionManager()Lorg/eclipse/jetty/server/SessionManager;
> at org.apache.hadoop.hbase.client.TestHCM.setUpBeforeClass(TestHCM.java:251)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18962) Support atomic BatchOperations through batchMutate()
[
https://issues.apache.org/jira/browse/HBASE-18962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245148#comment-16245148
]
Hadoop QA commented on HBASE-18962:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
10s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 6m
0s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
11s{color} | {color:green} hbase-server: The patch generated 0 new + 412
unchanged - 6 fixed = 412 total (was 418) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
36s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
52m 59s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 2.7.4 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}103m
16s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
19s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}176m 10s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-18962 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12896765/hbase-18962.master.003.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux f6db69236254 3.13.0-129-generic #178-Ubuntu SMP Fri Aug 11
12:48:20 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / cffbe4bef7 |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/9698/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/9698/console |
| Powered by | Apache Yetus 0.6.0 http://yetus.apache.org |
This message was automatically generated.
> Support atomic BatchOperations through batchMutate()
>
[jira] [Commented] (HBASE-18090) Improve TableSnapshotInputFormat to allow more multiple mappers per region
[
https://issues.apache.org/jira/browse/HBASE-18090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245137#comment-16245137
]
Hadoop QA commented on HBASE-18090:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 6s{color}
| {color:red} HBASE-18090 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.6.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HBASE-18090 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12896794/HBASE-18090.branch-1-V2.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/9701/console |
| Powered by | Apache Yetus 0.6.0 http://yetus.apache.org |
This message was automatically generated.
> Improve TableSnapshotInputFormat to allow more multiple mappers per region
> --
>
> Key: HBASE-18090
> URL: https://issues.apache.org/jira/browse/HBASE-18090
> Project: HBase
> Issue Type: Improvement
> Components: mapreduce
>Reporter: Mikhail Antonov
>Assignee: xinxin fan
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-18090-V3-master.patch,
> HBASE-18090-V4-master.patch, HBASE-18090-V5-master.patch,
> HBASE-18090-branch-1.3-v1.patch, HBASE-18090-branch-1.3-v2.patch,
> HBASE-18090.branch-1-V2.patch, HBASE-18090.branch-1.patch
>
>
> TableSnapshotInputFormat runs one map task per region in the table snapshot.
> This places unnecessary restriction that the region layout of the original
> table needs to take the processing resources available to MR job into
> consideration. Allowing to run multiple mappers per region (assuming
> reasonably even key distribution) would be useful.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-15075) Allow region split request to carry identification information
[ https://issues.apache.org/jira/browse/HBASE-15075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-15075: --- Fix Version/s: (was: 1.4.0) (was: 2.0.0) > Allow region split request to carry identification information > -- > > Key: HBASE-15075 > URL: https://issues.apache.org/jira/browse/HBASE-15075 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Ted Yu > Attachments: 15075-v0.txt, 15075-v1.txt, 15075-v2.txt, > HBASE-15075.v10.patch, HBASE-15075.v2.patch, HBASE-15075.v3.patch, > HBASE-15075.v4.patch, HBASE-15075.v5.patch, HBASE-15075.v6.patch, > HBASE-15075.v7.patch, HBASE-15075.v8.patch, HBASE-15075.v9.patch > > > During the process of improving region normalization feature, I found that if > region split request triggered by the execution of SplitNormalizationPlan > fails, there is no way of knowing whether the failed split originated from > region normalization. > The association of particular split request with outcome of split would give > RegionNormalizer information so that it can make better normalization > decisions in the subsequent invocations. > One approach is to embed metadata, such as a UUID, in SplitRequest which gets > passed through RegionStateTransitionContext when > RegionServerServices#reportRegionStateTransition() is called. > This way, RegionStateListener can be notified with the metadata (id of the > requester). > See discussion on dev mailing list > http://search-hadoop.com/m/YGbbCXdkivihp2 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-9899) for idempotent operation dups, return the result instead of throwing conflict exception
[ https://issues.apache.org/jira/browse/HBASE-9899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-9899: -- Fix Version/s: (was: 1.4.0) > for idempotent operation dups, return the result instead of throwing conflict > exception > --- > > Key: HBASE-9899 > URL: https://issues.apache.org/jira/browse/HBASE-9899 > Project: HBase > Issue Type: Improvement >Reporter: Sergey Shelukhin >Assignee: Guanghao Zhang > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-9899-addendum-branch-1.patch, > HBASE-9899-addendum.patch, HBASE-9899-branch-1.patch, > HBASE-9899-branch-1.patch, HBASE-9899-branch-1.patch, HBASE-9899-v1.patch, > HBASE-9899-v2.patch, HBASE-9899-v3.patch, HBASE-9899-v3.patch, > HBASE-9899-v4.patch, HBASE-9899-v4.patch > > > After HBASE-3787, we could store mvcc in operation context, and use it to > convert the modification request into read on dups instead of throwing > OperationConflictException. > MVCC tracking will have to be aware of such MVCC numbers present. Given that > scanners are usually relatively short-lived, that would prevent low watermark > from advancing for quite a bit more time -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18090) Improve TableSnapshotInputFormat to allow more multiple mappers per region
[ https://issues.apache.org/jira/browse/HBASE-18090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xinxin fan updated HBASE-18090: --- Attachment: HBASE-18090.branch-1-V2.patch > Improve TableSnapshotInputFormat to allow more multiple mappers per region > -- > > Key: HBASE-18090 > URL: https://issues.apache.org/jira/browse/HBASE-18090 > Project: HBase > Issue Type: Improvement > Components: mapreduce >Reporter: Mikhail Antonov >Assignee: xinxin fan > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-18090-V3-master.patch, > HBASE-18090-V4-master.patch, HBASE-18090-V5-master.patch, > HBASE-18090-branch-1.3-v1.patch, HBASE-18090-branch-1.3-v2.patch, > HBASE-18090.branch-1-V2.patch, HBASE-18090.branch-1.patch > > > TableSnapshotInputFormat runs one map task per region in the table snapshot. > This places unnecessary restriction that the region layout of the original > table needs to take the processing resources available to MR job into > consideration. Allowing to run multiple mappers per region (assuming > reasonably even key distribution) would be useful. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-15134) Add visibility into Flush and Compaction queues
[ https://issues.apache.org/jira/browse/HBASE-15134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-15134: --- Fix Version/s: (was: 1.5.0) > Add visibility into Flush and Compaction queues > --- > > Key: HBASE-15134 > URL: https://issues.apache.org/jira/browse/HBASE-15134 > Project: HBase > Issue Type: New Feature > Components: Compaction, metrics, regionserver >Reporter: Elliott Clark >Assignee: Abhishek Singh Chouhan > Fix For: 3.0.0, 1.4.0, 2.0.0-alpha-2 > > Attachments: HBASE-15134.branch-1.001.patch, > HBASE-15134.branch-1.001.patch, HBASE-15134.master.001.patch, > HBASE-15134.master.002.patch, HBASE-15134.master.003.patch, > HBASE-15134.patch, HBASE-15134.patch > > > On busy spurts we can see regionservers start to see large queues for > compaction. It's really hard to tell if the server is queueing a lot of > compactions for the same region, lots of compactions for lots of regions, or > just falling behind. > For flushes much the same. There can be flushes in queue that aren't being > run because of delayed flushes. There's no way to know from the metrics how > many flushes are for each region, how many are delayed. Etc. > We should add either more metrics around this ( num per region, max per > region, min per region ) or add on a UI page that has the list of compactions > and flushes. > Or both. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-15119) Include git SHA in check_compatibility reports
[ https://issues.apache.org/jira/browse/HBASE-15119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-15119: --- Fix Version/s: (was: 1.4.0) > Include git SHA in check_compatibility reports > -- > > Key: HBASE-15119 > URL: https://issues.apache.org/jira/browse/HBASE-15119 > Project: HBase > Issue Type: Improvement > Components: build >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > Fix For: 2.0.0, 1.3.0, 1.2.2, 1.1.6, 0.98.21 > > Attachments: HBASE-15119.v00.patch > > > Since some refs change over time (ie, branches), it would be nice to include > git shas in the version info included in check compatibility reports. It'll > also help interested parties to be sure of what they're looking at. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-15348) Fix tests broken by recent metrics re-work
[ https://issues.apache.org/jira/browse/HBASE-15348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-15348: --- Fix Version/s: (was: 1.4.0) > Fix tests broken by recent metrics re-work > -- > > Key: HBASE-15348 > URL: https://issues.apache.org/jira/browse/HBASE-15348 > Project: HBase > Issue Type: Bug > Components: metrics, test >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 2.0.0, 1.3.0 > > > Counts are appoximate and go away. We should re-work the tests or test utils > to make them work now. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-15328) Unvalidated Redirect in HMaster
[
https://issues.apache.org/jira/browse/HBASE-15328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-15328:
---
Fix Version/s: (was: 1.4.0)
> Unvalidated Redirect in HMaster
> ---
>
> Key: HBASE-15328
> URL: https://issues.apache.org/jira/browse/HBASE-15328
> Project: HBase
> Issue Type: Bug
> Components: security
>Reporter: stack
>Assignee: Sean Busbey
>Priority: Minor
> Fix For: 2.0.0, 1.3.1, 1.2.5, 1.1.10
>
> Attachments: HBASE-15328.0.patch, HBASE-15328.1.patch
>
>
> See OWASP page on why we should clean it up someday:
> https://www.owasp.org/index.php/Unvalidated_Redirects_and_Forwards_Cheat_Sheet
> Here is where we do the redirect:
> {code}
> @Override
> public void doGet(HttpServletRequest request,
> HttpServletResponse response) throws ServletException, IOException {
> String redirectUrl = request.getScheme() + "://"
> + request.getServerName() + ":" + regionServerInfoPort
> + request.getRequestURI();
> response.sendRedirect(redirectUrl);
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-15386) PREFETCH_BLOCKS_ON_OPEN in HColumnDescriptor is ignored
[ https://issues.apache.org/jira/browse/HBASE-15386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-15386: --- Fix Version/s: (was: 1.4.0) > PREFETCH_BLOCKS_ON_OPEN in HColumnDescriptor is ignored > --- > > Key: HBASE-15386 > URL: https://issues.apache.org/jira/browse/HBASE-15386 > Project: HBase > Issue Type: Sub-task > Components: BucketCache >Reporter: stack >Assignee: stack > Fix For: 2.0.0, 0.98.19, 1.2.2, 1.3.1 > > Attachments: 15386.branch-1.2.patch, 15386.branch-1.2.patch, > 15386.branch-1.patch, 15386.patch > > > We use the global flag hbase.rs.prefetchblocksonopen only and ignore the HCD > setting. > Purge from HCD or hook it up again (it probably worked once). > Thanks to Daniel Pol for finding this one. Let me fix. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-15429) Add a split policy for busy regions
[ https://issues.apache.org/jira/browse/HBASE-15429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-15429: --- Fix Version/s: (was: 1.4.0) > Add a split policy for busy regions > --- > > Key: HBASE-15429 > URL: https://issues.apache.org/jira/browse/HBASE-15429 > Project: HBase > Issue Type: Improvement > Components: regionserver >Reporter: Ashu Pachauri >Assignee: Ashu Pachauri > Fix For: 2.0.0, 1.3.1 > > Attachments: HBASE-15429-V1.patch, HBASE-15429-V2.patch, > HBASE-15429.branch-1.patch, HBASE-15429.patch > > > Currently, all region split policies are based on size. However, in certain > cases, it is a wise choice to make a split decision based on number of > requests to the region and split busy regions. > A crude metric is that if a region blocks writes often and throws > RegionTooBusyExceoption, it's probably a good idea to split it. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-15370) Backport Moderate Object Storage (MOB) to branch-1
[ https://issues.apache.org/jira/browse/HBASE-15370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-15370: --- Fix Version/s: (was: 1.4.0) > Backport Moderate Object Storage (MOB) to branch-1 > -- > > Key: HBASE-15370 > URL: https://issues.apache.org/jira/browse/HBASE-15370 > Project: HBase > Issue Type: Task >Reporter: Ted Yu > Attachments: 15370-test.out, HBASE-15370-branch-1.v1.patch, > HBASE-15370-branch-1.v2.patch, merge-conflict.list, mob-cmmits.txt, > mob-commits-v2.txt > > > MOB feature was integrated to master branch half a year ago. > Since then there have been bug fixes which stabilize the feature. > Some customers have been using it at PB scale. > Here is discussion thread on dev mailing list: > http://search-hadoop.com/m/YGbbDSqxD1PYXK62/hbase+MOB+in+branch-1=Re+MOB+in+branch+1+Re+RESULT+VOTE+Merge+branch+hbase+11339+HBase+MOB+to+trunk+ > This issue is to backport MOB feature to branch-1. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-15713) Backport "HBASE-15477 Do not save 'next block header' when we cache hfileblocks"
[ https://issues.apache.org/jira/browse/HBASE-15713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-15713: --- Fix Version/s: (was: 1.4.0) > Backport "HBASE-15477 Do not save 'next block header' when we cache > hfileblocks" > > > Key: HBASE-15713 > URL: https://issues.apache.org/jira/browse/HBASE-15713 > Project: HBase > Issue Type: Sub-task > Components: BucketCache >Affects Versions: 1.3.0 >Reporter: stack >Assignee: stack > Fix For: 1.3.0 > > Attachments: 15477.backport.branch-1.v7.patch, > 15477.backport.branch-1.v8.patch > > > Backport "HBASE-15477 Do not save 'next block header' when we cache > hfileblocks" > The backport involves removing support for hfile v1/hfileblockv1 support > which was our default before 0.92. The backport patch also removes tests that > depended on hfile v1, tests that were deprecated post 0.96 and to be removed > (0.96 was the release that would only run if the whole cluster was hfiles > > v1). > [~mantonov] Ok to commit the above to 1.3? Thanks boss. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-15729) Remove old JDiff wrapper scripts in dev-support
[ https://issues.apache.org/jira/browse/HBASE-15729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-15729: --- Fix Version/s: (was: 1.4.0) > Remove old JDiff wrapper scripts in dev-support > --- > > Key: HBASE-15729 > URL: https://issues.apache.org/jira/browse/HBASE-15729 > Project: HBase > Issue Type: Task > Components: build, community >Reporter: Dima Spivak >Assignee: Dima Spivak >Priority: Minor > Fix For: 2.0.0, 1.3.0, 1.2.2, 1.1.6, 0.98.21 > > Attachments: HBASE-15729.patch > > > Since HBASE-12808, we've been using the Java API Compliance Checker instead > of JDiff to look at API compatibility. Probably makes sense to remove the old > wrapper scripts that aren't being used anymore. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-15713) Backport "HBASE-15477 Do not save 'next block header' when we cache hfileblocks"
[ https://issues.apache.org/jira/browse/HBASE-15713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-15713: --- Affects Version/s: (was: 1.4.0) > Backport "HBASE-15477 Do not save 'next block header' when we cache > hfileblocks" > > > Key: HBASE-15713 > URL: https://issues.apache.org/jira/browse/HBASE-15713 > Project: HBase > Issue Type: Sub-task > Components: BucketCache >Affects Versions: 1.3.0 >Reporter: stack >Assignee: stack > Fix For: 1.3.0 > > Attachments: 15477.backport.branch-1.v7.patch, > 15477.backport.branch-1.v8.patch > > > Backport "HBASE-15477 Do not save 'next block header' when we cache > hfileblocks" > The backport involves removing support for hfile v1/hfileblockv1 support > which was our default before 0.92. The backport patch also removes tests that > depended on hfile v1, tests that were deprecated post 0.96 and to be removed > (0.96 was the release that would only run if the whole cluster was hfiles > > v1). > [~mantonov] Ok to commit the above to 1.3? Thanks boss. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-15631) Backport Regionserver Groups (HBASE-6721) to branch-1
[ https://issues.apache.org/jira/browse/HBASE-15631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-15631: --- Fix Version/s: (was: 1.5.0) > Backport Regionserver Groups (HBASE-6721) to branch-1 > -- > > Key: HBASE-15631 > URL: https://issues.apache.org/jira/browse/HBASE-15631 > Project: HBase > Issue Type: New Feature >Affects Versions: 1.4.0 >Reporter: Francis Liu >Assignee: Andrew Purtell > Fix For: 1.4.0 > > Attachments: HBASE-15631-branch-1-addendum.patch, > HBASE-15631-branch-1.patch, HBASE-15631-branch-1.patch, > HBASE-15631-branch-1.patch, HBASE-15631.branch-1.patch, HBASE-15631.patch > > > Based on dev list discussion backporting region server group should not be an > issue as it does not: 1. destabilize the code. 2. cause backward > incompatibility. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-15855) TestSplitTransactionOnCluster#testFailedSplit may fail due to pending dead server processing
[
https://issues.apache.org/jira/browse/HBASE-15855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-15855:
---
Fix Version/s: (was: 1.4.0)
> TestSplitTransactionOnCluster#testFailedSplit may fail due to pending dead
> server processing
>
>
> Key: HBASE-15855
> URL: https://issues.apache.org/jira/browse/HBASE-15855
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Minor
> Attachments: 15855-branch-1.v1.patch, HBASE-15855.branch-1.patch,
> testFailedSplit.err
>
>
> Sometimes TestSplitTransactionOnCluster#testFailedSplit fails with:
> {code}
> java.lang.AssertionError: null
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertTrue(Assert.java:52)
> at
> org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster.testFailedSplit(TestSplitTransactionOnCluster.java:1339)
> {code}
> Here was the reason:
> {code}
> 2016-05-12 14:38:31,022 INFO
> [RS:5;c66-slave-20176e25-5:40721-splits-1463063900610]
> regionserver.SplitRequest(143): Split transaction journal:
> STARTED at 1463063910621
> PREPARED at 1463063910628
> BEFORE_PRE_SPLIT_HOOK at 1463063910628
> AFTER_PRE_SPLIT_HOOK at 1463063910628
> SET_SPLITTING at 1463063910632
> CREATE_SPLIT_DIR at 1463063910743
> CLOSED_PARENT_REGION at 1463063910768
> OFFLINED_PARENT at 1463063910768
> STARTED_REGION_A_CREATION at 1463063910839
> STARTED_REGION_B_CREATION at 1463063910889
> 2016-05-12 14:38:31,023 DEBUG [Thread-1689-EventThread]
> zookeeper.ZooKeeperWatcher(511): hbase-admin-on-hconnection-0x9755dd10x0,
> quorum=localhost:49482, baseZNode=/hbase Received ZooKeeper Event, type=None,
> state=SyncConnected, path=null
> 2016-05-12 14:38:31,025 DEBUG [Thread-1689-EventThread]
> zookeeper.ZooKeeperWatcher(574):
> hbase-admin-on-hconnection-0x9755dd1-0x154a566b247001f connected
> 2016-05-12 14:38:31,052 DEBUG
> [B.defaultRpcServer.handler=3,queue=0,port=54033] master.HMaster(1373): Not
> running balancer because processing dead regionserver(s):
> c66-slave-20176e25-5.novalocal,49562,1463063863793
> 2016-05-12 14:38:31,054 INFO [Thread-1689]
> client.ConnectionManager$HConnectionImplementation(1684): Closing zookeeper
> sessionid=0x154a566b247001c
> {code}
> We should account for dead server processing before making the assertion.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-15801) Upgrade checkstyle for all branches
[ https://issues.apache.org/jira/browse/HBASE-15801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-15801: --- Fix Version/s: (was: 1.4.0) > Upgrade checkstyle for all branches > --- > > Key: HBASE-15801 > URL: https://issues.apache.org/jira/browse/HBASE-15801 > Project: HBase > Issue Type: Bug > Components: build >Affects Versions: 2.0.0, 1.3.0, 1.2.1, 1.0.3, 0.98.19, 1.4.0, 1.1.5 >Reporter: Duo Zhang >Assignee: Duo Zhang > Labels: checkstyle > Fix For: 2.0.0, 1.3.0, 1.0.4, 1.2.2, 0.98.20, 1.1.6 > > Attachments: HBASE-15801-0.98.patch, HBASE-15801-branch-1.1.patch, > HBASE-15801-branch-1.patch, HBASE-15801.patch > > > We should use the same checkstyle for all branches. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-15816) Provide client with ability to set priority on Operations
[ https://issues.apache.org/jira/browse/HBASE-15816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-15816: --- Fix Version/s: (was: 1.5.0) > Provide client with ability to set priority on Operations > -- > > Key: HBASE-15816 > URL: https://issues.apache.org/jira/browse/HBASE-15816 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: churro morales >Assignee: churro morales > Fix For: 2.0.0, 3.0.0, 1.4.0 > > Attachments: HBASE-15816-v1.patch, HBASE-15816.patch, > HBASE-15816.v1.branch-1.patch, HBASE-15816.v2.patch > > > First round will just be to expose the ability to set priorities for client > operations. For more background: > http://mail-archives.apache.org/mod_mbox/hbase-dev/201604.mbox/%3CCA+RK=_BG_o=q8HMptcP2WauAinmEsL+15f3YEJuz=qbpcya...@mail.gmail.com%3E > Next step would be to remove AnnotationReadingPriorityFunction and have the > client send priorities explicitly. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-15913) Sasl encryption doesn't work with AsyncRpcChannel
[ https://issues.apache.org/jira/browse/HBASE-15913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-15913: --- Fix Version/s: (was: 1.4.0) > Sasl encryption doesn't work with AsyncRpcChannel > - > > Key: HBASE-15913 > URL: https://issues.apache.org/jira/browse/HBASE-15913 > Project: HBase > Issue Type: Bug >Reporter: Colin Ma >Assignee: Colin Ma > Fix For: 1.3.0 > > Attachments: HBASE-15913.001-branch-1.patch > > > The async RPC code in branch-1 will have problems with qop != auth. It's the > same as HBASE-15830. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-15913) Sasl encryption doesn't work with AsyncRpcChannel
[ https://issues.apache.org/jira/browse/HBASE-15913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-15913: --- Affects Version/s: (was: 1.4.0) > Sasl encryption doesn't work with AsyncRpcChannel > - > > Key: HBASE-15913 > URL: https://issues.apache.org/jira/browse/HBASE-15913 > Project: HBase > Issue Type: Bug >Reporter: Colin Ma >Assignee: Colin Ma > Fix For: 1.3.0 > > Attachments: HBASE-15913.001-branch-1.patch > > > The async RPC code in branch-1 will have problems with qop != auth. It's the > same as HBASE-15830. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-15895) remove unmaintained jenkins build analysis tool.
[ https://issues.apache.org/jira/browse/HBASE-15895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-15895: --- Fix Version/s: (was: 1.4.0) > remove unmaintained jenkins build analysis tool. > > > Key: HBASE-15895 > URL: https://issues.apache.org/jira/browse/HBASE-15895 > Project: HBase > Issue Type: Task > Components: build >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Minor > Fix For: 2.0.0, 1.3.0, 1.0.4, 1.2.2, 0.98.20, 1.1.6 > > Attachments: HBASE-15895.1.patch > > > See HBASE-15889. We don't actually maintain the "buildstats" module any more. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-15941) HBCK repair should not unsplit healthy splitted region
[ https://issues.apache.org/jira/browse/HBASE-15941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-15941: --- Fix Version/s: (was: 1.4.0) > HBCK repair should not unsplit healthy splitted region > -- > > Key: HBASE-15941 > URL: https://issues.apache.org/jira/browse/HBASE-15941 > Project: HBase > Issue Type: Improvement > Components: hbck >Affects Versions: 1.0.0 >Reporter: Stephen Yuan Jiang >Assignee: Esteban Gutierrez > Fix For: 2.0.0, 1.3.1, 1.2.6 > > Attachments: HBASE-15941.branch-1.001.patch, > HBASE-15941.master.001.patch, HBASE-15941.master.001.patch, > HBASE-15941.master.002.patch > > > Currently HBCK design in branch-1 has a flaw when repair option (the > -fixHdfsOverlaps option specifically) is specified: it would wrongly merge > split region (by looking at HDFS, it thinks that there exists overlapped > regions - parent region and daughter regions covers the same key range, of > course). See HBASE-15940 for details. > This JIRA tracks the improvement not-to-merge split region in HBCK repair. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-16124) Make check_compatibility.sh less verbose when building HBase
[
https://issues.apache.org/jira/browse/HBASE-16124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-16124:
---
Fix Version/s: (was: 1.4.0)
> Make check_compatibility.sh less verbose when building HBase
>
>
> Key: HBASE-16124
> URL: https://issues.apache.org/jira/browse/HBASE-16124
> Project: HBase
> Issue Type: Improvement
> Components: build, test
>Reporter: Dima Spivak
>Assignee: Dima Spivak
>Priority: Minor
> Fix For: 2.0.0, 1.3.0, 1.2.2, 1.1.6, 0.98.21
>
> Attachments: HBASE-16124_v1.patch
>
>
> {{[check_compatibility.sh|https://github.com/apache/hbase/blob/master/dev-support/check_compatibility.sh]}}
> is a bit verbose when building HBase JARs, which makes it kind of a
> nightmare when used in a Jenkins job. Let's run those steps in Maven's batch
> mode, which means less unnecessary output and no expectation of user
> interaction.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-16073) update compatibility_checker for jacc dropping comma sep args
[ https://issues.apache.org/jira/browse/HBASE-16073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-16073: --- Fix Version/s: (was: 1.4.0) > update compatibility_checker for jacc dropping comma sep args > - > > Key: HBASE-16073 > URL: https://issues.apache.org/jira/browse/HBASE-16073 > Project: HBase > Issue Type: Task > Components: build, documentation >Reporter: Sean Busbey >Assignee: Dima Spivak >Priority: Critical > Fix For: 2.0.0, 1.3.0, 1.2.2, 1.1.6, 0.98.21 > > Attachments: HBASE-16073_v1.patch, HBASE-16073_v2.patch > > > the japi-compliance-checker has a change in place (post the 1.7 release) that > removes the ability to give a comma separated list of jars on the cli. > we should switch to generating descriptor xml docs since that will still be > supported, or update to use the expanded tooling suggested in the issue: > https://github.com/lvc/japi-compliance-checker/issues/27 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-16129) check_compatibility.sh is broken when using Java API Compliance Checker v1.7
[ https://issues.apache.org/jira/browse/HBASE-16129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-16129: --- Fix Version/s: (was: 1.4.0) > check_compatibility.sh is broken when using Java API Compliance Checker v1.7 > > > Key: HBASE-16129 > URL: https://issues.apache.org/jira/browse/HBASE-16129 > Project: HBase > Issue Type: Bug > Components: test >Reporter: Dima Spivak >Assignee: Dima Spivak > Fix For: 2.0.0, 1.3.0, 1.2.2, 1.1.6, 0.98.21 > > Attachments: HBASE-16129_v1.patch, HBASE-16129_v2.patch, > HBASE-16129_v3.patch > > > As part of HBASE-16073, we hardcoded check_compatiblity.sh to check out the > v1.7 tag of Java ACC. Unfortunately, just running it between two branches > that I know have incompatibilities, I get 0 incompatibilities (and 0 classes > read). Looks like this version doesn't properly traverse through JARs. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-16074) ITBLL fails, reports lost big or tiny families
[ https://issues.apache.org/jira/browse/HBASE-16074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-16074: --- Fix Version/s: (was: 1.4.0) > ITBLL fails, reports lost big or tiny families > -- > > Key: HBASE-16074 > URL: https://issues.apache.org/jira/browse/HBASE-16074 > Project: HBase > Issue Type: Bug > Components: integration tests >Affects Versions: 1.3.0, 0.98.20 >Reporter: Mikhail Antonov >Assignee: stack >Priority: Blocker > Fix For: 2.0.0, 1.3.0 > > Attachments: 16074.test.branch-1.3.patch, 16074.test.patch, > HBASE-16074.branch-1.3.001.patch, HBASE-16074.branch-1.3.002.patch, > HBASE-16074.branch-1.3.003.patch, HBASE-16074.branch-1.3.003.patch, > changes_to_stress_ITBLL.patch, changes_to_stress_ITBLL__a_bit_relaxed_.patch, > itbll log with failure, itbll log with success > > > Underlying MR jobs succeed but I'm seeing the following in the logs (mid-size > distributed test cluster): > ERROR test.IntegrationTestBigLinkedList$Verify: Found nodes which lost big or > tiny families, count=164 > I do not know exactly yet whether it's a bug, a test issue or env setup > issue, but need figure it out. Opening this to raise awareness and see if > someone saw that recently. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-16051) TestScannerHeartbeatMessages fails on some machines
[ https://issues.apache.org/jira/browse/HBASE-16051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-16051: --- Fix Version/s: (was: 1.4.0) > TestScannerHeartbeatMessages fails on some machines > --- > > Key: HBASE-16051 > URL: https://issues.apache.org/jira/browse/HBASE-16051 > Project: HBase > Issue Type: Test > Components: test >Affects Versions: 1.3.0 >Reporter: Mikhail Antonov >Assignee: Phil Yang > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-16051-v1.patch > > > I can see below on my Linux box (reproduces consistently). It passes on OSX > laptop. > T E S T S > --- > Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=256m; > support was removed in 8.0 > Running org.apache.hadoop.hbase.regionserver.TestScannerHeartbeatMessages > Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 27.907 sec > <<< FAILURE! - in > org.apache.hadoop.hbase.regionserver.TestScannerHeartbeatMessages > testScannerHeartbeatMessages(org.apache.hadoop.hbase.regionserver.TestScannerHeartbeatMessages) > Time elapsed: 12.95 sec <<< FAILURE! > java.lang.AssertionError: Heartbeats messages are disabled, an exception > should be thrown. If an exception is not thrown, the test case is not > testing the importance of heartbeat messages > at org.junit.Assert.fail(Assert.java:88) > at > org.apache.hadoop.hbase.regionserver.TestScannerHeartbeatMessages.testImportanceOfHeartbeats(TestScannerHeartbeatMessages.java:233) > at > org.apache.hadoop.hbase.regionserver.TestScannerHeartbeatMessages.testScannerHeartbeatMessages(TestScannerHeartbeatMessages.java:204) > Results : > Failed tests: > > TestScannerHeartbeatMessages.testScannerHeartbeatMessages:204->testImportanceOfHeartbeats:233 > Heartbeats messages are disabled, an exception should be thrown. If an > exception is not thrown, the test case is not testing the importance of > heartbeat messages > Tests run: 1, Failures: 1, Errors: 0, Skipped: 0 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-16373) precommit needs a dockerfile with hbase prereqs
[ https://issues.apache.org/jira/browse/HBASE-16373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-16373: --- Fix Version/s: (was: 1.4.0) > precommit needs a dockerfile with hbase prereqs > --- > > Key: HBASE-16373 > URL: https://issues.apache.org/jira/browse/HBASE-16373 > Project: HBase > Issue Type: Bug > Components: build >Affects Versions: 1.3.0, 1.4.0, 1.1.6, 1.2.3, 0.98.22 >Reporter: Sean Busbey >Assignee: Duo Zhang >Priority: Critical > Fix For: 1.3.0, 1.1.7, 0.98.23, 1.2.4 > > Attachments: HBASE-16373-0.98.patch, HBASE-16373-branch-1.patch > > > specifically, we need protoc. starting with the dockerfile used by default in > yetus and adding it will probably suffice. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-16450) Shell tool to dump replication queues
[ https://issues.apache.org/jira/browse/HBASE-16450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-16450: --- Fix Version/s: (was: 1.4.0) > Shell tool to dump replication queues > -- > > Key: HBASE-16450 > URL: https://issues.apache.org/jira/browse/HBASE-16450 > Project: HBase > Issue Type: Improvement > Components: Operability, Replication >Affects Versions: 2.0.0, 1.3.0, 1.1.5, 1.2.2 >Reporter: Esteban Gutierrez >Assignee: Esteban Gutierrez > Fix For: 2.0.0, 1.3.0, 0.98.24 > > Attachments: HBASE-16450.branch-1.001.patch, > HBASE-16450.branch-1.002.patch, HBASE-16450.master.001.patch, > HBASE-16450.master.002.patch, HBASE-16450.master.003.patch > > > Currently there is no way to dump list of the configured queues and the > replication queues when replication is enabled. Unfortunately the HBase > master only offers an option to dump the whole content of the znodes but not > details on the queues being processed on each RS. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-16611) Flakey org.apache.hadoop.hbase.client.TestReplicasClient.testCancelOfMultiGet
[
https://issues.apache.org/jira/browse/HBASE-16611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-16611:
---
Fix Version/s: (was: 1.5.0)
> Flakey org.apache.hadoop.hbase.client.TestReplicasClient.testCancelOfMultiGet
> -
>
> Key: HBASE-16611
> URL: https://issues.apache.org/jira/browse/HBASE-16611
> Project: HBase
> Issue Type: Bug
>Reporter: Heng Chen
>Assignee: Heng Chen
> Fix For: 2.0.0, 3.0.0, 1.4.0
>
> Attachments: HBASE-16611-branch-1.patch, HBASE-16611.patch,
> HBASE-16611.v1.patch, HBASE-16611.v1.patch, HBASE-16611.v2.patch
>
>
> see
> https://builds.apache.org/job/PreCommit-HBASE-Build/3494/artifact/patchprocess/patch-unit-hbase-server.txt
> {code}
> testCancelOfMultiGet(org.apache.hadoop.hbase.client.TestReplicasClient) Time
> elapsed: 4.026 sec <<< FAILURE!
> java.lang.AssertionError: null
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertTrue(Assert.java:52)
> at
> org.apache.hadoop.hbase.client.TestReplicasClient.testCancelOfMultiGet(TestReplicasClient.java:579)
> Tests run: 26, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 94.401 sec -
> in org.apache.hadoop.hbase.client.TestAdmin2
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 11.861 sec -
> in org.apache.hadoop.hbase.client.TestClientScannerRPCTimeout
> Running
> org.apache.hadoop.hbase.client.TestRestoreSnapshotFromClientWithRegionReplicas
> Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 261.925 sec
> <<< FAILURE! - in org.apache.hadoop.hbase.client.TestReplicasClient
> testCancelOfMultiGet(org.apache.hadoop.hbase.client.TestReplicasClient) Time
> elapsed: 4.522 sec <<< FAILURE!
> java.lang.AssertionError: null
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertTrue(Assert.java:52)
> at
> org.apache.hadoop.hbase.client.TestReplicasClient.testCancelOfMultiGet(TestReplicasClient.java:581)
> Running org.apache.hadoop.hbase.client.TestFastFail
> Tests run: 2, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 3.648 sec -
> in org.apache.hadoop.hbase.client.TestFastFail
> Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 277.894 sec
> <<< FAILURE! - in org.apache.hadoop.hbase.client.TestReplicasClient
> testCancelOfMultiGet(org.apache.hadoop.hbase.client.TestReplicasClient) Time
> elapsed: 5.359 sec <<< FAILURE!
> java.lang.AssertionError: null
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertTrue(Assert.java:52)
> at
> org.apache.hadoop.hbase.client.TestReplicasClient.testCancelOfMultiGet(TestReplicasClient.java:579)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-19200) Remove Registry and use AsyncRegistry directly
[ https://issues.apache.org/jira/browse/HBASE-19200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-19200: -- Summary: Remove Registry and use AsyncRegistry directly (was: Let ZooKeeperRegistry depend on ZKAsyncRegistry to get data from zookeeper) > Remove Registry and use AsyncRegistry directly > -- > > Key: HBASE-19200 > URL: https://issues.apache.org/jira/browse/HBASE-19200 > Project: HBase > Issue Type: Task > Components: Client, Zookeeper >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19200-v1.patch, HBASE-19200.patch > > > So that we can move most of the zookeeper related code out of hbase-client > module. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-16518) Remove old .arcconfig file
[ https://issues.apache.org/jira/browse/HBASE-16518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-16518: --- Fix Version/s: (was: 1.4.0) > Remove old .arcconfig file > -- > > Key: HBASE-16518 > URL: https://issues.apache.org/jira/browse/HBASE-16518 > Project: HBase > Issue Type: Task > Components: tooling >Reporter: Gary Helmling >Assignee: Gary Helmling >Priority: Trivial > Fix For: 2.0.0, 1.3.0, 1.0.4, 0.98.22, 1.1.7, 1.2.4 > > Attachments: HBASE-16518.001.patch > > > The project .arcconfig file points to a project that no longer exists on a no > longer supported phabricator instance. Since it is no longer used for > reviews, let's drop it. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-16630) Fragmentation in long running Bucket Cache
[ https://issues.apache.org/jira/browse/HBASE-16630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-16630: --- Fix Version/s: (was: 1.4.0) > Fragmentation in long running Bucket Cache > -- > > Key: HBASE-16630 > URL: https://issues.apache.org/jira/browse/HBASE-16630 > Project: HBase > Issue Type: Bug > Components: BucketCache >Affects Versions: 2.0.0, 1.1.6, 1.3.1, 1.2.3 >Reporter: deepankar >Assignee: deepankar >Priority: Critical > Fix For: 2.0.0, 1.3.1, 1.2.6 > > Attachments: 16630-v2-suggest.patch, 16630-v3-suggest.patch, > HBASE-16630-v2.patch, HBASE-16630-v3-branch-1.X.patch, > HBASE-16630-v3-branch-1.patch, HBASE-16630-v3.patch, > HBASE-16630-v4-branch-1.X.patch, HBASE-16630.patch > > > As we are running bucket cache for a long time in our system, we are > observing cases where some nodes after some time does not fully utilize the > bucket cache, in some cases it is even worse in the sense they get stuck at a > value < 0.25 % of the bucket cache (DEFAULT_MEMORY_FACTOR as all our tables > are configured in-memory for simplicity sake). > We took a heap dump and analyzed what is happening and saw that is classic > case of fragmentation, current implementation of BucketCache (mainly > BucketAllocator) relies on the logic that fullyFreeBuckets are available for > switching/adjusting cache usage between different bucketSizes . But once a > compaction / bulkload happens and the blocks are evicted from a bucket size , > these are usually evicted from random places of the buckets of a bucketSize > and thus locking the number of buckets associated with a bucketSize and in > the worst case of the fragmentation we have seen some bucketSizes with > occupancy ratio of < 10 % But they dont have any completelyFreeBuckets to > share with the other bucketSize. > Currently the existing eviction logic helps in the cases where cache used is > more the MEMORY_FACTOR or MULTI_FACTOR and once those evictions are also > done, the eviction (freeSpace function) will not evict anything and the cache > utilization will be stuck at that value without any allocations for other > required sizes. > The fix for this we came up with is simple that we do deFragmentation ( > compaction) of the bucketSize and thus increasing the occupancy ratio and > also freeing up the buckets to be fullyFree, this logic itself is not > complicated as the bucketAllocator takes care of packing the blocks in the > buckets, we need evict and re-allocate the blocks for all the BucketSizes > that dont fit the criteria. > I am attaching an initial patch just to give an idea of what we are thinking > and I'll improve it based on the comments from the community. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-16664) Timeout logic in AsyncProcess is broken
[ https://issues.apache.org/jira/browse/HBASE-16664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-16664: --- Fix Version/s: (was: 1.4.0) > Timeout logic in AsyncProcess is broken > --- > > Key: HBASE-16664 > URL: https://issues.apache.org/jira/browse/HBASE-16664 > Project: HBase > Issue Type: Bug >Reporter: Phil Yang >Assignee: Phil Yang > Fix For: 2.0.0, 1.3.0 > > Attachments: 1.patch, HBASE-16664-branch-1-v1.patch, > HBASE-16664-branch-1-v1.patch, HBASE-16664-branch-1-v2.patch, > HBASE-16664-branch-1.1-v1.patch, HBASE-16664-branch-1.2-v1.patch, > HBASE-16664-branch-1.3-v1.patch, HBASE-16664-branch-1.3-v2.patch, > HBASE-16664-branch-1.3.v3.patch, HBASE-16664-branch-1.v3.patch, > HBASE-16664-v1.patch, HBASE-16664-v2.patch, HBASE-16664-v3.patch, > HBASE-16664-v4.patch, HBASE-16664-v5.patch, HBASE-16664-v6.patch, > HBASE-16664-v7.patch, testhcm.patch > > > Rpc/operation timeout logic in AsyncProcess is broken. And Table's > set*Timeout does not take effect in its AP or BufferedMutator. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-16752) Upgrading from 1.2 to 1.3 can lead to replication failures due to difference in RPC size limit
[ https://issues.apache.org/jira/browse/HBASE-16752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-16752: --- Fix Version/s: (was: 1.4.0) > Upgrading from 1.2 to 1.3 can lead to replication failures due to difference > in RPC size limit > -- > > Key: HBASE-16752 > URL: https://issues.apache.org/jira/browse/HBASE-16752 > Project: HBase > Issue Type: Bug > Components: Replication, rpc >Affects Versions: 2.0.0, 1.3.0 >Reporter: Ashu Pachauri >Assignee: Ashu Pachauri > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-16752.V1.patch, HBASE-16752.V2.patch, > HBASE-16752.addendum.patch > > > In HBase 1.2, we don't limit size of a single RPC but in 1.3 we limit it by > default to 256 MB. This means that during upgrade scenarios (or when source > is 1.2 peer is already on 1.3), it's possible to encounter a situation where > we try to send an rpc with size greater than 256 MB because we never unroll a > WALEdit while sending replication traffic. > RpcServer throws the underlying exception locally, but closes the connection > with returning the underlying error to the client, and client only sees a > "Broken pipe" error. > I am not sure what is the proper fix here (or if one is needed) to make sure > this does not happen, but we should return the underlying exception to the > RpcClient, because without it, it can be difficult to diagnose the problem, > especially for someone new to HBase. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-16807) RegionServer will fail to report new active Hmaster until HMaster/RegionServer failover
[
https://issues.apache.org/jira/browse/HBASE-16807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-16807:
---
Fix Version/s: (was: 1.4.0)
> RegionServer will fail to report new active Hmaster until
> HMaster/RegionServer failover
> ---
>
> Key: HBASE-16807
> URL: https://issues.apache.org/jira/browse/HBASE-16807
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
> Fix For: 2.0.0, 1.3.0, 1.2.4, 0.98.24, 1.1.8
>
> Attachments: HBASE-16807-0.98.patch, HBASE-16807-branch-1.1.patch,
> HBASE-16807-branch-1.2.patch, HBASE-16807-branch-1.3.patch,
> HBASE-16807-branch-1.patch, HBASE-16807.patch
>
>
> It's little weird, but it happened in the product environment that few
> RegionServer missed master znode create notification on master failover. In
> that case ZooKeeperNodeTracker will not refresh the cached data and
> MasterAddressTracker will always return old active HM detail to Region server
> on ServiceException.
> Though We create region server stub on failure but without refreshing the
> MasterAddressTracker data.
> In HRegionServer.createRegionServerStatusStub()
> {code}
> boolean refresh = false; // for the first time, use cached data
> RegionServerStatusService.BlockingInterface intf = null;
> boolean interrupted = false;
> try {
> while (keepLooping()) {
> sn = this.masterAddressTracker.getMasterAddress(refresh);
> if (sn == null) {
> if (!keepLooping()) {
> // give up with no connection.
> LOG.debug("No master found and cluster is stopped; bailing out");
> return null;
> }
> if (System.currentTimeMillis() > (previousLogTime + 1000)) {
> LOG.debug("No master found; retry");
> previousLogTime = System.currentTimeMillis();
> }
> refresh = true; // let's try pull it from ZK directly
> if (sleep(200)) {
> interrupted = true;
> }
> continue;
> }
> {code}
> Here we refresh node only when 'sn' is NULL otherwise it will use same cached
> data.
> So in above case RegionServer will never report active HMaster successfully
> until HMaster failover or RegionServer restart.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-16710) Add ZStandard Codec to Compression.java
[ https://issues.apache.org/jira/browse/HBASE-16710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-16710: --- Fix Version/s: (was: 1.4.0) > Add ZStandard Codec to Compression.java > --- > > Key: HBASE-16710 > URL: https://issues.apache.org/jira/browse/HBASE-16710 > Project: HBase > Issue Type: Task >Affects Versions: 2.0.0 >Reporter: churro morales >Assignee: churro morales >Priority: Minor > Fix For: 2.0.0, 1.3.0, 1.2.5, 1.1.9 > > Attachments: HBASE-16710-0.98.patch, HBASE-16710-1.2.patch, > HBASE-16710.patch > > > HADOOP-13578 is adding the ZStandardCodec to hadoop. This is a placeholder > to ensure it gets added to hbase once this gets upstream. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-16754) Regions failing compaction due to referencing non-existent store file
[
https://issues.apache.org/jira/browse/HBASE-16754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-16754:
---
Fix Version/s: (was: 1.4.0)
> Regions failing compaction due to referencing non-existent store file
> -
>
> Key: HBASE-16754
> URL: https://issues.apache.org/jira/browse/HBASE-16754
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.2.3
>Reporter: Gary Helmling
>Assignee: Gary Helmling
>Priority: Blocker
> Fix For: 2.0.0, 1.3.0, 1.2.4
>
> Attachments: HBASE-16754.001.patch, HBASE-16754.branch-1.001.patch,
> HBASE-16754.branch-1.2.001.patch
>
>
> Running a mixed read write workload on a recent build off branch-1.3, we are
> seeing compactions occasionally fail with errors like the following (actual
> filenames replaced with placeholders):
> {noformat}
> 16/09/27 16:57:28 ERROR regionserver.CompactSplitThread: Compaction selection
> failed Store = XXX, pri = 116
> java.io.FileNotFoundException: File does not exist:
> hdfs://.../hbase/data/ns/table/region/cf/XXfilenameXX
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1309)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1301)
> at
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1317)
> at
> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:421)
>
> at
> org.apache.hadoop.hbase.regionserver.StoreFileInfo.getReferencedFileStatus(StoreFileInfo.java:342)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileInfo.getFileStatus(StoreFileInfo.java:355)
>
> at
> org.apache.hadoop.hbase.regionserver.StoreFileInfo.getModificationTime(StoreFileInfo.java:360)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.getModificationTimeStamp(StoreFile.java:321)
>
> at
> org.apache.hadoop.hbase.regionserver.StoreUtils.getLowestTimestamp(StoreUtils.java:63)
> at
> org.apache.hadoop.hbase.regionserver.compactions.RatioBasedCompactionPolicy.shouldPerformMajorCompaction(RatioBasedCompactionPolicy.java:63)
> at
> org.apache.hadoop.hbase.regionserver.compactions.SortedCompactionPolicy.selectCompaction(SortedCompactionPolicy.java:82)
>
> at
> org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.select(DefaultStoreEngine.java:107)
>
> at
> org.apache.hadoop.hbase.regionserver.HStore.requestCompaction(HStore.java:1644)
> at
> org.apache.hadoop.hbase.regionserver.CompactSplitThread.selectCompaction(CompactSplitThread.java:373)
> at
> org.apache.hadoop.hbase.regionserver.CompactSplitThread.access$100(CompactSplitThread.java:59)
> at
> org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.doCompaction(CompactSplitThread.java:498)
> at
> org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.run(CompactSplitThread.java:568)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> 16/09/27 17:01:31 ERROR regionserver.CompactSplitThread: Compaction selection
> failed Store = XXX, pri = 115
> java.io.FileNotFoundException: File does not exist:
> hdfs://.../hbase/data/ns/table/region/cf/XXfilenameXX
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1309)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1301)
> at
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1317)
> at
> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:421)
>
> at
> org.apache.hadoop.hbase.regionserver.StoreFileInfo.getReferencedFileStatus(StoreFileInfo.java:342)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileInfo.getFileStatus(StoreFileInfo.java:355)
>
> at
> org.apache.hadoop.hbase.regionserver.StoreFileInfo.getModificationTime(StoreFileInfo.java:360)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.getModificationTimeStamp(StoreFile.java:321)
>
> at
> org.apache.hadoop.hbase.regionserver.StoreUtils.getLowestTimestamp(StoreUtils.java:63)
> at
>
[jira] [Updated] (HBASE-16765) New SteppingRegionSplitPolicy, avoid too aggressive spread of regions for small tables.
[ https://issues.apache.org/jira/browse/HBASE-16765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-16765: --- Fix Version/s: (was: 1.4.0) > New SteppingRegionSplitPolicy, avoid too aggressive spread of regions for > small tables. > --- > > Key: HBASE-16765 > URL: https://issues.apache.org/jira/browse/HBASE-16765 > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl >Priority: Critical > Fix For: 2.0.0, 1.3.0, 1.2.5, 0.98.24, 1.1.8 > > Attachments: 16765-0.98.txt > > > We just did some experiments on some larger clusters and found that while > using IncreasingToUpperBoundRegionSplitPolicy generally works well and is > very convenient, it does tend to produce too many regions. > Since the logic is - by design - local, checking the number of regions of the > table in question on the local server only, we end with more regions then > necessary. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
