[jira] [Updated] (HBASE-20095) Redesign single instance pool in CleanerChore
[ https://issues.apache.org/jira/browse/HBASE-20095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-20095: -- Status: Patch Available (was: Open) v1 for review. > Redesign single instance pool in CleanerChore > - > > Key: HBASE-20095 > URL: https://issues.apache.org/jira/browse/HBASE-20095 > Project: HBase > Issue Type: Improvement >Reporter: Reid Chan >Assignee: Reid Chan >Priority: Critical > Attachments: HBASE-20095.master.001.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20095) Redesign single instance pool in CleanerChore
[ https://issues.apache.org/jira/browse/HBASE-20095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-20095: -- Attachment: HBASE-20095.master.001.patch > Redesign single instance pool in CleanerChore > - > > Key: HBASE-20095 > URL: https://issues.apache.org/jira/browse/HBASE-20095 > Project: HBase > Issue Type: Improvement >Reporter: Reid Chan >Assignee: Reid Chan >Priority: Critical > Attachments: HBASE-20095.master.001.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20110) Findbugs in zk and mr caused nightly #409 branch-2 to fail
[
https://issues.apache.org/jira/browse/HBASE-20110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16381631#comment-16381631
]
stack commented on HBASE-20110:
---
Arghhh... #411 failed for same dumb findbugs issues
{code}
07:36:20 | Vote | Subsystem | Runtime | Comment
07:36:20
07:36:20 | 0 |reexec | 0m 19s | Docker mode activated.
07:36:20 | ||| Prechecks
07:36:20 | ||| Compile Tests
07:36:20 | 0 |mvndep | 0m 11s | Maven dependency ordering
07:36:20 | +1 |mvninstall | 3m 40s | the source passed
07:36:20 | +1 | compile | 3m 27s | the source passed
07:36:20 | -0 | javac | 3m 27s | root has 32 issues.
07:36:20 | 0 | findbugs | 0m 0s | Skipped 15 modules in the
source tree
07:36:20 | ||| with no Java source.
07:36:20 | -1 | findbugs | 0m 27s | hbase-zookeeper in branch-2 has
3 extant
07:36:20 | ||| Findbugs warnings.
07:36:20 | -1 | findbugs | 0m 38s | hbase-mapreduce in branch-2 has
1 extant
07:36:20 | ||| Findbugs warnings.
07:36:20 | ||| Other Tests
07:36:20 | +1 | unit | 157m 5s | root in the source passed.
07:36:20 | || 181m 19s |
{code}
... we are close to a successful hbase2 nightly build.
> Findbugs in zk and mr caused nightly #409 branch-2 to fail
> --
>
> Key: HBASE-20110
> URL: https://issues.apache.org/jira/browse/HBASE-20110
> Project: HBase
> Issue Type: Bug
> Components: findbugs
>Reporter: stack
>Assignee: stack
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-20110.branch-2.001.patch
>
>
> Build nightly #409 on branch-2 passed all unit tests on hadoop2 and hadoop3
> and then failed because of extant findbugs issues in zk and mr.
> Here was report from the nightly:
> {code}
> 01:24:40 | Vote | Subsystem | Runtime | Comment
> 01:24:40
>
> 01:24:40 | 0 |reexec | 0m 17s | Docker mode activated.
> 01:24:40 | ||| Prechecks
> 01:24:40 | ||| Compile Tests
> 01:24:40 | 0 |mvndep | 0m 10s | Maven dependency ordering
> 01:24:40 | +1 |mvninstall | 3m 23s | the source passed
> 01:24:40 | +1 | compile | 3m 17s | the source passed
> 01:24:40 | -0 | javac | 3m 17s | root has 32 issues.
> 01:24:40 | 0 | findbugs | 0m 1s | Skipped 15 modules in the
> source tree
> 01:24:40 | ||| with no Java source.
> 01:24:40 | -1 | findbugs | 0m 22s | hbase-zookeeper in branch-2
> has 3 extant
> 01:24:40 | ||| Findbugs warnings.
> 01:24:40 | -1 | findbugs | 0m 32s | hbase-mapreduce in branch-2
> has 1 extant
> 01:24:40 | ||| Findbugs warnings.
> 01:24:40 | ||| Other Tests
> 01:24:40 | +1 | unit | 158m 41s | root in the source passed.
> 01:24:40 | || 180m 39s |
> {code}
> A couple of dead assigns and a compareTo w/o an equals ignored in original
> class but copy-paste was missing the findbugs exemption.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20110) Findbugs in zk and mr caused nightly #409 branch-2 to fail
[
https://issues.apache.org/jira/browse/HBASE-20110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-20110:
--
Description:
Build nightly #409 on branch-2 passed all unit tests on hadoop2 and hadoop3 and
then failed because of extant findbugs issues in zk and mr.
Here was report from the nightly:
{code}
01:24:40 | Vote | Subsystem | Runtime | Comment
01:24:40
01:24:40 | 0 |reexec | 0m 17s | Docker mode activated.
01:24:40 | ||| Prechecks
01:24:40 | ||| Compile Tests
01:24:40 | 0 |mvndep | 0m 10s | Maven dependency ordering
01:24:40 | +1 |mvninstall | 3m 23s | the source passed
01:24:40 | +1 | compile | 3m 17s | the source passed
01:24:40 | -0 | javac | 3m 17s | root has 32 issues.
01:24:40 | 0 | findbugs | 0m 1s | Skipped 15 modules in the
source tree
01:24:40 | ||| with no Java source.
01:24:40 | -1 | findbugs | 0m 22s | hbase-zookeeper in branch-2 has
3 extant
01:24:40 | ||| Findbugs warnings.
01:24:40 | -1 | findbugs | 0m 32s | hbase-mapreduce in branch-2 has
1 extant
01:24:40 | ||| Findbugs warnings.
01:24:40 | ||| Other Tests
01:24:40 | +1 | unit | 158m 41s | root in the source passed.
01:24:40 | || 180m 39s |
{code}
A couple of dead assigns and a compareTo w/o an equals ignored in original
class but copy-paste was missing the findbugs exemption.
was:
Build nightly #409 on branch-2 passed all unit tests on hadoop2 and hadoop3 and
then failed because of extant findbugs issues in zk and mr.
> Findbugs in zk and mr caused nightly #409 branch-2 to fail
> --
>
> Key: HBASE-20110
> URL: https://issues.apache.org/jira/browse/HBASE-20110
> Project: HBase
> Issue Type: Bug
> Components: findbugs
>Reporter: stack
>Assignee: stack
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-20110.branch-2.001.patch
>
>
> Build nightly #409 on branch-2 passed all unit tests on hadoop2 and hadoop3
> and then failed because of extant findbugs issues in zk and mr.
> Here was report from the nightly:
> {code}
> 01:24:40 | Vote | Subsystem | Runtime | Comment
> 01:24:40
>
> 01:24:40 | 0 |reexec | 0m 17s | Docker mode activated.
> 01:24:40 | ||| Prechecks
> 01:24:40 | ||| Compile Tests
> 01:24:40 | 0 |mvndep | 0m 10s | Maven dependency ordering
> 01:24:40 | +1 |mvninstall | 3m 23s | the source passed
> 01:24:40 | +1 | compile | 3m 17s | the source passed
> 01:24:40 | -0 | javac | 3m 17s | root has 32 issues.
> 01:24:40 | 0 | findbugs | 0m 1s | Skipped 15 modules in the
> source tree
> 01:24:40 | ||| with no Java source.
> 01:24:40 | -1 | findbugs | 0m 22s | hbase-zookeeper in branch-2
> has 3 extant
> 01:24:40 | ||| Findbugs warnings.
> 01:24:40 | -1 | findbugs | 0m 32s | hbase-mapreduce in branch-2
> has 1 extant
> 01:24:40 | ||| Findbugs warnings.
> 01:24:40 | ||| Other Tests
> 01:24:40 | +1 | unit | 158m 41s | root in the source passed.
> 01:24:40 | || 180m 39s |
> {code}
> A couple of dead assigns and a compareTo w/o an equals ignored in original
> class but copy-paste was missing the findbugs exemption.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (HBASE-20110) Findbugs in zk and mr caused nightly #409 branch-2 to fail
[ https://issues.apache.org/jira/browse/HBASE-20110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-20110. --- Resolution: Fixed Assignee: stack Fix Version/s: 2.0.0-beta-2 .001 is what I pushed on master and branch-2. Fixes are basic, non-controversial. > Findbugs in zk and mr caused nightly #409 branch-2 to fail > -- > > Key: HBASE-20110 > URL: https://issues.apache.org/jira/browse/HBASE-20110 > Project: HBase > Issue Type: Bug > Components: findbugs >Reporter: stack >Assignee: stack >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-20110.branch-2.001.patch > > > Build nightly #409 on branch-2 passed all unit tests on hadoop2 and hadoop3 > and then failed because of extant findbugs issues in zk and mr. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20110) Findbugs in zk and mr caused nightly #409 branch-2 to fail
[ https://issues.apache.org/jira/browse/HBASE-20110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-20110: -- Attachment: HBASE-20110.branch-2.001.patch > Findbugs in zk and mr caused nightly #409 branch-2 to fail > -- > > Key: HBASE-20110 > URL: https://issues.apache.org/jira/browse/HBASE-20110 > Project: HBase > Issue Type: Bug > Components: findbugs >Reporter: stack >Priority: Major > Attachments: HBASE-20110.branch-2.001.patch > > > Build nightly #409 on branch-2 passed all unit tests on hadoop2 and hadoop3 > and then failed because of extant findbugs issues in zk and mr. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-20110) Findbugs in zk and mr caused nightly #409 branch-2 to fail
stack created HBASE-20110: - Summary: Findbugs in zk and mr caused nightly #409 branch-2 to fail Key: HBASE-20110 URL: https://issues.apache.org/jira/browse/HBASE-20110 Project: HBase Issue Type: Bug Components: findbugs Reporter: stack Build nightly #409 on branch-2 passed all unit tests on hadoop2 and hadoop3 and then failed because of extant findbugs issues in zk and mr. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19598) Fix TestAssignmentManagerMetrics flaky test
[ https://issues.apache.org/jira/browse/HBASE-19598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16381578#comment-16381578 ] stack commented on HBASE-19598: --- .002 rebase, address [~busbey] comment from rb. Test does not hang any more but metrics compares fail asserts. Progress? > Fix TestAssignmentManagerMetrics flaky test > --- > > Key: HBASE-19598 > URL: https://issues.apache.org/jira/browse/HBASE-19598 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0-beta-1 >Reporter: Balazs Meszaros >Assignee: stack >Priority: Major > Attachments: HBASE-19598.master.001.patch, > HBASE-19598.master.002.patch, HBASE-19598.master.003.patch, > HBASE-19598.master.003.patch, HBASE-19598.master.004.patch, TestUtil.java > > > TestAssignmentManagerMetrics fails constantly. After bisecting, it seems that > commit 010012cbcb broke it (HBASE-18946). > The test method runs successfully, but it cannot shut the minicluster down, > and hangs forever. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19598) Fix TestAssignmentManagerMetrics flaky test
[ https://issues.apache.org/jira/browse/HBASE-19598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-19598: -- Attachment: HBASE-19598.master.004.patch > Fix TestAssignmentManagerMetrics flaky test > --- > > Key: HBASE-19598 > URL: https://issues.apache.org/jira/browse/HBASE-19598 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0-beta-1 >Reporter: Balazs Meszaros >Assignee: stack >Priority: Major > Attachments: HBASE-19598.master.001.patch, > HBASE-19598.master.002.patch, HBASE-19598.master.003.patch, > HBASE-19598.master.003.patch, HBASE-19598.master.004.patch, TestUtil.java > > > TestAssignmentManagerMetrics fails constantly. After bisecting, it seems that > commit 010012cbcb broke it (HBASE-18946). > The test method runs successfully, but it cannot shut the minicluster down, > and hangs forever. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20050) Reimplement updateReplicationPositions logic in serial replication based on the newly introduced replication storage layer
[
https://issues.apache.org/jira/browse/HBASE-20050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16381558#comment-16381558
]
Hadoop QA commented on HBASE-20050:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
22s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 6m
8s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
42s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
14s{color} | {color:red} hbase-replication: The patch generated 6 new + 0
unchanged - 0 fixed = 6 total (was 0) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
18s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
22m 11s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
17s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
13s{color} | {color:red} hbase-replication generated 1 new + 0 unchanged - 0
fixed = 1 total (was 0) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
24s{color} | {color:green} hbase-replication in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}118m 50s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
40s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}169m 45s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.replication.TestReplicationDroppedTables |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-20050 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12912521/HBASE-20050.v1.patch |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 52e444760878 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
[jira] [Reopened] (HBASE-19656) Disable TestAssignmentManagerMetrics for beta-1
[ https://issues.apache.org/jira/browse/HBASE-19656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack reopened HBASE-19656: --- Reopening to push same disable to master branch; test is failing there also. > Disable TestAssignmentManagerMetrics for beta-1 > --- > > Key: HBASE-19656 > URL: https://issues.apache.org/jira/browse/HBASE-19656 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack >Priority: Major > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19656.branch-2.001.patch > > > TestAssignmentManagerMetrics fails reliably but up on apache and locally. > [~balazs.meszaros] is working on the issue over in HBASE-19598. For now I'm > going to disable it because it seems to be only test that always fails on > branch-2. Will follow this w/ an issue to reenable for beta-2. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-19656) Disable TestAssignmentManagerMetrics for beta-1
[ https://issues.apache.org/jira/browse/HBASE-19656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-19656. --- Resolution: Fixed I pushed this disabling to master branch also. See HBASE-19598 for figuring proper fix for this test. > Disable TestAssignmentManagerMetrics for beta-1 > --- > > Key: HBASE-19656 > URL: https://issues.apache.org/jira/browse/HBASE-19656 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack >Priority: Major > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19656.branch-2.001.patch > > > TestAssignmentManagerMetrics fails reliably but up on apache and locally. > [~balazs.meszaros] is working on the issue over in HBASE-19598. For now I'm > going to disable it because it seems to be only test that always fails on > branch-2. Will follow this w/ an issue to reenable for beta-2. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19598) Fix TestAssignmentManagerMetrics flaky test
[
https://issues.apache.org/jira/browse/HBASE-19598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16381550#comment-16381550
]
stack commented on HBASE-19598:
---
So, to bring this issue up to date, TestAssignmentManagerMetrics has not been
fixed on branch-2 nor on master branch. It was disabled by HBASE-19656 so could
cut hbase-2.0.0-beta-1. It was not disabled in master at the time. I just
cherry-picked the HBASE-19656 disabling JIRA to master so hopefully master
builds will start passing. In meantime, we can figure out what is going on in
this issue.
It fails for me locally. We are supposed to be shutting down but in thread
dump, there is a Master trying to become active Master
{code}
"M:0;localhost:52152" #169 daemon prio=5 os_prio=31 tid=0x7fe16556e800
nid=0x13203 in Object.wait() [0x7a6ca000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:92)
- locked <0x00071519c020> (a java.lang.Object)
at org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:56)
at
org.apache.hadoop.hbase.master.HMaster.waitForMasterActive(HMaster.java:664)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.initializeZooKeeper(HRegionServer.java:881)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.preRegistrationInitialization(HRegionServer.java:825)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:927)
at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:571)
at java.lang.Thread.run(Thread.java:745)
{code}
... and a Master thread is latched trying to run a modify table procedure.
There is a missing check on whether cluster/master is up in here somewhere.
TODO.
Let me update the current state of this patch.
> Fix TestAssignmentManagerMetrics flaky test
> ---
>
> Key: HBASE-19598
> URL: https://issues.apache.org/jira/browse/HBASE-19598
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0-beta-1
>Reporter: Balazs Meszaros
>Assignee: stack
>Priority: Major
> Attachments: HBASE-19598.master.001.patch,
> HBASE-19598.master.002.patch, HBASE-19598.master.003.patch,
> HBASE-19598.master.003.patch, TestUtil.java
>
>
> TestAssignmentManagerMetrics fails constantly. After bisecting, it seems that
> commit 010012cbcb broke it (HBASE-18946).
> The test method runs successfully, but it cannot shut the minicluster down,
> and hangs forever.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19598) Fix TestAssignmentManagerMetrics flaky test
[ https://issues.apache.org/jira/browse/HBASE-19598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16381539#comment-16381539 ] stack commented on HBASE-19598: --- Linking JIRA that DISABLED this test. > Fix TestAssignmentManagerMetrics flaky test > --- > > Key: HBASE-19598 > URL: https://issues.apache.org/jira/browse/HBASE-19598 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0-beta-1 >Reporter: Balazs Meszaros >Assignee: stack >Priority: Major > Attachments: HBASE-19598.master.001.patch, > HBASE-19598.master.002.patch, HBASE-19598.master.003.patch, > HBASE-19598.master.003.patch, TestUtil.java > > > TestAssignmentManagerMetrics fails constantly. After bisecting, it seems that > commit 010012cbcb broke it (HBASE-18946). > The test method runs successfully, but it cannot shut the minicluster down, > and hangs forever. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20107) Add a test case for HBASE-14317
[ https://issues.apache.org/jira/browse/HBASE-20107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-20107: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: (was: 3.0.0) 2.0.0-beta-2 Status: Resolved (was: Patch Available) Thanks for the patch, Zephyr > Add a test case for HBASE-14317 > --- > > Key: HBASE-20107 > URL: https://issues.apache.org/jira/browse/HBASE-20107 > Project: HBase > Issue Type: Test > Components: wal >Affects Versions: 3.0.0 >Reporter: Zephyr Guo >Assignee: Zephyr Guo >Priority: Minor > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-20107-v1.patch, HBASE-20107-v2.patch > > > RS might get stuck because of HBASE-14317. I met same problem in my hbase, > and HBASE-14317 can solve this problmen very well. > But I don't get stuck when I run > _TestWalLockup.testLockupWhenSyncInMiddleOfZigZagSetup_. > We'll never see stuck in this test case, because > FailedSyncBeforeLogCloseException will break > _zigzagLatch.waitSafePoint(SyncFuture)_. Finally, > _zigzagLatch.releaseSafePoint()_ will be call, and _attainSafePoint(long)_ > get out stuck. > Here, we add a new test case to simulate stuck RS correctly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HBASE-20107) Add a test case for HBASE-14317
[ https://issues.apache.org/jira/browse/HBASE-20107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu reassigned HBASE-20107: -- Assignee: Zephyr Guo > Add a test case for HBASE-14317 > --- > > Key: HBASE-20107 > URL: https://issues.apache.org/jira/browse/HBASE-20107 > Project: HBase > Issue Type: Test > Components: wal >Affects Versions: 3.0.0 >Reporter: Zephyr Guo >Assignee: Zephyr Guo >Priority: Minor > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-20107-v1.patch, HBASE-20107-v2.patch > > > RS might get stuck because of HBASE-14317. I met same problem in my hbase, > and HBASE-14317 can solve this problmen very well. > But I don't get stuck when I run > _TestWalLockup.testLockupWhenSyncInMiddleOfZigZagSetup_. > We'll never see stuck in this test case, because > FailedSyncBeforeLogCloseException will break > _zigzagLatch.waitSafePoint(SyncFuture)_. Finally, > _zigzagLatch.releaseSafePoint()_ will be call, and _attainSafePoint(long)_ > get out stuck. > Here, we add a new test case to simulate stuck RS correctly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20070) website generation is failing
[
https://issues.apache.org/jira/browse/HBASE-20070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16381525#comment-16381525
]
Sean Busbey commented on HBASE-20070:
-
what's your maven and git versions?
> website generation is failing
> -
>
> Key: HBASE-20070
> URL: https://issues.apache.org/jira/browse/HBASE-20070
> Project: HBase
> Issue Type: Bug
> Components: website
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Blocker
> Attachments: HBASE-20070-misty.patch, HBASE-20070-misty.patch.1,
> HBASE-20070-misty.patch.3, HBASE-20070.0.patch, HBASE-20070.1.patch,
> HBASE-20070.2.patch, HBASE-20070.3.patch, HBASE-20070.4.patch,
> HBASE-20070.5.patch,
> hbase-install-log-a29b3caf4dbc7b8833474ef5da5438f7f6907e00.txt
>
>
> website generation has been failing since Feb 20th
> {code}
> Checking out files: 100% (68971/68971), done.
> Usage: grep [OPTION]... PATTERN [FILE]...
> Try 'grep --help' for more information.
> PUSHED is 2
> is not yet mentioned in the hbase-site commit log. Assuming we don't have it
> yet. 2
> Building HBase
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=256m;
> support was removed in 8.0
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=256m;
> support was removed in 8.0
> Failure: mvn clean site
> Build step 'Execute shell' marked build as failure
> {code}
> The status email says
> {code}
> Build status: Still Failing
> The HBase website has not been updated to incorporate HBase commit
> ${CURRENT_HBASE_COMMIT}.
> {code}
> Looking at the code where that grep happens, it looks like the env variable
> CURRENT_HBASE_COMMIT isn't getting set. That comes from some git command. I'm
> guessing the version of git changed on the build hosts and upended our
> assumptions.
> we should fix this to 1) rely on git's porcelain interface, and 2) fail as
> soon as that git command fails
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20107) Add a test case for HBASE-14317
[
https://issues.apache.org/jira/browse/HBASE-20107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16381526#comment-16381526
]
Hadoop QA commented on HBASE-20107:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
48s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
44s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
18m 24s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}116m
30s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
23s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}157m 59s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-20107 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12912516/HBASE-20107-v2.patch |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux c41779c67152 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / bdedcc5631 |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/11735/testReport/ |
| Max. process+thread count | 4833 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/11735/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> Add a test case for HBASE-14317
>
[jira] [Commented] (HBASE-19769) IllegalAccessError on package-private Hadoop metrics2 classes in MapReduce jobs
[
https://issues.apache.org/jira/browse/HBASE-19769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16381524#comment-16381524
]
stack commented on HBASE-19769:
---
bq. Would have been useful to get the reports about issues regarding
HBASE-17448 expressed over there during review, or at least expressed over
there when this got pulled from 2.0.
Yes. Should have updated the original issue [~apurtell].
Do you know if this an issue in 1.4x [~elserj] also sir?
> IllegalAccessError on package-private Hadoop metrics2 classes in MapReduce
> jobs
> ---
>
> Key: HBASE-19769
> URL: https://issues.apache.org/jira/browse/HBASE-19769
> Project: HBase
> Issue Type: Bug
> Components: mapreduce, metrics
>Affects Versions: 2.0.0-beta-1
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19769.001.branch-2.patch
>
>
> issues for context: HBASE-17170, HBASE-17448, TEZ-3299, HADOOP-10893
> Since Hadoop 2.6.0, the {{yarn jar}} entry point to submit a YARN job has
> been using a custom classloader to separate Hadoop dependencies from the
> user's JAR being run. A separate classloader is created for the user-provided
> jar, and then this classloader is set as the contextClassLoader before the
> Tool is executed by Hadoop's RunJar class. This has been (mostly?) fine for
> us to date because we don't try to access any Hadoop internal classes
> client-side.
> However, with the ZK metrics, clients are pushing ZK metrics to metrics2. The
> problem is that Hadoop metrics2 implementations which we reference from the
> same package are loaded by a different classloader than our HBase code is
> loaded from. This makes the expected package-private access of these Metrics2
> classes (e.g. MetricsInfoImpl) fail with an IllegalAccessError.
> {noformat}
> java.lang.RuntimeException: Could not create interface
> org.apache.hadoop.hbase.zookeeper.MetricsZooKeeperSource Is the hadoop
> compatibility jar on the classpath?
> at
> org.apache.hadoop.hbase.CompatibilitySingletonFactory.getInstance(CompatibilitySingletonFactory.java:75)
> at
> org.apache.hadoop.hbase.zookeeper.ZKMetrics.(ZKMetrics.java:36)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.(RecoverableZooKeeper.java:115)
> at org.apache.hadoop.hbase.zookeeper.ZKUtil.connect(ZKUtil.java:139)
> at
> org.apache.hadoop.hbase.zookeeper.ZKWatcher.(ZKWatcher.java:128)
> at
> org.apache.hadoop.hbase.zookeeper.ZKWatcher.(ZKWatcher.java:102)
> at
> org.apache.hadoop.hbase.security.token.TokenUtil.getAuthToken(TokenUtil.java:293)
> at
> org.apache.hadoop.hbase.security.token.TokenUtil.addTokenForJob(TokenUtil.java:259)
> at
> org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.initCredentials(TableMapReduceUtil.java:535)
> at
> org.apache.phoenix.mapreduce.MultiHfileOutputFormat.configureIncrementalLoad(MultiHfileOutputFormat.java:712)
> at
> org.apache.phoenix.mapreduce.AbstractBulkLoadTool.submitJob(AbstractBulkLoadTool.java:300)
> at
> org.apache.phoenix.mapreduce.AbstractBulkLoadTool.loadData(AbstractBulkLoadTool.java:267)
> at
> org.apache.phoenix.mapreduce.AbstractBulkLoadTool.run(AbstractBulkLoadTool.java:180)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
> at
> org.apache.phoenix.mapreduce.CsvBulkLoadTool.main(CsvBulkLoadTool.java:109)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:239)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:153)
> Caused by: java.util.ServiceConfigurationError:
> org.apache.hadoop.hbase.zookeeper.MetricsZooKeeperSource: Provider
> org.apache.hadoop.hbase.zookeeper.MetricsZooKeeperSourceImpl could not be
> instantiated
> at java.util.ServiceLoader.fail(ServiceLoader.java:232)
> at java.util.ServiceLoader.access$100(ServiceLoader.java:185)
> at
> java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384)
> at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
> at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
> at
> org.apache.hadoop.hbase.CompatibilitySingletonFactory.getInstance(CompatibilitySingletonFactory.java:59)
> ... 21 more
> Caused by:
[jira] [Commented] (HBASE-20055) Remove declaration of un-thrown exceptions and unused setRegionStateBackToOpen() from MergeTableRegionsProcedure
[ https://issues.apache.org/jira/browse/HBASE-20055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16381516#comment-16381516 ] stack commented on HBASE-20055: --- What made the tests pass [~uagashe] ? Patch looks good. But so did the other one (smile). > Remove declaration of un-thrown exceptions and unused > setRegionStateBackToOpen() from MergeTableRegionsProcedure > > > Key: HBASE-20055 > URL: https://issues.apache.org/jira/browse/HBASE-20055 > Project: HBase > Issue Type: Improvement > Components: amv2 >Affects Versions: 2.0.0-beta-1 >Reporter: Umesh Agashe >Assignee: Umesh Agashe >Priority: Minor > Fix For: 2.0.0-beta-2 > > Attachments: hbase-20055.master.001.patch, > hbase-20055.master.001.patch, hbase-20055.master.002.patch > > > A few methods declare exceptions in throws statement that are not thrown and > method setRegionStateBackToOpen() is not used in MergeTableRegionsProcedure. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20070) website generation is failing
[
https://issues.apache.org/jira/browse/HBASE-20070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16381484#comment-16381484
]
Hadoop QA commented on HBASE-20070:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
9s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} shelldocs {color} | {color:blue} 0m
3s{color} | {color:blue} Shelldocs was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 6m
0s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m
29s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 4m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
0s{color} | {color:green} The patch generated 0 new + 0 unchanged - 1 fixed =
0 total (was 1) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
46s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
19m 12s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m
27s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}148m 6s{color}
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
31s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}203m 26s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.client.TestBlockEvictionFromClient |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-20070 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12912509/HBASE-20070.5.patch |
| Optional Tests | asflicense shellcheck shelldocs javac javadoc unit
shadedjars hadoopcheck xml compile |
| uname | Linux 2d676dfd98e4 3.13.0-133-generic #182-Ubuntu SMP Tue Sep 19
15:49:21 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / bdedcc5631 |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| shellcheck | v0.4.4 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/11734/artifact/patchprocess/patch-unit-root.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/11734/testReport/ |
| Max. process+thread count | 4073 (vs. ulimit of 1) |
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/11734/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org
[jira] [Comment Edited] (HBASE-20109) Add Admin#getMasterLocation API for lightweight discovery of the active master location
[
https://issues.apache.org/jira/browse/HBASE-20109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16381473#comment-16381473
]
Reid Chan edited comment on HBASE-20109 at 3/1/18 3:50 AM:
---
{code}
default ServerName getMaster() throws IOException {
return getClusterMetrics(EnumSet.of(Option.MASTER)).getMasterName();
}
{code}
Ya, it's lightweight, {{ServerName}} also contains host and port.
But i doubt that the option feature is in branch-1?
was (Author: reidchan):
{code}
default ServerName getMaster() throws IOException {
return getClusterMetrics(EnumSet.of(Option.MASTER)).getMasterName();
}
{code}
Ya, it's lightweight, {ServerName} also contains host and port.
But i doubt that the option feature is in branch-1?
> Add Admin#getMasterLocation API for lightweight discovery of the active
> master location
> ---
>
> Key: HBASE-20109
> URL: https://issues.apache.org/jira/browse/HBASE-20109
> Project: HBase
> Issue Type: Improvement
> Components: Client
>Affects Versions: 1.4.2
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Minor
> Fix For: 2.0.0, 1.5.0
>
>
> Right now the only public API available to the client to learn the server
> name of the active master is Admin#getClusterStatus#getMaster, returning
> ServerName. On a cluster of any size getClusterStatus is expensive,
> especially if used only to retrieve the active master name.
> Let's add a simple API
> {code}
> ServerName Admin#getMasterLocation()
> {code}
> for lightweight discovery of the active master location. This makes sense
> because, weirdly, Admin already has a method getMasterInfoPort(), returning
> int.
> Internally the client has a notion of the active master because there is a
> connection open to it, or one that can be reopened, or if for some reason
> it's not easy to make a ServerName for that state, the ServerName can be
> deserialized out of the znode tracking the active master location.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20109) Add Admin#getMasterLocation API for lightweight discovery of the active master location
[
https://issues.apache.org/jira/browse/HBASE-20109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16381473#comment-16381473
]
Reid Chan commented on HBASE-20109:
---
{code}
default ServerName getMaster() throws IOException {
return getClusterMetrics(EnumSet.of(Option.MASTER)).getMasterName();
}
{code}
Ya, it's lightweight, {ServerName} also contains host and port.
But i doubt that the option feature is in branch-1?
> Add Admin#getMasterLocation API for lightweight discovery of the active
> master location
> ---
>
> Key: HBASE-20109
> URL: https://issues.apache.org/jira/browse/HBASE-20109
> Project: HBase
> Issue Type: Improvement
> Components: Client
>Affects Versions: 1.4.2
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Minor
> Fix For: 2.0.0, 1.5.0
>
>
> Right now the only public API available to the client to learn the server
> name of the active master is Admin#getClusterStatus#getMaster, returning
> ServerName. On a cluster of any size getClusterStatus is expensive,
> especially if used only to retrieve the active master name.
> Let's add a simple API
> {code}
> ServerName Admin#getMasterLocation()
> {code}
> for lightweight discovery of the active master location. This makes sense
> because, weirdly, Admin already has a method getMasterInfoPort(), returning
> int.
> Internally the client has a notion of the active master because there is a
> connection open to it, or one that can be reopened, or if for some reason
> it's not easy to make a ServerName for that state, the ServerName can be
> deserialized out of the znode tracking the active master location.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20104) Fix infinite loop of RIT when creating table on a rsgroup that has no online servers
[ https://issues.apache.org/jira/browse/HBASE-20104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16381463#comment-16381463 ] Xiaolin Ha commented on HBASE-20104: [~busbey] branch-1 and branch-1.4 also need to be fixed. I will make another patch that can be applied to them. > Fix infinite loop of RIT when creating table on a rsgroup that has no online > servers > > > Key: HBASE-20104 > URL: https://issues.apache.org/jira/browse/HBASE-20104 > Project: HBase > Issue Type: Bug > Components: rsgroup >Affects Versions: 2.0.0-beta-2 >Reporter: Xiaolin Ha >Assignee: Xiaolin Ha >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-20104.branch-2.001.patch > > > This error has been reported in > https://builds.apache.org/job/PreCommit-HBASE-Build/11635/testReport/org.apache.hadoop.hbase.rsgroup/TestRSGroups/org_apache_hadoop_hbase_rsgroup_TestRSGroups/ > Cases that creating tables on a rsgroup which has been stopped or > decommissioned all region servers can reproduce this error. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20109) Add Admin#getMasterLocation API for lightweight discovery of the active master location
[
https://issues.apache.org/jira/browse/HBASE-20109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16381459#comment-16381459
]
Chia-Ping Tsai commented on HBASE-20109:
The ClusterMetrics#Option can limit the scope so the data carried by the RPC
call are only the master name. Howeve, seems the method this issue try to add
is get the master location rather than master name? There is already a similar
API which can retrieve the master name - Admin#getMaster
> Add Admin#getMasterLocation API for lightweight discovery of the active
> master location
> ---
>
> Key: HBASE-20109
> URL: https://issues.apache.org/jira/browse/HBASE-20109
> Project: HBase
> Issue Type: Improvement
> Components: Client
>Affects Versions: 1.4.2
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Minor
> Fix For: 2.0.0, 1.5.0
>
>
> Right now the only public API available to the client to learn the server
> name of the active master is Admin#getClusterStatus#getMaster, returning
> ServerName. On a cluster of any size getClusterStatus is expensive,
> especially if used only to retrieve the active master name.
> Let's add a simple API
> {code}
> ServerName Admin#getMasterLocation()
> {code}
> for lightweight discovery of the active master location. This makes sense
> because, weirdly, Admin already has a method getMasterInfoPort(), returning
> int.
> Internally the client has a notion of the active master because there is a
> connection open to it, or one that can be reopened, or if for some reason
> it's not easy to make a ServerName for that state, the ServerName can be
> deserialized out of the znode tracking the active master location.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20107) Add a test case for HBASE-14317
[ https://issues.apache.org/jira/browse/HBASE-20107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16381455#comment-16381455 ] Allan Yang commented on HBASE-20107: +1 for adding this case. This case helped us to verify the fix of HBASE-14317 > Add a test case for HBASE-14317 > --- > > Key: HBASE-20107 > URL: https://issues.apache.org/jira/browse/HBASE-20107 > Project: HBase > Issue Type: Test > Components: wal >Affects Versions: 3.0.0 >Reporter: Zephyr Guo >Priority: Minor > Fix For: 3.0.0 > > Attachments: HBASE-20107-v1.patch, HBASE-20107-v2.patch > > > RS might get stuck because of HBASE-14317. I met same problem in my hbase, > and HBASE-14317 can solve this problmen very well. > But I don't get stuck when I run > _TestWalLockup.testLockupWhenSyncInMiddleOfZigZagSetup_. > We'll never see stuck in this test case, because > FailedSyncBeforeLogCloseException will break > _zigzagLatch.waitSafePoint(SyncFuture)_. Finally, > _zigzagLatch.releaseSafePoint()_ will be call, and _attainSafePoint(long)_ > get out stuck. > Here, we add a new test case to simulate stuck RS correctly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20055) Remove declaration of un-thrown exceptions and unused setRegionStateBackToOpen() from MergeTableRegionsProcedure
[
https://issues.apache.org/jira/browse/HBASE-20055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16381444#comment-16381444
]
Hadoop QA commented on HBASE-20055:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
38s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 8m
13s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
51s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
57s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 6m
50s{color} | {color:red} The patch causes 10 errors with Hadoop v2.6.5. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 8m
45s{color} | {color:red} The patch causes 10 errors with Hadoop v2.7.4. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 10m
55s{color} | {color:red} The patch causes 10 errors with Hadoop v3.0.0. {color}
|
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}130m
8s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}174m 22s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-20055 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12912504/hbase-20055.master.002.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 067263385a46 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / bdedcc5631 |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/11733/testReport/ |
| Max. process+thread count | 4768 (vs.
[jira] [Updated] (HBASE-20050) Reimplement updateReplicationPositions logic in serial replication based on the newly introduced replication storage layer
[ https://issues.apache.org/jira/browse/HBASE-20050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Hu updated HBASE-20050: - Attachment: HBASE-20050.v1.patch > Reimplement updateReplicationPositions logic in serial replication based on > the newly introduced replication storage layer > -- > > Key: HBASE-20050 > URL: https://issues.apache.org/jira/browse/HBASE-20050 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Assignee: Zheng Hu >Priority: Major > Attachments: HBASE-20050.patch, HBASE-20050.v1.patch > > > Now, in serial replication, need to update the replication position both in > zookeeper and meta table, which may led to inconsistent problem. I'll > re-implement this part based on the uniform replication storage layer. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20109) Add Admin#getMasterLocation API for lightweight discovery of the active master location
[
https://issues.apache.org/jira/browse/HBASE-20109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16381440#comment-16381440
]
Andrew Purtell commented on HBASE-20109:
bq. Using clustermetrics + option seems a solution?
How so? We have a monitoring client that wants a Java API to call to get the
location of the current active master. Seems the HBase client API is the right
place to something like that to go.
> Add Admin#getMasterLocation API for lightweight discovery of the active
> master location
> ---
>
> Key: HBASE-20109
> URL: https://issues.apache.org/jira/browse/HBASE-20109
> Project: HBase
> Issue Type: Improvement
> Components: Client
>Affects Versions: 1.4.2
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Minor
> Fix For: 2.0.0, 1.5.0
>
>
> Right now the only public API available to the client to learn the server
> name of the active master is Admin#getClusterStatus#getMaster, returning
> ServerName. On a cluster of any size getClusterStatus is expensive,
> especially if used only to retrieve the active master name.
> Let's add a simple API
> {code}
> ServerName Admin#getMasterLocation()
> {code}
> for lightweight discovery of the active master location. This makes sense
> because, weirdly, Admin already has a method getMasterInfoPort(), returning
> int.
> Internally the client has a notion of the active master because there is a
> connection open to it, or one that can be reopened, or if for some reason
> it's not easy to make a ServerName for that state, the ServerName can be
> deserialized out of the znode tracking the active master location.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20109) Add Admin#getMasterLocation API for lightweight discovery of the active master location
[
https://issues.apache.org/jira/browse/HBASE-20109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16381432#comment-16381432
]
Chia-Ping Tsai commented on HBASE-20109:
Using clustermetrics + option seems a solution?
> Add Admin#getMasterLocation API for lightweight discovery of the active
> master location
> ---
>
> Key: HBASE-20109
> URL: https://issues.apache.org/jira/browse/HBASE-20109
> Project: HBase
> Issue Type: Improvement
> Components: Client
>Affects Versions: 1.4.2
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Minor
> Fix For: 2.0.0, 1.5.0
>
>
> Right now the only public API available to the client to learn the server
> name of the active master is Admin#getClusterStatus#getMaster, returning
> ServerName. On a cluster of any size getClusterStatus is expensive,
> especially if used only to retrieve the active master name.
> Let's add a simple API
> {code}
> ServerName Admin#getMasterLocation()
> {code}
> for lightweight discovery of the active master location. This makes sense
> because, weirdly, Admin already has a method getMasterInfoPort(), returning
> int.
> Internally the client has a notion of the active master because there is a
> connection open to it, or one that can be reopened, or if for some reason
> it's not easy to make a ServerName for that state, the ServerName can be
> deserialized out of the znode tracking the active master location.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20070) website generation is failing
[
https://issues.apache.org/jira/browse/HBASE-20070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16381430#comment-16381430
]
Misty Stanley-Jones commented on HBASE-20070:
-
Actually soft -1 on version 5. I forgot to mention that there is something
about the way you are adding files to git that is making it interactive (you
have to press {{q}} to get out of a pager). I didn't look into it to see what
the deal is but probably not appropriate for a Jenkins job.
> website generation is failing
> -
>
> Key: HBASE-20070
> URL: https://issues.apache.org/jira/browse/HBASE-20070
> Project: HBase
> Issue Type: Bug
> Components: website
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Blocker
> Attachments: HBASE-20070-misty.patch, HBASE-20070-misty.patch.1,
> HBASE-20070-misty.patch.3, HBASE-20070.0.patch, HBASE-20070.1.patch,
> HBASE-20070.2.patch, HBASE-20070.3.patch, HBASE-20070.4.patch,
> HBASE-20070.5.patch,
> hbase-install-log-a29b3caf4dbc7b8833474ef5da5438f7f6907e00.txt
>
>
> website generation has been failing since Feb 20th
> {code}
> Checking out files: 100% (68971/68971), done.
> Usage: grep [OPTION]... PATTERN [FILE]...
> Try 'grep --help' for more information.
> PUSHED is 2
> is not yet mentioned in the hbase-site commit log. Assuming we don't have it
> yet. 2
> Building HBase
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=256m;
> support was removed in 8.0
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=256m;
> support was removed in 8.0
> Failure: mvn clean site
> Build step 'Execute shell' marked build as failure
> {code}
> The status email says
> {code}
> Build status: Still Failing
> The HBase website has not been updated to incorporate HBase commit
> ${CURRENT_HBASE_COMMIT}.
> {code}
> Looking at the code where that grep happens, it looks like the env variable
> CURRENT_HBASE_COMMIT isn't getting set. That comes from some git command. I'm
> guessing the version of git changed on the build hosts and upended our
> assumptions.
> we should fix this to 1) rely on git's porcelain interface, and 2) fail as
> soon as that git command fails
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20070) website generation is failing
[
https://issues.apache.org/jira/browse/HBASE-20070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16381409#comment-16381409
]
Misty Stanley-Jones commented on HBASE-20070:
-
+1 on version 5.
> website generation is failing
> -
>
> Key: HBASE-20070
> URL: https://issues.apache.org/jira/browse/HBASE-20070
> Project: HBase
> Issue Type: Bug
> Components: website
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Blocker
> Attachments: HBASE-20070-misty.patch, HBASE-20070-misty.patch.1,
> HBASE-20070-misty.patch.3, HBASE-20070.0.patch, HBASE-20070.1.patch,
> HBASE-20070.2.patch, HBASE-20070.3.patch, HBASE-20070.4.patch,
> HBASE-20070.5.patch,
> hbase-install-log-a29b3caf4dbc7b8833474ef5da5438f7f6907e00.txt
>
>
> website generation has been failing since Feb 20th
> {code}
> Checking out files: 100% (68971/68971), done.
> Usage: grep [OPTION]... PATTERN [FILE]...
> Try 'grep --help' for more information.
> PUSHED is 2
> is not yet mentioned in the hbase-site commit log. Assuming we don't have it
> yet. 2
> Building HBase
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=256m;
> support was removed in 8.0
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=256m;
> support was removed in 8.0
> Failure: mvn clean site
> Build step 'Execute shell' marked build as failure
> {code}
> The status email says
> {code}
> Build status: Still Failing
> The HBase website has not been updated to incorporate HBase commit
> ${CURRENT_HBASE_COMMIT}.
> {code}
> Looking at the code where that grep happens, it looks like the env variable
> CURRENT_HBASE_COMMIT isn't getting set. That comes from some git command. I'm
> guessing the version of git changed on the build hosts and upended our
> assumptions.
> we should fix this to 1) rely on git's porcelain interface, and 2) fail as
> soon as that git command fails
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20107) Add a test case for HBASE-14317
[ https://issues.apache.org/jira/browse/HBASE-20107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16381406#comment-16381406 ] Zephyr Guo commented on HBASE-20107: Attached HBASE-20107-v2.patch to fix the checkstyle warmings. > Add a test case for HBASE-14317 > --- > > Key: HBASE-20107 > URL: https://issues.apache.org/jira/browse/HBASE-20107 > Project: HBase > Issue Type: Test > Components: wal >Affects Versions: 3.0.0 >Reporter: Zephyr Guo >Priority: Minor > Fix For: 3.0.0 > > Attachments: HBASE-20107-v1.patch, HBASE-20107-v2.patch > > > RS might get stuck because of HBASE-14317. I met same problem in my hbase, > and HBASE-14317 can solve this problmen very well. > But I don't get stuck when I run > _TestWalLockup.testLockupWhenSyncInMiddleOfZigZagSetup_. > We'll never see stuck in this test case, because > FailedSyncBeforeLogCloseException will break > _zigzagLatch.waitSafePoint(SyncFuture)_. Finally, > _zigzagLatch.releaseSafePoint()_ will be call, and _attainSafePoint(long)_ > get out stuck. > Here, we add a new test case to simulate stuck RS correctly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20107) Add a test case for HBASE-14317
[ https://issues.apache.org/jira/browse/HBASE-20107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zephyr Guo updated HBASE-20107: --- Attachment: HBASE-20107-v2.patch > Add a test case for HBASE-14317 > --- > > Key: HBASE-20107 > URL: https://issues.apache.org/jira/browse/HBASE-20107 > Project: HBase > Issue Type: Test > Components: wal >Affects Versions: 3.0.0 >Reporter: Zephyr Guo >Priority: Minor > Fix For: 3.0.0 > > Attachments: HBASE-20107-v1.patch, HBASE-20107-v2.patch > > > RS might get stuck because of HBASE-14317. I met same problem in my hbase, > and HBASE-14317 can solve this problmen very well. > But I don't get stuck when I run > _TestWalLockup.testLockupWhenSyncInMiddleOfZigZagSetup_. > We'll never see stuck in this test case, because > FailedSyncBeforeLogCloseException will break > _zigzagLatch.waitSafePoint(SyncFuture)_. Finally, > _zigzagLatch.releaseSafePoint()_ will be call, and _attainSafePoint(long)_ > get out stuck. > Here, we add a new test case to simulate stuck RS correctly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20108) `hbase zkcli` falls into a non-interactive prompt after HBASE-15199
[
https://issues.apache.org/jira/browse/HBASE-20108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16381380#comment-16381380
]
Hadoop QA commented on HBASE-20108:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} branch-2 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
3s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
3s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 13m
3s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
24s{color} | {color:green} branch-2 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} shadedjars {color} | {color:red} 3m
56s{color} | {color:red} patch has 13 errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
14m 19s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
15s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}172m
20s{color} | {color:green} root in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
29s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}211m 49s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:9f2f2db |
| JIRA Issue | HBASE-20108 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12912495/HBASE-20108.001.branch-2.patch
|
| Optional Tests | asflicense javac javadoc unit shadedjars hadoopcheck
xml compile |
| uname | Linux 4f79089cb915 3.13.0-137-generic #186-Ubuntu SMP Mon Dec 4
19:09:19 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | branch-2 / 313464f007 |
| maven | version: Apache Maven 3.5.2
(138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/11730/testReport/ |
| Max. process+thread count | 4384 (vs. ulimit of 1) |
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/11730/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> `hbase zkcli` falls into a non-interactive prompt after HBASE-15199
> ---
>
> Key: HBASE-20108
> URL: https://issues.apache.org/jira/browse/HBASE-20108
> Project: HBase
> Issue Type: Bug
> Components: Usability
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
> Attachments:
[jira] [Commented] (HBASE-19863) java.lang.IllegalStateException: isDelete failed when SingleColumnValueFilter is used
[
https://issues.apache.org/jira/browse/HBASE-19863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16381333#comment-16381333
]
Hudson commented on HBASE-19863:
FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4667 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/4667/])
HBASE-19863 java.lang.IllegalStateException: isDelete failed when (elserj: rev
393ab302ab08b70a839ec87e75fcf4b165765db2)
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
* (add)
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestIsDeleteFailure.java
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
> java.lang.IllegalStateException: isDelete failed when SingleColumnValueFilter
> is used
> -
>
> Key: HBASE-19863
> URL: https://issues.apache.org/jira/browse/HBASE-19863
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 1.4.1
>Reporter: Sergey Soldatov
>Assignee: Sergey Soldatov
>Priority: Major
> Fix For: 3.0.0, 1.5.0, 2.0.0-beta-2, 1.4.3
>
> Attachments: HBASE-19863-branch-2.patch, HBASE-19863-branch1.patch,
> HBASE-19863-test.patch, HBASE-19863.v2-branch-2.patch,
> HBASE-19863.v3-branch-2.patch, HBASE-19863.v4-branch-2.patch,
> HBASE-19863.v4-master.patch, HBASE-19863.v5-branch-1.4.patch,
> HBASE-19863.v5-branch-1.patch, HBASE-19863.v5-branch-2.patch
>
>
> Under some circumstances scan with SingleColumnValueFilter may fail with an
> exception
> {noformat}
> java.lang.IllegalStateException: isDelete failed: deleteBuffer=C3,
> qualifier=C2, timestamp=1516433595543, comparison result: 1
> at
> org.apache.hadoop.hbase.regionserver.ScanDeleteTracker.isDeleted(ScanDeleteTracker.java:149)
> at
> org.apache.hadoop.hbase.regionserver.ScanQueryMatcher.match(ScanQueryMatcher.java:386)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:545)
> at
> org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:147)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.populateResult(HRegion.java:5876)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:6027)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:5814)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2552)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32385)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2150)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:112)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:187)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:167)
> {noformat}
> Conditions:
> table T with a single column family 0 that uses ROWCOL bloom filter
> (important) and column qualifiers C1,C2,C3,C4,C5.
> When we fill the table for every row we put deleted cell for C3.
> The table has a single region with two HStore:
> A: start row: 0, stop row: 99
> B: start row: 10 stop row: 99
> B has newer versions of rows 10-99. Store files have several blocks each
> (important).
> Store A is the result of major compaction, so it doesn't have any deleted
> cells (important).
> So, we are running a scan like:
> {noformat}
> scan 'T', { COLUMNS => ['0:C3','0:C5'], FILTER => "SingleColumnValueFilter
> ('0','C5',=,'binary:whatever')"}
> {noformat}
> How the scan performs:
> First, we iterate A for rows 0 and 1 without any problems.
> Next, we start to iterate A for row 10, so read the first cell and set hfs
> scanner to A :
> 10:0/C1/0/Put/x but found that we have a newer version of the cell in B :
> 10:0/C1/1/Put/x,
> so we make B as our current store scanner. Since we are looking for
> particular columns
> C3 and C5, we perform the optimization StoreScanner.seekOrSkipToNextColumn
> which
> would run reseek for all store scanners.
> For store A the following magic would happen in requestSeek:
> 1. bloom filter check passesGeneralBloomFilter would set haveToSeek to
> false because row 10 doesn't have C3 qualifier in store A.
> 2. Since we don't have to seek we just create a fake row
> 10:0/C3/OLDEST_TIMESTAMP/Maximum, an optimization that is quite important for
> us and it commented with :
> {noformat}
> // Multi-column Bloom filter optimization.
> // Create a fake key/value, so that this scanner only bubbles up to the
> top
> // of the KeyValueHeap in StoreScanner after we scanned this row/column in
> // all other store files. The
[jira] [Commented] (HBASE-18133) Low-latency space quota size reports
[ https://issues.apache.org/jira/browse/HBASE-18133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16381334#comment-16381334 ] Hudson commented on HBASE-18133: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4667 (See [https://builds.apache.org/job/HBase-Trunk_matrix/4667/]) HBASE-18133 Decrease quota reaction latency by HBase (elserj: rev bdedcc5631fa9c8c400d4daa01b8a1947d4a12dd) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/RegionServerSpaceQuotaManager.java * (add) hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/RegionSize.java * (add) hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/RegionSizeImpl.java * (add) hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/RegionSizeStoreFactory.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/policies/MissingSnapshotViolationPolicyEnforcement.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * (add) hbase-hadoop2-compat/src/main/resources/META-INF/services/org.apache.hadoop.hbase.regionserver.MetricsRegionServerQuotaSource * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/policies/DefaultViolationPolicyEnforcement.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerServices.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/quotas/SpaceQuotaHelperForTests.java * (add) hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerQuotaSourceImpl.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/policies/AbstractViolationPolicyEnforcement.java * (add) hbase-server/src/test/java/org/apache/hadoop/hbase/quotas/TestRegionSizeImpl.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/MockRegionServerServices.java * (edit) hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerQuotaSource.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerRegionSpaceUseReport.java * (add) hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/NoOpRegionSizeStore.java * (add) hbase-server/src/test/java/org/apache/hadoop/hbase/quotas/TestLowLatencySpaceQuotas.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/quotas/TestQuotaObserverChoreRegionReports.java * (add) hbase-server/src/test/java/org/apache/hadoop/hbase/quotas/TestRegionSizeStoreImpl.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/quotas/TestSpaceQuotas.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHStore.java * (add) hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/RegionSizeStore.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/SpaceViolationPolicyEnforcement.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/quotas/TestFileSystemUtilizationChore.java * (add) hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/RegionSizeStoreImpl.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServer.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/FileSystemUtilizationChore.java * (add) hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/RegionSizeReportingChore.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/quotas/policies/TestBulkLoadCheckingViolationPolicyEnforcement.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java * (add) hbase-server/src/test/java/org/apache/hadoop/hbase/quotas/TestRegionSizeReportingChore.java > Low-latency space quota size reports > > > Key: HBASE-18133 > URL: https://issues.apache.org/jira/browse/HBASE-18133 > Project: HBase > Issue Type: Improvement >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-18133.001.patch, HBASE-18133.002.patch, > HBASE-18133.003.patch, HBASE-18133.004.patch, HBASE-18133.005.patch, > HBASE-18133.006.patch, HBASE-18133.007.patch, HBASE-18133.008.patch, > HBASE-18133.009.patch > > > Presently space quota enforcement relies on RegionServers sending reports to > the master about each Region that they host. This is done by periodically, > reading the cached size of each HFile in each Region (which was ultimately > computed from HDFS). > This means that the Master is unaware of Region size growth until the the > next time this chore in a RegionServer
[jira] [Commented] (HBASE-20106) API Compliance checker should fall back to specifying origin as remote repo
[
https://issues.apache.org/jira/browse/HBASE-20106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16381332#comment-16381332
]
Hudson commented on HBASE-20106:
FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4667 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/4667/])
HBASE-20106 [api compliance chacker] Fix Bug Where Branch Isn't Found (busbey:
rev 96ebab748fae24582cfaea0d0f8c318f79bf1681)
* (edit) dev-support/checkcompatibility.py
> API Compliance checker should fall back to specifying origin as remote repo
> ---
>
> Key: HBASE-20106
> URL: https://issues.apache.org/jira/browse/HBASE-20106
> Project: HBase
> Issue Type: Bug
> Components: API, community
>Affects Versions: 2.0.0, 1.4.0, 1.3.2, 1.2.7
>Reporter: Sean Busbey
>Assignee: Alex Leblang
>Priority: Major
> Fix For: 2.0.0, 3.0.0, 1.3.2, 1.5.0, 1.2.7, 1.4.3
>
> Attachments: HBASE-20106.0.patch
>
>
> Since HBASE-18020 has been in the 1.4.0 release, handling the addendum here.
> {quote}
> Andrew Purtell added a comment - 08/Aug/17 19:34
> Does there need to be a documentation update too. This didn't work for me. On
> branch-1.4
> {code}
> ./dev-support/checkcompatibility.py --annotation
> org.apache.hadoop.hbase.classification.InterfaceAudience.Public --annotation
> org.apache.hadoop.hbase.classification.InterfaceAudience.LimitedPrivate
> --include-file "hbase-*" rel/1.3.1 branch-1.4
> INFO:root:Source revision: rel/1.3.1
> INFO:root:Destination revision: branch-1.4
> INFO:root:Applying JAR filename include filter: hbase-*
> INFO:root:Filtering classes using 2 annotation(s):
> INFO:root:org.apache.hadoop.hbase.classification.InterfaceAudience.Public
> INFO:root:
> org.apache.hadoop.hbase.classification.InterfaceAudience.LimitedPrivate
> INFO:root:Downloading Java ACC...
> INFO:root:Removing scratch dir /data/src/hbase/target/compat-check
> INFO:root:Creating empty scratch dir /data/src/hbase/target/compat-check
> INFO:root:Checking out 1b8ca49af872b63c7b1fbc20a6fdbe71516dd04c in
> /data/src/hbase/target/compat-check/src
> INFO:root:Checking out e9e16b59420c5d9a47b9b014b3bb3cb3421b1de9 in
> /data/src/hbase/target/compat-check/dst
> INFO:root:Building in /data/src/hbase/target/compat-check/src
> ...
> INFO:root:Will check compatibility between original jars:
>
> /data/src/hbase/target/compat-check/src/hbase-archetypes/hbase-client-project/target/hbase-client-project-1.3.1.jar
> ...
> and new jars:
>
> /data/src/hbase/target/compat-check/dst/hbase-archetypes/hbase-shaded-client-project/target/hbase-shaded-client-project-1.4.0-SNAPSHOT.jar
> ...
> Traceback (most recent call last):
> File "./dev-support/checkcompatibility.py", line 514, in
> main()
> File "./dev-support/checkcompatibility.py", line 508, in main
> dst_jars, args.annotations, skip_annotations)
> File "./dev-support/checkcompatibility.py", line 283, in run_java_acc
> "-l", get_repo_name(),
> File "./dev-support/checkcompatibility.py", line 126, in get_repo_name
> cwd=get_repo_dir()).strip()
> File "./dev-support/checkcompatibility.py", line 71, in check_output
> raise error
> subprocess.CalledProcessError: Command '['git', 'config', '--get',
> 'remote.origin.url']' returned non-zero exit status 1
> [apurtell@ip-172-31-47-35 hbase]$ git --version
> git version 1.7.1
> {code}
> {quote}
> {quote}
> awleblang Alex Leblang added a comment - 05/Sep/17 08:47
> For some reason, perhaps due to the change from bash to python, the code
> needed to explicitly state "origin/branch_name" on clean checkouts. I went
> back to look at the bash version and we use the same command, git rev-parse,
> so I'm not sure why this only just starting failing but this fixed the issue
> in all cases where I was able to replicate the failure
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20070) website generation is failing
[
https://issues.apache.org/jira/browse/HBASE-20070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16381328#comment-16381328
]
Sean Busbey commented on HBASE-20070:
-
-5
- update use of cp to use flags that also work on mac.
I updated cp to use {{-pPR}}, which should be "recursively copy everything
while maintaining file metadata (owner, timestamp, access)" on both Mac
(10.11.6 built-in tested) and Linux (CentOS 7.4 tested).
> website generation is failing
> -
>
> Key: HBASE-20070
> URL: https://issues.apache.org/jira/browse/HBASE-20070
> Project: HBase
> Issue Type: Bug
> Components: website
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Blocker
> Attachments: HBASE-20070-misty.patch, HBASE-20070-misty.patch.1,
> HBASE-20070-misty.patch.3, HBASE-20070.0.patch, HBASE-20070.1.patch,
> HBASE-20070.2.patch, HBASE-20070.3.patch, HBASE-20070.4.patch,
> HBASE-20070.5.patch,
> hbase-install-log-a29b3caf4dbc7b8833474ef5da5438f7f6907e00.txt
>
>
> website generation has been failing since Feb 20th
> {code}
> Checking out files: 100% (68971/68971), done.
> Usage: grep [OPTION]... PATTERN [FILE]...
> Try 'grep --help' for more information.
> PUSHED is 2
> is not yet mentioned in the hbase-site commit log. Assuming we don't have it
> yet. 2
> Building HBase
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=256m;
> support was removed in 8.0
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=256m;
> support was removed in 8.0
> Failure: mvn clean site
> Build step 'Execute shell' marked build as failure
> {code}
> The status email says
> {code}
> Build status: Still Failing
> The HBase website has not been updated to incorporate HBase commit
> ${CURRENT_HBASE_COMMIT}.
> {code}
> Looking at the code where that grep happens, it looks like the env variable
> CURRENT_HBASE_COMMIT isn't getting set. That comes from some git command. I'm
> guessing the version of git changed on the build hosts and upended our
> assumptions.
> we should fix this to 1) rely on git's porcelain interface, and 2) fail as
> soon as that git command fails
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20070) website generation is failing
[
https://issues.apache.org/jira/browse/HBASE-20070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-20070:
Status: Patch Available (was: In Progress)
> website generation is failing
> -
>
> Key: HBASE-20070
> URL: https://issues.apache.org/jira/browse/HBASE-20070
> Project: HBase
> Issue Type: Bug
> Components: website
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Blocker
> Attachments: HBASE-20070-misty.patch, HBASE-20070-misty.patch.1,
> HBASE-20070-misty.patch.3, HBASE-20070.0.patch, HBASE-20070.1.patch,
> HBASE-20070.2.patch, HBASE-20070.3.patch, HBASE-20070.4.patch,
> HBASE-20070.5.patch,
> hbase-install-log-a29b3caf4dbc7b8833474ef5da5438f7f6907e00.txt
>
>
> website generation has been failing since Feb 20th
> {code}
> Checking out files: 100% (68971/68971), done.
> Usage: grep [OPTION]... PATTERN [FILE]...
> Try 'grep --help' for more information.
> PUSHED is 2
> is not yet mentioned in the hbase-site commit log. Assuming we don't have it
> yet. 2
> Building HBase
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=256m;
> support was removed in 8.0
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=256m;
> support was removed in 8.0
> Failure: mvn clean site
> Build step 'Execute shell' marked build as failure
> {code}
> The status email says
> {code}
> Build status: Still Failing
> The HBase website has not been updated to incorporate HBase commit
> ${CURRENT_HBASE_COMMIT}.
> {code}
> Looking at the code where that grep happens, it looks like the env variable
> CURRENT_HBASE_COMMIT isn't getting set. That comes from some git command. I'm
> guessing the version of git changed on the build hosts and upended our
> assumptions.
> we should fix this to 1) rely on git's porcelain interface, and 2) fail as
> soon as that git command fails
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20070) website generation is failing
[
https://issues.apache.org/jira/browse/HBASE-20070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16381325#comment-16381325
]
Misty Stanley-Jones commented on HBASE-20070:
-
Testing latest patch now.
> website generation is failing
> -
>
> Key: HBASE-20070
> URL: https://issues.apache.org/jira/browse/HBASE-20070
> Project: HBase
> Issue Type: Bug
> Components: website
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Blocker
> Attachments: HBASE-20070-misty.patch, HBASE-20070-misty.patch.1,
> HBASE-20070-misty.patch.3, HBASE-20070.0.patch, HBASE-20070.1.patch,
> HBASE-20070.2.patch, HBASE-20070.3.patch, HBASE-20070.4.patch,
> HBASE-20070.5.patch,
> hbase-install-log-a29b3caf4dbc7b8833474ef5da5438f7f6907e00.txt
>
>
> website generation has been failing since Feb 20th
> {code}
> Checking out files: 100% (68971/68971), done.
> Usage: grep [OPTION]... PATTERN [FILE]...
> Try 'grep --help' for more information.
> PUSHED is 2
> is not yet mentioned in the hbase-site commit log. Assuming we don't have it
> yet. 2
> Building HBase
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=256m;
> support was removed in 8.0
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=256m;
> support was removed in 8.0
> Failure: mvn clean site
> Build step 'Execute shell' marked build as failure
> {code}
> The status email says
> {code}
> Build status: Still Failing
> The HBase website has not been updated to incorporate HBase commit
> ${CURRENT_HBASE_COMMIT}.
> {code}
> Looking at the code where that grep happens, it looks like the env variable
> CURRENT_HBASE_COMMIT isn't getting set. That comes from some git command. I'm
> guessing the version of git changed on the build hosts and upended our
> assumptions.
> we should fix this to 1) rely on git's porcelain interface, and 2) fail as
> soon as that git command fails
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20109) Add Admin#getMasterLocation API for lightweight discovery of the active master location
[
https://issues.apache.org/jira/browse/HBASE-20109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-20109:
---
Description:
Right now the only public API available to the client to learn the server name
of the active master is Admin#getClusterStatus#getMaster, returning ServerName.
On a cluster of any size getClusterStatus is expensive, especially if used only
to retrieve the active master name.
Let's add a simple API
{code}
ServerName Admin#getMasterLocation()
{code}
for lightweight discovery of the active master location. This makes sense
because, weirdly, Admin already has a method getMasterInfoPort(), returning
int.
Internally the client has a notion of the active master because there is a
connection open to it, or one that can be reopened, or if for some reason it's
not easy to make a ServerName for that state, the ServerName can be
deserialized out of the znode tracking the active master location.
was:
Right now the only public API available to the client to learn the server name
of the active master is Admin#getClusterStatus#getMaster, returning ServerName.
On a cluster of any size getClusterStatus is expensive, especially if used only
to retrieve the active master name.
Let's add a simple API
{code}
ServerName Admin#getMasterAddress()
{code}
for lightweight discovery of the active master location. This makes sense
because, weirdly, Admin already has a method getMasterInfoPort(), returning
int.
Internally the client has a notion of the active master because there is a
connection open to it, or one that can be reopened, or if for some reason it's
not easy to make a ServerName for that state, the ServerName can be
deserialized out of the znode tracking the active master location.
> Add Admin#getMasterLocation API for lightweight discovery of the active
> master location
> ---
>
> Key: HBASE-20109
> URL: https://issues.apache.org/jira/browse/HBASE-20109
> Project: HBase
> Issue Type: Improvement
> Components: Client
>Affects Versions: 1.4.2
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Minor
> Fix For: 2.0.0, 1.5.0
>
>
> Right now the only public API available to the client to learn the server
> name of the active master is Admin#getClusterStatus#getMaster, returning
> ServerName. On a cluster of any size getClusterStatus is expensive,
> especially if used only to retrieve the active master name.
> Let's add a simple API
> {code}
> ServerName Admin#getMasterLocation()
> {code}
> for lightweight discovery of the active master location. This makes sense
> because, weirdly, Admin already has a method getMasterInfoPort(), returning
> int.
> Internally the client has a notion of the active master because there is a
> connection open to it, or one that can be reopened, or if for some reason
> it's not easy to make a ServerName for that state, the ServerName can be
> deserialized out of the znode tracking the active master location.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20109) Add Admin#getMasterLocation API for lightweight discovery of the active master location
[
https://issues.apache.org/jira/browse/HBASE-20109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-20109:
---
Summary: Add Admin#getMasterLocation API for lightweight discovery of the
active master location (was: Add Admin#getMasterAddress API for lightweight
discovery of the active master location)
> Add Admin#getMasterLocation API for lightweight discovery of the active
> master location
> ---
>
> Key: HBASE-20109
> URL: https://issues.apache.org/jira/browse/HBASE-20109
> Project: HBase
> Issue Type: Improvement
> Components: Client
>Affects Versions: 1.4.2
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Minor
> Fix For: 2.0.0, 1.5.0
>
>
> Right now the only public API available to the client to learn the server
> name of the active master is Admin#getClusterStatus#getMaster, returning
> ServerName. On a cluster of any size getClusterStatus is expensive,
> especially if used only to retrieve the active master name.
> Let's add a simple API
> {code}
> ServerName Admin#getMasterAddress()
> {code}
> for lightweight discovery of the active master location. This makes sense
> because, weirdly, Admin already has a method getMasterInfoPort(), returning
> int.
> Internally the client has a notion of the active master because there is a
> connection open to it, or one that can be reopened, or if for some reason
> it's not easy to make a ServerName for that state, the ServerName can be
> deserialized out of the znode tracking the active master location.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20070) website generation is failing
[
https://issues.apache.org/jira/browse/HBASE-20070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-20070:
Attachment: HBASE-20070.5.patch
> website generation is failing
> -
>
> Key: HBASE-20070
> URL: https://issues.apache.org/jira/browse/HBASE-20070
> Project: HBase
> Issue Type: Bug
> Components: website
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Blocker
> Attachments: HBASE-20070-misty.patch, HBASE-20070-misty.patch.1,
> HBASE-20070-misty.patch.3, HBASE-20070.0.patch, HBASE-20070.1.patch,
> HBASE-20070.2.patch, HBASE-20070.3.patch, HBASE-20070.4.patch,
> HBASE-20070.5.patch,
> hbase-install-log-a29b3caf4dbc7b8833474ef5da5438f7f6907e00.txt
>
>
> website generation has been failing since Feb 20th
> {code}
> Checking out files: 100% (68971/68971), done.
> Usage: grep [OPTION]... PATTERN [FILE]...
> Try 'grep --help' for more information.
> PUSHED is 2
> is not yet mentioned in the hbase-site commit log. Assuming we don't have it
> yet. 2
> Building HBase
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=256m;
> support was removed in 8.0
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=256m;
> support was removed in 8.0
> Failure: mvn clean site
> Build step 'Execute shell' marked build as failure
> {code}
> The status email says
> {code}
> Build status: Still Failing
> The HBase website has not been updated to incorporate HBase commit
> ${CURRENT_HBASE_COMMIT}.
> {code}
> Looking at the code where that grep happens, it looks like the env variable
> CURRENT_HBASE_COMMIT isn't getting set. That comes from some git command. I'm
> guessing the version of git changed on the build hosts and upended our
> assumptions.
> we should fix this to 1) rely on git's porcelain interface, and 2) fail as
> soon as that git command fails
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-18370) Port HBASE-16209 to branch-1.3
[
https://issues.apache.org/jira/browse/HBASE-18370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16381307#comment-16381307
]
Hadoop QA commented on HBASE-18370:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} branch-1.3 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
38s{color} | {color:green} branch-1.3 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
35s{color} | {color:green} branch-1.3 passed with JDK v1.8.0_163 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
34s{color} | {color:green} branch-1.3 passed with JDK v1.7.0_171 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
23s{color} | {color:green} branch-1.3 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
50s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
31s{color} | {color:green} branch-1.3 passed with JDK v1.8.0_163 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
35s{color} | {color:green} branch-1.3 passed with JDK v1.7.0_171 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
32s{color} | {color:green} the patch passed with JDK v1.8.0_163 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
34s{color} | {color:green} the patch passed with JDK v1.7.0_171 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
18s{color} | {color:red} hbase-server: The patch generated 3 new + 284
unchanged - 1 fixed = 287 total (was 285) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
22s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 12s{color} | {color:green} Patch does not cause any errors with Hadoop 2.4.1
2.5.2 2.6.5 2.7.4. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
25s{color} | {color:green} the patch passed with JDK v1.8.0_163 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
34s{color} | {color:green} the patch passed with JDK v1.7.0_171 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}104m 48s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
23s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}132m 36s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hbase.replication.regionserver.TestRegionReplicaReplicationEndpoint |
| | hadoop.hbase.replication.multiwal.TestReplicationEndpointWithMultipleWAL |
| | hadoop.hbase.regionserver.TestEndToEndSplitTransaction |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce
[jira] [Commented] (HBASE-20101) HBase should provide a way to re-validate locality
[ https://issues.apache.org/jira/browse/HBASE-20101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16381294#comment-16381294 ] Jean-Marc Spaggiari commented on HBASE-20101: - Kind of! There is 2 things. One Chore that can check, eventually, on a specified frequence. One API call that can trigger a reload for a specific region/CF... I need this 2nd option because I manually move some blocks to restore locality. I don't want to force HBase to re-check everything. Since I know which file the blocks are belonging too, I just need to say to HBase than this specific Region/CF changed... > HBase should provide a way to re-validate locality > -- > > Key: HBASE-20101 > URL: https://issues.apache.org/jira/browse/HBASE-20101 > Project: HBase > Issue Type: New Feature >Reporter: Jean-Marc Spaggiari >Priority: Major > > HDFS blocks can move for many reasons. HDFS balancing, lost of a disk, of a > node, etc. However, today, locality seems to be calculated when the files are > opened for the first time. Even disabling and re-enabling the regions doesn't > trigger a re-calculation of the locality. > We should provide a way to let the user ask for this number to be > re-calculated. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-18862) backport HBASE-15109 to branch-1.1,branch-1.2,branch-1.3
[
https://issues.apache.org/jira/browse/HBASE-18862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16381287#comment-16381287
]
Hadoop QA commented on HBASE-18862:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} branch-1.3 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
36s{color} | {color:green} branch-1.3 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
41s{color} | {color:green} branch-1.3 passed with JDK v1.8.0_162 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
36s{color} | {color:green} branch-1.3 passed with JDK v1.7.0_171 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
20s{color} | {color:green} branch-1.3 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
53s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
36s{color} | {color:green} branch-1.3 passed with JDK v1.8.0_162 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
37s{color} | {color:green} branch-1.3 passed with JDK v1.7.0_171 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
36s{color} | {color:green} the patch passed with JDK v1.8.0_162 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
36s{color} | {color:green} the patch passed with JDK v1.7.0_171 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
32s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 49s{color} | {color:green} Patch does not cause any errors with Hadoop 2.4.1
2.5.2 2.6.5 2.7.4. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} the patch passed with JDK v1.8.0_162 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
39s{color} | {color:green} the patch passed with JDK v1.7.0_171 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 84m 16s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}113m 15s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.regionserver.TestEndToEndSplitTransaction |
| | hadoop.hbase.regionserver.TestZKLessMergeOnCluster |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:dca6535 |
| JIRA Issue | HBASE-18862 |
| JIRA Patch URL |
[jira] [Updated] (HBASE-20055) Remove declaration of un-thrown exceptions and unused setRegionStateBackToOpen() from MergeTableRegionsProcedure
[ https://issues.apache.org/jira/browse/HBASE-20055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Umesh Agashe updated HBASE-20055: - Attachment: hbase-20055.master.002.patch > Remove declaration of un-thrown exceptions and unused > setRegionStateBackToOpen() from MergeTableRegionsProcedure > > > Key: HBASE-20055 > URL: https://issues.apache.org/jira/browse/HBASE-20055 > Project: HBase > Issue Type: Improvement > Components: amv2 >Affects Versions: 2.0.0-beta-1 >Reporter: Umesh Agashe >Assignee: Umesh Agashe >Priority: Minor > Fix For: 2.0.0-beta-2 > > Attachments: hbase-20055.master.001.patch, > hbase-20055.master.001.patch, hbase-20055.master.002.patch > > > A few methods declare exceptions in throws statement that are not thrown and > method setRegionStateBackToOpen() is not used in MergeTableRegionsProcedure. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20105) Allow flushes to target SSD storage
[ https://issues.apache.org/jira/browse/HBASE-20105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16381240#comment-16381240 ] Jean-Marc Spaggiari commented on HBASE-20105: - Blocked by HDFS-13209 > Allow flushes to target SSD storage > --- > > Key: HBASE-20105 > URL: https://issues.apache.org/jira/browse/HBASE-20105 > Project: HBase > Issue Type: New Feature > Components: Performance, regionserver >Affects Versions: hbase-2.0.0-alpha-4 >Reporter: Jean-Marc Spaggiari >Priority: Major > > On heavy writes usecases, flushes are compactes together pretty quickly. > Allowing flushes to go on SSD allows faster flush and faster first > compactions. Subsequent compactions going on regular storage. > > I will be interesting to have an option to target SSD for flushes. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20109) Add Admin#getMasterAddress API for lightweight discovery of the active master location
[
https://issues.apache.org/jira/browse/HBASE-20109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-20109:
---
Affects Version/s: 1.4.2
> Add Admin#getMasterAddress API for lightweight discovery of the active master
> location
> --
>
> Key: HBASE-20109
> URL: https://issues.apache.org/jira/browse/HBASE-20109
> Project: HBase
> Issue Type: Improvement
> Components: Client
>Affects Versions: 1.4.2
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Minor
> Fix For: 2.0.0, 1.5.0
>
>
> Right now the only public API available to the client to learn the server
> name of the active master is Admin#getClusterStatus#getMaster, returning
> ServerName. On a cluster of any size getClusterStatus is expensive,
> especially if used only to retrieve the active master name.
> Let's add a simple API
> {code}
> ServerName Admin#getMasterAddress()
> {code}
> for lightweight discovery of the active master location. This makes sense
> because, weirdly, Admin already has a method getMasterInfoPort(), returning
> int.
> Internally the client has a notion of the active master because there is a
> connection open to it, or one that can be reopened, or if for some reason
> it's not easy to make a ServerName for that state, the ServerName can be
> deserialized out of the znode tracking the active master location.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-20109) Add Admin#getMasterAddress API for lightweight discovery of the active master location
Andrew Purtell created HBASE-20109:
--
Summary: Add Admin#getMasterAddress API for lightweight discovery
of the active master location
Key: HBASE-20109
URL: https://issues.apache.org/jira/browse/HBASE-20109
Project: HBase
Issue Type: Improvement
Components: Client
Reporter: Andrew Purtell
Assignee: Andrew Purtell
Fix For: 2.0.0, 1.5.0
Right now the only public API available to the client to learn the server name
of the active master is Admin#getClusterStatus#getMaster, returning ServerName.
On a cluster of any size getClusterStatus is expensive, especially if used only
to retrieve the active master name.
Let's add a simple API
{code}
ServerName Admin#getMasterAddress()
{code}
for lightweight discovery of the active master location. This makes sense
because, weirdly, Admin already has a method getMasterInfoPort(), returning
int.
Internally the client has a notion of the active master because there is a
connection open to it, or one that can be reopened, or if for some reason it's
not easy to make a ServerName for that state, the ServerName can be
deserialized out of the znode tracking the active master location.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20093) Replace ServerLoad by ServerMetrics for ServerManager
[ https://issues.apache.org/jira/browse/HBASE-20093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16381214#comment-16381214 ] Ted Yu commented on HBASE-20093: Can you do this exercise: In 1.x cluster, upgrade (and restart) the master to 2.0, see if 1.x region server report is properly handled Thanks > Replace ServerLoad by ServerMetrics for ServerManager > - > > Key: HBASE-20093 > URL: https://issues.apache.org/jira/browse/HBASE-20093 > Project: HBase > Issue Type: Task >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Major > Fix For: 2.0.0 > > Attachments: HBASE-20093.v0.patch, HBASE-20093.v1.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20093) Replace ServerLoad by ServerMetrics for ServerManager
[
https://issues.apache.org/jira/browse/HBASE-20093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16381204#comment-16381204
]
Chia-Ping Tsai commented on HBASE-20093:
Thanks [~yuzhih...@gmail.com] for the reports.
{quote}what if master runs 2.0 but some region servers are running 1.x ?
{quote}
The report from RS to Master is based on protobuf, and the new class
ServerMetricsImpl can process the protobuf request correctly.
{quote}java.lang.ClassCastException:
org.apache.hadoop.hbase.ServerMetricsBuilder$ServerMetricsImpl cannot be cast
to org.apache.hadoop.hbase.ServerLoad
at
org.apache.hadoop.hbase.master.HMaster.lambda$getClusterMetricsWithoutCoprocessor$2(HMaster.java:2438){quote}
The master won't use ServerLoad anymore (excluding
serverManager.checkAndRecordNewServer...I will push a addendum to fix it). Not
sure why casting still happen. How could I reproduce the error?
> Replace ServerLoad by ServerMetrics for ServerManager
> -
>
> Key: HBASE-20093
> URL: https://issues.apache.org/jira/browse/HBASE-20093
> Project: HBase
> Issue Type: Task
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Major
> Fix For: 2.0.0
>
> Attachments: HBASE-20093.v0.patch, HBASE-20093.v1.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HBASE-17448) Export metrics from RecoverableZooKeeper
[ https://issues.apache.org/jira/browse/HBASE-17448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell reassigned HBASE-17448: -- Assignee: Chinmay Kulkarni (was: Andrew Purtell) > Export metrics from RecoverableZooKeeper > > > Key: HBASE-17448 > URL: https://issues.apache.org/jira/browse/HBASE-17448 > Project: HBase > Issue Type: Improvement > Components: Zookeeper >Affects Versions: 1.3.1 >Reporter: Andrew Purtell >Assignee: Chinmay Kulkarni >Priority: Major > Labels: patch > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-17448-branch-1.patch, HBASE-17448.patch, > HBASE-17448.patch > > > Consider adding instrumentation to RecoverableZooKeeper that exposes metrics > on the performance and health of the embedded ZooKeeper client: latency > histograms for each op type, number of reconnections, number of ops where a > reconnection was necessary to proceed, number of failed ops due to > CONNECTIONLOSS, number of failed ops due to SESSIONEXIPRED, number of failed > ops due to OPERATIONTIMEOUT. > RecoverableZooKeeper is a class in hbase-client so we can hook up the new > metrics to both client- and server-side metrics reporters. Probably this > metrics source should be a process singleton. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Reopened] (HBASE-17448) Export metrics from RecoverableZooKeeper
[ https://issues.apache.org/jira/browse/HBASE-17448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell reopened HBASE-17448: Assignee: Andrew Purtell (was: Chinmay Kulkarni) > Export metrics from RecoverableZooKeeper > > > Key: HBASE-17448 > URL: https://issues.apache.org/jira/browse/HBASE-17448 > Project: HBase > Issue Type: Improvement > Components: Zookeeper >Affects Versions: 1.3.1 >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Major > Labels: patch > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-17448-branch-1.patch, HBASE-17448.patch, > HBASE-17448.patch > > > Consider adding instrumentation to RecoverableZooKeeper that exposes metrics > on the performance and health of the embedded ZooKeeper client: latency > histograms for each op type, number of reconnections, number of ops where a > reconnection was necessary to proceed, number of failed ops due to > CONNECTIONLOSS, number of failed ops due to SESSIONEXIPRED, number of failed > ops due to OPERATIONTIMEOUT. > RecoverableZooKeeper is a class in hbase-client so we can hook up the new > metrics to both client- and server-side metrics reporters. Probably this > metrics source should be a process singleton. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-17448) Export metrics from RecoverableZooKeeper
[ https://issues.apache.org/jira/browse/HBASE-17448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16381184#comment-16381184 ] Andrew Purtell commented on HBASE-17448: HBASE-19769 pulled this from 2.0. The following were discussed as motivations: - Issues with service loading in YARN, IllegalAccess exceptions - Too verbose (? - they are metrics) - Dont' want stuff based on hadoop-metrics All valid. Although this shipped in 1.4.0, 1.4.1, and 1.4.2, I have no problem pulling it from 1.5 if problematic, or even from 1.4.3 if more than problematic approaching broken. Please advise. My thinking at this time is it should get pulled from 1.5 since it has already been pulled from 2.0. > Export metrics from RecoverableZooKeeper > > > Key: HBASE-17448 > URL: https://issues.apache.org/jira/browse/HBASE-17448 > Project: HBase > Issue Type: Improvement > Components: Zookeeper >Affects Versions: 1.3.1 >Reporter: Andrew Purtell >Assignee: Chinmay Kulkarni >Priority: Major > Labels: patch > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-17448-branch-1.patch, HBASE-17448.patch, > HBASE-17448.patch > > > Consider adding instrumentation to RecoverableZooKeeper that exposes metrics > on the performance and health of the embedded ZooKeeper client: latency > histograms for each op type, number of reconnections, number of ops where a > reconnection was necessary to proceed, number of failed ops due to > CONNECTIONLOSS, number of failed ops due to SESSIONEXIPRED, number of failed > ops due to OPERATIONTIMEOUT. > RecoverableZooKeeper is a class in hbase-client so we can hook up the new > metrics to both client- and server-side metrics reporters. Probably this > metrics source should be a process singleton. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-19769) IllegalAccessError on package-private Hadoop metrics2 classes in MapReduce jobs
[
https://issues.apache.org/jira/browse/HBASE-19769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-19769:
---
Comment: was deleted
(was: So the decision was to (permanently?) revert HBASE-17448 here.
I don't see a notice on HBASE-17448 informing folks there this was going to
happen.
Since it is out from 2.0, should we revert it from 1.5 too?)
> IllegalAccessError on package-private Hadoop metrics2 classes in MapReduce
> jobs
> ---
>
> Key: HBASE-19769
> URL: https://issues.apache.org/jira/browse/HBASE-19769
> Project: HBase
> Issue Type: Bug
> Components: mapreduce, metrics
>Affects Versions: 2.0.0-beta-1
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19769.001.branch-2.patch
>
>
> issues for context: HBASE-17170, HBASE-17448, TEZ-3299, HADOOP-10893
> Since Hadoop 2.6.0, the {{yarn jar}} entry point to submit a YARN job has
> been using a custom classloader to separate Hadoop dependencies from the
> user's JAR being run. A separate classloader is created for the user-provided
> jar, and then this classloader is set as the contextClassLoader before the
> Tool is executed by Hadoop's RunJar class. This has been (mostly?) fine for
> us to date because we don't try to access any Hadoop internal classes
> client-side.
> However, with the ZK metrics, clients are pushing ZK metrics to metrics2. The
> problem is that Hadoop metrics2 implementations which we reference from the
> same package are loaded by a different classloader than our HBase code is
> loaded from. This makes the expected package-private access of these Metrics2
> classes (e.g. MetricsInfoImpl) fail with an IllegalAccessError.
> {noformat}
> java.lang.RuntimeException: Could not create interface
> org.apache.hadoop.hbase.zookeeper.MetricsZooKeeperSource Is the hadoop
> compatibility jar on the classpath?
> at
> org.apache.hadoop.hbase.CompatibilitySingletonFactory.getInstance(CompatibilitySingletonFactory.java:75)
> at
> org.apache.hadoop.hbase.zookeeper.ZKMetrics.(ZKMetrics.java:36)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.(RecoverableZooKeeper.java:115)
> at org.apache.hadoop.hbase.zookeeper.ZKUtil.connect(ZKUtil.java:139)
> at
> org.apache.hadoop.hbase.zookeeper.ZKWatcher.(ZKWatcher.java:128)
> at
> org.apache.hadoop.hbase.zookeeper.ZKWatcher.(ZKWatcher.java:102)
> at
> org.apache.hadoop.hbase.security.token.TokenUtil.getAuthToken(TokenUtil.java:293)
> at
> org.apache.hadoop.hbase.security.token.TokenUtil.addTokenForJob(TokenUtil.java:259)
> at
> org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.initCredentials(TableMapReduceUtil.java:535)
> at
> org.apache.phoenix.mapreduce.MultiHfileOutputFormat.configureIncrementalLoad(MultiHfileOutputFormat.java:712)
> at
> org.apache.phoenix.mapreduce.AbstractBulkLoadTool.submitJob(AbstractBulkLoadTool.java:300)
> at
> org.apache.phoenix.mapreduce.AbstractBulkLoadTool.loadData(AbstractBulkLoadTool.java:267)
> at
> org.apache.phoenix.mapreduce.AbstractBulkLoadTool.run(AbstractBulkLoadTool.java:180)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
> at
> org.apache.phoenix.mapreduce.CsvBulkLoadTool.main(CsvBulkLoadTool.java:109)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:239)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:153)
> Caused by: java.util.ServiceConfigurationError:
> org.apache.hadoop.hbase.zookeeper.MetricsZooKeeperSource: Provider
> org.apache.hadoop.hbase.zookeeper.MetricsZooKeeperSourceImpl could not be
> instantiated
> at java.util.ServiceLoader.fail(ServiceLoader.java:232)
> at java.util.ServiceLoader.access$100(ServiceLoader.java:185)
> at
> java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384)
> at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
> at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
> at
> org.apache.hadoop.hbase.CompatibilitySingletonFactory.getInstance(CompatibilitySingletonFactory.java:59)
> ... 21 more
> Caused by: java.lang.IllegalAccessError: tried to access class
>
[jira] [Commented] (HBASE-19769) IllegalAccessError on package-private Hadoop metrics2 classes in MapReduce jobs
[
https://issues.apache.org/jira/browse/HBASE-19769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16381180#comment-16381180
]
Andrew Purtell commented on HBASE-19769:
I feel this should come out in 1.5 since it is already out in 2.0, at least.
> IllegalAccessError on package-private Hadoop metrics2 classes in MapReduce
> jobs
> ---
>
> Key: HBASE-19769
> URL: https://issues.apache.org/jira/browse/HBASE-19769
> Project: HBase
> Issue Type: Bug
> Components: mapreduce, metrics
>Affects Versions: 2.0.0-beta-1
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19769.001.branch-2.patch
>
>
> issues for context: HBASE-17170, HBASE-17448, TEZ-3299, HADOOP-10893
> Since Hadoop 2.6.0, the {{yarn jar}} entry point to submit a YARN job has
> been using a custom classloader to separate Hadoop dependencies from the
> user's JAR being run. A separate classloader is created for the user-provided
> jar, and then this classloader is set as the contextClassLoader before the
> Tool is executed by Hadoop's RunJar class. This has been (mostly?) fine for
> us to date because we don't try to access any Hadoop internal classes
> client-side.
> However, with the ZK metrics, clients are pushing ZK metrics to metrics2. The
> problem is that Hadoop metrics2 implementations which we reference from the
> same package are loaded by a different classloader than our HBase code is
> loaded from. This makes the expected package-private access of these Metrics2
> classes (e.g. MetricsInfoImpl) fail with an IllegalAccessError.
> {noformat}
> java.lang.RuntimeException: Could not create interface
> org.apache.hadoop.hbase.zookeeper.MetricsZooKeeperSource Is the hadoop
> compatibility jar on the classpath?
> at
> org.apache.hadoop.hbase.CompatibilitySingletonFactory.getInstance(CompatibilitySingletonFactory.java:75)
> at
> org.apache.hadoop.hbase.zookeeper.ZKMetrics.(ZKMetrics.java:36)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.(RecoverableZooKeeper.java:115)
> at org.apache.hadoop.hbase.zookeeper.ZKUtil.connect(ZKUtil.java:139)
> at
> org.apache.hadoop.hbase.zookeeper.ZKWatcher.(ZKWatcher.java:128)
> at
> org.apache.hadoop.hbase.zookeeper.ZKWatcher.(ZKWatcher.java:102)
> at
> org.apache.hadoop.hbase.security.token.TokenUtil.getAuthToken(TokenUtil.java:293)
> at
> org.apache.hadoop.hbase.security.token.TokenUtil.addTokenForJob(TokenUtil.java:259)
> at
> org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.initCredentials(TableMapReduceUtil.java:535)
> at
> org.apache.phoenix.mapreduce.MultiHfileOutputFormat.configureIncrementalLoad(MultiHfileOutputFormat.java:712)
> at
> org.apache.phoenix.mapreduce.AbstractBulkLoadTool.submitJob(AbstractBulkLoadTool.java:300)
> at
> org.apache.phoenix.mapreduce.AbstractBulkLoadTool.loadData(AbstractBulkLoadTool.java:267)
> at
> org.apache.phoenix.mapreduce.AbstractBulkLoadTool.run(AbstractBulkLoadTool.java:180)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
> at
> org.apache.phoenix.mapreduce.CsvBulkLoadTool.main(CsvBulkLoadTool.java:109)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:239)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:153)
> Caused by: java.util.ServiceConfigurationError:
> org.apache.hadoop.hbase.zookeeper.MetricsZooKeeperSource: Provider
> org.apache.hadoop.hbase.zookeeper.MetricsZooKeeperSourceImpl could not be
> instantiated
> at java.util.ServiceLoader.fail(ServiceLoader.java:232)
> at java.util.ServiceLoader.access$100(ServiceLoader.java:185)
> at
> java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384)
> at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
> at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
> at
> org.apache.hadoop.hbase.CompatibilitySingletonFactory.getInstance(CompatibilitySingletonFactory.java:59)
> ... 21 more
> Caused by: java.lang.IllegalAccessError: tried to access class
> org.apache.hadoop.metrics2.lib.MetricsInfoImpl from class
> org.apache.hadoop.metrics2.lib.DynamicMetricsRegistry
> at
>
[jira] [Assigned] (HBASE-19598) Fix TestAssignmentManagerMetrics flaky test
[ https://issues.apache.org/jira/browse/HBASE-19598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack reassigned HBASE-19598: - Assignee: stack (was: Balazs Meszaros) > Fix TestAssignmentManagerMetrics flaky test > --- > > Key: HBASE-19598 > URL: https://issues.apache.org/jira/browse/HBASE-19598 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0-beta-1 >Reporter: Balazs Meszaros >Assignee: stack >Priority: Major > Attachments: HBASE-19598.master.001.patch, > HBASE-19598.master.002.patch, HBASE-19598.master.003.patch, > HBASE-19598.master.003.patch, TestUtil.java > > > TestAssignmentManagerMetrics fails constantly. After bisecting, it seems that > commit 010012cbcb broke it (HBASE-18946). > The test method runs successfully, but it cannot shut the minicluster down, > and hangs forever. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19769) IllegalAccessError on package-private Hadoop metrics2 classes in MapReduce jobs
[
https://issues.apache.org/jira/browse/HBASE-19769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16381173#comment-16381173
]
Andrew Purtell commented on HBASE-19769:
Would have been useful to get the reports about issues regarding HBASE-17448
expressed over there during review, or at least expressed over there when this
got pulled from 2.0. I see the following:
- Issues with service loading in YARN
- Too verbose (? - they are metrics)
- Dont' want stuff based on hadoop-metrics
All valid.
Although this shipped in 1.4.0, 1.4.1, and 1.4.2, I have no problem pulling it
from 1.5 if problematic, or even from 1.4.3 if more than problematic
approaching broken. Please advise.
[~stack] [~elserj]
> IllegalAccessError on package-private Hadoop metrics2 classes in MapReduce
> jobs
> ---
>
> Key: HBASE-19769
> URL: https://issues.apache.org/jira/browse/HBASE-19769
> Project: HBase
> Issue Type: Bug
> Components: mapreduce, metrics
>Affects Versions: 2.0.0-beta-1
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19769.001.branch-2.patch
>
>
> issues for context: HBASE-17170, HBASE-17448, TEZ-3299, HADOOP-10893
> Since Hadoop 2.6.0, the {{yarn jar}} entry point to submit a YARN job has
> been using a custom classloader to separate Hadoop dependencies from the
> user's JAR being run. A separate classloader is created for the user-provided
> jar, and then this classloader is set as the contextClassLoader before the
> Tool is executed by Hadoop's RunJar class. This has been (mostly?) fine for
> us to date because we don't try to access any Hadoop internal classes
> client-side.
> However, with the ZK metrics, clients are pushing ZK metrics to metrics2. The
> problem is that Hadoop metrics2 implementations which we reference from the
> same package are loaded by a different classloader than our HBase code is
> loaded from. This makes the expected package-private access of these Metrics2
> classes (e.g. MetricsInfoImpl) fail with an IllegalAccessError.
> {noformat}
> java.lang.RuntimeException: Could not create interface
> org.apache.hadoop.hbase.zookeeper.MetricsZooKeeperSource Is the hadoop
> compatibility jar on the classpath?
> at
> org.apache.hadoop.hbase.CompatibilitySingletonFactory.getInstance(CompatibilitySingletonFactory.java:75)
> at
> org.apache.hadoop.hbase.zookeeper.ZKMetrics.(ZKMetrics.java:36)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.(RecoverableZooKeeper.java:115)
> at org.apache.hadoop.hbase.zookeeper.ZKUtil.connect(ZKUtil.java:139)
> at
> org.apache.hadoop.hbase.zookeeper.ZKWatcher.(ZKWatcher.java:128)
> at
> org.apache.hadoop.hbase.zookeeper.ZKWatcher.(ZKWatcher.java:102)
> at
> org.apache.hadoop.hbase.security.token.TokenUtil.getAuthToken(TokenUtil.java:293)
> at
> org.apache.hadoop.hbase.security.token.TokenUtil.addTokenForJob(TokenUtil.java:259)
> at
> org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.initCredentials(TableMapReduceUtil.java:535)
> at
> org.apache.phoenix.mapreduce.MultiHfileOutputFormat.configureIncrementalLoad(MultiHfileOutputFormat.java:712)
> at
> org.apache.phoenix.mapreduce.AbstractBulkLoadTool.submitJob(AbstractBulkLoadTool.java:300)
> at
> org.apache.phoenix.mapreduce.AbstractBulkLoadTool.loadData(AbstractBulkLoadTool.java:267)
> at
> org.apache.phoenix.mapreduce.AbstractBulkLoadTool.run(AbstractBulkLoadTool.java:180)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
> at
> org.apache.phoenix.mapreduce.CsvBulkLoadTool.main(CsvBulkLoadTool.java:109)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:239)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:153)
> Caused by: java.util.ServiceConfigurationError:
> org.apache.hadoop.hbase.zookeeper.MetricsZooKeeperSource: Provider
> org.apache.hadoop.hbase.zookeeper.MetricsZooKeeperSourceImpl could not be
> instantiated
> at java.util.ServiceLoader.fail(ServiceLoader.java:232)
> at java.util.ServiceLoader.access$100(ServiceLoader.java:185)
> at
> java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384)
> at
[jira] [Commented] (HBASE-19598) Fix TestAssignmentManagerMetrics flaky test
[ https://issues.apache.org/jira/browse/HBASE-19598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16381172#comment-16381172 ] stack commented on HBASE-19598: --- Doesn't fail in branch-2. Looking at the thread-dump, I see master in startup (probably active master trying to take over) and a procedure stuck trying to alter table. I have a bunch of practice on these issue types [~busbey] and I should cleanup the mess I've left here. > Fix TestAssignmentManagerMetrics flaky test > --- > > Key: HBASE-19598 > URL: https://issues.apache.org/jira/browse/HBASE-19598 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0-beta-1 >Reporter: Balazs Meszaros >Assignee: Balazs Meszaros >Priority: Major > Attachments: HBASE-19598.master.001.patch, > HBASE-19598.master.002.patch, HBASE-19598.master.003.patch, > HBASE-19598.master.003.patch, TestUtil.java > > > TestAssignmentManagerMetrics fails constantly. After bisecting, it seems that > commit 010012cbcb broke it (HBASE-18946). > The test method runs successfully, but it cannot shut the minicluster down, > and hangs forever. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19769) IllegalAccessError on package-private Hadoop metrics2 classes in MapReduce jobs
[
https://issues.apache.org/jira/browse/HBASE-19769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16381169#comment-16381169
]
Andrew Purtell commented on HBASE-19769:
So the decision was to (permanently?) revert HBASE-17448 here.
I don't see a notice on HBASE-17448 informing folks there this was going to
happen.
Since it is out from 2.0, should we revert it from 1.5 too?
> IllegalAccessError on package-private Hadoop metrics2 classes in MapReduce
> jobs
> ---
>
> Key: HBASE-19769
> URL: https://issues.apache.org/jira/browse/HBASE-19769
> Project: HBase
> Issue Type: Bug
> Components: mapreduce, metrics
>Affects Versions: 2.0.0-beta-1
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19769.001.branch-2.patch
>
>
> issues for context: HBASE-17170, HBASE-17448, TEZ-3299, HADOOP-10893
> Since Hadoop 2.6.0, the {{yarn jar}} entry point to submit a YARN job has
> been using a custom classloader to separate Hadoop dependencies from the
> user's JAR being run. A separate classloader is created for the user-provided
> jar, and then this classloader is set as the contextClassLoader before the
> Tool is executed by Hadoop's RunJar class. This has been (mostly?) fine for
> us to date because we don't try to access any Hadoop internal classes
> client-side.
> However, with the ZK metrics, clients are pushing ZK metrics to metrics2. The
> problem is that Hadoop metrics2 implementations which we reference from the
> same package are loaded by a different classloader than our HBase code is
> loaded from. This makes the expected package-private access of these Metrics2
> classes (e.g. MetricsInfoImpl) fail with an IllegalAccessError.
> {noformat}
> java.lang.RuntimeException: Could not create interface
> org.apache.hadoop.hbase.zookeeper.MetricsZooKeeperSource Is the hadoop
> compatibility jar on the classpath?
> at
> org.apache.hadoop.hbase.CompatibilitySingletonFactory.getInstance(CompatibilitySingletonFactory.java:75)
> at
> org.apache.hadoop.hbase.zookeeper.ZKMetrics.(ZKMetrics.java:36)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.(RecoverableZooKeeper.java:115)
> at org.apache.hadoop.hbase.zookeeper.ZKUtil.connect(ZKUtil.java:139)
> at
> org.apache.hadoop.hbase.zookeeper.ZKWatcher.(ZKWatcher.java:128)
> at
> org.apache.hadoop.hbase.zookeeper.ZKWatcher.(ZKWatcher.java:102)
> at
> org.apache.hadoop.hbase.security.token.TokenUtil.getAuthToken(TokenUtil.java:293)
> at
> org.apache.hadoop.hbase.security.token.TokenUtil.addTokenForJob(TokenUtil.java:259)
> at
> org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.initCredentials(TableMapReduceUtil.java:535)
> at
> org.apache.phoenix.mapreduce.MultiHfileOutputFormat.configureIncrementalLoad(MultiHfileOutputFormat.java:712)
> at
> org.apache.phoenix.mapreduce.AbstractBulkLoadTool.submitJob(AbstractBulkLoadTool.java:300)
> at
> org.apache.phoenix.mapreduce.AbstractBulkLoadTool.loadData(AbstractBulkLoadTool.java:267)
> at
> org.apache.phoenix.mapreduce.AbstractBulkLoadTool.run(AbstractBulkLoadTool.java:180)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
> at
> org.apache.phoenix.mapreduce.CsvBulkLoadTool.main(CsvBulkLoadTool.java:109)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:239)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:153)
> Caused by: java.util.ServiceConfigurationError:
> org.apache.hadoop.hbase.zookeeper.MetricsZooKeeperSource: Provider
> org.apache.hadoop.hbase.zookeeper.MetricsZooKeeperSourceImpl could not be
> instantiated
> at java.util.ServiceLoader.fail(ServiceLoader.java:232)
> at java.util.ServiceLoader.access$100(ServiceLoader.java:185)
> at
> java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384)
> at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
> at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
> at
> org.apache.hadoop.hbase.CompatibilitySingletonFactory.getInstance(CompatibilitySingletonFactory.java:59)
> ... 21 more
> Caused by: java.lang.IllegalAccessError: tried to access class
>
[jira] [Commented] (HBASE-19423) Replication entries are not filtered correctly when replication scope is set through WAL Co-processor
[
https://issues.apache.org/jira/browse/HBASE-19423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16381163#comment-16381163
]
Hadoop QA commented on HBASE-19423:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
29s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} branch-1.4 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
6s{color} | {color:green} branch-1.4 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
43s{color} | {color:green} branch-1.4 passed with JDK v1.8.0_162 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
39s{color} | {color:green} branch-1.4 passed with JDK v1.7.0_171 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
21s{color} | {color:green} branch-1.4 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
10s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
39s{color} | {color:green} branch-1.4 passed with JDK v1.8.0_162 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
37s{color} | {color:green} branch-1.4 passed with JDK v1.7.0_171 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 1m
0s{color} | {color:red} root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
25s{color} | {color:red} hbase-server in the patch failed with JDK v1.8.0_162.
{color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 25s{color}
| {color:red} hbase-server in the patch failed with JDK v1.8.0_162. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
25s{color} | {color:red} hbase-server in the patch failed with JDK v1.7.0_171.
{color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 25s{color}
| {color:red} hbase-server in the patch failed with JDK v1.7.0_171. {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
20s{color} | {color:red} hbase-server: The patch generated 1 new + 6 unchanged
- 0 fixed = 7 total (was 6) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} shadedjars {color} | {color:red} 1m
31s{color} | {color:red} patch has 41 errors when building our shaded
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 2m
29s{color} | {color:red} The patch causes 41 errors with Hadoop v2.4.1. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 3m
26s{color} | {color:red} The patch causes 41 errors with Hadoop v2.5.2. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 4m
19s{color} | {color:red} The patch causes 41 errors with Hadoop v2.6.5. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 5m
11s{color} | {color:red} The patch causes 41 errors with Hadoop v2.7.4. {color}
|
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
32s{color} | {color:red} hbase-server-jdk1.8.0_162 with JDK v1.8.0_162
generated 6 new + 3 unchanged - 0 fixed = 9 total (was 3) {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
37s{color} | {color:red} hbase-server-jdk1.7.0_171 with JDK v1.7.0_171
generated 6 new + 3 unchanged - 0 fixed = 9 total (was 3) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 25s{color}
| {color:red} hbase-server in the patch
[jira] [Commented] (HBASE-20070) website generation is failing
[
https://issues.apache.org/jira/browse/HBASE-20070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16381162#comment-16381162
]
Misty Stanley-Jones commented on HBASE-20070:
-
FYI with that one change, I got a totally clean run of all phases including
adding the files, generating the patch, etc. I can't reproduce the dependency
problem you were seeing.
> website generation is failing
> -
>
> Key: HBASE-20070
> URL: https://issues.apache.org/jira/browse/HBASE-20070
> Project: HBase
> Issue Type: Bug
> Components: website
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Blocker
> Attachments: HBASE-20070-misty.patch, HBASE-20070-misty.patch.1,
> HBASE-20070-misty.patch.3, HBASE-20070.0.patch, HBASE-20070.1.patch,
> HBASE-20070.2.patch, HBASE-20070.3.patch, HBASE-20070.4.patch,
> hbase-install-log-a29b3caf4dbc7b8833474ef5da5438f7f6907e00.txt
>
>
> website generation has been failing since Feb 20th
> {code}
> Checking out files: 100% (68971/68971), done.
> Usage: grep [OPTION]... PATTERN [FILE]...
> Try 'grep --help' for more information.
> PUSHED is 2
> is not yet mentioned in the hbase-site commit log. Assuming we don't have it
> yet. 2
> Building HBase
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=256m;
> support was removed in 8.0
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=256m;
> support was removed in 8.0
> Failure: mvn clean site
> Build step 'Execute shell' marked build as failure
> {code}
> The status email says
> {code}
> Build status: Still Failing
> The HBase website has not been updated to incorporate HBase commit
> ${CURRENT_HBASE_COMMIT}.
> {code}
> Looking at the code where that grep happens, it looks like the env variable
> CURRENT_HBASE_COMMIT isn't getting set. That comes from some git command. I'm
> guessing the version of git changed on the build hosts and upended our
> assumptions.
> we should fix this to 1) rely on git's porcelain interface, and 2) fail as
> soon as that git command fails
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-18135) Track file archival for low latency space quota with snapshots
[
https://issues.apache.org/jira/browse/HBASE-18135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16381152#comment-16381152
]
Hadoop QA commented on HBASE-18135:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 5 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
23s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
54s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
37s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
12s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
25s{color} | {color:red} hbase-client: The patch generated 2 new + 7 unchanged
- 1 fixed = 9 total (was 8) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
1s{color} | {color:red} hbase-server: The patch generated 26 new + 333
unchanged - 2 fixed = 359 total (was 335) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 2 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
11s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 6m
7s{color} | {color:red} The patch causes 10 errors with Hadoop v2.6.5. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 8m
6s{color} | {color:red} The patch causes 10 errors with Hadoop v2.7.4. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 10m
13s{color} | {color:red} The patch causes 10 errors with Hadoop v3.0.0. {color}
|
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green}
1m 5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
27s{color} | {color:green} hbase-protocol-shaded in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
57s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 94m
13s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green}
[jira] [Commented] (HBASE-20070) website generation is failing
[
https://issues.apache.org/jira/browse/HBASE-20070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16381149#comment-16381149
]
Misty Stanley-Jones commented on HBASE-20070:
-
You might be using {{cp}} from Homebrew. If it is in {{/usr/local/bin}} that's
probably what's going on. Yes, that's the line. I'm pretty sure that Linux
{{cp}} supporting {{-r}} is new so it is possible Jenkins won't work with this,
in which case you'll have to somehow check what flags are supported or just use
{{mv}} or something (even {{rsync}} but that's more complicated).
> website generation is failing
> -
>
> Key: HBASE-20070
> URL: https://issues.apache.org/jira/browse/HBASE-20070
> Project: HBase
> Issue Type: Bug
> Components: website
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Blocker
> Attachments: HBASE-20070-misty.patch, HBASE-20070-misty.patch.1,
> HBASE-20070-misty.patch.3, HBASE-20070.0.patch, HBASE-20070.1.patch,
> HBASE-20070.2.patch, HBASE-20070.3.patch, HBASE-20070.4.patch,
> hbase-install-log-a29b3caf4dbc7b8833474ef5da5438f7f6907e00.txt
>
>
> website generation has been failing since Feb 20th
> {code}
> Checking out files: 100% (68971/68971), done.
> Usage: grep [OPTION]... PATTERN [FILE]...
> Try 'grep --help' for more information.
> PUSHED is 2
> is not yet mentioned in the hbase-site commit log. Assuming we don't have it
> yet. 2
> Building HBase
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=256m;
> support was removed in 8.0
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=256m;
> support was removed in 8.0
> Failure: mvn clean site
> Build step 'Execute shell' marked build as failure
> {code}
> The status email says
> {code}
> Build status: Still Failing
> The HBase website has not been updated to incorporate HBase commit
> ${CURRENT_HBASE_COMMIT}.
> {code}
> Looking at the code where that grep happens, it looks like the env variable
> CURRENT_HBASE_COMMIT isn't getting set. That comes from some git command. I'm
> guessing the version of git changed on the build hosts and upended our
> assumptions.
> we should fix this to 1) rely on git's porcelain interface, and 2) fail as
> soon as that git command fails
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20070) website generation is failing
[
https://issues.apache.org/jira/browse/HBASE-20070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16381120#comment-16381120
]
Sean Busbey commented on HBASE-20070:
-
Just to confirm, this is the line I'm updating, right?
{code}
cp -au "${component_dir}"/target/staging/* .
{code}
> website generation is failing
> -
>
> Key: HBASE-20070
> URL: https://issues.apache.org/jira/browse/HBASE-20070
> Project: HBase
> Issue Type: Bug
> Components: website
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Blocker
> Attachments: HBASE-20070-misty.patch, HBASE-20070-misty.patch.1,
> HBASE-20070-misty.patch.3, HBASE-20070.0.patch, HBASE-20070.1.patch,
> HBASE-20070.2.patch, HBASE-20070.3.patch, HBASE-20070.4.patch,
> hbase-install-log-a29b3caf4dbc7b8833474ef5da5438f7f6907e00.txt
>
>
> website generation has been failing since Feb 20th
> {code}
> Checking out files: 100% (68971/68971), done.
> Usage: grep [OPTION]... PATTERN [FILE]...
> Try 'grep --help' for more information.
> PUSHED is 2
> is not yet mentioned in the hbase-site commit log. Assuming we don't have it
> yet. 2
> Building HBase
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=256m;
> support was removed in 8.0
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=256m;
> support was removed in 8.0
> Failure: mvn clean site
> Build step 'Execute shell' marked build as failure
> {code}
> The status email says
> {code}
> Build status: Still Failing
> The HBase website has not been updated to incorporate HBase commit
> ${CURRENT_HBASE_COMMIT}.
> {code}
> Looking at the code where that grep happens, it looks like the env variable
> CURRENT_HBASE_COMMIT isn't getting set. That comes from some git command. I'm
> guessing the version of git changed on the build hosts and upended our
> assumptions.
> we should fix this to 1) rely on git's porcelain interface, and 2) fail as
> soon as that git command fails
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20070) website generation is failing
[
https://issues.apache.org/jira/browse/HBASE-20070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16381117#comment-16381117
]
Sean Busbey commented on HBASE-20070:
-
curious. I'm using a mac locally, so I wonder why this didn't fail for me.
> website generation is failing
> -
>
> Key: HBASE-20070
> URL: https://issues.apache.org/jira/browse/HBASE-20070
> Project: HBase
> Issue Type: Bug
> Components: website
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Blocker
> Attachments: HBASE-20070-misty.patch, HBASE-20070-misty.patch.1,
> HBASE-20070-misty.patch.3, HBASE-20070.0.patch, HBASE-20070.1.patch,
> HBASE-20070.2.patch, HBASE-20070.3.patch, HBASE-20070.4.patch,
> hbase-install-log-a29b3caf4dbc7b8833474ef5da5438f7f6907e00.txt
>
>
> website generation has been failing since Feb 20th
> {code}
> Checking out files: 100% (68971/68971), done.
> Usage: grep [OPTION]... PATTERN [FILE]...
> Try 'grep --help' for more information.
> PUSHED is 2
> is not yet mentioned in the hbase-site commit log. Assuming we don't have it
> yet. 2
> Building HBase
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=256m;
> support was removed in 8.0
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=256m;
> support was removed in 8.0
> Failure: mvn clean site
> Build step 'Execute shell' marked build as failure
> {code}
> The status email says
> {code}
> Build status: Still Failing
> The HBase website has not been updated to incorporate HBase commit
> ${CURRENT_HBASE_COMMIT}.
> {code}
> Looking at the code where that grep happens, it looks like the env variable
> CURRENT_HBASE_COMMIT isn't getting set. That comes from some git command. I'm
> guessing the version of git changed on the build hosts and upended our
> assumptions.
> we should fix this to 1) rely on git's porcelain interface, and 2) fail as
> soon as that git command fails
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20108) `hbase zkcli` falls into a non-interactive prompt after HBASE-15199
[
https://issues.apache.org/jira/browse/HBASE-20108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16381116#comment-16381116
]
Josh Elser commented on HBASE-20108:
bq. could we only include jline when zkcli is invoked? hate to add another
runtime dependency that's only used in one place.
Can try to do that!
> `hbase zkcli` falls into a non-interactive prompt after HBASE-15199
> ---
>
> Key: HBASE-20108
> URL: https://issues.apache.org/jira/browse/HBASE-20108
> Project: HBase
> Issue Type: Bug
> Components: Usability
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-20108.001.branch-2.patch
>
>
> HBASE-15199 pulls the jruby-complete jar out of the normal classpath for
> commands run in HBase. Jruby-complete bundles jline inside. ZK uses jline for
> its nice shell-like usage.
> The problem is that this uncovered a bug where we're not explicitly bundling
> a version of jline to make sure that {{hbase zkcli}} actually works. As long
> as we're expecting {{zkcli}} to be there, we should provide jline on the
> classpath to make sure the users get a real cli.
> Thanks to [~sergey.soldatov] for getting to the bottom of it quickly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-17825) Backup: further optimizations
[ https://issues.apache.org/jira/browse/HBASE-17825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16381115#comment-16381115 ] Hudson commented on HBASE-17825: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4666 (See [https://builds.apache.org/job/HBase-Trunk_matrix/4666/]) HBASE-17825: Backup further optimizations (elserj: rev 6cfa208add6ea424e17cae00114ebd3e7d7967f1) * (edit) hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/WALPlayer.java * (edit) hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat2.java * (edit) hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/impl/IncrementalTableBackupClient.java * (edit) hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/mapreduce/MapReduceBackupMergeJob.java * (edit) hbase-backup/src/test/java/org/apache/hadoop/hbase/backup/TestIncrementalBackupMergeWithFailures.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSTableDescriptors.java > Backup: further optimizations > - > > Key: HBASE-17825 > URL: https://issues.apache.org/jira/browse/HBASE-17825 > Project: HBase > Issue Type: Improvement >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Critical > Labels: backup > Fix For: 3.0.0 > > Attachments: HBASE-17825-v1.patch, HBASE-17825-v2.patch, > HBASE-17825-v3.patch, HBASE-17825-v4.patch, HBASE-17825-v5.patch > > > Some phases of backup and restore can be optimized: > # WALPlayer support for multiple tables > # Run DistCp once per all tables during backup/restore > The eventual goal: > # 2 M/R jobs per backup/restore -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20070) website generation is failing
[
https://issues.apache.org/jira/browse/HBASE-20070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1638#comment-1638
]
Misty Stanley-Jones commented on HBASE-20070:
-
-1 on patch 4, because it won't run on Mac out of the box, due to differences
in the {{cp}} command. I solved a similar thing with sed here:
[https://github.com/docker/docker.github.io/blob/master/_scripts/fetch-upstream-resources.sh#L10]
However it looks like Linux honors the {{-r}} flag for {{cp}} now, so maybe you
can just switch the {{-au}} to {{-r}} everywhere (after verifying on a Jenkins
worker that it has a version of {{cp}} that supports the flag?)
I've changed the {{cp}} locally and will do more testing today.
(I sent this hours ago but it didn't post because the session was expired but
the page doesn't have the smarts to realize that.)
> website generation is failing
> -
>
> Key: HBASE-20070
> URL: https://issues.apache.org/jira/browse/HBASE-20070
> Project: HBase
> Issue Type: Bug
> Components: website
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Blocker
> Attachments: HBASE-20070-misty.patch, HBASE-20070-misty.patch.1,
> HBASE-20070-misty.patch.3, HBASE-20070.0.patch, HBASE-20070.1.patch,
> HBASE-20070.2.patch, HBASE-20070.3.patch, HBASE-20070.4.patch,
> hbase-install-log-a29b3caf4dbc7b8833474ef5da5438f7f6907e00.txt
>
>
> website generation has been failing since Feb 20th
> {code}
> Checking out files: 100% (68971/68971), done.
> Usage: grep [OPTION]... PATTERN [FILE]...
> Try 'grep --help' for more information.
> PUSHED is 2
> is not yet mentioned in the hbase-site commit log. Assuming we don't have it
> yet. 2
> Building HBase
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=256m;
> support was removed in 8.0
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=256m;
> support was removed in 8.0
> Failure: mvn clean site
> Build step 'Execute shell' marked build as failure
> {code}
> The status email says
> {code}
> Build status: Still Failing
> The HBase website has not been updated to incorporate HBase commit
> ${CURRENT_HBASE_COMMIT}.
> {code}
> Looking at the code where that grep happens, it looks like the env variable
> CURRENT_HBASE_COMMIT isn't getting set. That comes from some git command. I'm
> guessing the version of git changed on the build hosts and upended our
> assumptions.
> we should fix this to 1) rely on git's porcelain interface, and 2) fail as
> soon as that git command fails
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19598) Fix TestAssignmentManagerMetrics flaky test
[ https://issues.apache.org/jira/browse/HBASE-19598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16381110#comment-16381110 ] Sean Busbey commented on HBASE-19598: - for ~2 weeks this has been the test holding up nightlies on master. I could run it down if folks are busy elsewhere? > Fix TestAssignmentManagerMetrics flaky test > --- > > Key: HBASE-19598 > URL: https://issues.apache.org/jira/browse/HBASE-19598 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0-beta-1 >Reporter: Balazs Meszaros >Assignee: Balazs Meszaros >Priority: Major > Attachments: HBASE-19598.master.001.patch, > HBASE-19598.master.002.patch, HBASE-19598.master.003.patch, > HBASE-19598.master.003.patch, TestUtil.java > > > TestAssignmentManagerMetrics fails constantly. After bisecting, it seems that > commit 010012cbcb broke it (HBASE-18946). > The test method runs successfully, but it cannot shut the minicluster down, > and hangs forever. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19423) Replication entries are not filtered correctly when replication scope is set through WAL Co-processor
[
https://issues.apache.org/jira/browse/HBASE-19423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francis Liu updated HBASE-19423:
Fix Version/s: (was: 1.3.2)
1.3.3
> Replication entries are not filtered correctly when replication scope is set
> through WAL Co-processor
> -
>
> Key: HBASE-19423
> URL: https://issues.apache.org/jira/browse/HBASE-19423
> Project: HBase
> Issue Type: Bug
>Reporter: Mohammad Arshad
>Priority: Major
> Labels: Replication, WAL
> Fix For: 1.4.0, 1.3.3
>
> Attachments: HBASE-19423-branch-1.3-001.patch,
> HBASE-19423-branch-1.4-001.patch, HBASE-19423-master-001-test.patch
>
>
> Replicaion scope set in WALObserver is getting reset in
> Replication.scopeWALEdits().
> Because of this problem custom implementation of WALObserver can not be used
> as a replication filter.
> Suppose WALObserver implementation has logic to filter all entries from
> family f2
> {code}
> // Filter all family f2 rows
> public static class ReplicationFilterWALCoprocessor extends BaseWALObserver
> {
> @Override
> public boolean preWALWrite(ObserverContext WALCoprocessorEnvironment> ctx,
> HRegionInfo info, WALKey logKey, WALEdit logEdit) throws IOException {
> ArrayList cells = logEdit.getCells();
> for (Cell cell : cells) {
> byte[] fam = CellUtil.cloneFamily(cell);
> if ("f2".equals(Bytes.toString(fam))) {
> NavigableMap scopes = logKey.getScopes();
> if (scopes == null) {
> logKey.setScopes(new TreeMap Integer>(Bytes.BYTES_COMPARATOR));
> }
> logKey.getScopes().put(fam, HConstants.REPLICATION_SCOPE_LOCAL);
> }
> }
> return false;
> }
> }
> {code}
> This logic can not work as
> {{org.apache.hadoop.hbase.replication.regionserver.Replication.scopeWALEdits()}}
> recreates and populates scopes.
> *SOLUTION:*
> In Replication.scopeWALEdits(), create scopes map only if WALKey does not
> have it.
> {code}
> NavigableMap scopes = logKey.getScopes();
> if (scopes == null) {
> scopes = new TreeMap(Bytes.BYTES_COMPARATOR);
> }
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20075) remove logic for branch-1.1 nightly testing
[ https://issues.apache.org/jira/browse/HBASE-20075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francis Liu updated HBASE-20075: Fix Version/s: (was: 1.3.2) 1.3.3 > remove logic for branch-1.1 nightly testing > --- > > Key: HBASE-20075 > URL: https://issues.apache.org/jira/browse/HBASE-20075 > Project: HBase > Issue Type: Task > Components: test >Affects Versions: 1.1.13 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Minor > Fix For: 2.0.0, 3.0.0, 1.5.0, 1.3.3, 1.2.8, 1.4.3 > > Attachments: HBASE-20075.0.patch > > > since branch-1.1 is EOM, remove the branch-1.1 specific logic from our > Jenkinsfile and delete the Jenkinsfile in branch-1.1 so we'll stop running > nightlies for it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18370) Port HBASE-16209 to branch-1.3
[ https://issues.apache.org/jira/browse/HBASE-18370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francis Liu updated HBASE-18370: Fix Version/s: (was: 1.3.2) 1.3.3 > Port HBASE-16209 to branch-1.3 > -- > > Key: HBASE-18370 > URL: https://issues.apache.org/jira/browse/HBASE-18370 > Project: HBase > Issue Type: Improvement >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Major > Fix For: 1.3.3 > > Attachments: HBASE-18370-branch-1.3.patch > > > Currently once a region goes into FAILED_OPEN state this requires operator > intervention. With some underlying causes, this is necessary. With others, > the master could eventually successfully deploy the region without humans in > the loop. The master should optionally attempt automatic resolution of > FAILED_OPEN states with a strategy of: delay, unassign, reassign. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-17890) FuzzyRowFilter fail if unaligned support is false
[
https://issues.apache.org/jira/browse/HBASE-17890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francis Liu updated HBASE-17890:
Fix Version/s: (was: 1.3.2)
1.3.3
> FuzzyRowFilter fail if unaligned support is false
> -
>
> Key: HBASE-17890
> URL: https://issues.apache.org/jira/browse/HBASE-17890
> Project: HBase
> Issue Type: Sub-task
> Components: util
>Affects Versions: 2.0.0, 1.2.5
>Reporter: Jerry He
>Assignee: Chia-Ping Tsai
>Priority: Major
> Fix For: 2.0.0, 3.0.0, 1.3.3, 1.2.8, 1.4.3
>
> Attachments: HBASE-17890.v0.branch-1.patch, HBASE-17890.v0.patch,
> HBASE-17890.v1.branch-1.patch, HBASE-17890.v1.patch, HBASE-17890.v2.patch,
> HBASE-17890.v3.patch, HBASE-17890.v3.patch, HBASE-17890.v3.patch,
> HBASE-17890.v3.patch, HBASE-17890.v3.patch
>
>
> When unaligned support is false, FuzzyRow tests fail:
> {noformat}
> Failed tests:
> TestFuzzyRowAndColumnRangeFilter.Test:134->runTest:157->runScanner:186
> expected:<10> but was:<0>
> TestFuzzyRowFilter.testSatisfiesForward:81 expected: but was:
> TestFuzzyRowFilter.testSatisfiesReverse:121 expected: but
> was:
> TestFuzzyRowFilterEndToEnd.testEndToEnd:247->runTest1:278->runScanner:343
> expected:<6250> but was:<0>
> TestFuzzyRowFilterEndToEnd.testFilterList:385->runTest:417->runScanner:445
> expected:<5> but was:<0>
> TestFuzzyRowFilterEndToEnd.testHBASE14782:204 expected:<6> but was:<0>
> {noformat}
> This can be reproduced in the case described in HBASE-17869. Or on a platform
> really without unaligned support.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-13838) Fix shared TaskStatusTmpl.jamon issues (coloring, content, etc.)
[ https://issues.apache.org/jira/browse/HBASE-13838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francis Liu updated HBASE-13838: Fix Version/s: (was: 1.3.2) 1.3.3 > Fix shared TaskStatusTmpl.jamon issues (coloring, content, etc.) > > > Key: HBASE-13838 > URL: https://issues.apache.org/jira/browse/HBASE-13838 > Project: HBase > Issue Type: Bug > Components: UI >Affects Versions: 1.1.0 >Reporter: Lars George >Assignee: Matt Warhaftig >Priority: Major > Labels: beginner > Fix For: 2.0.0, 1.5.0, 1.3.3, 1.4.3 > > Attachments: hbase-13838-v1.patch, hbase-13838_post.tiff, > hbase-13838_post_put_command.txt, hbase-13838_pre.tiff > > > There are a few issues with the shared TaskStatusTmpl: > - "Client operations" tab is always empty > For Master this is expected, but for RegionServers there is never anything > listed either. Fix for RS status page (probably caused by params not > containing Operation subclass anymore, but some PB generated classes?) > - Hide “Client Operations” tab for master UI > Since operations are RS only. Or we fix this and make other calls show here. > - The "alert-error" for aborted tasks is not set in CSS at all > When a task was aborted it should be amber or red, but the assigned style is > not in any of the linked stylesheets (abort-error). Add. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20108) `hbase zkcli` falls into a non-interactive prompt after HBASE-15199
[
https://issues.apache.org/jira/browse/HBASE-20108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16381104#comment-16381104
]
Sean Busbey commented on HBASE-20108:
-
could we only include jline when zkcli is invoked? hate to add another runtime
dependency that's only used in one place.
> `hbase zkcli` falls into a non-interactive prompt after HBASE-15199
> ---
>
> Key: HBASE-20108
> URL: https://issues.apache.org/jira/browse/HBASE-20108
> Project: HBase
> Issue Type: Bug
> Components: Usability
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-20108.001.branch-2.patch
>
>
> HBASE-15199 pulls the jruby-complete jar out of the normal classpath for
> commands run in HBase. Jruby-complete bundles jline inside. ZK uses jline for
> its nice shell-like usage.
> The problem is that this uncovered a bug where we're not explicitly bundling
> a version of jline to make sure that {{hbase zkcli}} actually works. As long
> as we're expecting {{zkcli}} to be there, we should provide jline on the
> classpath to make sure the users get a real cli.
> Thanks to [~sergey.soldatov] for getting to the bottom of it quickly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-8963) Add configuration option to skip HFile archiving
[ https://issues.apache.org/jira/browse/HBASE-8963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-8963: -- Labels: usability (was: ) > Add configuration option to skip HFile archiving > > > Key: HBASE-8963 > URL: https://issues.apache.org/jira/browse/HBASE-8963 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Critical > Labels: usability > Fix For: 2.0.0 > > Attachments: 8963-v10.txt, HBASE-8963.trunk.v1.patch, > HBASE-8963.trunk.v2.patch, HBASE-8963.trunk.v3.patch, > HBASE-8963.trunk.v4.patch, HBASE-8963.trunk.v5.patch, > HBASE-8963.trunk.v6.patch, HBASE-8963.trunk.v7.patch, > HBASE-8963.trunk.v8.patch, HBASE-8963.trunk.v9.patch > > > Currently HFileArchiver is always called when a table is dropped or compacted. > A configuration option (either global or per table) should be provided so > that archiving can be skipped when table is deleted or compacted. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18862) backport HBASE-15109 to branch-1.1,branch-1.2,branch-1.3
[ https://issues.apache.org/jira/browse/HBASE-18862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francis Liu updated HBASE-18862: Fix Version/s: (was: 1.3.2) 1.3.3 > backport HBASE-15109 to branch-1.1,branch-1.2,branch-1.3 > > > Key: HBASE-18862 > URL: https://issues.apache.org/jira/browse/HBASE-18862 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 1.3.1, 1.2.6, 1.1.12 >Reporter: Yechao Chen >Assignee: Yechao Chen >Priority: Critical > Fix For: 1.3.3, 1.2.8 > > Attachments: HBASE-18862-branch-1.1-v1.patch, > HBASE-18862-branch-1.1.patch, HBASE-18862-branch-1.2-v1.patch, > HBASE-18862-branch-1.2.patch, HBASE-18862-branch-1.3-v1.patch, > HBASE-18862-branch-1.3.patch, HBASE-18862-branch-1.patch > > > HBASE-15109 should apply to branch-1.1,branch-1.2,branch-1.3 also. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20108) `hbase zkcli` falls into a non-interactive prompt after HBASE-15199
[
https://issues.apache.org/jira/browse/HBASE-20108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16381101#comment-16381101
]
Josh Elser commented on HBASE-20108:
We exclude jline from the zookeeper dependency. I think we just need to undo
that..
> `hbase zkcli` falls into a non-interactive prompt after HBASE-15199
> ---
>
> Key: HBASE-20108
> URL: https://issues.apache.org/jira/browse/HBASE-20108
> Project: HBase
> Issue Type: Bug
> Components: Usability
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-20108.001.branch-2.patch
>
>
> HBASE-15199 pulls the jruby-complete jar out of the normal classpath for
> commands run in HBase. Jruby-complete bundles jline inside. ZK uses jline for
> its nice shell-like usage.
> The problem is that this uncovered a bug where we're not explicitly bundling
> a version of jline to make sure that {{hbase zkcli}} actually works. As long
> as we're expecting {{zkcli}} to be there, we should provide jline on the
> classpath to make sure the users get a real cli.
> Thanks to [~sergey.soldatov] for getting to the bottom of it quickly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20108) `hbase zkcli` falls into a non-interactive prompt after HBASE-15199
[
https://issues.apache.org/jira/browse/HBASE-20108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser updated HBASE-20108:
---
Attachment: HBASE-20108.001.branch-2.patch
> `hbase zkcli` falls into a non-interactive prompt after HBASE-15199
> ---
>
> Key: HBASE-20108
> URL: https://issues.apache.org/jira/browse/HBASE-20108
> Project: HBase
> Issue Type: Bug
> Components: Usability
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-20108.001.branch-2.patch
>
>
> HBASE-15199 pulls the jruby-complete jar out of the normal classpath for
> commands run in HBase. Jruby-complete bundles jline inside. ZK uses jline for
> its nice shell-like usage.
> The problem is that this uncovered a bug where we're not explicitly bundling
> a version of jline to make sure that {{hbase zkcli}} actually works. As long
> as we're expecting {{zkcli}} to be there, we should provide jline on the
> classpath to make sure the users get a real cli.
> Thanks to [~sergey.soldatov] for getting to the bottom of it quickly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20108) `hbase zkcli` falls into a non-interactive prompt after HBASE-15199
[
https://issues.apache.org/jira/browse/HBASE-20108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser updated HBASE-20108:
---
Status: Patch Available (was: Open)
> `hbase zkcli` falls into a non-interactive prompt after HBASE-15199
> ---
>
> Key: HBASE-20108
> URL: https://issues.apache.org/jira/browse/HBASE-20108
> Project: HBase
> Issue Type: Bug
> Components: Usability
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Critical
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-20108.001.branch-2.patch
>
>
> HBASE-15199 pulls the jruby-complete jar out of the normal classpath for
> commands run in HBase. Jruby-complete bundles jline inside. ZK uses jline for
> its nice shell-like usage.
> The problem is that this uncovered a bug where we're not explicitly bundling
> a version of jline to make sure that {{hbase zkcli}} actually works. As long
> as we're expecting {{zkcli}} to be there, we should provide jline on the
> classpath to make sure the users get a real cli.
> Thanks to [~sergey.soldatov] for getting to the bottom of it quickly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-13603) Write test asserting desired priority of RS->Master RPCs
[
https://issues.apache.org/jira/browse/HBASE-13603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francis Liu updated HBASE-13603:
Fix Version/s: (was: 1.3.2)
1.3.3
> Write test asserting desired priority of RS->Master RPCs
>
>
> Key: HBASE-13603
> URL: https://issues.apache.org/jira/browse/HBASE-13603
> Project: HBase
> Issue Type: Test
> Components: IPC/RPC, test
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Minor
> Fix For: 2.0.0, 1.5.0, 1.3.3, 1.2.8, 1.4.3
>
>
> From HBASE-13351:
> {quote}
> Any way we can write a FT test to assert that the RS->Master APIs are treated
> with higher priority. I see your UT for asserting the annotation.
> {quote}
> Write a test that verifies expected RPCs are run on the correct pools in as
> real of an environment possible.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-16071) The VisibilityLabelFilter and AccessControlFilter should not count the "delete cell"
[
https://issues.apache.org/jira/browse/HBASE-16071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francis Liu updated HBASE-16071:
Fix Version/s: (was: 1.3.2)
1.3.3
> The VisibilityLabelFilter and AccessControlFilter should not count the
> "delete cell"
>
>
> Key: HBASE-16071
> URL: https://issues.apache.org/jira/browse/HBASE-16071
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Minor
> Fix For: 2.0.0, 1.5.0, 1.3.3, 1.4.3
>
> Attachments: HBASE-16071-v1.patch, HBASE-16071-v2.patch,
> HBASE-16071-v3.patch
>
>
> The VisibilityLabelFilter will see and count the "delete cell" if the
> scan.isRaw() returns true, so the (put) cell will be skipped if it has lower
> version than "delete cell"
> The critical code is shown below:
> {code:title=VisibilityLabelFilter.java|borderStyle=solid}
> public ReturnCode filterKeyValue(Cell cell) throws IOException {
> if (curFamily.getBytes() == null
> || !(CellUtil.matchingFamily(cell, curFamily.getBytes(),
> curFamily.getOffset(),
> curFamily.getLength( {
> curFamily.set(cell.getFamilyArray(), cell.getFamilyOffset(),
> cell.getFamilyLength());
> // For this family, all the columns can have max of
> curFamilyMaxVersions versions. No need to
> // consider the older versions for visibility label check.
> // Ideally this should have been done at a lower layer by HBase (?)
> curFamilyMaxVersions = cfVsMaxVersions.get(curFamily);
> // Family is changed. Just unset curQualifier.
> curQualifier.unset();
> }
> if (curQualifier.getBytes() == null
> || !(CellUtil.matchingQualifier(cell, curQualifier.getBytes(),
> curQualifier.getOffset(),
> curQualifier.getLength( {
> curQualifier.set(cell.getQualifierArray(), cell.getQualifierOffset(),
> cell.getQualifierLength());
> curQualMetVersions = 0;
> }
> curQualMetVersions++;
> if (curQualMetVersions > curFamilyMaxVersions) {
> return ReturnCode.SKIP;
> }
> return this.expEvaluator.evaluate(cell) ? ReturnCode.INCLUDE :
> ReturnCode.SKIP;
> }
> {code}
> [VisibilityLabelFilter.java|https://github.com/apache/hbase/blob/d7a4499dfc8b3936a0eca867589fc2b23b597866/hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityLabelFilter.java]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-18910) Backport HBASE-17292 "Add observer notification before bulk loaded hfile is moved to region directory" to 1.3
[ https://issues.apache.org/jira/browse/HBASE-18910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francis Liu updated HBASE-18910: Fix Version/s: (was: 1.3.2) 1.3.3 > Backport HBASE-17292 "Add observer notification before bulk loaded hfile is > moved to region directory" to 1.3 > - > > Key: HBASE-18910 > URL: https://issues.apache.org/jira/browse/HBASE-18910 > Project: HBase > Issue Type: Bug >Reporter: Guangxu Cheng >Assignee: Guangxu Cheng >Priority: Major > Fix For: 1.3.3 > > Attachments: HBASE-18910.branch-1.3.v1.patch > > > HBASE-18900 will backport HBASE-17290 to branch-1.3.But HBASE-17290 is > dependent on HBASE-17292.so this issue will backport HBASE-17292 to > branch-1.3. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-19458) Allow building HBase 1.3.x against Hadoop 2.8.2
[ https://issues.apache.org/jira/browse/HBASE-19458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francis Liu updated HBASE-19458: Fix Version/s: (was: 1.3.2) 1.3.3 > Allow building HBase 1.3.x against Hadoop 2.8.2 > --- > > Key: HBASE-19458 > URL: https://issues.apache.org/jira/browse/HBASE-19458 > Project: HBase > Issue Type: Bug >Affects Versions: 1.3.1 >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl >Priority: Major > Fix For: 1.3.3 > > Attachments: 19458.txt > > > Currently the 1.3 branch cannot be built against Hadoop 2.8.2. (that latest > stable 2.x release). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18415) The local timeout may cause Admin to submit duplicate request
[ https://issues.apache.org/jira/browse/HBASE-18415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francis Liu updated HBASE-18415: Fix Version/s: (was: 1.3.2) 1.3.3 > The local timeout may cause Admin to submit duplicate request > - > > Key: HBASE-18415 > URL: https://issues.apache.org/jira/browse/HBASE-18415 > Project: HBase > Issue Type: Bug >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Major > Fix For: 2.0.0, 1.5.0, 1.3.3, 1.4.3 > > Attachments: HBASE-18415.branch-1.ut.patch, > HBASE-18415.branch-1.v0.patch, HBASE-18415.branch-1.v1.patch, > HBASE-18415.branch-1.v2.patch, HBASE-18415.branch-1.v3.patch, > HBASE-18415.branch-1.v3.patch, HBASE-18415.branch-1.v3.patch, > HBASE-18415.branch-1.v4.patch, HBASE-18415.branch-1.v4.patch, > HBASE-18415.branch-1.v4.patch > > > After a timeout occurs on first request, client will retry the request with > distinct group/nonce. The second request may bring the TableXXXException back > if the first request have changed the table state. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18182) Setting HBASE_REGIONSERVER_OPTS with UseG1GC will cause regionserver start error
[
https://issues.apache.org/jira/browse/HBASE-18182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francis Liu updated HBASE-18182:
Fix Version/s: (was: 1.3.2)
1.3.3
> Setting HBASE_REGIONSERVER_OPTS with UseG1GC will cause regionserver start
> error
>
>
> Key: HBASE-18182
> URL: https://issues.apache.org/jira/browse/HBASE-18182
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 2.0.0, 1.1.6, 1.3.1, 1.2.6
>Reporter: Fangyuan Deng
>Assignee: Fangyuan Deng
>Priority: Major
> Fix For: 2.0.0, 1.5.0, 1.3.3, 1.2.8, 1.4.3
>
> Attachments: HBASE-18182.0.patch
>
>
> when we set in hbase-env.sh to use G1GC
> HBASE_REGIONSERVER_OPTS="$HBASE_REGIONSERVER_OPTS -XX:+UseG1GC -Xms96g
> -Xmx96g -XX:MaxGCPauseMillis=100"
> then run ./hbase-daemon.sh start regionserver
> Conflicting collector combinations in option list; please refer to the
> release notes for the combinations allowed
> Error: Could not create the Java Virtual Machine.
> Error: A fatal exception has occurred. Program will exit.
> because the HBASE_OPTS default use CMS and is conflicted with
> HBASE_REGIONSERVER_OPTS
> so I add a patch to modify bin/hbase like this
> elif [ "$COMMAND" = "regionserver" ] ; then
> CLASS='org.apache.hadoop.hbase.regionserver.HRegionServer'
> if [ "$1" != "stop" ] ; then
> HBASE_OPTS="$HBASE_OPTS $HBASE_REGIONSERVER_OPTS"
> if [[ $HBASE_REGIONSERVER_OPTS == *UseG1GC* ]] ; then
> HBASE_OPTS=${HBASE_OPTS/'-XX:+UseConcMarkSweepGC'/''}
> fi
> fi
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-18639) nightly jobs that need to do jdk7 + jdk8 need more than 6 hours
[ https://issues.apache.org/jira/browse/HBASE-18639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francis Liu updated HBASE-18639: Fix Version/s: (was: 1.3.2) 1.3.3 > nightly jobs that need to do jdk7 + jdk8 need more than 6 hours > --- > > Key: HBASE-18639 > URL: https://issues.apache.org/jira/browse/HBASE-18639 > Project: HBase > Issue Type: Task > Components: test >Reporter: Sean Busbey >Priority: Major > Labels: beginner > Fix For: 1.5.0, 1.3.3, 1.2.8, 1.4.3 > > > looks like the branches that need to do 2 jdks will need more than 6 hours, > at least until we can parallelize things. > e.g. > * > [branch-1.2|https://builds.apache.org/view/H-L/view/HBase/job/HBase%20Nightly/job/branch-1.2/buildTimeTrend] > * > [branch-1.3|https://builds.apache.org/view/H-L/view/HBase/job/HBase%20Nightly/job/branch-1.3/buildTimeTrend] > branch-1.4 and branch-1 are failing in jdk7 checks, so they don't go that > long yet, but once we get those tests fixed presumably they'll need the time > as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-17960) IntegrationTestReplication fails in successive runs due to lack of appropriate cleanup
[ https://issues.apache.org/jira/browse/HBASE-17960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francis Liu updated HBASE-17960: Fix Version/s: (was: 1.3.2) 1.3.3 > IntegrationTestReplication fails in successive runs due to lack of > appropriate cleanup > -- > > Key: HBASE-17960 > URL: https://issues.apache.org/jira/browse/HBASE-17960 > Project: HBase > Issue Type: Bug > Components: integration tests >Reporter: Ashu Pachauri >Priority: Major > Fix For: 2.0.0, 3.0.0, 1.5.0, 1.3.3, 1.4.3 > > > The way ITR works right now is that it adds a peer named 'TestPeer' for the > replication destination cluster. The name of the peer is same across runs. > Also, it removes the peer in the beginning of each run. However, it does not > wait for the queues corresponding to the peer to get cleaned up (which is an > asynchronous operation and can take 10s of seconds). This causes the next run > to fail and so on. > The test setup should wait for a non-trivial amount of time to cleanup the > queues corresponding to the peer. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-17885) Backport HBASE-15871 to branch-1
[ https://issues.apache.org/jira/browse/HBASE-17885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francis Liu updated HBASE-17885: Fix Version/s: (was: 1.3.2) 1.3.3 > Backport HBASE-15871 to branch-1 > > > Key: HBASE-17885 > URL: https://issues.apache.org/jira/browse/HBASE-17885 > Project: HBase > Issue Type: Bug > Components: Scanners >Affects Versions: 1.3.1, 1.2.5, 1.1.8 >Reporter: ramkrishna.s.vasudevan >Priority: Major > Fix For: 1.3.3, 1.2.8, 1.4.3 > > > Will try to rebase the branch-1 patch at the earliest. Hope the fix versions > are correct. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-17406) Occasional failed splits on branch-1.3
[
https://issues.apache.org/jira/browse/HBASE-17406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francis Liu updated HBASE-17406:
Fix Version/s: (was: 1.3.2)
1.3.3
> Occasional failed splits on branch-1.3
> --
>
> Key: HBASE-17406
> URL: https://issues.apache.org/jira/browse/HBASE-17406
> Project: HBase
> Issue Type: Sub-task
> Components: Compaction, regionserver
>Affects Versions: 1.3.0
>Reporter: Mikhail Antonov
>Priority: Major
> Fix For: 1.3.3
>
>
> We observed occasional (rare) failed splits on branch-1.3 builds, that might
> be another echo of HBASE-13082.
> Essentially here's what seems to be happening:
> First on the RS hosting to-be-split parent seems to see some
> FileNotFoundException's in the logs. It could be simple file not found on
> some scanner path:
> {quote}
> 16/11/21 07:19:28 WARN hdfs.DFSClient: DFS Read
> java.io.FileNotFoundException: File does not exist:
> at
> org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:71)
> at
> org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:61)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocationsInt(FSNamesystem.java:1828)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1799)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1712)
>
> at
> org.apache.hadoop.hbase.io.hfile.HFileBlock.readWithExtra(HFileBlock.java:733)
> at
> org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1461)
> at
> org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderImpl.readBlockDataInternal(HFileBlock.java:1715)
> at
> org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderImpl.readBlockData(HFileBlock.java:1560)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:454)
> at
> org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.loadDataBlockWithScanInfo(HFileBlockIndex.java:271)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.seekTo(HFileReaderV2.java:651)
> at
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.reseekTo(HFileReaderV2.java:631)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseekAtOrAfter(StoreFileScanner.java:292)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseek(StoreFileScanner.java:201)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.enforceSeek(StoreFileScanner.java:412)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.requestSeek(StoreFileScanner.java:375)
> at
> org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:310)
> at
> org.apache.hadoop.hbase.regionserver.KeyValueHeap.requestSeek(KeyValueHeap.java:268)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:889)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.seekToNextRow(StoreScanner.java:867)
> at org.apache.hadoop.hbase.regionserver.Stor
> {quote}
> Or it could be warning from HFileArchiver:
> {quote}
> 16/11/21 07:20:44 WARN backup.HFileArchiver: Failed to archive class
> org.apache.hadoop.hbase.backup.HFileArchiver$FileableStoreFile,
> because it does not exist! Skipping and continuing on.
> java.io.FileNotFoundException: File/Directory does not exist.
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirAttrOp.setTimes(FSDirAttrOp.java:121)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.setTimes(FSNamesystem.java:1910)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.setTimes(NameNodeRpcServer.java:1223)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.setTimes(ClientNamenodeProtocolServerSideTranslatorPB.java:915)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:982)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2049)
> at or
> {quote}
> Then on the RS hosting parent I'm seeing:
> {quote}16/11/21 18:03:17 ERROR regionserver.HRegion: Could not initialize all
> stores for the region=
> 16/11/21 18:03:17 ERROR regionserver.HRegion: Could not initialize all stores
> for the
> 16/11/21 18:03:17 WARN ipc.Client: interrupted waiting to send rpc request to
> server
> java.lang.InterruptedException
> at
[jira] [Updated] (HBASE-15678) Normalize RetryingCallable cache clearing and implementations
[ https://issues.apache.org/jira/browse/HBASE-15678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francis Liu updated HBASE-15678: Fix Version/s: (was: 1.3.2) 1.3.3 > Normalize RetryingCallable cache clearing and implementations > - > > Key: HBASE-15678 > URL: https://issues.apache.org/jira/browse/HBASE-15678 > Project: HBase > Issue Type: Sub-task > Components: Client >Reporter: Gary Helmling >Assignee: Gary Helmling >Priority: Major > Fix For: 1.3.3 > > > This is a fair amount of duplication and inconsistency in the meta cache > handling of RetryingCallable implementations: > * meta cache is often cleared in prepare() when reload=true, in addition to > being cleared in throwable() > * each RetryingCallable implementation does this slightly differently, > leading to inconsistencies and potential bugs > * RegionServerCallable and RegionAdminServiceCallable duplicate a lot of > code, but with small, seemingly unnecessary inconsistencies. We should clean > these up into a common base with subclasses doing only the necessary > differentiation. > The main goal here is to establish some common handling, to the extent > possible, for the meta cache interactions by the different implementations. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-17055) Disabling table not getting enabled after clean cluster restart.
[
https://issues.apache.org/jira/browse/HBASE-17055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francis Liu updated HBASE-17055:
Fix Version/s: (was: 1.3.2)
1.3.3
> Disabling table not getting enabled after clean cluster restart.
>
>
> Key: HBASE-17055
> URL: https://issues.apache.org/jira/browse/HBASE-17055
> Project: HBase
> Issue Type: Bug
> Components: Region Assignment
>Affects Versions: 1.3.0
>Reporter: Y. SREENIVASULU REDDY
>Assignee: Stephen Yuan Jiang
>Priority: Major
> Fix For: 1.3.3
>
>
> scenario:
> 1. Disable the table, while disabling the table is in progress.
> 2. Restart whole HBase service.
> 3. Then enable the table.
> the above operation leads to RIT continously.
> pls find the below logs for understanding.
> while disabling the table whole hbase service went down.
> the following is the master logs
> {noformat}
> 2016-11-09 19:32:55,102 INFO
> [RpcServer.FifoWFPBQ.default.handler=49,queue=4,port=16000] master.HMaster:
> Client=seenu//host-1 disable testTable
> 2016-11-09 19:32:55,257 DEBUG
> [RpcServer.FifoWFPBQ.default.handler=49,queue=4,port=16000]
> procedure2.ProcedureExecutor: Procedure DisableTableProcedure
> (table=testTable) id=8 owner=seenu state=RUNNABLE:DISABLE_TABLE_PREPARE added
> to the store.
> 2016-11-09 19:32:55,264 DEBUG [ProcedureExecutor-5]
> lock.ZKInterProcessLockBase: Acquired a lock for
> /hbase/table-lock/testTable/write-master:165
> 2016-11-09 19:32:55,285 DEBUG
> [RpcServer.FifoWFPBQ.default.handler=49,queue=4,port=16000]
> master.MasterRpcServices: Checking to see if procedure is done procId=8
> 2016-11-09 19:32:55,386 DEBUG
> [RpcServer.FifoWFPBQ.default.handler=49,queue=4,port=16000]
> master.MasterRpcServices: Checking to see if procedure is done procId=8
> 2016-11-09 19:32:55,513 INFO [ProcedureExecutor-5]
> zookeeper.ZKTableStateManager: Moving table testTable state from DISABLING to
> DISABLING
> 2016-11-09 19:32:55,587 DEBUG
> [RpcServer.FifoWFPBQ.default.handler=49,queue=4,port=16000]
> master.MasterRpcServices: Checking to see if procedure is done procId=8
> 2016-11-09 19:32:55,628 INFO [ProcedureExecutor-5]
> procedure.DisableTableProcedure: Offlining 1 regions.
> .
> .
> .
> .
> .
> .
> .
> .
> 2016-11-09 19:33:02,871 INFO [AM.ZK.Worker-pool2-t7] master.RegionStates:
> Offlined 1890fa9c085dcc2ee0602f4bab069d10 from host-1,16040,1478690163056
> Wed Nov 9 19:33:02 CST 2016 Terminating master
> {noformat}
> here we need to observe
> {color:red} Offlined 1890fa9c085dcc2ee0602f4bab069d10 from
> host-1,16040,1478690163056 {color}
> then hmaster went down, all regionServers also made down.
> After hmaster and regionserver are restarted
> executed enable Table operation on the table.
> {panel:title=HMaster
> Logs|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1|bgColor=#CE}
> {noformat}
> 2016-11-09 19:49:57,059 INFO
> [RpcServer.FifoWFPBQ.default.handler=49,queue=4,port=16000] master.HMaster:
> Client=seenu//host-1 enable testTable
> 2016-11-09 19:49:57,325 DEBUG
> [RpcServer.FifoWFPBQ.default.handler=49,queue=4,port=16000]
> procedure2.ProcedureExecutor: Procedure EnableTableProcedure
> (table=testTable) id=9 owner=seenu state=RUNNABLE:ENABLE_TABLE_PREPARE added
> to the store.
> 2016-11-09 19:49:57,333 DEBUG [ProcedureExecutor-2]
> lock.ZKInterProcessLockBase: Acquired a lock for
> /hbase/table-lock/testTable/write-master:168
> 2016-11-09 19:49:57,335 DEBUG [hconnection-0x745317ee-shared--pool3-t11]
> ipc.RpcClientImpl: Use SIMPLE authentication for service ClientService,
> sasl=false
> 2016-11-09 19:49:57,335 DEBUG [hconnection-0x745317ee-shared--pool3-t11]
> ipc.RpcClientImpl: Connecting to host-1:16040
> 2016-11-09 19:49:57,347 DEBUG
> [RpcServer.FifoWFPBQ.default.handler=49,queue=4,port=16000]
> master.MasterRpcServices: Checking to see if procedure is done procId=9
> 2016-11-09 19:49:57,449 DEBUG
> [RpcServer.FifoWFPBQ.default.handler=49,queue=4,port=16000]
> master.MasterRpcServices: Checking to see if procedure is done procId=9
> 2016-11-09 19:49:57,579 INFO [ProcedureExecutor-2]
> procedure.EnableTableProcedure: Attempting to enable the table testTable
> 2016-11-09 19:49:57,580 INFO [ProcedureExecutor-2]
> zookeeper.ZKTableStateManager: Moving table testTable state from DISABLED to
> ENABLING
> 2016-11-09 19:49:57,655 DEBUG
> [RpcServer.FifoWFPBQ.default.handler=49,queue=4,port=16000]
> master.MasterRpcServices: Checking to see if procedure is done procId=9
> 2016-11-09 19:49:57,707 INFO [ProcedureExecutor-2]
> procedure.EnableTableProcedure: Table 'testTable' has 1 regions, of which 1
> are offline.
> 2016-11-09 19:49:57,707 INFO [ProcedureExecutor-2]
> procedure.EnableTableProcedure:
[jira] [Updated] (HBASE-14391) Empty regionserver WAL will never be deleted although the coresponding regionserver has been stale
[
https://issues.apache.org/jira/browse/HBASE-14391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francis Liu updated HBASE-14391:
Fix Version/s: (was: 1.3.2)
1.3.3
> Empty regionserver WAL will never be deleted although the coresponding
> regionserver has been stale
> --
>
> Key: HBASE-14391
> URL: https://issues.apache.org/jira/browse/HBASE-14391
> Project: HBase
> Issue Type: Bug
> Components: wal
>Affects Versions: 1.0.2
>Reporter: Qianxi Zhang
>Assignee: Qianxi Zhang
>Priority: Major
> Fix For: 2.0.0, 1.5.0, 1.3.3, 1.2.8, 1.4.3
>
> Attachments: HBASE-14391-master-v3.patch,
> HBASE_14391_master_v4.patch, HBASE_14391_trunk_v1.patch,
> HBASE_14391_trunk_v2.patch, WALs-leftover-dir.txt
>
>
> When I restarted the hbase cluster in which there was few data, I found there
> are two directories for one host with different timestamp which indicates
> that the old regionserver wal directory is not deleted.
> FHLog#989
> {code}
> @Override
> public void close() throws IOException {
> shutdown();
> final FileStatus[] files = getFiles();
> if (null != files && 0 != files.length) {
> for (FileStatus file : files) {
> Path p = getWALArchivePath(this.fullPathArchiveDir, file.getPath());
> // Tell our listeners that a log is going to be archived.
> if (!this.listeners.isEmpty()) {
> for (WALActionsListener i : this.listeners) {
> i.preLogArchive(file.getPath(), p);
> }
> }
> if (!FSUtils.renameAndSetModifyTime(fs, file.getPath(), p)) {
> throw new IOException("Unable to rename " + file.getPath() + " to "
> + p);
> }
> // Tell our listeners that a log was archived.
> if (!this.listeners.isEmpty()) {
> for (WALActionsListener i : this.listeners) {
> i.postLogArchive(file.getPath(), p);
> }
> }
> }
> LOG.debug("Moved " + files.length + " WAL file(s) to " +
> FSUtils.getPath(this.fullPathArchiveDir));
> }
> LOG.info("Closed WAL: " + toString());
> }
> {code}
> When regionserver is stopped, the hlog will be archived, so wal/regionserver
> is empty in hdfs.
> MasterFileSystem#252
> {code}
> if (curLogFiles == null || curLogFiles.length == 0) {
> // Empty log folder. No recovery needed
> continue;
> }
> {code}
> The regionserver directory will be not splitted, it makes sense. But it will
> be not deleted.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-14223) Meta WALs are not cleared if meta region was closed and RS aborts
[
https://issues.apache.org/jira/browse/HBASE-14223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francis Liu updated HBASE-14223:
Fix Version/s: (was: 1.3.2)
1.3.3
> Meta WALs are not cleared if meta region was closed and RS aborts
> -
>
> Key: HBASE-14223
> URL: https://issues.apache.org/jira/browse/HBASE-14223
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
>Priority: Major
> Fix For: 2.0.0, 1.5.0, 1.3.3, 1.2.8, 1.4.3
>
> Attachments: HBASE-14223logs, hbase-14223_v0.patch,
> hbase-14223_v1-branch-1.patch, hbase-14223_v2-branch-1.patch,
> hbase-14223_v3-branch-1.patch, hbase-14223_v3-branch-1.patch,
> hbase-14223_v3-master.patch
>
>
> When an RS opens meta, and later closes it, the WAL(FSHlog) is not closed.
> The last WAL file just sits there in the RS WAL directory. If RS stops
> gracefully, the WAL file for meta is deleted. Otherwise if RS aborts, WAL for
> meta is not cleaned. It is also not split (which is correct) since master
> determines that the RS no longer hosts meta at the time of RS abort.
> From a cluster after running ITBLL with CM, I see a lot of {{-splitting}}
> directories left uncleaned:
> {code}
> [root@os-enis-dal-test-jun-4-7 cluster-os]# sudo -u hdfs hadoop fs -ls
> /apps/hbase/data/WALs
> Found 31 items
> drwxr-xr-x - hbase hadoop 0 2015-06-05 01:14
> /apps/hbase/data/WALs/hregion-58203265
> drwxr-xr-x - hbase hadoop 0 2015-06-05 07:54
> /apps/hbase/data/WALs/os-enis-dal-test-jun-4-1.openstacklocal,16020,1433489308745-splitting
> drwxr-xr-x - hbase hadoop 0 2015-06-05 09:28
> /apps/hbase/data/WALs/os-enis-dal-test-jun-4-1.openstacklocal,16020,1433494382959-splitting
> drwxr-xr-x - hbase hadoop 0 2015-06-05 10:01
> /apps/hbase/data/WALs/os-enis-dal-test-jun-4-1.openstacklocal,16020,1433498252205-splitting
> ...
> {code}
> The directories contain WALs from meta:
> {code}
> [root@os-enis-dal-test-jun-4-7 cluster-os]# sudo -u hdfs hadoop fs -ls
> /apps/hbase/data/WALs/os-enis-dal-test-jun-4-5.openstacklocal,16020,1433466904285-splitting
> Found 2 items
> -rw-r--r-- 3 hbase hadoop 201608 2015-06-05 03:15
> /apps/hbase/data/WALs/os-enis-dal-test-jun-4-5.openstacklocal,16020,1433466904285-splitting/os-enis-dal-test-jun-4-5.openstacklocal%2C16020%2C1433466904285..meta.1433470511501.meta
> -rw-r--r-- 3 hbase hadoop 44420 2015-06-05 04:36
> /apps/hbase/data/WALs/os-enis-dal-test-jun-4-5.openstacklocal,16020,1433466904285-splitting/os-enis-dal-test-jun-4-5.openstacklocal%2C16020%2C1433466904285..meta.1433474111645.meta
> {code}
> The RS hosted the meta region for some time:
> {code}
> 2015-06-05 03:14:28,692 INFO [PostOpenDeployTasks:1588230740]
> zookeeper.MetaTableLocator: Setting hbase:meta region location in ZooKeeper
> as os-enis-dal-test-jun-4-5.openstacklocal,16020,1433466904285
> ...
> 2015-06-05 03:15:17,302 INFO
> [RS_CLOSE_META-os-enis-dal-test-jun-4-5:16020-0] regionserver.HRegion: Closed
> hbase:meta,,1.1588230740
> {code}
> In between, a WAL is created:
> {code}
> 2015-06-05 03:15:11,707 INFO
> [RS_OPEN_META-os-enis-dal-test-jun-4-5:16020-0-MetaLogRoller] wal.FSHLog:
> Rolled WAL
> /apps/hbase/data/WALs/os-enis-dal-test-jun-4-5.openstacklocal,16020,1433466904285/os-enis-dal-test-jun-4-5.openstacklocal%2C16020%2C1433466904285..meta.1433470511501.meta
> with entries=385, filesize=196.88 KB; new WAL
> /apps/hbase/data/WALs/os-enis-dal-test-jun-4-5.openstacklocal,16020,1433466904285/os-enis-dal-test-jun-4-5.openstacklocal%2C16020%2C1433466904285..meta.1433474111645.meta
> {code}
> When CM killed the region server later master did not see these WAL files:
> {code}
> ./hbase-hbase-master-os-enis-dal-test-jun-4-3.log:2015-06-05 03:36:46,075
> INFO [MASTER_SERVER_OPERATIONS-os-enis-dal-test-jun-4-3:16000-0]
> master.SplitLogManager: started splitting 2 logs in
> [hdfs://os-enis-dal-test-jun-4-1.openstacklocal:8020/apps/hbase/data/WALs/os-enis-dal-test-jun-4-5.openstacklocal,16020,1433466904285-splitting]
> for [os-enis-dal-test-jun-4-5.openstacklocal,16020,1433466904285]
> ./hbase-hbase-master-os-enis-dal-test-jun-4-3.log:2015-06-05 03:36:47,300
> INFO [main-EventThread] wal.WALSplitter: Archived processed log
> hdfs://os-enis-dal-test-jun-4-1.openstacklocal:8020/apps/hbase/data/WALs/os-enis-dal-test-jun-4-5.openstacklocal,16020,1433466904285-splitting/os-enis-dal-test-jun-4-5.openstacklocal%2C16020%2C1433466904285.default.1433475074436
> to
> hdfs://os-enis-dal-test-jun-4-1.openstacklocal:8020/apps/hbase/data/oldWALs/os-enis-dal-test-jun-4-5.openstacklocal%2C16020%2C1433466904285.default.1433475074436
> ./hbase-hbase-master-os-enis-dal-test-jun-4-3.log:2015-06-05 03:36:50,497
> INFO
[jira] [Updated] (HBASE-13160) SplitLogWorker does not pick up the task immediately
[
https://issues.apache.org/jira/browse/HBASE-13160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francis Liu updated HBASE-13160:
Fix Version/s: (was: 1.3.2)
1.3.3
> SplitLogWorker does not pick up the task immediately
>
>
> Key: HBASE-13160
> URL: https://issues.apache.org/jira/browse/HBASE-13160
> Project: HBase
> Issue Type: Improvement
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
>Priority: Major
> Fix For: 2.0.0, 1.5.0, 1.3.3, 1.4.3
>
> Attachments: hbase-13160_v1.patch
>
>
> We were reading some code with Jeffrey, and we realized that the
> SplitLogWorker's internal task loop is weird. It does {{ls}} every second and
> sleeps, but have another mechanism to learn about new tasks, but does not
> make affective use of the zk notification.
> I have a simple patch which might improve this area.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-12943) Set sun.net.inetaddr.ttl in HBase
[ https://issues.apache.org/jira/browse/HBASE-12943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francis Liu updated HBASE-12943: Fix Version/s: (was: 1.3.2) 1.3.3 > Set sun.net.inetaddr.ttl in HBase > - > > Key: HBASE-12943 > URL: https://issues.apache.org/jira/browse/HBASE-12943 > Project: HBase > Issue Type: Bug >Reporter: Liu Shaohui >Assignee: Liu Shaohui >Priority: Major > Fix For: 2.0.0, 1.5.0, 1.3.3, 1.4.3 > > Attachments: 12943-1-master.txt > > > The default value of config: sun.net.inetaddr.ttl is -1 and the java > processes will cache the mapping of hostname to ip address forever, See: > http://docs.oracle.com/javase/7/docs/technotes/guides/net/properties.html > But things go wrong when a regionserver with same hostname and different ip > address rejoins the hbase cluster. The HMaster will get wrong ip address of > the regionserver from this cache and every region assignment to this > regionserver will be blocked for a time because the HMaster can't communicate > with the regionserver. > A tradeoff is to set the sun.net.inetaddr.ttl to 10m or 1h and make the wrong > cache expired. > Suggestions are welcomed. Thanks~ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-14583) Enabled client-side metrics by default
[ https://issues.apache.org/jira/browse/HBASE-14583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francis Liu updated HBASE-14583: Fix Version/s: (was: 1.3.2) 1.3.3 > Enabled client-side metrics by default > -- > > Key: HBASE-14583 > URL: https://issues.apache.org/jira/browse/HBASE-14583 > Project: HBase > Issue Type: Task > Components: Client, Operability, Performance >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Critical > Fix For: 2.0.0, 1.5.0, 1.3.3, 1.4.3 > > Attachments: 14583.00.branch-1.patch, 14583.00.patch > > > Enabling this feature by default for master and branch-1. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
