[jira] [Commented] (HBASE-21476) Support for nanosecond timestamps
[ https://issues.apache.org/jira/browse/HBASE-21476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16705697#comment-16705697 ] Sean Busbey commented on HBASE-21476: - I see the WIP patches are starting to address MOB handling, but I don't see it mentioned in the scope document at all. Should call out in the scope document the impact of having tables configured for MOB both with and without nanosecond support enabled. > Support for nanosecond timestamps > - > > Key: HBASE-21476 > URL: https://issues.apache.org/jira/browse/HBASE-21476 > Project: HBase > Issue Type: New Feature >Affects Versions: 2.1.1 >Reporter: Andrey Elenskiy >Assignee: Andrey Elenskiy >Priority: Major > Labels: features, patch > Attachments: Apache HBase - Nanosecond Timestamps v1.pdf, > nanosecond_timestamps_v1.patch, nanosecond_timestamps_v2.patch > > > Introducing a new table attribute "NANOSECOND_TIMESTAMPS" to tell HBase to > handle timestamps with nanosecond precision. This is useful for applications > that timestamp updates at the source with nanoseconds and still want features > like column family TTL and "hbase.hstore.time.to.purge.deletes" to work. > The attribute should be specified either on new tables or on existing tables > which have timestamps only with nanosecond precision. There's no migration > from milliseconds to nanoseconds for already existing tables. We could add > this migration as part of compaction if you think that would be useful, but > that would obviously make the change more complex. > I've added a new EnvironmentEdge method "currentTimeNano()" that uses > [java.time.Instant|https://docs.oracle.com/javase/8/docs/api/java/time/Instant.html] > to get time in nanoseconds which means it will only work with Java 8. The > idea is to gradually replace all places where "EnvironmentEdge.currentTime()" > is used to have HBase working purely with nanoseconds (which is a > prerequisite for HBASE-14070). Also, I've refactored ScanInfo and > PartitionedMobCompactor to expect TableDescriptor as an argument which makes > code a little cleaner and easier to extend. > Couple more points: > - column family TTL (specified in seconds) and > "hbase.hstore.time.to.purge.deletes" (specified in milliseconds) options > don't need to be changed, those are adjusted automatically. > - Per cell TTL needs to be scaled by clients accordingly after > "NANOSECOND_TIMESTAMPS" table attribute is specified. > Looking for everyone's feedback to know if that's a worthwhile direction. > Will add more comprehensive tests in a later patch. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21534) TestAssignmentManager is flakey
[
https://issues.apache.org/jira/browse/HBASE-21534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16705685#comment-16705685
]
Hadoop QA commented on HBASE-21534:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
12s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
50s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
49s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 27s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}129m
12s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
27s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}165m 29s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21534 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12950131/HBASE-21534.patch |
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 949cf349eb60 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 766aa1bfcc |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/15164/testReport/ |
| Max. process+thread count | 4946 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/15164/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> TestAssignmentMana
[jira] [Commented] (HBASE-21476) Support for nanosecond timestamps
[ https://issues.apache.org/jira/browse/HBASE-21476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16705695#comment-16705695 ] Sean Busbey commented on HBASE-21476: - Do we need to account for this table attribute when bulk loading? What about snapshots? do they retain information on wether their contents use nanoseconds? Do tables cloned from a snapshot have to have the same nanosecond config as the snapshot? > Support for nanosecond timestamps > - > > Key: HBASE-21476 > URL: https://issues.apache.org/jira/browse/HBASE-21476 > Project: HBase > Issue Type: New Feature >Affects Versions: 2.1.1 >Reporter: Andrey Elenskiy >Assignee: Andrey Elenskiy >Priority: Major > Labels: features, patch > Attachments: Apache HBase - Nanosecond Timestamps v1.pdf, > nanosecond_timestamps_v1.patch, nanosecond_timestamps_v2.patch > > > Introducing a new table attribute "NANOSECOND_TIMESTAMPS" to tell HBase to > handle timestamps with nanosecond precision. This is useful for applications > that timestamp updates at the source with nanoseconds and still want features > like column family TTL and "hbase.hstore.time.to.purge.deletes" to work. > The attribute should be specified either on new tables or on existing tables > which have timestamps only with nanosecond precision. There's no migration > from milliseconds to nanoseconds for already existing tables. We could add > this migration as part of compaction if you think that would be useful, but > that would obviously make the change more complex. > I've added a new EnvironmentEdge method "currentTimeNano()" that uses > [java.time.Instant|https://docs.oracle.com/javase/8/docs/api/java/time/Instant.html] > to get time in nanoseconds which means it will only work with Java 8. The > idea is to gradually replace all places where "EnvironmentEdge.currentTime()" > is used to have HBase working purely with nanoseconds (which is a > prerequisite for HBASE-14070). Also, I've refactored ScanInfo and > PartitionedMobCompactor to expect TableDescriptor as an argument which makes > code a little cleaner and easier to extend. > Couple more points: > - column family TTL (specified in seconds) and > "hbase.hstore.time.to.purge.deletes" (specified in milliseconds) options > don't need to be changed, those are adjusted automatically. > - Per cell TTL needs to be scaled by clients accordingly after > "NANOSECOND_TIMESTAMPS" table attribute is specified. > Looking for everyone's feedback to know if that's a worthwhile direction. > Will add more comprehensive tests in a later patch. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21476) Support for nanosecond timestamps
[
https://issues.apache.org/jira/browse/HBASE-21476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16705694#comment-16705694
]
Sean Busbey commented on HBASE-21476:
-
{quote}
HBase clients that don’t provide timestamps to their requests, don’t need to
change anything.
The clients looking to write into a table that supports nanoseconds will have
to provide
nanoseconds via “setTimestamp()” using Java 8’s Instant API. Same goes for per
cell TTL and
Get/Scan requests with “setTimeRange()”
{quote}
What happens if a client that doesn't support nanoseconds attempts to write to
a table that is configured for nanoseconds?
> Support for nanosecond timestamps
> -
>
> Key: HBASE-21476
> URL: https://issues.apache.org/jira/browse/HBASE-21476
> Project: HBase
> Issue Type: New Feature
>Affects Versions: 2.1.1
>Reporter: Andrey Elenskiy
>Assignee: Andrey Elenskiy
>Priority: Major
> Labels: features, patch
> Attachments: Apache HBase - Nanosecond Timestamps v1.pdf,
> nanosecond_timestamps_v1.patch, nanosecond_timestamps_v2.patch
>
>
> Introducing a new table attribute "NANOSECOND_TIMESTAMPS" to tell HBase to
> handle timestamps with nanosecond precision. This is useful for applications
> that timestamp updates at the source with nanoseconds and still want features
> like column family TTL and "hbase.hstore.time.to.purge.deletes" to work.
> The attribute should be specified either on new tables or on existing tables
> which have timestamps only with nanosecond precision. There's no migration
> from milliseconds to nanoseconds for already existing tables. We could add
> this migration as part of compaction if you think that would be useful, but
> that would obviously make the change more complex.
> I've added a new EnvironmentEdge method "currentTimeNano()" that uses
> [java.time.Instant|https://docs.oracle.com/javase/8/docs/api/java/time/Instant.html]
> to get time in nanoseconds which means it will only work with Java 8. The
> idea is to gradually replace all places where "EnvironmentEdge.currentTime()"
> is used to have HBase working purely with nanoseconds (which is a
> prerequisite for HBASE-14070). Also, I've refactored ScanInfo and
> PartitionedMobCompactor to expect TableDescriptor as an argument which makes
> code a little cleaner and easier to extend.
> Couple more points:
> - column family TTL (specified in seconds) and
> "hbase.hstore.time.to.purge.deletes" (specified in milliseconds) options
> don't need to be changed, those are adjusted automatically.
> - Per cell TTL needs to be scaled by clients accordingly after
> "NANOSECOND_TIMESTAMPS" table attribute is specified.
> Looking for everyone's feedback to know if that's a worthwhile direction.
> Will add more comprehensive tests in a later patch.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21476) Support for nanosecond timestamps
[
https://issues.apache.org/jira/browse/HBASE-21476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-21476:
Status: Patch Available (was: Open)
moving to patch available so QABot can give it a spin. Please use {{git
format-patch}} to create future patches.
> Support for nanosecond timestamps
> -
>
> Key: HBASE-21476
> URL: https://issues.apache.org/jira/browse/HBASE-21476
> Project: HBase
> Issue Type: New Feature
>Affects Versions: 2.1.1
>Reporter: Andrey Elenskiy
>Assignee: Andrey Elenskiy
>Priority: Major
> Labels: features, patch
> Attachments: Apache HBase - Nanosecond Timestamps v1.pdf,
> nanosecond_timestamps_v1.patch, nanosecond_timestamps_v2.patch
>
>
> Introducing a new table attribute "NANOSECOND_TIMESTAMPS" to tell HBase to
> handle timestamps with nanosecond precision. This is useful for applications
> that timestamp updates at the source with nanoseconds and still want features
> like column family TTL and "hbase.hstore.time.to.purge.deletes" to work.
> The attribute should be specified either on new tables or on existing tables
> which have timestamps only with nanosecond precision. There's no migration
> from milliseconds to nanoseconds for already existing tables. We could add
> this migration as part of compaction if you think that would be useful, but
> that would obviously make the change more complex.
> I've added a new EnvironmentEdge method "currentTimeNano()" that uses
> [java.time.Instant|https://docs.oracle.com/javase/8/docs/api/java/time/Instant.html]
> to get time in nanoseconds which means it will only work with Java 8. The
> idea is to gradually replace all places where "EnvironmentEdge.currentTime()"
> is used to have HBase working purely with nanoseconds (which is a
> prerequisite for HBASE-14070). Also, I've refactored ScanInfo and
> PartitionedMobCompactor to expect TableDescriptor as an argument which makes
> code a little cleaner and easier to extend.
> Couple more points:
> - column family TTL (specified in seconds) and
> "hbase.hstore.time.to.purge.deletes" (specified in milliseconds) options
> don't need to be changed, those are adjusted automatically.
> - Per cell TTL needs to be scaled by clients accordingly after
> "NANOSECOND_TIMESTAMPS" table attribute is specified.
> Looking for everyone's feedback to know if that's a worthwhile direction.
> Will add more comprehensive tests in a later patch.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-21538) Rewrite RegionReplicaFlushHandler to use AsyncClusterConnection
Duo Zhang created HBASE-21538: - Summary: Rewrite RegionReplicaFlushHandler to use AsyncClusterConnection Key: HBASE-21538 URL: https://issues.apache.org/jira/browse/HBASE-21538 Project: HBase Issue Type: Sub-task Reporter: Duo Zhang -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-21537) Rewrite ServerManager.closeRegionSilentlyAndWait to use AsyncClusterConnection
Duo Zhang created HBASE-21537: - Summary: Rewrite ServerManager.closeRegionSilentlyAndWait to use AsyncClusterConnection Key: HBASE-21537 URL: https://issues.apache.org/jira/browse/HBASE-21537 Project: HBase Issue Type: Sub-task Reporter: Duo Zhang -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21526) Use AsyncClusterConnection in ServerManager for getRsAdmin
[ https://issues.apache.org/jira/browse/HBASE-21526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21526: -- Assignee: Duo Zhang Status: Patch Available (was: Open) > Use AsyncClusterConnection in ServerManager for getRsAdmin > -- > > Key: HBASE-21526 > URL: https://issues.apache.org/jira/browse/HBASE-21526 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Attachments: HBASE-21516-HBASE-21512.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21526) Use AsyncClusterConnection in ServerManager for getRsAdmin
[ https://issues.apache.org/jira/browse/HBASE-21526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21526: -- Attachment: HBASE-21516-HBASE-21512.patch > Use AsyncClusterConnection in ServerManager for getRsAdmin > -- > > Key: HBASE-21526 > URL: https://issues.apache.org/jira/browse/HBASE-21526 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Priority: Major > Attachments: HBASE-21516-HBASE-21512.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21515) Also initialize an AsyncClusterConnection in HRegionServer
[ https://issues.apache.org/jira/browse/HBASE-21515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21515: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: HBASE-21512 Status: Resolved (was: Patch Available) Pushed to branch HBASE-21512. Thanks [~zghaobac] for reviewing. > Also initialize an AsyncClusterConnection in HRegionServer > -- > > Key: HBASE-21515 > URL: https://issues.apache.org/jira/browse/HBASE-21515 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: HBASE-21512 > > Attachments: HBASE-21515-v1.patch, HBASE-21515-v2.patch, > HBASE-21515-v3.patch, HBASE-21515-v4.patch, HBASE-21515-v4.patch, > HBASE-21515.patch > > > Plan to do this incrementally, so first we will only introduce the new > AsyncClusterConnection, without deleting the old ClusterConnection. And we > can move the code which uses the ClusterConnection to use > AsyncClusterConnection, through different sub tasks here. And finally we can > remove the ClusterConnection completely. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21516) Use AsyncConnection instead of Connection in SecureBulkLoadManager
[ https://issues.apache.org/jira/browse/HBASE-21516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21516: -- Assignee: Duo Zhang Status: Patch Available (was: Open) > Use AsyncConnection instead of Connection in SecureBulkLoadManager > -- > > Key: HBASE-21516 > URL: https://issues.apache.org/jira/browse/HBASE-21516 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Attachments: HBASE-21516-HBASE-21512.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21516) Use AsyncConnection instead of Connection in SecureBulkLoadManager
[ https://issues.apache.org/jira/browse/HBASE-21516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21516: -- Attachment: HBASE-21516-HBASE-21512.patch > Use AsyncConnection instead of Connection in SecureBulkLoadManager > -- > > Key: HBASE-21516 > URL: https://issues.apache.org/jira/browse/HBASE-21516 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: Duo Zhang >Priority: Major > Attachments: HBASE-21516-HBASE-21512.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21486) The current replication implementation for peer in STANDBY state breaks serial replication
[ https://issues.apache.org/jira/browse/HBASE-21486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21486: -- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Pushed to master. Thanks [~zghaobac] for reviewing. > The current replication implementation for peer in STANDBY state breaks > serial replication > -- > > Key: HBASE-21486 > URL: https://issues.apache.org/jira/browse/HBASE-21486 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Critical > Fix For: 3.0.0 > > Attachments: HBASE-21486.patch > > > As we will drop recovered queues in STANDBY state, and also when transiting > to DA, we will drain the replication queue, we may miss to update the last > pushed sequence id, and cause the serial replication to be stuck. Need to > find a way to deal with this. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21534) TestAssignmentManager is flakey
[
https://issues.apache.org/jira/browse/HBASE-21534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-21534:
--
Assignee: Duo Zhang
Status: Patch Available (was: Open)
> TestAssignmentManager is flakey
> ---
>
> Key: HBASE-21534
> URL: https://issues.apache.org/jira/browse/HBASE-21534
> Project: HBase
> Issue Type: Task
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Attachments: HBASE-21534.patch
>
>
> See this in the outout and then the test hang
> {noformat}
> 2018-11-29 20:47:50,061 WARN [MockRSProcedureDispatcher-pool5-t10]
> assignment.AssignmentManager(894): The region server localhost,102,1 is
> already dead, skip reportRegionStateTransition call
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21530) Abort_Procedure should be able to take a list of proc IDs
[ https://issues.apache.org/jira/browse/HBASE-21530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16705589#comment-16705589 ] Allan Yang commented on HBASE-21530: Aborting is not very useful to get out of stuck, since some procedure is not abortable. So in 2.x, we have implemented another mechanism to bypass a stuck procedure. abort_procedure is removed from shell. See HBASE-21223 "Remove this command. It will cause more damage than it could ever solve. It should exist, it should be out in hbck2, not here in user-space." > Abort_Procedure should be able to take a list of proc IDs > - > > Key: HBASE-21530 > URL: https://issues.apache.org/jira/browse/HBASE-21530 > Project: HBase > Issue Type: Improvement >Reporter: Geoffrey Jacoby >Priority: Minor > > As a convenience, it would be helpful if the HBase shell's abort_procedure > call had the option of taking in multiple procedure ids at the same time, > rather than relying on operators to use a loop in an external script. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21464) Splitting blocked with meta NSRE during split transaction
[
https://issues.apache.org/jira/browse/HBASE-21464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16705577#comment-16705577
]
Hadoop QA commented on HBASE-21464:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} branch-1 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
40s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
15s{color} | {color:green} branch-1 passed with JDK v1.8.0_192 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
19s{color} | {color:green} branch-1 passed with JDK v1.7.0_201 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
31s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
50s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} branch-1 passed with JDK v1.8.0_192 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
23s{color} | {color:green} branch-1 passed with JDK v1.7.0_201 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
16s{color} | {color:green} the patch passed with JDK v1.8.0_192 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
18s{color} | {color:green} the patch passed with JDK v1.7.0_201 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
47s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
1m 38s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
15s{color} | {color:green} the patch passed with JDK v1.8.0_192 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
21s{color} | {color:green} the patch passed with JDK v1.7.0_201 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
24s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
7s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 17m 17s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:61288f8 |
| JIRA Issue | HBASE-21464 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12950254/HBASE-21464-branch-1.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux
[jira] [Comment Edited] (HBASE-21464) Splitting blocked with meta NSRE during split transaction
[
https://issues.apache.org/jira/browse/HBASE-21464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16705565#comment-16705565
]
Andrew Purtell edited comment on HBASE-21464 at 12/1/18 1:43 AM:
-
I don't think recursive region relocation works the way we are all expecting,
that when we NSRE on meta we will always end up in ConnectionManager#locateMeta
with useCache = false. The sum of recursive region relocation code is hard to
understand and should be rewritten. I'm not going to do that today. What I do
have is a patch that works reliably to fix the issue in my test environment
when meta is moved during split activity while preserving the intents of
HBASE-10785 (don't overload zookeeper with lookups by looking up meta every
time) and HBASE-19260 (don't overload zookeeper with unnecessary concurrent
lookups). There is a new limit on cache entry age for meta, hardcoded to 10
seconds (should it be configurable? I don't think it matters much...), to
prevent getting stuck on a stale meta location. Consider it a safety valve we
need while continuing to look at this problem.
How to reproduce:
* Run a load client. I use YCSB with 100 threads. The test table is named
"test".
* In the HBase shell: while true ; do sleep 30 ; balancer ; flush 'test';
compact 'test' ; split 'test' ; balancer ; done
You've hit the problem when the result of the shell 'balancer' command is
always false. Go to the master, you'll find a split in progress that can't
finish. Go to the regionserver attempting the split and you'll find the split
worker going back again and again to the regionserver no longer hosting meta
looking for meta.
was (Author: apurtell):
I don't think recursive region relocation works the way we are all expecting,
that when we NSRE on meta we will always end up in
ConnectionManager#locateRegion with useCache = false. The sum of recursive
region relocation code is hard to understand and should be rewritten. I'm not
going to do that today. What I do have is a patch that works reliably to fix
the issue in my test environment when meta is moved during split activity while
preserving the intents of HBASE-10785 (don't overload zookeeper with lookups by
looking up meta every time) and HBASE-19260 (don't overload zookeeper with
unnecessary concurrent lookups). There is a new limit on cache entry age for
meta, hardcoded to 10 seconds (should it be configurable? I don't think it
matters much...), to prevent getting stuck on a stale meta location. Consider
it a safety valve we need while continuing to look at this problem.
How to reproduce:
* Run a load client. I use YCSB with 100 threads. The test table is named
"test".
* In the HBase shell: while true ; do sleep 30 ; balancer ; flush 'test';
compact 'test' ; split 'test' ; balancer ; done
You've hit the problem when the result of the shell 'balancer' command is
always false. Go to the master, you'll find a split in progress that can't
finish. Go to the regionserver attempting the split and you'll find the split
worker going back again and again to the regionserver no longer hosting meta
looking for meta.
> Splitting blocked with meta NSRE during split transaction
> -
>
> Key: HBASE-21464
> URL: https://issues.apache.org/jira/browse/HBASE-21464
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.5.0, 1.4.3, 1.4.4, 1.4.5, 1.4.6, 1.4.8, 1.4.7
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Blocker
> Fix For: 1.5.0, 1.4.9
>
> Attachments: HBASE-21464-branch-1.patch, HBASE-21464-branch-1.patch,
> HBASE-21464-branch-1.patch, HBASE-21464-branch-1.patch
>
>
> Splitting is blocked during split transaction. The split worker is trying to
> update meta but isn't able to relocate it after NSRE:
> {noformat}
> 2018-11-09 17:50:45,277 INFO
> [regionserver/ip-172-31-5-92.us-west-2.compute.internal/172.31.5.92:8120-splits-1541785709434]
> client.RpcRetryingCaller: Call exception, tries=13, retries=350,
> started=88590 ms ago, cancelled=false,
> msg=org.apache.hadoop.hbase.NotServingRegionException: Region hbase:meta,,1
> is not online on ip-172-31-13-83.us-west-2.compute.internal,8120,1541785618832
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:3088)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.getRegion(RSRpcServices.java:1271)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:2198)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:36617)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2396)
> at org.apache.hadoop.
[jira] [Commented] (HBASE-21464) Splitting blocked with meta NSRE during split transaction
[
https://issues.apache.org/jira/browse/HBASE-21464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16705566#comment-16705566
]
Andrew Purtell commented on HBASE-21464:
Attached a new patch for QA. Difference between this and the one posted a
couple of minutes ago is just addition of a comment I dropped that is still
relevant, so I put the comment back.
> Splitting blocked with meta NSRE during split transaction
> -
>
> Key: HBASE-21464
> URL: https://issues.apache.org/jira/browse/HBASE-21464
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.5.0, 1.4.3, 1.4.4, 1.4.5, 1.4.6, 1.4.8, 1.4.7
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Blocker
> Fix For: 1.5.0, 1.4.9
>
> Attachments: HBASE-21464-branch-1.patch, HBASE-21464-branch-1.patch,
> HBASE-21464-branch-1.patch, HBASE-21464-branch-1.patch
>
>
> Splitting is blocked during split transaction. The split worker is trying to
> update meta but isn't able to relocate it after NSRE:
> {noformat}
> 2018-11-09 17:50:45,277 INFO
> [regionserver/ip-172-31-5-92.us-west-2.compute.internal/172.31.5.92:8120-splits-1541785709434]
> client.RpcRetryingCaller: Call exception, tries=13, retries=350,
> started=88590 ms ago, cancelled=false,
> msg=org.apache.hadoop.hbase.NotServingRegionException: Region hbase:meta,,1
> is not online on ip-172-31-13-83.us-west-2.compute.internal,8120,1541785618832
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:3088)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.getRegion(RSRpcServices.java:1271)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:2198)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:36617)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2396)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:297)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:277)row
> 'test,,1541785709452.5ba6596f0050c2dab969d152829227c6.44' on table
> 'hbase:meta' at region=hbase:meta,1.1588230740,
> hostname=ip-172-31-15-225.us-west-2.compute.internal,8120,1541785640586,
> seqNum=0{noformat}
> Clients, in this case YCSB, are hung with part of the keyspace missing:
> {noformat}
> 2018-11-09 17:51:06,033 DEBUG [hconnection-0x5739e567-shared--pool1-t165]
> client.ConnectionManager$HConnectionImplementation: locateRegionInMeta
> parentTable=hbase:meta, metaLocation=, attempt=14 of 35 failed; retrying
> after sleep of 20158 because: No server address listed in hbase:meta for
> region
> test,user307326104267982763,1541785754600.ef90030b05cb02305b75e9bfbc3ee081.
> containing row user3301635648728421323{noformat}
> Balancing cannot run indefinitely because the split transaction is stuck
> {noformat}
> 2018-11-09 17:49:55,478 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=8100] master.HMaster:
> Not running balancer because 3 region(s) in transition:
> [{ef90030b05cb02305b75e9bfbc3ee081 state=SPLITTING_NEW, ts=1541785754606,
> server=ip-172-31-5-92.us-west-2.compute.internal,8120,1541785626417},
> {5ba6596f0050c2dab969d152829227c6 state=SPLITTING, ts=1541785754606,
> server=ip-172-31-5-92.us-west-2.compute{noformat}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21464) Splitting blocked with meta NSRE during split transaction
[
https://issues.apache.org/jira/browse/HBASE-21464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-21464:
---
Attachment: HBASE-21464-branch-1.patch
> Splitting blocked with meta NSRE during split transaction
> -
>
> Key: HBASE-21464
> URL: https://issues.apache.org/jira/browse/HBASE-21464
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.5.0, 1.4.3, 1.4.4, 1.4.5, 1.4.6, 1.4.8, 1.4.7
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Blocker
> Fix For: 1.5.0, 1.4.9
>
> Attachments: HBASE-21464-branch-1.patch, HBASE-21464-branch-1.patch,
> HBASE-21464-branch-1.patch, HBASE-21464-branch-1.patch
>
>
> Splitting is blocked during split transaction. The split worker is trying to
> update meta but isn't able to relocate it after NSRE:
> {noformat}
> 2018-11-09 17:50:45,277 INFO
> [regionserver/ip-172-31-5-92.us-west-2.compute.internal/172.31.5.92:8120-splits-1541785709434]
> client.RpcRetryingCaller: Call exception, tries=13, retries=350,
> started=88590 ms ago, cancelled=false,
> msg=org.apache.hadoop.hbase.NotServingRegionException: Region hbase:meta,,1
> is not online on ip-172-31-13-83.us-west-2.compute.internal,8120,1541785618832
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:3088)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.getRegion(RSRpcServices.java:1271)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:2198)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:36617)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2396)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:297)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:277)row
> 'test,,1541785709452.5ba6596f0050c2dab969d152829227c6.44' on table
> 'hbase:meta' at region=hbase:meta,1.1588230740,
> hostname=ip-172-31-15-225.us-west-2.compute.internal,8120,1541785640586,
> seqNum=0{noformat}
> Clients, in this case YCSB, are hung with part of the keyspace missing:
> {noformat}
> 2018-11-09 17:51:06,033 DEBUG [hconnection-0x5739e567-shared--pool1-t165]
> client.ConnectionManager$HConnectionImplementation: locateRegionInMeta
> parentTable=hbase:meta, metaLocation=, attempt=14 of 35 failed; retrying
> after sleep of 20158 because: No server address listed in hbase:meta for
> region
> test,user307326104267982763,1541785754600.ef90030b05cb02305b75e9bfbc3ee081.
> containing row user3301635648728421323{noformat}
> Balancing cannot run indefinitely because the split transaction is stuck
> {noformat}
> 2018-11-09 17:49:55,478 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=8100] master.HMaster:
> Not running balancer because 3 region(s) in transition:
> [{ef90030b05cb02305b75e9bfbc3ee081 state=SPLITTING_NEW, ts=1541785754606,
> server=ip-172-31-5-92.us-west-2.compute.internal,8120,1541785626417},
> {5ba6596f0050c2dab969d152829227c6 state=SPLITTING, ts=1541785754606,
> server=ip-172-31-5-92.us-west-2.compute{noformat}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21464) Splitting blocked with meta NSRE during split transaction
[
https://issues.apache.org/jira/browse/HBASE-21464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16705565#comment-16705565
]
Andrew Purtell commented on HBASE-21464:
I don't think recursive region relocation works the way we are all expecting,
that when we NSRE on meta we will always end up in
ConnectionManager#locateRegion with useCache = false. The sum of recursive
region relocation code is hard to understand and should be rewritten. I'm not
going to do that today. What I do have is a patch that works reliably to fix
the issue in my test environment when meta is moved during split activity while
preserving the intents of HBASE-10785 (don't overload zookeeper with lookups by
looking up meta every time) and HBASE-19260 (don't overload zookeeper with
unnecessary concurrent lookups). There is a new limit on cache entry age for
meta, hardcoded to 10 seconds (should it be configurable? I don't think it
matters much...), to prevent getting stuck on a stale meta location. Consider
it a safety valve we need while continuing to look at this problem.
How to reproduce:
* Run a load client. I use YCSB with 100 threads. The test table is named
"test".
* In the HBase shell: while true ; do sleep 30 ; balancer ; flush 'test';
compact 'test' ; split 'test' ; balancer ; done
You've hit the problem when the result of the shell 'balancer' command is
always false. Go to the master, you'll find a split in progress that can't
finish. Go to the regionserver attempting the split and you'll find the split
worker going back again and again to the regionserver no longer hosting meta
looking for meta.
> Splitting blocked with meta NSRE during split transaction
> -
>
> Key: HBASE-21464
> URL: https://issues.apache.org/jira/browse/HBASE-21464
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.5.0, 1.4.3, 1.4.4, 1.4.5, 1.4.6, 1.4.8, 1.4.7
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Blocker
> Fix For: 1.5.0, 1.4.9
>
> Attachments: HBASE-21464-branch-1.patch, HBASE-21464-branch-1.patch,
> HBASE-21464-branch-1.patch
>
>
> Splitting is blocked during split transaction. The split worker is trying to
> update meta but isn't able to relocate it after NSRE:
> {noformat}
> 2018-11-09 17:50:45,277 INFO
> [regionserver/ip-172-31-5-92.us-west-2.compute.internal/172.31.5.92:8120-splits-1541785709434]
> client.RpcRetryingCaller: Call exception, tries=13, retries=350,
> started=88590 ms ago, cancelled=false,
> msg=org.apache.hadoop.hbase.NotServingRegionException: Region hbase:meta,,1
> is not online on ip-172-31-13-83.us-west-2.compute.internal,8120,1541785618832
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:3088)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.getRegion(RSRpcServices.java:1271)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:2198)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:36617)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2396)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:297)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:277)row
> 'test,,1541785709452.5ba6596f0050c2dab969d152829227c6.44' on table
> 'hbase:meta' at region=hbase:meta,1.1588230740,
> hostname=ip-172-31-15-225.us-west-2.compute.internal,8120,1541785640586,
> seqNum=0{noformat}
> Clients, in this case YCSB, are hung with part of the keyspace missing:
> {noformat}
> 2018-11-09 17:51:06,033 DEBUG [hconnection-0x5739e567-shared--pool1-t165]
> client.ConnectionManager$HConnectionImplementation: locateRegionInMeta
> parentTable=hbase:meta, metaLocation=, attempt=14 of 35 failed; retrying
> after sleep of 20158 because: No server address listed in hbase:meta for
> region
> test,user307326104267982763,1541785754600.ef90030b05cb02305b75e9bfbc3ee081.
> containing row user3301635648728421323{noformat}
> Balancing cannot run indefinitely because the split transaction is stuck
> {noformat}
> 2018-11-09 17:49:55,478 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=8100] master.HMaster:
> Not running balancer because 3 region(s) in transition:
> [{ef90030b05cb02305b75e9bfbc3ee081 state=SPLITTING_NEW, ts=1541785754606,
> server=ip-172-31-5-92.us-west-2.compute.internal,8120,1541785626417},
> {5ba6596f0050c2dab969d152829227c6 state=SPLITTING, ts=1541785754606,
> server=ip-172-31-5-92.us-west-2.compute{noformat}
>
--
This message was sent by Atlassian
[jira] [Updated] (HBASE-21464) Splitting blocked with meta NSRE during split transaction
[
https://issues.apache.org/jira/browse/HBASE-21464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-21464:
---
Status: Patch Available (was: Open)
> Splitting blocked with meta NSRE during split transaction
> -
>
> Key: HBASE-21464
> URL: https://issues.apache.org/jira/browse/HBASE-21464
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.4.7, 1.4.8, 1.4.6, 1.4.5, 1.4.4, 1.4.3, 1.5.0
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Blocker
> Fix For: 1.5.0, 1.4.9
>
> Attachments: HBASE-21464-branch-1.patch, HBASE-21464-branch-1.patch,
> HBASE-21464-branch-1.patch
>
>
> Splitting is blocked during split transaction. The split worker is trying to
> update meta but isn't able to relocate it after NSRE:
> {noformat}
> 2018-11-09 17:50:45,277 INFO
> [regionserver/ip-172-31-5-92.us-west-2.compute.internal/172.31.5.92:8120-splits-1541785709434]
> client.RpcRetryingCaller: Call exception, tries=13, retries=350,
> started=88590 ms ago, cancelled=false,
> msg=org.apache.hadoop.hbase.NotServingRegionException: Region hbase:meta,,1
> is not online on ip-172-31-13-83.us-west-2.compute.internal,8120,1541785618832
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:3088)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.getRegion(RSRpcServices.java:1271)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:2198)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:36617)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2396)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:297)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:277)row
> 'test,,1541785709452.5ba6596f0050c2dab969d152829227c6.44' on table
> 'hbase:meta' at region=hbase:meta,1.1588230740,
> hostname=ip-172-31-15-225.us-west-2.compute.internal,8120,1541785640586,
> seqNum=0{noformat}
> Clients, in this case YCSB, are hung with part of the keyspace missing:
> {noformat}
> 2018-11-09 17:51:06,033 DEBUG [hconnection-0x5739e567-shared--pool1-t165]
> client.ConnectionManager$HConnectionImplementation: locateRegionInMeta
> parentTable=hbase:meta, metaLocation=, attempt=14 of 35 failed; retrying
> after sleep of 20158 because: No server address listed in hbase:meta for
> region
> test,user307326104267982763,1541785754600.ef90030b05cb02305b75e9bfbc3ee081.
> containing row user3301635648728421323{noformat}
> Balancing cannot run indefinitely because the split transaction is stuck
> {noformat}
> 2018-11-09 17:49:55,478 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=8100] master.HMaster:
> Not running balancer because 3 region(s) in transition:
> [{ef90030b05cb02305b75e9bfbc3ee081 state=SPLITTING_NEW, ts=1541785754606,
> server=ip-172-31-5-92.us-west-2.compute.internal,8120,1541785626417},
> {5ba6596f0050c2dab969d152829227c6 state=SPLITTING, ts=1541785754606,
> server=ip-172-31-5-92.us-west-2.compute{noformat}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21464) Splitting blocked with meta NSRE during split transaction
[
https://issues.apache.org/jira/browse/HBASE-21464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-21464:
---
Attachment: HBASE-21464-branch-1.patch
> Splitting blocked with meta NSRE during split transaction
> -
>
> Key: HBASE-21464
> URL: https://issues.apache.org/jira/browse/HBASE-21464
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.5.0, 1.4.3, 1.4.4, 1.4.5, 1.4.6, 1.4.8, 1.4.7
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Blocker
> Fix For: 1.5.0, 1.4.9
>
> Attachments: HBASE-21464-branch-1.patch, HBASE-21464-branch-1.patch,
> HBASE-21464-branch-1.patch
>
>
> Splitting is blocked during split transaction. The split worker is trying to
> update meta but isn't able to relocate it after NSRE:
> {noformat}
> 2018-11-09 17:50:45,277 INFO
> [regionserver/ip-172-31-5-92.us-west-2.compute.internal/172.31.5.92:8120-splits-1541785709434]
> client.RpcRetryingCaller: Call exception, tries=13, retries=350,
> started=88590 ms ago, cancelled=false,
> msg=org.apache.hadoop.hbase.NotServingRegionException: Region hbase:meta,,1
> is not online on ip-172-31-13-83.us-west-2.compute.internal,8120,1541785618832
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:3088)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.getRegion(RSRpcServices.java:1271)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:2198)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:36617)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2396)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:297)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:277)row
> 'test,,1541785709452.5ba6596f0050c2dab969d152829227c6.44' on table
> 'hbase:meta' at region=hbase:meta,1.1588230740,
> hostname=ip-172-31-15-225.us-west-2.compute.internal,8120,1541785640586,
> seqNum=0{noformat}
> Clients, in this case YCSB, are hung with part of the keyspace missing:
> {noformat}
> 2018-11-09 17:51:06,033 DEBUG [hconnection-0x5739e567-shared--pool1-t165]
> client.ConnectionManager$HConnectionImplementation: locateRegionInMeta
> parentTable=hbase:meta, metaLocation=, attempt=14 of 35 failed; retrying
> after sleep of 20158 because: No server address listed in hbase:meta for
> region
> test,user307326104267982763,1541785754600.ef90030b05cb02305b75e9bfbc3ee081.
> containing row user3301635648728421323{noformat}
> Balancing cannot run indefinitely because the split transaction is stuck
> {noformat}
> 2018-11-09 17:49:55,478 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=8100] master.HMaster:
> Not running balancer because 3 region(s) in transition:
> [{ef90030b05cb02305b75e9bfbc3ee081 state=SPLITTING_NEW, ts=1541785754606,
> server=ip-172-31-5-92.us-west-2.compute.internal,8120,1541785626417},
> {5ba6596f0050c2dab969d152829227c6 state=SPLITTING, ts=1541785754606,
> server=ip-172-31-5-92.us-west-2.compute{noformat}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Issue Comment Deleted] (HBASE-21464) Splitting blocked with meta NSRE during split transaction
[
https://issues.apache.org/jira/browse/HBASE-21464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-21464:
---
Comment: was deleted
(was: | (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
34s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} branch-1 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
17s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
38s{color} | {color:green} branch-1 passed with JDK v1.8.0_192 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
18s{color} | {color:green} branch-1 passed with JDK v1.7.0_201 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
31s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
38s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} branch-1 passed with JDK v1.8.0_192 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
22s{color} | {color:green} branch-1 passed with JDK v1.7.0_201 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
15s{color} | {color:green} the patch passed with JDK v1.8.0_192 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
18s{color} | {color:green} the patch passed with JDK v1.7.0_201 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
29s{color} | {color:red} hbase-client: The patch generated 1 new + 66 unchanged
- 0 fixed = 67 total (was 66) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
35s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
1m 39s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} the patch passed with JDK v1.8.0_192 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
22s{color} | {color:green} the patch passed with JDK v1.7.0_201 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
28s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 32s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:61288f8 |
| JIRA Issue | HBASE-21464 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12950248/HBASE-21464-branch-1.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbasea
[jira] [Commented] (HBASE-21514) Refactor CacheConfig
[
https://issues.apache.org/jira/browse/HBASE-21514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16705562#comment-16705562
]
Hadoop QA commented on HBASE-21514:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 25 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
51s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
8s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
31s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
56s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 1m 56s{color}
| {color:red} hbase-server generated 3 new + 185 unchanged - 3 fixed = 188
total (was 188) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
18s{color} | {color:red} hbase-server: The patch generated 24 new + 419
unchanged - 8 fixed = 443 total (was 427) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
12s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
9m 29s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m
19s{color} | {color:red} hbase-server generated 1 new + 0 unchanged - 0 fixed =
1 total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 24m 27s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 64m 33s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hbase-server |
| | Inconsistent synchronization of
org.apache.hadoop.hbase.regionserver.MetricsRegionServerWrapperImpl.blockCache;
locked 43% of time Unsynchronized access at
MetricsRegionServerWrapperImpl.java:43% of time Unsynchronized access at
MetricsRegionServerWrapperImpl.java:[line 306] |
| Failed junit tests | hadoop.hbase.regionserver.TestRowPrefixBloomFilter |
| | hadoop.hbase.io.TestHalfStoreFileReader |
| | hadoop.hbase.io.hfile.TestHFileInlineToRootChunkConversion |
| | hadoop.hbase.regionserver.TestHStoreFile |
| | hadoop.hbase.io.hfile.TestHFile |
| | hadoop.hbase.regionserver.TestBlocksScanned |
| | hadoop.hbase.regionserver.TestDataBlockEncodingTool |
| | hadoop.hbase.mob.TestMobFileCache |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21514 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12950147/HBASE-21514.master.001.patch
|
| Optional Tests | dupname asflicense javac
[jira] [Commented] (HBASE-21536) Fix completebulkload usage instructions
[
https://issues.apache.org/jira/browse/HBASE-21536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16705557#comment-16705557
]
Hadoop QA commented on HBASE-21536:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
58s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} refguide {color} | {color:blue} 5m
21s{color} | {color:blue} branch has no errors when building the reference
guide. See footer for rendered docs, which you should manually inspect. {color}
|
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
51s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source: . {color}
|
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
58s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 2m
59s{color} | {color:red} root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 3m
36s{color} | {color:red} root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 3m 36s{color}
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:blue}0{color} | {color:blue} refguide {color} | {color:blue} 4m
56s{color} | {color:blue} patch has no errors when building the reference
guide. See footer for rendered docs, which you should manually inspect. {color}
|
| {color:red}-1{color} | {color:red} shadedjars {color} | {color:red} 3m
9s{color} | {color:red} patch has 16 errors when building our shaded downstream
artifacts. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 1m
54s{color} | {color:red} The patch causes 16 errors with Hadoop v2.7.4. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 3m
49s{color} | {color:red} The patch causes 16 errors with Hadoop v3.0.0. {color}
|
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source: . {color}
|
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
57s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}253m 27s{color}
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
52s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{c
[jira] [Commented] (HBASE-21464) Splitting blocked with meta NSRE during split transaction
[
https://issues.apache.org/jira/browse/HBASE-21464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16705551#comment-16705551
]
Hadoop QA commented on HBASE-21464:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
34s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} branch-1 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
17s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
38s{color} | {color:green} branch-1 passed with JDK v1.8.0_192 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
18s{color} | {color:green} branch-1 passed with JDK v1.7.0_201 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
31s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
38s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} branch-1 passed with JDK v1.8.0_192 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
22s{color} | {color:green} branch-1 passed with JDK v1.7.0_201 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
15s{color} | {color:green} the patch passed with JDK v1.8.0_192 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
18s{color} | {color:green} the patch passed with JDK v1.7.0_201 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
29s{color} | {color:red} hbase-client: The patch generated 1 new + 66 unchanged
- 0 fixed = 67 total (was 66) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
35s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
1m 39s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} the patch passed with JDK v1.8.0_192 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
22s{color} | {color:green} the patch passed with JDK v1.7.0_201 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
28s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 32s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:61288f8 |
| JIRA Issue | HBASE-21464 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12950248/HBASE-21464-branch-1.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars had
[jira] [Issue Comment Deleted] (HBASE-21464) Splitting blocked with meta NSRE during split transaction
[
https://issues.apache.org/jira/browse/HBASE-21464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-21464:
---
Comment: was deleted
(was: | (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} branch-1 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
50s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
17s{color} | {color:green} branch-1 passed with JDK v1.8.0_192 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
18s{color} | {color:green} branch-1 passed with JDK v1.7.0_201 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
34s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
45s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
22s{color} | {color:green} branch-1 passed with JDK v1.8.0_192 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
20s{color} | {color:green} branch-1 passed with JDK v1.7.0_201 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
15s{color} | {color:green} the patch passed with JDK v1.8.0_192 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
17s{color} | {color:green} the patch passed with JDK v1.7.0_201 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
29s{color} | {color:red} hbase-client: The patch generated 1 new + 65 unchanged
- 1 fixed = 66 total (was 66) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
46s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
1m 43s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
16s{color} | {color:green} the patch passed with JDK v1.8.0_192 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
21s{color} | {color:green} the patch passed with JDK v1.7.0_201 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
30s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 42s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:61288f8 |
| JIRA Issue | HBASE-21464 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12950245/HBASE-21464-branch-1.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbasea
[jira] [Updated] (HBASE-21464) Splitting blocked with meta NSRE during split transaction
[
https://issues.apache.org/jira/browse/HBASE-21464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-21464:
---
Attachment: (was: HBASE-21464-branch-1.patch)
> Splitting blocked with meta NSRE during split transaction
> -
>
> Key: HBASE-21464
> URL: https://issues.apache.org/jira/browse/HBASE-21464
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.5.0, 1.4.3, 1.4.4, 1.4.5, 1.4.6, 1.4.8, 1.4.7
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Blocker
> Fix For: 1.5.0, 1.4.9
>
> Attachments: HBASE-21464-branch-1.patch, HBASE-21464-branch-1.patch
>
>
> Splitting is blocked during split transaction. The split worker is trying to
> update meta but isn't able to relocate it after NSRE:
> {noformat}
> 2018-11-09 17:50:45,277 INFO
> [regionserver/ip-172-31-5-92.us-west-2.compute.internal/172.31.5.92:8120-splits-1541785709434]
> client.RpcRetryingCaller: Call exception, tries=13, retries=350,
> started=88590 ms ago, cancelled=false,
> msg=org.apache.hadoop.hbase.NotServingRegionException: Region hbase:meta,,1
> is not online on ip-172-31-13-83.us-west-2.compute.internal,8120,1541785618832
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:3088)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.getRegion(RSRpcServices.java:1271)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:2198)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:36617)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2396)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:297)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:277)row
> 'test,,1541785709452.5ba6596f0050c2dab969d152829227c6.44' on table
> 'hbase:meta' at region=hbase:meta,1.1588230740,
> hostname=ip-172-31-15-225.us-west-2.compute.internal,8120,1541785640586,
> seqNum=0{noformat}
> Clients, in this case YCSB, are hung with part of the keyspace missing:
> {noformat}
> 2018-11-09 17:51:06,033 DEBUG [hconnection-0x5739e567-shared--pool1-t165]
> client.ConnectionManager$HConnectionImplementation: locateRegionInMeta
> parentTable=hbase:meta, metaLocation=, attempt=14 of 35 failed; retrying
> after sleep of 20158 because: No server address listed in hbase:meta for
> region
> test,user307326104267982763,1541785754600.ef90030b05cb02305b75e9bfbc3ee081.
> containing row user3301635648728421323{noformat}
> Balancing cannot run indefinitely because the split transaction is stuck
> {noformat}
> 2018-11-09 17:49:55,478 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=8100] master.HMaster:
> Not running balancer because 3 region(s) in transition:
> [{ef90030b05cb02305b75e9bfbc3ee081 state=SPLITTING_NEW, ts=1541785754606,
> server=ip-172-31-5-92.us-west-2.compute.internal,8120,1541785626417},
> {5ba6596f0050c2dab969d152829227c6 state=SPLITTING, ts=1541785754606,
> server=ip-172-31-5-92.us-west-2.compute{noformat}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21464) Splitting blocked with meta NSRE during split transaction
[
https://issues.apache.org/jira/browse/HBASE-21464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-21464:
---
Status: Open (was: Patch Available)
> Splitting blocked with meta NSRE during split transaction
> -
>
> Key: HBASE-21464
> URL: https://issues.apache.org/jira/browse/HBASE-21464
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.4.7, 1.4.8, 1.4.6, 1.4.5, 1.4.4, 1.4.3, 1.5.0
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Blocker
> Fix For: 1.5.0, 1.4.9
>
> Attachments: HBASE-21464-branch-1.patch, HBASE-21464-branch-1.patch
>
>
> Splitting is blocked during split transaction. The split worker is trying to
> update meta but isn't able to relocate it after NSRE:
> {noformat}
> 2018-11-09 17:50:45,277 INFO
> [regionserver/ip-172-31-5-92.us-west-2.compute.internal/172.31.5.92:8120-splits-1541785709434]
> client.RpcRetryingCaller: Call exception, tries=13, retries=350,
> started=88590 ms ago, cancelled=false,
> msg=org.apache.hadoop.hbase.NotServingRegionException: Region hbase:meta,,1
> is not online on ip-172-31-13-83.us-west-2.compute.internal,8120,1541785618832
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:3088)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.getRegion(RSRpcServices.java:1271)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:2198)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:36617)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2396)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:297)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:277)row
> 'test,,1541785709452.5ba6596f0050c2dab969d152829227c6.44' on table
> 'hbase:meta' at region=hbase:meta,1.1588230740,
> hostname=ip-172-31-15-225.us-west-2.compute.internal,8120,1541785640586,
> seqNum=0{noformat}
> Clients, in this case YCSB, are hung with part of the keyspace missing:
> {noformat}
> 2018-11-09 17:51:06,033 DEBUG [hconnection-0x5739e567-shared--pool1-t165]
> client.ConnectionManager$HConnectionImplementation: locateRegionInMeta
> parentTable=hbase:meta, metaLocation=, attempt=14 of 35 failed; retrying
> after sleep of 20158 because: No server address listed in hbase:meta for
> region
> test,user307326104267982763,1541785754600.ef90030b05cb02305b75e9bfbc3ee081.
> containing row user3301635648728421323{noformat}
> Balancing cannot run indefinitely because the split transaction is stuck
> {noformat}
> 2018-11-09 17:49:55,478 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=8100] master.HMaster:
> Not running balancer because 3 region(s) in transition:
> [{ef90030b05cb02305b75e9bfbc3ee081 state=SPLITTING_NEW, ts=1541785754606,
> server=ip-172-31-5-92.us-west-2.compute.internal,8120,1541785626417},
> {5ba6596f0050c2dab969d152829227c6 state=SPLITTING, ts=1541785754606,
> server=ip-172-31-5-92.us-west-2.compute{noformat}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21464) Splitting blocked with meta NSRE during split transaction
[
https://issues.apache.org/jira/browse/HBASE-21464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16705536#comment-16705536
]
Hadoop QA commented on HBASE-21464:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} branch-1 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
50s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
17s{color} | {color:green} branch-1 passed with JDK v1.8.0_192 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
18s{color} | {color:green} branch-1 passed with JDK v1.7.0_201 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
34s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
45s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
22s{color} | {color:green} branch-1 passed with JDK v1.8.0_192 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
20s{color} | {color:green} branch-1 passed with JDK v1.7.0_201 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
15s{color} | {color:green} the patch passed with JDK v1.8.0_192 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
17s{color} | {color:green} the patch passed with JDK v1.7.0_201 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
29s{color} | {color:red} hbase-client: The patch generated 1 new + 65 unchanged
- 1 fixed = 66 total (was 66) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
46s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
1m 43s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
16s{color} | {color:green} the patch passed with JDK v1.8.0_192 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
21s{color} | {color:green} the patch passed with JDK v1.7.0_201 {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
30s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 42s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:61288f8 |
| JIRA Issue | HBASE-21464 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12950245/HBASE-21464-branch-1.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars had
[jira] [Updated] (HBASE-21464) Splitting blocked with meta NSRE during split transaction
[
https://issues.apache.org/jira/browse/HBASE-21464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-21464:
---
Attachment: (was: HBASE-21464-branch-1.patch)
> Splitting blocked with meta NSRE during split transaction
> -
>
> Key: HBASE-21464
> URL: https://issues.apache.org/jira/browse/HBASE-21464
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.5.0, 1.4.3, 1.4.4, 1.4.5, 1.4.6, 1.4.8, 1.4.7
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Blocker
> Fix For: 1.5.0, 1.4.9
>
> Attachments: HBASE-21464-branch-1.patch, HBASE-21464-branch-1.patch
>
>
> Splitting is blocked during split transaction. The split worker is trying to
> update meta but isn't able to relocate it after NSRE:
> {noformat}
> 2018-11-09 17:50:45,277 INFO
> [regionserver/ip-172-31-5-92.us-west-2.compute.internal/172.31.5.92:8120-splits-1541785709434]
> client.RpcRetryingCaller: Call exception, tries=13, retries=350,
> started=88590 ms ago, cancelled=false,
> msg=org.apache.hadoop.hbase.NotServingRegionException: Region hbase:meta,,1
> is not online on ip-172-31-13-83.us-west-2.compute.internal,8120,1541785618832
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:3088)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.getRegion(RSRpcServices.java:1271)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:2198)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:36617)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2396)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:297)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:277)row
> 'test,,1541785709452.5ba6596f0050c2dab969d152829227c6.44' on table
> 'hbase:meta' at region=hbase:meta,1.1588230740,
> hostname=ip-172-31-15-225.us-west-2.compute.internal,8120,1541785640586,
> seqNum=0{noformat}
> Clients, in this case YCSB, are hung with part of the keyspace missing:
> {noformat}
> 2018-11-09 17:51:06,033 DEBUG [hconnection-0x5739e567-shared--pool1-t165]
> client.ConnectionManager$HConnectionImplementation: locateRegionInMeta
> parentTable=hbase:meta, metaLocation=, attempt=14 of 35 failed; retrying
> after sleep of 20158 because: No server address listed in hbase:meta for
> region
> test,user307326104267982763,1541785754600.ef90030b05cb02305b75e9bfbc3ee081.
> containing row user3301635648728421323{noformat}
> Balancing cannot run indefinitely because the split transaction is stuck
> {noformat}
> 2018-11-09 17:49:55,478 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=8100] master.HMaster:
> Not running balancer because 3 region(s) in transition:
> [{ef90030b05cb02305b75e9bfbc3ee081 state=SPLITTING_NEW, ts=1541785754606,
> server=ip-172-31-5-92.us-west-2.compute.internal,8120,1541785626417},
> {5ba6596f0050c2dab969d152829227c6 state=SPLITTING, ts=1541785754606,
> server=ip-172-31-5-92.us-west-2.compute{noformat}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21464) Splitting blocked with meta NSRE during split transaction
[
https://issues.apache.org/jira/browse/HBASE-21464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-21464:
---
Attachment: HBASE-21464-branch-1.patch
> Splitting blocked with meta NSRE during split transaction
> -
>
> Key: HBASE-21464
> URL: https://issues.apache.org/jira/browse/HBASE-21464
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.5.0, 1.4.3, 1.4.4, 1.4.5, 1.4.6, 1.4.8, 1.4.7
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Blocker
> Fix For: 1.5.0, 1.4.9
>
> Attachments: HBASE-21464-branch-1.patch, HBASE-21464-branch-1.patch,
> HBASE-21464-branch-1.patch
>
>
> Splitting is blocked during split transaction. The split worker is trying to
> update meta but isn't able to relocate it after NSRE:
> {noformat}
> 2018-11-09 17:50:45,277 INFO
> [regionserver/ip-172-31-5-92.us-west-2.compute.internal/172.31.5.92:8120-splits-1541785709434]
> client.RpcRetryingCaller: Call exception, tries=13, retries=350,
> started=88590 ms ago, cancelled=false,
> msg=org.apache.hadoop.hbase.NotServingRegionException: Region hbase:meta,,1
> is not online on ip-172-31-13-83.us-west-2.compute.internal,8120,1541785618832
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:3088)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.getRegion(RSRpcServices.java:1271)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:2198)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:36617)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2396)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:297)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:277)row
> 'test,,1541785709452.5ba6596f0050c2dab969d152829227c6.44' on table
> 'hbase:meta' at region=hbase:meta,1.1588230740,
> hostname=ip-172-31-15-225.us-west-2.compute.internal,8120,1541785640586,
> seqNum=0{noformat}
> Clients, in this case YCSB, are hung with part of the keyspace missing:
> {noformat}
> 2018-11-09 17:51:06,033 DEBUG [hconnection-0x5739e567-shared--pool1-t165]
> client.ConnectionManager$HConnectionImplementation: locateRegionInMeta
> parentTable=hbase:meta, metaLocation=, attempt=14 of 35 failed; retrying
> after sleep of 20158 because: No server address listed in hbase:meta for
> region
> test,user307326104267982763,1541785754600.ef90030b05cb02305b75e9bfbc3ee081.
> containing row user3301635648728421323{noformat}
> Balancing cannot run indefinitely because the split transaction is stuck
> {noformat}
> 2018-11-09 17:49:55,478 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=8100] master.HMaster:
> Not running balancer because 3 region(s) in transition:
> [{ef90030b05cb02305b75e9bfbc3ee081 state=SPLITTING_NEW, ts=1541785754606,
> server=ip-172-31-5-92.us-west-2.compute.internal,8120,1541785626417},
> {5ba6596f0050c2dab969d152829227c6 state=SPLITTING, ts=1541785754606,
> server=ip-172-31-5-92.us-west-2.compute{noformat}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21464) Splitting blocked with meta NSRE during split transaction
[
https://issues.apache.org/jira/browse/HBASE-21464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16705526#comment-16705526
]
Andrew Purtell commented on HBASE-21464:
A bit tricky to reproduce. A revert of HBASE-19260 doesn't help actually. So
let me attach the one change that seems to really work (so far)
> Splitting blocked with meta NSRE during split transaction
> -
>
> Key: HBASE-21464
> URL: https://issues.apache.org/jira/browse/HBASE-21464
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.5.0, 1.4.3, 1.4.4, 1.4.5, 1.4.6, 1.4.8, 1.4.7
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Blocker
> Fix For: 1.5.0, 1.4.9
>
> Attachments: HBASE-21464-branch-1.patch, HBASE-21464-branch-1.patch,
> HBASE-21464-branch-1.patch
>
>
> Splitting is blocked during split transaction. The split worker is trying to
> update meta but isn't able to relocate it after NSRE:
> {noformat}
> 2018-11-09 17:50:45,277 INFO
> [regionserver/ip-172-31-5-92.us-west-2.compute.internal/172.31.5.92:8120-splits-1541785709434]
> client.RpcRetryingCaller: Call exception, tries=13, retries=350,
> started=88590 ms ago, cancelled=false,
> msg=org.apache.hadoop.hbase.NotServingRegionException: Region hbase:meta,,1
> is not online on ip-172-31-13-83.us-west-2.compute.internal,8120,1541785618832
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:3088)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.getRegion(RSRpcServices.java:1271)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:2198)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:36617)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2396)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:297)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:277)row
> 'test,,1541785709452.5ba6596f0050c2dab969d152829227c6.44' on table
> 'hbase:meta' at region=hbase:meta,1.1588230740,
> hostname=ip-172-31-15-225.us-west-2.compute.internal,8120,1541785640586,
> seqNum=0{noformat}
> Clients, in this case YCSB, are hung with part of the keyspace missing:
> {noformat}
> 2018-11-09 17:51:06,033 DEBUG [hconnection-0x5739e567-shared--pool1-t165]
> client.ConnectionManager$HConnectionImplementation: locateRegionInMeta
> parentTable=hbase:meta, metaLocation=, attempt=14 of 35 failed; retrying
> after sleep of 20158 because: No server address listed in hbase:meta for
> region
> test,user307326104267982763,1541785754600.ef90030b05cb02305b75e9bfbc3ee081.
> containing row user3301635648728421323{noformat}
> Balancing cannot run indefinitely because the split transaction is stuck
> {noformat}
> 2018-11-09 17:49:55,478 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=8100] master.HMaster:
> Not running balancer because 3 region(s) in transition:
> [{ef90030b05cb02305b75e9bfbc3ee081 state=SPLITTING_NEW, ts=1541785754606,
> server=ip-172-31-5-92.us-west-2.compute.internal,8120,1541785626417},
> {5ba6596f0050c2dab969d152829227c6 state=SPLITTING, ts=1541785754606,
> server=ip-172-31-5-92.us-west-2.compute{noformat}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21464) Splitting blocked with meta NSRE during split transaction
[
https://issues.apache.org/jira/browse/HBASE-21464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-21464:
---
Attachment: (was: HBASE-21464-branch-1.patch)
> Splitting blocked with meta NSRE during split transaction
> -
>
> Key: HBASE-21464
> URL: https://issues.apache.org/jira/browse/HBASE-21464
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.5.0, 1.4.3, 1.4.4, 1.4.5, 1.4.6, 1.4.8, 1.4.7
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Blocker
> Fix For: 1.5.0, 1.4.9
>
> Attachments: HBASE-21464-branch-1.patch, HBASE-21464-branch-1.patch
>
>
> Splitting is blocked during split transaction. The split worker is trying to
> update meta but isn't able to relocate it after NSRE:
> {noformat}
> 2018-11-09 17:50:45,277 INFO
> [regionserver/ip-172-31-5-92.us-west-2.compute.internal/172.31.5.92:8120-splits-1541785709434]
> client.RpcRetryingCaller: Call exception, tries=13, retries=350,
> started=88590 ms ago, cancelled=false,
> msg=org.apache.hadoop.hbase.NotServingRegionException: Region hbase:meta,,1
> is not online on ip-172-31-13-83.us-west-2.compute.internal,8120,1541785618832
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:3088)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.getRegion(RSRpcServices.java:1271)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:2198)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:36617)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2396)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:297)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:277)row
> 'test,,1541785709452.5ba6596f0050c2dab969d152829227c6.44' on table
> 'hbase:meta' at region=hbase:meta,1.1588230740,
> hostname=ip-172-31-15-225.us-west-2.compute.internal,8120,1541785640586,
> seqNum=0{noformat}
> Clients, in this case YCSB, are hung with part of the keyspace missing:
> {noformat}
> 2018-11-09 17:51:06,033 DEBUG [hconnection-0x5739e567-shared--pool1-t165]
> client.ConnectionManager$HConnectionImplementation: locateRegionInMeta
> parentTable=hbase:meta, metaLocation=, attempt=14 of 35 failed; retrying
> after sleep of 20158 because: No server address listed in hbase:meta for
> region
> test,user307326104267982763,1541785754600.ef90030b05cb02305b75e9bfbc3ee081.
> containing row user3301635648728421323{noformat}
> Balancing cannot run indefinitely because the split transaction is stuck
> {noformat}
> 2018-11-09 17:49:55,478 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=8100] master.HMaster:
> Not running balancer because 3 region(s) in transition:
> [{ef90030b05cb02305b75e9bfbc3ee081 state=SPLITTING_NEW, ts=1541785754606,
> server=ip-172-31-5-92.us-west-2.compute.internal,8120,1541785626417},
> {5ba6596f0050c2dab969d152829227c6 state=SPLITTING, ts=1541785754606,
> server=ip-172-31-5-92.us-west-2.compute{noformat}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21464) Splitting blocked with meta NSRE during split transaction
[
https://issues.apache.org/jira/browse/HBASE-21464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-21464:
---
Status: Patch Available (was: Open)
> Splitting blocked with meta NSRE during split transaction
> -
>
> Key: HBASE-21464
> URL: https://issues.apache.org/jira/browse/HBASE-21464
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.4.7, 1.4.8, 1.4.6, 1.4.5, 1.4.4, 1.4.3, 1.5.0
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Blocker
> Fix For: 1.5.0, 1.4.9
>
> Attachments: HBASE-21464-branch-1.patch, HBASE-21464-branch-1.patch,
> HBASE-21464-branch-1.patch
>
>
> Splitting is blocked during split transaction. The split worker is trying to
> update meta but isn't able to relocate it after NSRE:
> {noformat}
> 2018-11-09 17:50:45,277 INFO
> [regionserver/ip-172-31-5-92.us-west-2.compute.internal/172.31.5.92:8120-splits-1541785709434]
> client.RpcRetryingCaller: Call exception, tries=13, retries=350,
> started=88590 ms ago, cancelled=false,
> msg=org.apache.hadoop.hbase.NotServingRegionException: Region hbase:meta,,1
> is not online on ip-172-31-13-83.us-west-2.compute.internal,8120,1541785618832
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:3088)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.getRegion(RSRpcServices.java:1271)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:2198)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:36617)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2396)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:297)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:277)row
> 'test,,1541785709452.5ba6596f0050c2dab969d152829227c6.44' on table
> 'hbase:meta' at region=hbase:meta,1.1588230740,
> hostname=ip-172-31-15-225.us-west-2.compute.internal,8120,1541785640586,
> seqNum=0{noformat}
> Clients, in this case YCSB, are hung with part of the keyspace missing:
> {noformat}
> 2018-11-09 17:51:06,033 DEBUG [hconnection-0x5739e567-shared--pool1-t165]
> client.ConnectionManager$HConnectionImplementation: locateRegionInMeta
> parentTable=hbase:meta, metaLocation=, attempt=14 of 35 failed; retrying
> after sleep of 20158 because: No server address listed in hbase:meta for
> region
> test,user307326104267982763,1541785754600.ef90030b05cb02305b75e9bfbc3ee081.
> containing row user3301635648728421323{noformat}
> Balancing cannot run indefinitely because the split transaction is stuck
> {noformat}
> 2018-11-09 17:49:55,478 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=8100] master.HMaster:
> Not running balancer because 3 region(s) in transition:
> [{ef90030b05cb02305b75e9bfbc3ee081 state=SPLITTING_NEW, ts=1541785754606,
> server=ip-172-31-5-92.us-west-2.compute.internal,8120,1541785626417},
> {5ba6596f0050c2dab969d152829227c6 state=SPLITTING, ts=1541785754606,
> server=ip-172-31-5-92.us-west-2.compute{noformat}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21464) Splitting blocked with meta NSRE during split transaction
[
https://issues.apache.org/jira/browse/HBASE-21464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-21464:
---
Attachment: HBASE-21464-branch-1.patch
> Splitting blocked with meta NSRE during split transaction
> -
>
> Key: HBASE-21464
> URL: https://issues.apache.org/jira/browse/HBASE-21464
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.5.0, 1.4.3, 1.4.4, 1.4.5, 1.4.6, 1.4.8, 1.4.7
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Blocker
> Fix For: 1.5.0, 1.4.9
>
> Attachments: HBASE-21464-branch-1.patch, HBASE-21464-branch-1.patch,
> HBASE-21464-branch-1.patch
>
>
> Splitting is blocked during split transaction. The split worker is trying to
> update meta but isn't able to relocate it after NSRE:
> {noformat}
> 2018-11-09 17:50:45,277 INFO
> [regionserver/ip-172-31-5-92.us-west-2.compute.internal/172.31.5.92:8120-splits-1541785709434]
> client.RpcRetryingCaller: Call exception, tries=13, retries=350,
> started=88590 ms ago, cancelled=false,
> msg=org.apache.hadoop.hbase.NotServingRegionException: Region hbase:meta,,1
> is not online on ip-172-31-13-83.us-west-2.compute.internal,8120,1541785618832
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:3088)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.getRegion(RSRpcServices.java:1271)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:2198)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:36617)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2396)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:297)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:277)row
> 'test,,1541785709452.5ba6596f0050c2dab969d152829227c6.44' on table
> 'hbase:meta' at region=hbase:meta,1.1588230740,
> hostname=ip-172-31-15-225.us-west-2.compute.internal,8120,1541785640586,
> seqNum=0{noformat}
> Clients, in this case YCSB, are hung with part of the keyspace missing:
> {noformat}
> 2018-11-09 17:51:06,033 DEBUG [hconnection-0x5739e567-shared--pool1-t165]
> client.ConnectionManager$HConnectionImplementation: locateRegionInMeta
> parentTable=hbase:meta, metaLocation=, attempt=14 of 35 failed; retrying
> after sleep of 20158 because: No server address listed in hbase:meta for
> region
> test,user307326104267982763,1541785754600.ef90030b05cb02305b75e9bfbc3ee081.
> containing row user3301635648728421323{noformat}
> Balancing cannot run indefinitely because the split transaction is stuck
> {noformat}
> 2018-11-09 17:49:55,478 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=8100] master.HMaster:
> Not running balancer because 3 region(s) in transition:
> [{ef90030b05cb02305b75e9bfbc3ee081 state=SPLITTING_NEW, ts=1541785754606,
> server=ip-172-31-5-92.us-west-2.compute.internal,8120,1541785626417},
> {5ba6596f0050c2dab969d152829227c6 state=SPLITTING, ts=1541785754606,
> server=ip-172-31-5-92.us-west-2.compute{noformat}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21464) Splitting blocked with meta NSRE during split transaction
[
https://issues.apache.org/jira/browse/HBASE-21464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-21464:
---
Status: Open (was: Patch Available)
> Splitting blocked with meta NSRE during split transaction
> -
>
> Key: HBASE-21464
> URL: https://issues.apache.org/jira/browse/HBASE-21464
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.4.7, 1.4.8, 1.4.6, 1.4.5, 1.4.4, 1.4.3, 1.5.0
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Blocker
> Fix For: 1.5.0, 1.4.9
>
> Attachments: HBASE-21464-branch-1.patch, HBASE-21464-branch-1.patch
>
>
> Splitting is blocked during split transaction. The split worker is trying to
> update meta but isn't able to relocate it after NSRE:
> {noformat}
> 2018-11-09 17:50:45,277 INFO
> [regionserver/ip-172-31-5-92.us-west-2.compute.internal/172.31.5.92:8120-splits-1541785709434]
> client.RpcRetryingCaller: Call exception, tries=13, retries=350,
> started=88590 ms ago, cancelled=false,
> msg=org.apache.hadoop.hbase.NotServingRegionException: Region hbase:meta,,1
> is not online on ip-172-31-13-83.us-west-2.compute.internal,8120,1541785618832
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:3088)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.getRegion(RSRpcServices.java:1271)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:2198)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:36617)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2396)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:297)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:277)row
> 'test,,1541785709452.5ba6596f0050c2dab969d152829227c6.44' on table
> 'hbase:meta' at region=hbase:meta,1.1588230740,
> hostname=ip-172-31-15-225.us-west-2.compute.internal,8120,1541785640586,
> seqNum=0{noformat}
> Clients, in this case YCSB, are hung with part of the keyspace missing:
> {noformat}
> 2018-11-09 17:51:06,033 DEBUG [hconnection-0x5739e567-shared--pool1-t165]
> client.ConnectionManager$HConnectionImplementation: locateRegionInMeta
> parentTable=hbase:meta, metaLocation=, attempt=14 of 35 failed; retrying
> after sleep of 20158 because: No server address listed in hbase:meta for
> region
> test,user307326104267982763,1541785754600.ef90030b05cb02305b75e9bfbc3ee081.
> containing row user3301635648728421323{noformat}
> Balancing cannot run indefinitely because the split transaction is stuck
> {noformat}
> 2018-11-09 17:49:55,478 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=8100] master.HMaster:
> Not running balancer because 3 region(s) in transition:
> [{ef90030b05cb02305b75e9bfbc3ee081 state=SPLITTING_NEW, ts=1541785754606,
> server=ip-172-31-5-92.us-west-2.compute.internal,8120,1541785626417},
> {5ba6596f0050c2dab969d152829227c6 state=SPLITTING, ts=1541785754606,
> server=ip-172-31-5-92.us-west-2.compute{noformat}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21464) Splitting blocked with meta NSRE during split transaction
[
https://issues.apache.org/jira/browse/HBASE-21464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16705513#comment-16705513
]
Andrew Purtell commented on HBASE-21464:
I attempted a revert the HBASE-19239 change to RegionLocations to see if this
broke cache lookups. It did not.
I have a patch that works but it goes to zookeeper every time, retaining the
HBASE-19260 lock that serializes lookups to zookeeper and does limited caching
so any waiters will return the result of the first thread through the lock.
A revert of HBASE-19260 also fixes this problem.
Of these options I prefer a revert of HBASE-19260. Attaching a new patch that
does this.
> Splitting blocked with meta NSRE during split transaction
> -
>
> Key: HBASE-21464
> URL: https://issues.apache.org/jira/browse/HBASE-21464
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.5.0, 1.4.3, 1.4.4, 1.4.5, 1.4.6, 1.4.8, 1.4.7
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Blocker
> Fix For: 1.5.0, 1.4.9
>
> Attachments: HBASE-21464-branch-1.patch, HBASE-21464-branch-1.patch,
> HBASE-21464-branch-1.patch
>
>
> Splitting is blocked during split transaction. The split worker is trying to
> update meta but isn't able to relocate it after NSRE:
> {noformat}
> 2018-11-09 17:50:45,277 INFO
> [regionserver/ip-172-31-5-92.us-west-2.compute.internal/172.31.5.92:8120-splits-1541785709434]
> client.RpcRetryingCaller: Call exception, tries=13, retries=350,
> started=88590 ms ago, cancelled=false,
> msg=org.apache.hadoop.hbase.NotServingRegionException: Region hbase:meta,,1
> is not online on ip-172-31-13-83.us-west-2.compute.internal,8120,1541785618832
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:3088)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.getRegion(RSRpcServices.java:1271)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:2198)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:36617)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2396)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:297)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:277)row
> 'test,,1541785709452.5ba6596f0050c2dab969d152829227c6.44' on table
> 'hbase:meta' at region=hbase:meta,1.1588230740,
> hostname=ip-172-31-15-225.us-west-2.compute.internal,8120,1541785640586,
> seqNum=0{noformat}
> Clients, in this case YCSB, are hung with part of the keyspace missing:
> {noformat}
> 2018-11-09 17:51:06,033 DEBUG [hconnection-0x5739e567-shared--pool1-t165]
> client.ConnectionManager$HConnectionImplementation: locateRegionInMeta
> parentTable=hbase:meta, metaLocation=, attempt=14 of 35 failed; retrying
> after sleep of 20158 because: No server address listed in hbase:meta for
> region
> test,user307326104267982763,1541785754600.ef90030b05cb02305b75e9bfbc3ee081.
> containing row user3301635648728421323{noformat}
> Balancing cannot run indefinitely because the split transaction is stuck
> {noformat}
> 2018-11-09 17:49:55,478 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=8100] master.HMaster:
> Not running balancer because 3 region(s) in transition:
> [{ef90030b05cb02305b75e9bfbc3ee081 state=SPLITTING_NEW, ts=1541785754606,
> server=ip-172-31-5-92.us-west-2.compute.internal,8120,1541785626417},
> {5ba6596f0050c2dab969d152829227c6 state=SPLITTING, ts=1541785754606,
> server=ip-172-31-5-92.us-west-2.compute{noformat}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21464) Splitting blocked with meta NSRE during split transaction
[
https://issues.apache.org/jira/browse/HBASE-21464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-21464:
---
Attachment: HBASE-21464-branch-1.patch
> Splitting blocked with meta NSRE during split transaction
> -
>
> Key: HBASE-21464
> URL: https://issues.apache.org/jira/browse/HBASE-21464
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.5.0, 1.4.3, 1.4.4, 1.4.5, 1.4.6, 1.4.8, 1.4.7
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Blocker
> Fix For: 1.5.0, 1.4.9
>
> Attachments: HBASE-21464-branch-1.patch, HBASE-21464-branch-1.patch,
> HBASE-21464-branch-1.patch
>
>
> Splitting is blocked during split transaction. The split worker is trying to
> update meta but isn't able to relocate it after NSRE:
> {noformat}
> 2018-11-09 17:50:45,277 INFO
> [regionserver/ip-172-31-5-92.us-west-2.compute.internal/172.31.5.92:8120-splits-1541785709434]
> client.RpcRetryingCaller: Call exception, tries=13, retries=350,
> started=88590 ms ago, cancelled=false,
> msg=org.apache.hadoop.hbase.NotServingRegionException: Region hbase:meta,,1
> is not online on ip-172-31-13-83.us-west-2.compute.internal,8120,1541785618832
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:3088)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.getRegion(RSRpcServices.java:1271)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:2198)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:36617)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2396)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:297)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:277)row
> 'test,,1541785709452.5ba6596f0050c2dab969d152829227c6.44' on table
> 'hbase:meta' at region=hbase:meta,1.1588230740,
> hostname=ip-172-31-15-225.us-west-2.compute.internal,8120,1541785640586,
> seqNum=0{noformat}
> Clients, in this case YCSB, are hung with part of the keyspace missing:
> {noformat}
> 2018-11-09 17:51:06,033 DEBUG [hconnection-0x5739e567-shared--pool1-t165]
> client.ConnectionManager$HConnectionImplementation: locateRegionInMeta
> parentTable=hbase:meta, metaLocation=, attempt=14 of 35 failed; retrying
> after sleep of 20158 because: No server address listed in hbase:meta for
> region
> test,user307326104267982763,1541785754600.ef90030b05cb02305b75e9bfbc3ee081.
> containing row user3301635648728421323{noformat}
> Balancing cannot run indefinitely because the split transaction is stuck
> {noformat}
> 2018-11-09 17:49:55,478 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=8100] master.HMaster:
> Not running balancer because 3 region(s) in transition:
> [{ef90030b05cb02305b75e9bfbc3ee081 state=SPLITTING_NEW, ts=1541785754606,
> server=ip-172-31-5-92.us-west-2.compute.internal,8120,1541785626417},
> {5ba6596f0050c2dab969d152829227c6 state=SPLITTING, ts=1541785754606,
> server=ip-172-31-5-92.us-west-2.compute{noformat}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21464) Splitting blocked with meta NSRE during split transaction
[
https://issues.apache.org/jira/browse/HBASE-21464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-21464:
---
Status: Patch Available (was: Reopened)
> Splitting blocked with meta NSRE during split transaction
> -
>
> Key: HBASE-21464
> URL: https://issues.apache.org/jira/browse/HBASE-21464
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.4.7, 1.4.8, 1.4.6, 1.4.5, 1.4.4, 1.4.3, 1.5.0
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Blocker
> Fix For: 1.5.0, 1.4.9
>
> Attachments: HBASE-21464-branch-1.patch, HBASE-21464-branch-1.patch,
> HBASE-21464-branch-1.patch
>
>
> Splitting is blocked during split transaction. The split worker is trying to
> update meta but isn't able to relocate it after NSRE:
> {noformat}
> 2018-11-09 17:50:45,277 INFO
> [regionserver/ip-172-31-5-92.us-west-2.compute.internal/172.31.5.92:8120-splits-1541785709434]
> client.RpcRetryingCaller: Call exception, tries=13, retries=350,
> started=88590 ms ago, cancelled=false,
> msg=org.apache.hadoop.hbase.NotServingRegionException: Region hbase:meta,,1
> is not online on ip-172-31-13-83.us-west-2.compute.internal,8120,1541785618832
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:3088)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.getRegion(RSRpcServices.java:1271)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:2198)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:36617)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2396)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:297)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:277)row
> 'test,,1541785709452.5ba6596f0050c2dab969d152829227c6.44' on table
> 'hbase:meta' at region=hbase:meta,1.1588230740,
> hostname=ip-172-31-15-225.us-west-2.compute.internal,8120,1541785640586,
> seqNum=0{noformat}
> Clients, in this case YCSB, are hung with part of the keyspace missing:
> {noformat}
> 2018-11-09 17:51:06,033 DEBUG [hconnection-0x5739e567-shared--pool1-t165]
> client.ConnectionManager$HConnectionImplementation: locateRegionInMeta
> parentTable=hbase:meta, metaLocation=, attempt=14 of 35 failed; retrying
> after sleep of 20158 because: No server address listed in hbase:meta for
> region
> test,user307326104267982763,1541785754600.ef90030b05cb02305b75e9bfbc3ee081.
> containing row user3301635648728421323{noformat}
> Balancing cannot run indefinitely because the split transaction is stuck
> {noformat}
> 2018-11-09 17:49:55,478 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=8100] master.HMaster:
> Not running balancer because 3 region(s) in transition:
> [{ef90030b05cb02305b75e9bfbc3ee081 state=SPLITTING_NEW, ts=1541785754606,
> server=ip-172-31-5-92.us-west-2.compute.internal,8120,1541785626417},
> {5ba6596f0050c2dab969d152829227c6 state=SPLITTING, ts=1541785754606,
> server=ip-172-31-5-92.us-west-2.compute{noformat}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21514) Refactor CacheConfig
[ https://issues.apache.org/jira/browse/HBASE-21514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-21514: --- Status: Patch Available (was: Open) > Refactor CacheConfig > > > Key: HBASE-21514 > URL: https://issues.apache.org/jira/browse/HBASE-21514 > Project: HBase > Issue Type: Improvement >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-21514.master.001.patch > > > # move the global cache instances from CacheConfig. Only keep config stuff in > CacheConfig. > # Move block cache to HRegionServer's member variable. One rs has one block > cache. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21476) Support for nanosecond timestamps
[ https://issues.apache.org/jira/browse/HBASE-21476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Elenskiy updated HBASE-21476: Attachment: nanosecond_timestamps_v2.patch > Support for nanosecond timestamps > - > > Key: HBASE-21476 > URL: https://issues.apache.org/jira/browse/HBASE-21476 > Project: HBase > Issue Type: New Feature >Affects Versions: 2.1.1 >Reporter: Andrey Elenskiy >Assignee: Andrey Elenskiy >Priority: Major > Labels: features, patch > Attachments: Apache HBase - Nanosecond Timestamps v1.pdf, > nanosecond_timestamps_v1.patch, nanosecond_timestamps_v2.patch > > > Introducing a new table attribute "NANOSECOND_TIMESTAMPS" to tell HBase to > handle timestamps with nanosecond precision. This is useful for applications > that timestamp updates at the source with nanoseconds and still want features > like column family TTL and "hbase.hstore.time.to.purge.deletes" to work. > The attribute should be specified either on new tables or on existing tables > which have timestamps only with nanosecond precision. There's no migration > from milliseconds to nanoseconds for already existing tables. We could add > this migration as part of compaction if you think that would be useful, but > that would obviously make the change more complex. > I've added a new EnvironmentEdge method "currentTimeNano()" that uses > [java.time.Instant|https://docs.oracle.com/javase/8/docs/api/java/time/Instant.html] > to get time in nanoseconds which means it will only work with Java 8. The > idea is to gradually replace all places where "EnvironmentEdge.currentTime()" > is used to have HBase working purely with nanoseconds (which is a > prerequisite for HBASE-14070). Also, I've refactored ScanInfo and > PartitionedMobCompactor to expect TableDescriptor as an argument which makes > code a little cleaner and easier to extend. > Couple more points: > - column family TTL (specified in seconds) and > "hbase.hstore.time.to.purge.deletes" (specified in milliseconds) options > don't need to be changed, those are adjusted automatically. > - Per cell TTL needs to be scaled by clients accordingly after > "NANOSECOND_TIMESTAMPS" table attribute is specified. > Looking for everyone's feedback to know if that's a worthwhile direction. > Will add more comprehensive tests in a later patch. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21518) TestMasterFailoverWithProcedures is flaky
[
https://issues.apache.org/jira/browse/HBASE-21518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16705466#comment-16705466
]
Hudson commented on HBASE-21518:
Results for branch branch-2
[build #1533 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1533/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1533//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1533//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1533//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> TestMasterFailoverWithProcedures is flaky
> -
>
> Key: HBASE-21518
> URL: https://issues.apache.org/jira/browse/HBASE-21518
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.2.0, 2.0.3, 2.1.2
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.2
>
> Attachments: HBASE-21518-v1.patch, output.txt
>
>
> TestMasterFailoverWithProcedures test is failing frequently, times out. I
> faced this failure on 2.0.3RC0 vote and it also appears on multiple flaky
> dashboards.
> branch-2:
> [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Flaky-Tests/job/branch-2/2007/]
> branch-2.1:
> [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Flaky-Tests/job/branch-2.1/2002/]
> branch-2.0:
> [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Flaky-Tests/job/branch-2.0/1988/]
>
> {noformat}
> [INFO] Running
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures
> [ERROR] Tests run: 4, Failures: 0, Errors: 2, Skipped: 0, Time elapsed:
> 780.648 s <<< FAILURE! - in
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures
> [ERROR]
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures
> Time elapsed: 749.024 s <<< ERROR!
> org.junit.runners.model.TestTimedOutException: test timed out after 780
> seconds
> at
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures.tearDown(TestMasterFailoverWithProcedures.java:86)
> [ERROR]
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures
> Time elapsed: 749.051 s <<< ERROR!
> java.lang.Exception: Appears to be stuck in thread RS-EventLoopGroup-3-2
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21464) Splitting blocked with meta NSRE during split transaction
[
https://issues.apache.org/jira/browse/HBASE-21464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16705378#comment-16705378
]
Andrew Purtell commented on HBASE-21464:
As a fallback I can remove the use of any cache for the location of the meta
region. We'd go to zookeeper every time we need to find it, so would be in
effect a revert of HBASE-10785. Will post a patch that does this if only to try
it out.
> Splitting blocked with meta NSRE during split transaction
> -
>
> Key: HBASE-21464
> URL: https://issues.apache.org/jira/browse/HBASE-21464
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.5.0, 1.4.3, 1.4.4, 1.4.5, 1.4.6, 1.4.8, 1.4.7
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Blocker
> Fix For: 1.5.0, 1.4.9
>
> Attachments: HBASE-21464-branch-1.patch, HBASE-21464-branch-1.patch
>
>
> Splitting is blocked during split transaction. The split worker is trying to
> update meta but isn't able to relocate it after NSRE:
> {noformat}
> 2018-11-09 17:50:45,277 INFO
> [regionserver/ip-172-31-5-92.us-west-2.compute.internal/172.31.5.92:8120-splits-1541785709434]
> client.RpcRetryingCaller: Call exception, tries=13, retries=350,
> started=88590 ms ago, cancelled=false,
> msg=org.apache.hadoop.hbase.NotServingRegionException: Region hbase:meta,,1
> is not online on ip-172-31-13-83.us-west-2.compute.internal,8120,1541785618832
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:3088)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.getRegion(RSRpcServices.java:1271)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:2198)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:36617)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2396)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:297)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:277)row
> 'test,,1541785709452.5ba6596f0050c2dab969d152829227c6.44' on table
> 'hbase:meta' at region=hbase:meta,1.1588230740,
> hostname=ip-172-31-15-225.us-west-2.compute.internal,8120,1541785640586,
> seqNum=0{noformat}
> Clients, in this case YCSB, are hung with part of the keyspace missing:
> {noformat}
> 2018-11-09 17:51:06,033 DEBUG [hconnection-0x5739e567-shared--pool1-t165]
> client.ConnectionManager$HConnectionImplementation: locateRegionInMeta
> parentTable=hbase:meta, metaLocation=, attempt=14 of 35 failed; retrying
> after sleep of 20158 because: No server address listed in hbase:meta for
> region
> test,user307326104267982763,1541785754600.ef90030b05cb02305b75e9bfbc3ee081.
> containing row user3301635648728421323{noformat}
> Balancing cannot run indefinitely because the split transaction is stuck
> {noformat}
> 2018-11-09 17:49:55,478 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=8100] master.HMaster:
> Not running balancer because 3 region(s) in transition:
> [{ef90030b05cb02305b75e9bfbc3ee081 state=SPLITTING_NEW, ts=1541785754606,
> server=ip-172-31-5-92.us-west-2.compute.internal,8120,1541785626417},
> {5ba6596f0050c2dab969d152829227c6 state=SPLITTING, ts=1541785754606,
> server=ip-172-31-5-92.us-west-2.compute{noformat}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Reopened] (HBASE-21464) Splitting blocked with meta NSRE during split transaction
[
https://issues.apache.org/jira/browse/HBASE-21464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell reopened HBASE-21464:
I reverted my commit because I was able to reproduce the problem again.
That part of the change we removed where the cache lookup for meta region was
modified was what was necessary. There is something wrong with the cache
lookup. I don't see a change between 1.4.2 and 1.4.3 that is any kind of
smoking gun so am a bit stumped here.
> Splitting blocked with meta NSRE during split transaction
> -
>
> Key: HBASE-21464
> URL: https://issues.apache.org/jira/browse/HBASE-21464
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.5.0, 1.4.3, 1.4.4, 1.4.5, 1.4.6, 1.4.8, 1.4.7
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Blocker
> Fix For: 1.5.0, 1.4.9
>
> Attachments: HBASE-21464-branch-1.patch, HBASE-21464-branch-1.patch
>
>
> Splitting is blocked during split transaction. The split worker is trying to
> update meta but isn't able to relocate it after NSRE:
> {noformat}
> 2018-11-09 17:50:45,277 INFO
> [regionserver/ip-172-31-5-92.us-west-2.compute.internal/172.31.5.92:8120-splits-1541785709434]
> client.RpcRetryingCaller: Call exception, tries=13, retries=350,
> started=88590 ms ago, cancelled=false,
> msg=org.apache.hadoop.hbase.NotServingRegionException: Region hbase:meta,,1
> is not online on ip-172-31-13-83.us-west-2.compute.internal,8120,1541785618832
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:3088)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.getRegion(RSRpcServices.java:1271)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:2198)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:36617)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2396)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:297)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:277)row
> 'test,,1541785709452.5ba6596f0050c2dab969d152829227c6.44' on table
> 'hbase:meta' at region=hbase:meta,1.1588230740,
> hostname=ip-172-31-15-225.us-west-2.compute.internal,8120,1541785640586,
> seqNum=0{noformat}
> Clients, in this case YCSB, are hung with part of the keyspace missing:
> {noformat}
> 2018-11-09 17:51:06,033 DEBUG [hconnection-0x5739e567-shared--pool1-t165]
> client.ConnectionManager$HConnectionImplementation: locateRegionInMeta
> parentTable=hbase:meta, metaLocation=, attempt=14 of 35 failed; retrying
> after sleep of 20158 because: No server address listed in hbase:meta for
> region
> test,user307326104267982763,1541785754600.ef90030b05cb02305b75e9bfbc3ee081.
> containing row user3301635648728421323{noformat}
> Balancing cannot run indefinitely because the split transaction is stuck
> {noformat}
> 2018-11-09 17:49:55,478 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=8100] master.HMaster:
> Not running balancer because 3 region(s) in transition:
> [{ef90030b05cb02305b75e9bfbc3ee081 state=SPLITTING_NEW, ts=1541785754606,
> server=ip-172-31-5-92.us-west-2.compute.internal,8120,1541785626417},
> {5ba6596f0050c2dab969d152829227c6 state=SPLITTING, ts=1541785754606,
> server=ip-172-31-5-92.us-west-2.compute{noformat}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20694) Consolidate warning on SecureBulkLoad directory permissions
[ https://issues.apache.org/jira/browse/HBASE-20694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-20694: --- Labels: beginner (was: ) > Consolidate warning on SecureBulkLoad directory permissions > --- > > Key: HBASE-20694 > URL: https://issues.apache.org/jira/browse/HBASE-20694 > Project: HBase > Issue Type: Improvement > Components: documentation >Reporter: Josh Elser >Priority: Minor > Labels: beginner > Fix For: 3.0.0, 1.5.0, 1.3.3, 2.2.0, 1.2.10, 1.4.10 > > > Follow-on from HBASE-20605: > HBase 1.x has a check which ignores a directory permission check if you're > using a specific filesystem which we think doesnt' do security properly. > HBase 2.x dropped this check. > Since the security of bulk-loaded data is dependent upon this directory > permission (and thus the capabilities of the FileSystem), it would be better > to have a consistent warning across branches. > [~busbey] suggested that we make a WARN message which points admins to our > Book (and write such a section if we don't have something sufficient already) > and our supported filesystems, coupled with an option to disable that warning > message. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HBASE-20694) Consolidate warning on SecureBulkLoad directory permissions
[ https://issues.apache.org/jira/browse/HBASE-20694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser reassigned HBASE-20694: -- Assignee: (was: Josh Elser) > Consolidate warning on SecureBulkLoad directory permissions > --- > > Key: HBASE-20694 > URL: https://issues.apache.org/jira/browse/HBASE-20694 > Project: HBase > Issue Type: Improvement > Components: documentation >Reporter: Josh Elser >Priority: Minor > Labels: beginner > Fix For: 3.0.0, 1.5.0, 1.3.3, 2.2.0, 1.2.10, 1.4.10 > > > Follow-on from HBASE-20605: > HBase 1.x has a check which ignores a directory permission check if you're > using a specific filesystem which we think doesnt' do security properly. > HBase 2.x dropped this check. > Since the security of bulk-loaded data is dependent upon this directory > permission (and thus the capabilities of the FileSystem), it would be better > to have a consistent warning across branches. > [~busbey] suggested that we make a WARN message which points admins to our > Book (and write such a section if we don't have something sufficient already) > and our supported filesystems, coupled with an option to disable that warning > message. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20694) Consolidate warning on SecureBulkLoad directory permissions
[ https://issues.apache.org/jira/browse/HBASE-20694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16705288#comment-16705288 ] Josh Elser commented on HBASE-20694: Thanks, Sean. I like the way you think. > Consolidate warning on SecureBulkLoad directory permissions > --- > > Key: HBASE-20694 > URL: https://issues.apache.org/jira/browse/HBASE-20694 > Project: HBase > Issue Type: Improvement > Components: documentation >Reporter: Josh Elser >Priority: Minor > Labels: beginner > Fix For: 3.0.0, 1.5.0, 1.3.3, 2.2.0, 1.2.10, 1.4.10 > > > Follow-on from HBASE-20605: > HBase 1.x has a check which ignores a directory permission check if you're > using a specific filesystem which we think doesnt' do security properly. > HBase 2.x dropped this check. > Since the security of bulk-loaded data is dependent upon this directory > permission (and thus the capabilities of the FileSystem), it would be better > to have a consistent warning across branches. > [~busbey] suggested that we make a WARN message which points admins to our > Book (and write such a section if we don't have something sufficient already) > and our supported filesystems, coupled with an option to disable that warning > message. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21536) Fix completebulkload usage instructions
[
https://issues.apache.org/jira/browse/HBASE-21536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Artem Ervits updated HBASE-21536:
-
Description:
Usage information upon invoking LoadIncrementalHFiles is misleading and
error-prone.
{code:java}
usage: completebulkload /path/to/hfileoutputformat-output tablename -loadTable
-Dcreate.table=no - can be used to avoid creation of table by this tool
Note: if you set this to 'no', then the target table must already exist in HBase
-loadTable implies your baseDirectory to store file has a depth of 3 ,you must
have an existing table
-Dignore.unmatched.families=yes - can be used to ignore unmatched column
families{code}
in case of invoking the class via hbase command, completebulkload argument is
unnecessary and only required via hadoop jar invocation. This is also an
attempt to clarify where <-loadTable> and <-Dargs> arguments go on the command
line.
Furthermore, since LoadIncrementalHFiles was recently moved out of
hbase-server.jar to hbase-mapreduce, updating ref guide to demonstrate as such.
was:
Usage information upon invoking LoadIncrementalHFiles is misleading and
error-prone.
{code:java}
usage: completebulkload /path/to/hfileoutputformat-output tablename -loadTable
-Dcreate.table=no - can be used to avoid creation of table by this tool
Note: if you set this to 'no', then the target table must already exist in HBase
-loadTable implies your baseDirectory to store file has a depth of 3 ,you must
have an existing table
-Dignore.unmatched.families=yes - can be used to ignore unmatched column
families{code}
in case of invoking the class via hbase command, completebulkload argument is
unnecessary and only required via hadoop jar invocation. This is also an
attempt to clarify where <-loadTable> and <-Dargs> arguments go on the command
line.
> Fix completebulkload usage instructions
> ---
>
> Key: HBASE-21536
> URL: https://issues.apache.org/jira/browse/HBASE-21536
> Project: HBase
> Issue Type: Task
> Components: documentation, mapreduce
>Reporter: Artem Ervits
>Assignee: Artem Ervits
>Priority: Trivial
> Attachments: HBASE-21536.v01.patch
>
>
> Usage information upon invoking LoadIncrementalHFiles is misleading and
> error-prone.
> {code:java}
> usage: completebulkload /path/to/hfileoutputformat-output tablename -loadTable
> -Dcreate.table=no - can be used to avoid creation of table by this tool
> Note: if you set this to 'no', then the target table must already exist in
> HBase
> -loadTable implies your baseDirectory to store file has a depth of 3 ,you
> must have an existing table
> -Dignore.unmatched.families=yes - can be used to ignore unmatched column
> families{code}
> in case of invoking the class via hbase command, completebulkload argument is
> unnecessary and only required via hadoop jar invocation. This is also an
> attempt to clarify where <-loadTable> and <-Dargs> arguments go on the
> command line.
> Furthermore, since LoadIncrementalHFiles was recently moved out of
> hbase-server.jar to hbase-mapreduce, updating ref guide to demonstrate as
> such.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20026) Add 1.4 release line to the JDK and Hadoop expectation tables
[ https://issues.apache.org/jira/browse/HBASE-20026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-20026: --- Fix Version/s: (was: 1.4.9) 1.4.10 > Add 1.4 release line to the JDK and Hadoop expectation tables > - > > Key: HBASE-20026 > URL: https://issues.apache.org/jira/browse/HBASE-20026 > Project: HBase > Issue Type: Task > Components: documentation >Affects Versions: 1.4.0 >Reporter: Sean Busbey >Priority: Critical > Fix For: 1.4.10 > > > the ref guide currently doesn't have any expectations listed for branch-1.4 > releases around JDK and Hadoop versions. > either add it, or maybe update the existing entries so we have "1.2, 1.3, > 1.4" in a single entry. unless we're ready to include something different > among them. (Maybe note the default Hadoop we ship with? Or Hadoop 2.8.2+ > moving to S maybe? if we've actually done any of the legwork.) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21536) Fix completebulkload usage instructions
[
https://issues.apache.org/jira/browse/HBASE-21536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Artem Ervits updated HBASE-21536:
-
Status: Patch Available (was: Open)
> Fix completebulkload usage instructions
> ---
>
> Key: HBASE-21536
> URL: https://issues.apache.org/jira/browse/HBASE-21536
> Project: HBase
> Issue Type: Task
> Components: documentation, mapreduce
>Reporter: Artem Ervits
>Assignee: Artem Ervits
>Priority: Trivial
> Attachments: HBASE-21536.v01.patch
>
>
> Usage information upon invoking LoadIncrementalHFiles is misleading and
> error-prone.
> {code:java}
> usage: completebulkload /path/to/hfileoutputformat-output tablename -loadTable
> -Dcreate.table=no - can be used to avoid creation of table by this tool
> Note: if you set this to 'no', then the target table must already exist in
> HBase
> -loadTable implies your baseDirectory to store file has a depth of 3 ,you
> must have an existing table
> -Dignore.unmatched.families=yes - can be used to ignore unmatched column
> families{code}
> in case of invoking the class via hbase command, completebulkload argument is
> unnecessary and only required via hadoop jar invocation. This is also an
> attempt to clarify where <-loadTable> and <-Dargs> arguments go on the
> command line.
> Furthermore, since LoadIncrementalHFiles was recently moved out of
> hbase-server.jar to hbase-mapreduce, updating ref guide to demonstrate as
> such.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21536) Fix completebulkload usage instructions
[
https://issues.apache.org/jira/browse/HBASE-21536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Artem Ervits updated HBASE-21536:
-
Attachment: HBASE-21536.v01.patch
> Fix completebulkload usage instructions
> ---
>
> Key: HBASE-21536
> URL: https://issues.apache.org/jira/browse/HBASE-21536
> Project: HBase
> Issue Type: Task
> Components: documentation, mapreduce
>Reporter: Artem Ervits
>Assignee: Artem Ervits
>Priority: Trivial
> Attachments: HBASE-21536.v01.patch
>
>
> Usage information upon invoking LoadIncrementalHFiles is misleading and
> error-prone.
> {code:java}
> usage: completebulkload /path/to/hfileoutputformat-output tablename -loadTable
> -Dcreate.table=no - can be used to avoid creation of table by this tool
> Note: if you set this to 'no', then the target table must already exist in
> HBase
> -loadTable implies your baseDirectory to store file has a depth of 3 ,you
> must have an existing table
> -Dignore.unmatched.families=yes - can be used to ignore unmatched column
> families{code}
> in case of invoking the class via hbase command, completebulkload argument is
> unnecessary and only required via hadoop jar invocation. This is also an
> attempt to clarify where <-loadTable> and <-Dargs> arguments go on the
> command line.
> Furthermore, since LoadIncrementalHFiles was recently moved out of
> hbase-server.jar to hbase-mapreduce, updating ref guide to demonstrate as
> such.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-21536) Fix completebulkload usage instructions
Artem Ervits created HBASE-21536:
Summary: Fix completebulkload usage instructions
Key: HBASE-21536
URL: https://issues.apache.org/jira/browse/HBASE-21536
Project: HBase
Issue Type: Task
Components: documentation, mapreduce
Reporter: Artem Ervits
Assignee: Artem Ervits
Usage information upon invoking LoadIncrementalHFiles is misleading and
error-prone.
{code:java}
usage: completebulkload /path/to/hfileoutputformat-output tablename -loadTable
-Dcreate.table=no - can be used to avoid creation of table by this tool
Note: if you set this to 'no', then the target table must already exist in HBase
-loadTable implies your baseDirectory to store file has a depth of 3 ,you must
have an existing table
-Dignore.unmatched.families=yes - can be used to ignore unmatched column
families{code}
in case of invoking the class via hbase command, completebulkload argument is
unnecessary and only required via hadoop jar invocation. This is also an
attempt to clarify where <-loadTable> and <-Dargs> arguments go on the command
line.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21013) Backport "read part" of HBASE-18754 to all active 1.x branches
[ https://issues.apache.org/jira/browse/HBASE-21013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-21013: --- Fix Version/s: (was: 1.4.9) 1.4.10 > Backport "read part" of HBASE-18754 to all active 1.x branches > -- > > Key: HBASE-21013 > URL: https://issues.apache.org/jira/browse/HBASE-21013 > Project: HBase > Issue Type: Sub-task >Reporter: Chia-Ping Tsai >Assignee: Mingdao Yang >Priority: Critical > Fix For: 1.5.0, 1.3.3, 1.2.10, 1.4.10 > > Attachments: HBASE-21013-branch-1.4.001.patch > > > The hfiles impacted by HBASE-18754 will have bytes of proto.TimeRangeTracker. > It makes all 1.x branches failed to read the hfile since all 1.x branches > can't deserialize the proto.TimeRangeTracker. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21359) Fix build problem against Hadoop 2.8.5
[ https://issues.apache.org/jira/browse/HBASE-21359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-21359: --- Resolution: Fixed Status: Resolved (was: Patch Available) > Fix build problem against Hadoop 2.8.5 > -- > > Key: HBASE-21359 > URL: https://issues.apache.org/jira/browse/HBASE-21359 > Project: HBase > Issue Type: Bug > Components: build >Affects Versions: 1.4.8 >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Trivial > Fix For: 1.5.0, 1.4.9 > > Attachments: HBASE-21359-branch-1.patch > > > 1.4.8 build fails against Hadoop 2.8.5. The fix is an easy change to > supplemental-models.xml. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21265) Split up TestRSGroups
[ https://issues.apache.org/jira/browse/HBASE-21265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-21265: --- Resolution: Fixed Status: Resolved (was: Patch Available) > Split up TestRSGroups > - > > Key: HBASE-21265 > URL: https://issues.apache.org/jira/browse/HBASE-21265 > Project: HBase > Issue Type: Task > Components: rsgroup, test >Affects Versions: 1.4.8 >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Labels: beginner, beginners > Fix For: 3.0.0, 1.5.0, 2.2.0, 1.4.9 > > Attachments: HBASE-21265-branch-1.patch, HBASE-21265-branch-1.patch, > HBASE-21265-branch-2.patch, HBASE-21265-branch-2.patch, > HBASE-21265.branch-2.1.001.patch, HBASE-21265.patch, HBASE-21265.patch > > > TestRSGroups is flaky. It is stable when run in isolation but when run as > part of the suite with concurrent executors it can fail. The current running > time of this unit on my dev box is ~240 seconds (4 minutes), which is far too > much time. This unit should be broken up 5 to 8 ways, grouped by > functionality under test. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20694) Consolidate warning on SecureBulkLoad directory permissions
[ https://issues.apache.org/jira/browse/HBASE-20694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-20694: --- Fix Version/s: (was: 1.4.9) 1.4.10 > Consolidate warning on SecureBulkLoad directory permissions > --- > > Key: HBASE-20694 > URL: https://issues.apache.org/jira/browse/HBASE-20694 > Project: HBase > Issue Type: Improvement > Components: documentation >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Minor > Fix For: 3.0.0, 1.5.0, 1.3.3, 2.2.0, 1.2.10, 1.4.10 > > > Follow-on from HBASE-20605: > HBase 1.x has a check which ignores a directory permission check if you're > using a specific filesystem which we think doesnt' do security properly. > HBase 2.x dropped this check. > Since the security of bulk-loaded data is dependent upon this directory > permission (and thus the capabilities of the FileSystem), it would be better > to have a consistent warning across branches. > [~busbey] suggested that we make a WARN message which points admins to our > Book (and write such a section if we don't have something sufficient already) > and our supported filesystems, coupled with an option to disable that warning > message. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HBASE-21306) website should have 1.4.x release docs
[ https://issues.apache.org/jira/browse/HBASE-21306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell reassigned HBASE-21306: -- Assignee: Andrew Purtell > website should have 1.4.x release docs > -- > > Key: HBASE-21306 > URL: https://issues.apache.org/jira/browse/HBASE-21306 > Project: HBase > Issue Type: Task > Components: community, documentation, website >Affects Versions: 1.4.7 >Reporter: Mike Drob >Assignee: Andrew Purtell >Priority: Major > > Since 1.4 is the stable line now, we should include the API docs on the > website instead of (in addition to?) the 1.2 docs we have now. > See also HBASE-21114 and HBASE-21119 for the process. > FYI: [~busbey], [~apurtell] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21306) website should have 1.4.x release docs
[ https://issues.apache.org/jira/browse/HBASE-21306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-21306: --- Fix Version/s: (was: 1.4.9) > website should have 1.4.x release docs > -- > > Key: HBASE-21306 > URL: https://issues.apache.org/jira/browse/HBASE-21306 > Project: HBase > Issue Type: Task > Components: community, documentation, website >Affects Versions: 1.4.7 >Reporter: Mike Drob >Priority: Major > > Since 1.4 is the stable line now, we should include the API docs on the > website instead of (in addition to?) the 1.2 docs we have now. > See also HBASE-21114 and HBASE-21119 for the process. > FYI: [~busbey], [~apurtell] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21518) TestMasterFailoverWithProcedures is flaky
[
https://issues.apache.org/jira/browse/HBASE-21518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16705195#comment-16705195
]
stack commented on HBASE-21518:
---
Nice find boys. Yes please for branch-2.0 [~psomogyi].
> TestMasterFailoverWithProcedures is flaky
> -
>
> Key: HBASE-21518
> URL: https://issues.apache.org/jira/browse/HBASE-21518
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.2.0, 2.0.3, 2.1.2
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.2
>
> Attachments: HBASE-21518-v1.patch, output.txt
>
>
> TestMasterFailoverWithProcedures test is failing frequently, times out. I
> faced this failure on 2.0.3RC0 vote and it also appears on multiple flaky
> dashboards.
> branch-2:
> [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Flaky-Tests/job/branch-2/2007/]
> branch-2.1:
> [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Flaky-Tests/job/branch-2.1/2002/]
> branch-2.0:
> [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Flaky-Tests/job/branch-2.0/1988/]
>
> {noformat}
> [INFO] Running
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures
> [ERROR] Tests run: 4, Failures: 0, Errors: 2, Skipped: 0, Time elapsed:
> 780.648 s <<< FAILURE! - in
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures
> [ERROR]
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures
> Time elapsed: 749.024 s <<< ERROR!
> org.junit.runners.model.TestTimedOutException: test timed out after 780
> seconds
> at
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures.tearDown(TestMasterFailoverWithProcedures.java:86)
> [ERROR]
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures
> Time elapsed: 749.051 s <<< ERROR!
> java.lang.Exception: Appears to be stuck in thread RS-EventLoopGroup-3-2
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21479) Individual tests in TestHRegionReplayEvents class are failing
[
https://issues.apache.org/jira/browse/HBASE-21479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16705188#comment-16705188
]
stack commented on HBASE-21479:
---
+1 Try it.
> Individual tests in TestHRegionReplayEvents class are failing
> -
>
> Key: HBASE-21479
> URL: https://issues.apache.org/jira/browse/HBASE-21479
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Peter Somogyi
>Priority: Major
> Attachments: HBASE-21479-v1.patch, testHRegionReplayEvents-output.txt
>
>
> The test fails in both master branch and branch-2 :
> {code}
> testSkippingEditsWithSmallerSeqIdAfterRegionOpenEvent(org.apache.hadoop.hbase.regionserver.TestHRegionReplayEvents)
> Time elapsed: 3.74 sec <<< ERROR!
> java.lang.IndexOutOfBoundsException: Index: 2, Size: 1
> at
> org.apache.hadoop.hbase.regionserver.TestHRegionReplayEvents.testSkippingEditsWithSmallerSeqIdAfterRegionOpenEvent(TestHRegionReplayEvents.java:1042)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21530) Abort_Procedure should be able to take a list of proc IDs
[ https://issues.apache.org/jira/browse/HBASE-21530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16705173#comment-16705173 ] Xu Cang commented on HBASE-21530: - I am throwing an idea here, should we also have some concept such as 'Procedure Expiration' that procedure in 'RUNNABLE' state will be automatically aborted when it expires. (And this should be an option that can be turned on/off procedure based, not system-wide) > Abort_Procedure should be able to take a list of proc IDs > - > > Key: HBASE-21530 > URL: https://issues.apache.org/jira/browse/HBASE-21530 > Project: HBase > Issue Type: Improvement >Reporter: Geoffrey Jacoby >Priority: Minor > > As a convenience, it would be helpful if the HBase shell's abort_procedure > call had the option of taking in multiple procedure ids at the same time, > rather than relying on operators to use a loop in an external script. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-21530) Abort_Procedure should be able to take a list of proc IDs
[
https://issues.apache.org/jira/browse/HBASE-21530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16705155#comment-16705155
]
Andrew Purtell edited comment on HBASE-21530 at 11/30/18 6:44 PM:
--
{quote}aborting a queued (RUNNABLE) procedure should always be OK.
{quote}
+1
I think we should un-deprecate at least the ability to cancel procedure
requests from queue. It could be fine to return an error for a cancel request
if it is already running. We could call it cancel instead of abort if you
prefer to better represent the semantic.
was (Author: apurtell):
{quote}aborting a queued (RUNNABLE) procedure should always be OK.
{quote}
+1
I think we should un-deprecate at least the ability to cancel procedure
requests from queue. It could be fine to return an error for a cancel request
if it is already running. We could call it cancel instead of abort if you
prefer to better represent the semantic.
> Abort_Procedure should be able to take a list of proc IDs
> -
>
> Key: HBASE-21530
> URL: https://issues.apache.org/jira/browse/HBASE-21530
> Project: HBase
> Issue Type: Improvement
>Reporter: Geoffrey Jacoby
>Priority: Minor
>
> As a convenience, it would be helpful if the HBase shell's abort_procedure
> call had the option of taking in multiple procedure ids at the same time,
> rather than relying on operators to use a loop in an external script.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21530) Abort_Procedure should be able to take a list of proc IDs
[
https://issues.apache.org/jira/browse/HBASE-21530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16705155#comment-16705155
]
Andrew Purtell commented on HBASE-21530:
{quote}aborting a queued (RUNNABLE) procedure should always be OK.
{quote}
+1
I think we should un-deprecate at least the ability to cancel procedure
requests from queue. It could be fine to return an error for a cancel request
if it is already running. We could call it cancel instead of abort if you
prefer to better represent the semantic.
> Abort_Procedure should be able to take a list of proc IDs
> -
>
> Key: HBASE-21530
> URL: https://issues.apache.org/jira/browse/HBASE-21530
> Project: HBase
> Issue Type: Improvement
>Reporter: Geoffrey Jacoby
>Priority: Minor
>
> As a convenience, it would be helpful if the HBase shell's abort_procedure
> call had the option of taking in multiple procedure ids at the same time,
> rather than relying on operators to use a loop in an external script.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21479) Individual tests in TestHRegionReplayEvents class are failing
[
https://issues.apache.org/jira/browse/HBASE-21479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16705152#comment-16705152
]
Hadoop QA commented on HBASE-21479:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
12s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
57s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
46s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
40s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
49s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 25s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}131m
3s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
28s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}169m 58s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21479 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12950173/HBASE-21479-v1.patch |
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 73c4491be61a 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / dfeab9f5c9 |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/15158/testReport/ |
| Max. process+thread count | 4934 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/15158/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Individual test
[jira] [Commented] (HBASE-21518) TestMasterFailoverWithProcedures is flaky
[
https://issues.apache.org/jira/browse/HBASE-21518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16704986#comment-16704986
]
Hudson commented on HBASE-21518:
Results for branch branch-2.1
[build #648 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/648/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/648//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/648//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/648//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> TestMasterFailoverWithProcedures is flaky
> -
>
> Key: HBASE-21518
> URL: https://issues.apache.org/jira/browse/HBASE-21518
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.2.0, 2.0.3, 2.1.2
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.2
>
> Attachments: HBASE-21518-v1.patch, output.txt
>
>
> TestMasterFailoverWithProcedures test is failing frequently, times out. I
> faced this failure on 2.0.3RC0 vote and it also appears on multiple flaky
> dashboards.
> branch-2:
> [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Flaky-Tests/job/branch-2/2007/]
> branch-2.1:
> [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Flaky-Tests/job/branch-2.1/2002/]
> branch-2.0:
> [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Flaky-Tests/job/branch-2.0/1988/]
>
> {noformat}
> [INFO] Running
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures
> [ERROR] Tests run: 4, Failures: 0, Errors: 2, Skipped: 0, Time elapsed:
> 780.648 s <<< FAILURE! - in
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures
> [ERROR]
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures
> Time elapsed: 749.024 s <<< ERROR!
> org.junit.runners.model.TestTimedOutException: test timed out after 780
> seconds
> at
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures.tearDown(TestMasterFailoverWithProcedures.java:86)
> [ERROR]
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures
> Time elapsed: 749.051 s <<< ERROR!
> java.lang.Exception: Appears to be stuck in thread RS-EventLoopGroup-3-2
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21479) Individual tests in TestHRegionReplayEvents class are failing
[
https://issues.apache.org/jira/browse/HBASE-21479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Somogyi updated HBASE-21479:
--
Attachment: HBASE-21479-v1.patch
> Individual tests in TestHRegionReplayEvents class are failing
> -
>
> Key: HBASE-21479
> URL: https://issues.apache.org/jira/browse/HBASE-21479
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Peter Somogyi
>Priority: Major
> Attachments: HBASE-21479-v1.patch, testHRegionReplayEvents-output.txt
>
>
> The test fails in both master branch and branch-2 :
> {code}
> testSkippingEditsWithSmallerSeqIdAfterRegionOpenEvent(org.apache.hadoop.hbase.regionserver.TestHRegionReplayEvents)
> Time elapsed: 3.74 sec <<< ERROR!
> java.lang.IndexOutOfBoundsException: Index: 2, Size: 1
> at
> org.apache.hadoop.hbase.regionserver.TestHRegionReplayEvents.testSkippingEditsWithSmallerSeqIdAfterRegionOpenEvent(TestHRegionReplayEvents.java:1042)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21479) Individual tests in TestHRegionReplayEvents class are failing
[
https://issues.apache.org/jira/browse/HBASE-21479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Somogyi updated HBASE-21479:
--
Status: Patch Available (was: In Progress)
> Individual tests in TestHRegionReplayEvents class are failing
> -
>
> Key: HBASE-21479
> URL: https://issues.apache.org/jira/browse/HBASE-21479
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Peter Somogyi
>Priority: Major
> Attachments: HBASE-21479-v1.patch, testHRegionReplayEvents-output.txt
>
>
> The test fails in both master branch and branch-2 :
> {code}
> testSkippingEditsWithSmallerSeqIdAfterRegionOpenEvent(org.apache.hadoop.hbase.regionserver.TestHRegionReplayEvents)
> Time elapsed: 3.74 sec <<< ERROR!
> java.lang.IndexOutOfBoundsException: Index: 2, Size: 1
> at
> org.apache.hadoop.hbase.regionserver.TestHRegionReplayEvents.testSkippingEditsWithSmallerSeqIdAfterRegionOpenEvent(TestHRegionReplayEvents.java:1042)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21479) Individual tests in TestHRegionReplayEvents class are failing
[
https://issues.apache.org/jira/browse/HBASE-21479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Somogyi updated HBASE-21479:
--
Summary: Individual tests in TestHRegionReplayEvents class are failing
(was:
TestHRegionReplayEvents#testSkippingEditsWithSmallerSeqIdAfterRegionOpenEvent
fails with IndexOutOfBoundsException)
> Individual tests in TestHRegionReplayEvents class are failing
> -
>
> Key: HBASE-21479
> URL: https://issues.apache.org/jira/browse/HBASE-21479
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Peter Somogyi
>Priority: Major
> Attachments: testHRegionReplayEvents-output.txt
>
>
> The test fails in both master branch and branch-2 :
> {code}
> testSkippingEditsWithSmallerSeqIdAfterRegionOpenEvent(org.apache.hadoop.hbase.regionserver.TestHRegionReplayEvents)
> Time elapsed: 3.74 sec <<< ERROR!
> java.lang.IndexOutOfBoundsException: Index: 2, Size: 1
> at
> org.apache.hadoop.hbase.regionserver.TestHRegionReplayEvents.testSkippingEditsWithSmallerSeqIdAfterRegionOpenEvent(TestHRegionReplayEvents.java:1042)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-21535) Zombie Master detector is not working
Pankaj Kumar created HBASE-21535:
Summary: Zombie Master detector is not working
Key: HBASE-21535
URL: https://issues.apache.org/jira/browse/HBASE-21535
Project: HBase
Issue Type: Bug
Components: master
Reporter: Pankaj Kumar
Assignee: Pankaj Kumar
We have InitializationMonitor thread in HMaster which detects Zombie Hmaster
based on _hbase.master.initializationmonitor.timeout _and halts if
_hbase.master.initializationmonitor.haltontimeout_ set _true_.
After HBASE-19694, HMaster initialization order was correted. Hmaster is set
active after Initializing ZK system trackers as follows,
{noformat}
status.setStatus("Initializing ZK system trackers");
initializeZKBasedSystemTrackers();
status.setStatus("Loading last flushed sequence id of regions");
try {
this.serverManager.loadLastFlushedSequenceIds();
} catch (IOException e) {
LOG.debug("Failed to load last flushed sequence id of regions"
+ " from file system", e);
}
// Set ourselves as active Master now our claim has succeeded up in zk.
this.activeMaster = true;
{noformat}
But Zombie detector thread is started at the begining phase of
finishActiveMasterInitialization(),
{noformat}
private void finishActiveMasterInitialization(MonitoredTask status) throws
IOException,
InterruptedException, KeeperException, ReplicationException {
Thread zombieDetector = new Thread(new InitializationMonitor(this),
"ActiveMasterInitializationMonitor-" + System.currentTimeMillis());
zombieDetector.setDaemon(true);
zombieDetector.start();
{noformat}
During zombieDetector execution "master.isActiveMaster()" will be false, so it
won't wait and cant detect zombie master.
{noformat}
@Override
public void run() {
try {
while (!master.isStopped() && master.isActiveMaster()) {
Thread.sleep(timeout);
if (master.isInitialized()) {
LOG.debug("Initialization completed within allotted tolerance. Monitor
exiting.");
} else {
LOG.error("Master failed to complete initialization after " + timeout + "ms.
Please"
+ " consider submitting a bug report including a thread dump of this
process.");
if (haltOnTimeout) {
LOG.error("Zombie Master exiting. Thread dump to stdout");
Threads.printThreadInfo(System.out, "Zombie HMaster");
System.exit(-1);
}
}
}
} catch (InterruptedException ie) {
LOG.trace("InitMonitor thread interrupted. Existing.");
}
}
}
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21535) Zombie Master detector is not working
[
https://issues.apache.org/jira/browse/HBASE-21535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pankaj Kumar updated HBASE-21535:
-
External issue URL: (was:
https://issues.apache.org/jira/browse/HBASE-19694)
> Zombie Master detector is not working
> -
>
> Key: HBASE-21535
> URL: https://issues.apache.org/jira/browse/HBASE-21535
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
>Priority: Critical
>
> We have InitializationMonitor thread in HMaster which detects Zombie Hmaster
> based on _hbase.master.initializationmonitor.timeout _and halts if
> _hbase.master.initializationmonitor.haltontimeout_ set _true_.
> After HBASE-19694, HMaster initialization order was correted. Hmaster is set
> active after Initializing ZK system trackers as follows,
> {noformat}
> status.setStatus("Initializing ZK system trackers");
> initializeZKBasedSystemTrackers();
> status.setStatus("Loading last flushed sequence id of regions");
> try {
> this.serverManager.loadLastFlushedSequenceIds();
> } catch (IOException e) {
> LOG.debug("Failed to load last flushed sequence id of regions"
> + " from file system", e);
> }
> // Set ourselves as active Master now our claim has succeeded up in zk.
> this.activeMaster = true;
> {noformat}
> But Zombie detector thread is started at the begining phase of
> finishActiveMasterInitialization(),
> {noformat}
> private void finishActiveMasterInitialization(MonitoredTask status) throws
> IOException,
> InterruptedException, KeeperException, ReplicationException {
> Thread zombieDetector = new Thread(new InitializationMonitor(this),
> "ActiveMasterInitializationMonitor-" + System.currentTimeMillis());
> zombieDetector.setDaemon(true);
> zombieDetector.start();
> {noformat}
> During zombieDetector execution "master.isActiveMaster()" will be false, so
> it won't wait and cant detect zombie master.
> {noformat}
> @Override
> public void run() {
> try {
> while (!master.isStopped() && master.isActiveMaster()) {
> Thread.sleep(timeout);
> if (master.isInitialized()) {
> LOG.debug("Initialization completed within allotted tolerance. Monitor
> exiting.");
> } else {
> LOG.error("Master failed to complete initialization after " + timeout + "ms.
> Please"
> + " consider submitting a bug report including a thread dump of this
> process.");
> if (haltOnTimeout) {
> LOG.error("Zombie Master exiting. Thread dump to stdout");
> Threads.printThreadInfo(System.out, "Zombie HMaster");
> System.exit(-1);
> }
> }
> }
> } catch (InterruptedException ie) {
> LOG.trace("InitMonitor thread interrupted. Existing.");
> }
> }
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21535) Zombie Master detector is not working
[
https://issues.apache.org/jira/browse/HBASE-21535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pankaj Kumar updated HBASE-21535:
-
External issue URL: https://issues.apache.org/jira/browse/HBASE-19694
> Zombie Master detector is not working
> -
>
> Key: HBASE-21535
> URL: https://issues.apache.org/jira/browse/HBASE-21535
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
>Priority: Critical
>
> We have InitializationMonitor thread in HMaster which detects Zombie Hmaster
> based on _hbase.master.initializationmonitor.timeout _and halts if
> _hbase.master.initializationmonitor.haltontimeout_ set _true_.
> After HBASE-19694, HMaster initialization order was correted. Hmaster is set
> active after Initializing ZK system trackers as follows,
> {noformat}
> status.setStatus("Initializing ZK system trackers");
> initializeZKBasedSystemTrackers();
> status.setStatus("Loading last flushed sequence id of regions");
> try {
> this.serverManager.loadLastFlushedSequenceIds();
> } catch (IOException e) {
> LOG.debug("Failed to load last flushed sequence id of regions"
> + " from file system", e);
> }
> // Set ourselves as active Master now our claim has succeeded up in zk.
> this.activeMaster = true;
> {noformat}
> But Zombie detector thread is started at the begining phase of
> finishActiveMasterInitialization(),
> {noformat}
> private void finishActiveMasterInitialization(MonitoredTask status) throws
> IOException,
> InterruptedException, KeeperException, ReplicationException {
> Thread zombieDetector = new Thread(new InitializationMonitor(this),
> "ActiveMasterInitializationMonitor-" + System.currentTimeMillis());
> zombieDetector.setDaemon(true);
> zombieDetector.start();
> {noformat}
> During zombieDetector execution "master.isActiveMaster()" will be false, so
> it won't wait and cant detect zombie master.
> {noformat}
> @Override
> public void run() {
> try {
> while (!master.isStopped() && master.isActiveMaster()) {
> Thread.sleep(timeout);
> if (master.isInitialized()) {
> LOG.debug("Initialization completed within allotted tolerance. Monitor
> exiting.");
> } else {
> LOG.error("Master failed to complete initialization after " + timeout + "ms.
> Please"
> + " consider submitting a bug report including a thread dump of this
> process.");
> if (haltOnTimeout) {
> LOG.error("Zombie Master exiting. Thread dump to stdout");
> Threads.printThreadInfo(System.out, "Zombie HMaster");
> System.exit(-1);
> }
> }
> }
> } catch (InterruptedException ie) {
> LOG.trace("InitMonitor thread interrupted. Existing.");
> }
> }
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21518) TestMasterFailoverWithProcedures is flaky
[
https://issues.apache.org/jira/browse/HBASE-21518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16704783#comment-16704783
]
Hudson commented on HBASE-21518:
Results for branch master
[build #638 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/638/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/638//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/638//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/638//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> TestMasterFailoverWithProcedures is flaky
> -
>
> Key: HBASE-21518
> URL: https://issues.apache.org/jira/browse/HBASE-21518
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.2.0, 2.0.3, 2.1.2
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.2
>
> Attachments: HBASE-21518-v1.patch, output.txt
>
>
> TestMasterFailoverWithProcedures test is failing frequently, times out. I
> faced this failure on 2.0.3RC0 vote and it also appears on multiple flaky
> dashboards.
> branch-2:
> [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Flaky-Tests/job/branch-2/2007/]
> branch-2.1:
> [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Flaky-Tests/job/branch-2.1/2002/]
> branch-2.0:
> [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Flaky-Tests/job/branch-2.0/1988/]
>
> {noformat}
> [INFO] Running
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures
> [ERROR] Tests run: 4, Failures: 0, Errors: 2, Skipped: 0, Time elapsed:
> 780.648 s <<< FAILURE! - in
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures
> [ERROR]
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures
> Time elapsed: 749.024 s <<< ERROR!
> org.junit.runners.model.TestTimedOutException: test timed out after 780
> seconds
> at
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures.tearDown(TestMasterFailoverWithProcedures.java:86)
> [ERROR]
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures
> Time elapsed: 749.051 s <<< ERROR!
> java.lang.Exception: Appears to be stuck in thread RS-EventLoopGroup-3-2
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21453) Convert ReadOnlyZKClient to DEBUG instead of INFO
[
https://issues.apache.org/jira/browse/HBASE-21453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-21453:
Component/s: Zookeeper
logging
> Convert ReadOnlyZKClient to DEBUG instead of INFO
> -
>
> Key: HBASE-21453
> URL: https://issues.apache.org/jira/browse/HBASE-21453
> Project: HBase
> Issue Type: Bug
> Components: logging, Zookeeper
>Reporter: stack
>Assignee: Sakthi
>Priority: Major
> Attachments: hbase-21453.master.001.patch
>
>
> Running commands in spark-shell, this is what it looks like on each
> invocation:
> {code}
> scala> val count = rdd.count()
> 2018-11-07 21:01:46,026 INFO [Executor task launch worker for task 1]
> zookeeper.ReadOnlyZKClient: Connect 0x18f3d868 to localhost:2181 with session
> timeout=9ms, retries 30, retry interval 1000ms, keepAlive=6ms
> 2018-11-07 21:01:46,027 INFO [ReadOnlyZKClient-localhost:2181@0x18f3d868]
> zookeeper.ZooKeeper: Initiating client connection,
> connectString=localhost:2181 sessionTimeout=9
> watcher=org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$20/1362339879@743dab9f
> 2018-11-07 21:01:46,030 INFO
> [ReadOnlyZKClient-localhost:2181@0x18f3d868-SendThread(localhost:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server
> localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL
> (unknown error)
> 2018-11-07 21:01:46,031 INFO
> [ReadOnlyZKClient-localhost:2181@0x18f3d868-SendThread(localhost:2181)]
> zookeeper.ClientCnxn: Socket connection established to
> localhost/127.0.0.1:2181, initiating session
> 2018-11-07 21:01:46,033 INFO
> [ReadOnlyZKClient-localhost:2181@0x18f3d868-SendThread(localhost:2181)]
> zookeeper.ClientCnxn: Session establishment complete on server
> localhost/127.0.0.1:2181, sessionid = 0x166f1b283080005, negotiated timeout =
> 4
> 2018-11-07 21:01:46,035 INFO [Executor task launch worker for task 1]
> mapreduce.TableInputFormatBase: Input split length: 0 bytes.
> [Stage 1:> (0 + 1) /
> 1]2018-11-07 21:01:48,074 INFO [Executor task launch worker for task 1]
> zookeeper.ReadOnlyZKClient: Close zookeeper connection 0x18f3d868 to
> localhost:2181
> 2018-11-07 21:01:48,075 INFO [ReadOnlyZKClient-localhost:2181@0x18f3d868]
> zookeeper.ZooKeeper: Session: 0x166f1b283080005 closed
> 2018-11-07 21:01:48,076 INFO [ReadOnlyZKClient
> -localhost:2181@0x18f3d868-EventThread] zookeeper.ClientCnxn: EventThread
> shut down for session: 0x166f1b283080005
> count: Long = 10
> {code}
> Let me shut down the ReadOnlyZKClient log level.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21486) The current replication implementation for peer in STANDBY state breaks serial replication
[ https://issues.apache.org/jira/browse/HBASE-21486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16704775#comment-16704775 ] Guanghao Zhang commented on HBASE-21486: +1. > The current replication implementation for peer in STANDBY state breaks > serial replication > -- > > Key: HBASE-21486 > URL: https://issues.apache.org/jira/browse/HBASE-21486 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Critical > Fix For: 3.0.0 > > Attachments: HBASE-21486.patch > > > As we will drop recovered queues in STANDBY state, and also when transiting > to DA, we will drain the replication queue, we may miss to update the last > pushed sequence id, and cause the serial replication to be stuck. Need to > find a way to deal with this. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21413) Empty meta log doesn't get split when restart whole cluster
[
https://issues.apache.org/jira/browse/HBASE-21413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16704761#comment-16704761
]
Hadoop QA commented on HBASE-21413:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 3m
26s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} branch-2.1 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
19s{color} | {color:green} branch-2.1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
57s{color} | {color:green} branch-2.1 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
8s{color} | {color:green} branch-2.1 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
20s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
8s{color} | {color:green} branch-2.1 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
36s{color} | {color:green} branch-2.1 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
42s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
0s{color} | {color:red} hbase-server: The patch generated 2 new + 5 unchanged -
0 fixed = 7 total (was 5) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
21s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 7s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}235m 7s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
29s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}275m 35s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.client.TestAdmin1 |
| | hadoop.hbase.regionserver.TestSplitTransactionOnCluster |
| | hadoop.hbase.client.TestAsyncTableGetMultiThreaded |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:42ca976 |
| JIRA Issue | HBASE-21413 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12950141/HBASE-21413.branch-2.1.001.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 1d9ff31418b5 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | branch-2.1 / 4ebbfa3d60 |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC3 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HBASE-Build/15157/artifact/patchprocess/diff-checkstyle-hb
[jira] [Commented] (HBASE-21524) Unnecessary DEBUG log in ConnectionImplementation#isTableEnabled
[
https://issues.apache.org/jira/browse/HBASE-21524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16704686#comment-16704686
]
Hudson commented on HBASE-21524:
Results for branch master
[build #637 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/637/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/637//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/637//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/637//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Unnecessary DEBUG log in ConnectionImplementation#isTableEnabled
>
>
> Key: HBASE-21524
> URL: https://issues.apache.org/jira/browse/HBASE-21524
> Project: HBase
> Issue Type: Improvement
> Components: Client
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.2
>
> Attachments: HBASE-21524.001.branch-2.0.patch
>
>
> Too much crap coming out of isTableEnabled for "normal" situations. Don't
> need a DEBUG message to tell us that the table is available.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-18735) Provide a fast mechanism for shutting down mini cluster
[
https://issues.apache.org/jira/browse/HBASE-18735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16704688#comment-16704688
]
Hudson commented on HBASE-18735:
Results for branch master
[build #637 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/637/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/637//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/637//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/637//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Provide a fast mechanism for shutting down mini cluster
> ---
>
> Key: HBASE-18735
> URL: https://issues.apache.org/jira/browse/HBASE-18735
> Project: HBase
> Issue Type: Wish
>Reporter: Samarth Jain
>Assignee: Artem Ervits
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.0.3, 2.1.2
>
> Attachments: HBASE-18735.v01.patch, HBASE-18735.v02.patch,
> HBASE-18735.v03.patch
>
>
> The current mechanism of shutting down a mini cluster through
> HBaseTestingUtility.shutDownMiniCluster can take a lot of time when the mini
> cluster almost has a lot of tables. A lot of this time is spent in closing
> all the user regions. It would be nice to have a mechanism where this
> shutdown can happen quickly without having to worry about closing these user
> regions. At the same time, this mechanism would need to make sure that all
> the critical system resources like file handles and network ports are still
> released so that subsequently initialized mini clusters on the same JVM or
> system won't run into resource issues. This would make testing using HBase
> mini clusters much faster and immensely help out test frameworks of dependent
> projects like Phoenix.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21523) Chatty DEBUG logging in Master log
[
https://issues.apache.org/jira/browse/HBASE-21523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16704687#comment-16704687
]
Hudson commented on HBASE-21523:
Results for branch master
[build #637 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/637/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/637//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/637//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/637//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Chatty DEBUG logging in Master log
> --
>
> Key: HBASE-21523
> URL: https://issues.apache.org/jira/browse/HBASE-21523
> Project: HBase
> Issue Type: Improvement
> Components: backup&restore
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-21523.001.patch, HBASE-21523.002.patch
>
>
> Noticed this in testing, a lot of spam in the Master log at DEBUG.
> {noformat}
> 2018-11-27 01:12:28,322 DEBUG [ForkJoinPool-1-worker-6] impl.BackupMetaTable:
> Backup table backup:system exists and available
> 2018-11-27 01:12:28,326 DEBUG [ForkJoinPool-1-worker-6] impl.BackupMetaTable:
> Backup table backup:system_bulk exists and available
> 2018-11-27 01:12:28,341 DEBUG [ForkJoinPool-1-worker-6] impl.BackupMetaTable:
> Backup table backup:system exists and available
> 2018-11-27 01:12:28,345 DEBUG [ForkJoinPool-1-worker-6] impl.BackupMetaTable:
> Backup table backup:system_bulk exists and available
> {noformat}
> FYI [~vrodionov]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21504) If enable FIFOCompactionPolicy, a compaction may write a "empty" hfile whose maxTimeStamp is long max. This kind of hfile will never be archived.
[
https://issues.apache.org/jira/browse/HBASE-21504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16704689#comment-16704689
]
Hudson commented on HBASE-21504:
Results for branch master
[build #637 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/637/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/637//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/637//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/637//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> If enable FIFOCompactionPolicy, a compaction may write a "empty" hfile whose
> maxTimeStamp is long max. This kind of hfile will never be archived.
> -
>
> Key: HBASE-21504
> URL: https://issues.apache.org/jira/browse/HBASE-21504
> Project: HBase
> Issue Type: Bug
> Components: Compaction
>Affects Versions: 2.1.0
>Reporter: xuming
>Assignee: Zheng Hu
>Priority: Critical
> Fix For: 3.0.0, 1.5.0, 1.3.3, 2.2.0, 1.4.9, 2.1.2, 2.0.4
>
> Attachments: 1.patch, HBASE-21504.v1.patch
>
>
> When i use FIFOCompactionPolicy, and if all hfiles(>1) are TTL expired in a
> region, once we do a compaction on the region, the compaction policy will
> select the latest hfile to do compaction.But beacuse the latest hfile is
> already TTL expired, compactor only write a "empty" hfile(whose entry counter
> is 0 and maxTimeStamp is Long.MAX_VALUE) finally. Because maxTimeStamp is
> long max, so the "empty" hfile will never be TTL expired, more seriously we
> can not archive it by FIFOCompactionPolicy forever.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21504) If enable FIFOCompactionPolicy, a compaction may write a "empty" hfile whose maxTimeStamp is long max. This kind of hfile will never be archived.
[
https://issues.apache.org/jira/browse/HBASE-21504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16704627#comment-16704627
]
Hudson commented on HBASE-21504:
Results for branch branch-2.1
[build #647 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/647/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/647//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/647//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/647//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> If enable FIFOCompactionPolicy, a compaction may write a "empty" hfile whose
> maxTimeStamp is long max. This kind of hfile will never be archived.
> -
>
> Key: HBASE-21504
> URL: https://issues.apache.org/jira/browse/HBASE-21504
> Project: HBase
> Issue Type: Bug
> Components: Compaction
>Affects Versions: 2.1.0
>Reporter: xuming
>Assignee: Zheng Hu
>Priority: Critical
> Fix For: 3.0.0, 1.5.0, 1.3.3, 2.2.0, 1.4.9, 2.1.2, 2.0.4
>
> Attachments: 1.patch, HBASE-21504.v1.patch
>
>
> When i use FIFOCompactionPolicy, and if all hfiles(>1) are TTL expired in a
> region, once we do a compaction on the region, the compaction policy will
> select the latest hfile to do compaction.But beacuse the latest hfile is
> already TTL expired, compactor only write a "empty" hfile(whose entry counter
> is 0 and maxTimeStamp is Long.MAX_VALUE) finally. Because maxTimeStamp is
> long max, so the "empty" hfile will never be TTL expired, more seriously we
> can not archive it by FIFOCompactionPolicy forever.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HBASE-21479) TestHRegionReplayEvents#testSkippingEditsWithSmallerSeqIdAfterRegionOpenEvent fails with IndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/HBASE-21479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Somogyi reassigned HBASE-21479:
-
Assignee: Peter Somogyi
> TestHRegionReplayEvents#testSkippingEditsWithSmallerSeqIdAfterRegionOpenEvent
> fails with IndexOutOfBoundsException
> --
>
> Key: HBASE-21479
> URL: https://issues.apache.org/jira/browse/HBASE-21479
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Peter Somogyi
>Priority: Major
> Attachments: testHRegionReplayEvents-output.txt
>
>
> The test fails in both master branch and branch-2 :
> {code}
> testSkippingEditsWithSmallerSeqIdAfterRegionOpenEvent(org.apache.hadoop.hbase.regionserver.TestHRegionReplayEvents)
> Time elapsed: 3.74 sec <<< ERROR!
> java.lang.IndexOutOfBoundsException: Index: 2, Size: 1
> at
> org.apache.hadoop.hbase.regionserver.TestHRegionReplayEvents.testSkippingEditsWithSmallerSeqIdAfterRegionOpenEvent(TestHRegionReplayEvents.java:1042)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work started] (HBASE-21479) TestHRegionReplayEvents#testSkippingEditsWithSmallerSeqIdAfterRegionOpenEvent fails with IndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/HBASE-21479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HBASE-21479 started by Peter Somogyi.
-
> TestHRegionReplayEvents#testSkippingEditsWithSmallerSeqIdAfterRegionOpenEvent
> fails with IndexOutOfBoundsException
> --
>
> Key: HBASE-21479
> URL: https://issues.apache.org/jira/browse/HBASE-21479
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Peter Somogyi
>Priority: Major
> Attachments: testHRegionReplayEvents-output.txt
>
>
> The test fails in both master branch and branch-2 :
> {code}
> testSkippingEditsWithSmallerSeqIdAfterRegionOpenEvent(org.apache.hadoop.hbase.regionserver.TestHRegionReplayEvents)
> Time elapsed: 3.74 sec <<< ERROR!
> java.lang.IndexOutOfBoundsException: Index: 2, Size: 1
> at
> org.apache.hadoop.hbase.regionserver.TestHRegionReplayEvents.testSkippingEditsWithSmallerSeqIdAfterRegionOpenEvent(TestHRegionReplayEvents.java:1042)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21504) If enable FIFOCompactionPolicy, a compaction may write a "empty" hfile whose maxTimeStamp is long max. This kind of hfile will never be archived.
[
https://issues.apache.org/jira/browse/HBASE-21504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16704589#comment-16704589
]
Hudson commented on HBASE-21504:
Results for branch branch-2.0
[build #1125 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1125/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1125//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1125//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1125//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> If enable FIFOCompactionPolicy, a compaction may write a "empty" hfile whose
> maxTimeStamp is long max. This kind of hfile will never be archived.
> -
>
> Key: HBASE-21504
> URL: https://issues.apache.org/jira/browse/HBASE-21504
> Project: HBase
> Issue Type: Bug
> Components: Compaction
>Affects Versions: 2.1.0
>Reporter: xuming
>Assignee: Zheng Hu
>Priority: Critical
> Fix For: 3.0.0, 1.5.0, 1.3.3, 2.2.0, 1.4.9, 2.1.2, 2.0.4
>
> Attachments: 1.patch, HBASE-21504.v1.patch
>
>
> When i use FIFOCompactionPolicy, and if all hfiles(>1) are TTL expired in a
> region, once we do a compaction on the region, the compaction policy will
> select the latest hfile to do compaction.But beacuse the latest hfile is
> already TTL expired, compactor only write a "empty" hfile(whose entry counter
> is 0 and maxTimeStamp is Long.MAX_VALUE) finally. Because maxTimeStamp is
> long max, so the "empty" hfile will never be TTL expired, more seriously we
> can not archive it by FIFOCompactionPolicy forever.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21504) If enable FIFOCompactionPolicy, a compaction may write a "empty" hfile whose maxTimeStamp is long max. This kind of hfile will never be archived.
[
https://issues.apache.org/jira/browse/HBASE-21504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16704572#comment-16704572
]
Hudson commented on HBASE-21504:
Results for branch branch-2
[build #1532 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1532/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1532//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1532//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1532//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> If enable FIFOCompactionPolicy, a compaction may write a "empty" hfile whose
> maxTimeStamp is long max. This kind of hfile will never be archived.
> -
>
> Key: HBASE-21504
> URL: https://issues.apache.org/jira/browse/HBASE-21504
> Project: HBase
> Issue Type: Bug
> Components: Compaction
>Affects Versions: 2.1.0
>Reporter: xuming
>Assignee: Zheng Hu
>Priority: Critical
> Fix For: 3.0.0, 1.5.0, 1.3.3, 2.2.0, 1.4.9, 2.1.2, 2.0.4
>
> Attachments: 1.patch, HBASE-21504.v1.patch
>
>
> When i use FIFOCompactionPolicy, and if all hfiles(>1) are TTL expired in a
> region, once we do a compaction on the region, the compaction policy will
> select the latest hfile to do compaction.But beacuse the latest hfile is
> already TTL expired, compactor only write a "empty" hfile(whose entry counter
> is 0 and maxTimeStamp is Long.MAX_VALUE) finally. Because maxTimeStamp is
> long max, so the "empty" hfile will never be TTL expired, more seriously we
> can not archive it by FIFOCompactionPolicy forever.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-18735) Provide a fast mechanism for shutting down mini cluster
[
https://issues.apache.org/jira/browse/HBASE-18735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16704571#comment-16704571
]
Hudson commented on HBASE-18735:
Results for branch branch-2
[build #1532 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1532/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1532//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1532//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1532//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Provide a fast mechanism for shutting down mini cluster
> ---
>
> Key: HBASE-18735
> URL: https://issues.apache.org/jira/browse/HBASE-18735
> Project: HBase
> Issue Type: Wish
>Reporter: Samarth Jain
>Assignee: Artem Ervits
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.0.3, 2.1.2
>
> Attachments: HBASE-18735.v01.patch, HBASE-18735.v02.patch,
> HBASE-18735.v03.patch
>
>
> The current mechanism of shutting down a mini cluster through
> HBaseTestingUtility.shutDownMiniCluster can take a lot of time when the mini
> cluster almost has a lot of tables. A lot of this time is spent in closing
> all the user regions. It would be nice to have a mechanism where this
> shutdown can happen quickly without having to worry about closing these user
> regions. At the same time, this mechanism would need to make sure that all
> the critical system resources like file handles and network ports are still
> released so that subsequently initialized mini clusters on the same JVM or
> system won't run into resource issues. This would make testing using HBase
> mini clusters much faster and immensely help out test frameworks of dependent
> projects like Phoenix.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20610) Procedure V2 - Distributed Log Splitting
[ https://issues.apache.org/jira/browse/HBASE-20610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16704568#comment-16704568 ] Jingyun Tian commented on HBASE-20610: -- Upload a poc of this issue. It's a draft right now. Please feel free to comment. [~Apache9] > Procedure V2 - Distributed Log Splitting > > > Key: HBASE-20610 > URL: https://issues.apache.org/jira/browse/HBASE-20610 > Project: HBase > Issue Type: Umbrella > Components: proc-v2 >Reporter: Guanghao Zhang >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-20610.master.001.patch > > > Now master and regionserver use zk to coordinate log split tasks. The split > log manager manages all log files which need to be scanned and split. Then > the split log manager places all the logs into the ZooKeeper splitWAL node > (/hbase/splitWAL) as tasks and monitors these task nodes and waits for them > to be processed. Each regionserver watch splitWAL znode and grab task when > node children changed. And regionserver does the work to split the logs. > Open this umbrella issue to move this "coordinate" work to use new procedure > v2 framework and reduce zk depencency. Plan to finish this before 3.0 > release. Any suggestions are welcomed. Thanks. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20610) Procedure V2 - Distributed Log Splitting
[ https://issues.apache.org/jira/browse/HBASE-20610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingyun Tian updated HBASE-20610: - Attachment: HBASE-20610.master.001.patch > Procedure V2 - Distributed Log Splitting > > > Key: HBASE-20610 > URL: https://issues.apache.org/jira/browse/HBASE-20610 > Project: HBase > Issue Type: Umbrella > Components: proc-v2 >Reporter: Guanghao Zhang >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-20610.master.001.patch > > > Now master and regionserver use zk to coordinate log split tasks. The split > log manager manages all log files which need to be scanned and split. Then > the split log manager places all the logs into the ZooKeeper splitWAL node > (/hbase/splitWAL) as tasks and monitors these task nodes and waits for them > to be processed. Each regionserver watch splitWAL znode and grab task when > node children changed. And regionserver does the work to split the logs. > Open this umbrella issue to move this "coordinate" work to use new procedure > v2 framework and reduce zk depencency. Plan to finish this before 3.0 > release. Any suggestions are welcomed. Thanks. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21514) Refactor CacheConfig
[ https://issues.apache.org/jira/browse/HBASE-21514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16704553#comment-16704553 ] Guanghao Zhang commented on HBASE-21514: Add a 001 patch to see the HADOOP QA result... > Refactor CacheConfig > > > Key: HBASE-21514 > URL: https://issues.apache.org/jira/browse/HBASE-21514 > Project: HBase > Issue Type: Improvement >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-21514.master.001.patch > > > # move the global cache instances from CacheConfig. Only keep config stuff in > CacheConfig. > # Move block cache to HRegionServer's member variable. One rs has one block > cache. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21514) Refactor CacheConfig
[ https://issues.apache.org/jira/browse/HBASE-21514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-21514: --- Attachment: HBASE-21514.master.001.patch > Refactor CacheConfig > > > Key: HBASE-21514 > URL: https://issues.apache.org/jira/browse/HBASE-21514 > Project: HBase > Issue Type: Improvement >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Fix For: 3.0.0 > > Attachments: HBASE-21514.master.001.patch > > > # move the global cache instances from CacheConfig. Only keep config stuff in > CacheConfig. > # Move block cache to HRegionServer's member variable. One rs has one block > cache. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21504) If enable FIFOCompactionPolicy, a compaction may write a "empty" hfile whose maxTimeStamp is long max. This kind of hfile will never be archived.
[
https://issues.apache.org/jira/browse/HBASE-21504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16704532#comment-16704532
]
Hudson commented on HBASE-21504:
Results for branch branch-1.4
[build #568 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/568/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/568//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/568//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/568//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> If enable FIFOCompactionPolicy, a compaction may write a "empty" hfile whose
> maxTimeStamp is long max. This kind of hfile will never be archived.
> -
>
> Key: HBASE-21504
> URL: https://issues.apache.org/jira/browse/HBASE-21504
> Project: HBase
> Issue Type: Bug
> Components: Compaction
>Affects Versions: 2.1.0
>Reporter: xuming
>Assignee: Zheng Hu
>Priority: Critical
> Fix For: 3.0.0, 1.5.0, 1.3.3, 2.2.0, 1.4.9, 2.1.2, 2.0.4
>
> Attachments: 1.patch, HBASE-21504.v1.patch
>
>
> When i use FIFOCompactionPolicy, and if all hfiles(>1) are TTL expired in a
> region, once we do a compaction on the region, the compaction policy will
> select the latest hfile to do compaction.But beacuse the latest hfile is
> already TTL expired, compactor only write a "empty" hfile(whose entry counter
> is 0 and maxTimeStamp is Long.MAX_VALUE) finally. Because maxTimeStamp is
> long max, so the "empty" hfile will never be TTL expired, more seriously we
> can not archive it by FIFOCompactionPolicy forever.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21504) If enable FIFOCompactionPolicy, a compaction may write a "empty" hfile whose maxTimeStamp is long max. This kind of hfile will never be archived.
[
https://issues.apache.org/jira/browse/HBASE-21504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16704528#comment-16704528
]
Hudson commented on HBASE-21504:
Results for branch branch-1
[build #571 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/571/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/571//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/571//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/571//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
> If enable FIFOCompactionPolicy, a compaction may write a "empty" hfile whose
> maxTimeStamp is long max. This kind of hfile will never be archived.
> -
>
> Key: HBASE-21504
> URL: https://issues.apache.org/jira/browse/HBASE-21504
> Project: HBase
> Issue Type: Bug
> Components: Compaction
>Affects Versions: 2.1.0
>Reporter: xuming
>Assignee: Zheng Hu
>Priority: Critical
> Fix For: 3.0.0, 1.5.0, 1.3.3, 2.2.0, 1.4.9, 2.1.2, 2.0.4
>
> Attachments: 1.patch, HBASE-21504.v1.patch
>
>
> When i use FIFOCompactionPolicy, and if all hfiles(>1) are TTL expired in a
> region, once we do a compaction on the region, the compaction policy will
> select the latest hfile to do compaction.But beacuse the latest hfile is
> already TTL expired, compactor only write a "empty" hfile(whose entry counter
> is 0 and maxTimeStamp is Long.MAX_VALUE) finally. Because maxTimeStamp is
> long max, so the "empty" hfile will never be TTL expired, more seriously we
> can not archive it by FIFOCompactionPolicy forever.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21504) If enable FIFOCompactionPolicy, a compaction may write a "empty" hfile whose maxTimeStamp is long max. This kind of hfile will never be archived.
[
https://issues.apache.org/jira/browse/HBASE-21504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16704501#comment-16704501
]
Hudson commented on HBASE-21504:
Results for branch branch-1.2
[build #571 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/571/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/571//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/571//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.2/571//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 source release artifact{color}
-- Something went wrong with this stage, [check relevant console
output|${BUILD_URL}/console].
> If enable FIFOCompactionPolicy, a compaction may write a "empty" hfile whose
> maxTimeStamp is long max. This kind of hfile will never be archived.
> -
>
> Key: HBASE-21504
> URL: https://issues.apache.org/jira/browse/HBASE-21504
> Project: HBase
> Issue Type: Bug
> Components: Compaction
>Affects Versions: 2.1.0
>Reporter: xuming
>Assignee: Zheng Hu
>Priority: Critical
> Fix For: 3.0.0, 1.5.0, 1.3.3, 2.2.0, 1.4.9, 2.1.2, 2.0.4
>
> Attachments: 1.patch, HBASE-21504.v1.patch
>
>
> When i use FIFOCompactionPolicy, and if all hfiles(>1) are TTL expired in a
> region, once we do a compaction on the region, the compaction policy will
> select the latest hfile to do compaction.But beacuse the latest hfile is
> already TTL expired, compactor only write a "empty" hfile(whose entry counter
> is 0 and maxTimeStamp is Long.MAX_VALUE) finally. Because maxTimeStamp is
> long max, so the "empty" hfile will never be TTL expired, more seriously we
> can not archive it by FIFOCompactionPolicy forever.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21515) Also initialize an AsyncClusterConnection in HRegionServer
[
https://issues.apache.org/jira/browse/HBASE-21515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16704487#comment-16704487
]
Hadoop QA commented on HBASE-21515:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 20 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
24s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
10s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
10s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
23s{color} | {color:green} The patch passed checkstyle in hbase-common {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
33s{color} | {color:green} hbase-client: The patch generated 0 new + 3
unchanged - 3 fixed = 3 total (was 6) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
20s{color} | {color:green} hbase-server: The patch generated 0 new + 285
unchanged - 2 fixed = 285 total (was 287) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
11s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
9m 18s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
29s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
8s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}135m 18s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
57s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}191m 39s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.regionserver.TestMultiColumnScanner |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBAS
[jira] [Commented] (HBASE-21504) If enable FIFOCompactionPolicy, a compaction may write a "empty" hfile whose maxTimeStamp is long max. This kind of hfile will never be archived.
[
https://issues.apache.org/jira/browse/HBASE-21504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16704482#comment-16704482
]
Hudson commented on HBASE-21504:
Results for branch branch-1.3
[build #560 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/560/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/560//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/560//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/560//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> If enable FIFOCompactionPolicy, a compaction may write a "empty" hfile whose
> maxTimeStamp is long max. This kind of hfile will never be archived.
> -
>
> Key: HBASE-21504
> URL: https://issues.apache.org/jira/browse/HBASE-21504
> Project: HBase
> Issue Type: Bug
> Components: Compaction
>Affects Versions: 2.1.0
>Reporter: xuming
>Assignee: Zheng Hu
>Priority: Critical
> Fix For: 3.0.0, 1.5.0, 1.3.3, 2.2.0, 1.4.9, 2.1.2, 2.0.4
>
> Attachments: 1.patch, HBASE-21504.v1.patch
>
>
> When i use FIFOCompactionPolicy, and if all hfiles(>1) are TTL expired in a
> region, once we do a compaction on the region, the compaction policy will
> select the latest hfile to do compaction.But beacuse the latest hfile is
> already TTL expired, compactor only write a "empty" hfile(whose entry counter
> is 0 and maxTimeStamp is Long.MAX_VALUE) finally. Because maxTimeStamp is
> long max, so the "empty" hfile will never be TTL expired, more seriously we
> can not archive it by FIFOCompactionPolicy forever.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21048) Get LogLevel is not working from console in secure environment
[
https://issues.apache.org/jira/browse/HBASE-21048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16704473#comment-16704473
]
Chandra Sekhar commented on HBASE-21048:
Any progress there for this issue ?
> Get LogLevel is not working from console in secure environment
> --
>
> Key: HBASE-21048
> URL: https://issues.apache.org/jira/browse/HBASE-21048
> Project: HBase
> Issue Type: Bug
>Reporter: Chandra Sekhar
>Priority: Major
>
> When we try to get log level of specific package in secure environment,
> getting SocketException.
> {code:java}
> hbase/master/bin# ./hbase org.apache.hadoop.hbase.http.log.LogLevel -getlevel
> host-:16010 org.apache.hadoop.hbase
> Connecting to http://host-:16010/logLevel?log=org.apache.hadoop.hbase
> java.net.SocketException: Unexpected end of file from server
> {code}
> It is trying to connect http instead of https
> code snippet that handling only http in *LogLevel.java*
> {code:java}
> public static void main(String[] args) {
> if (args.length == 3 && "-getlevel".equals(args[0])) {
> process("http://"; + args[1] + "/logLevel?log=" + args[2]);
> return;
> }
> else if (args.length == 4 && "-setlevel".equals(args[0])) {
> process("http://"; + args[1] + "/logLevel?log=" + args[2]
> + "&level=" + args[3]);
> return;
> }
> System.err.println(USAGES);
> System.exit(-1);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21413) Empty meta log doesn't get split when restart whole cluster
[ https://issues.apache.org/jira/browse/HBASE-21413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allan Yang updated HBASE-21413: --- Attachment: HBASE-21413.branch-2.1.001.patch > Empty meta log doesn't get split when restart whole cluster > --- > > Key: HBASE-21413 > URL: https://issues.apache.org/jira/browse/HBASE-21413 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.1.1, 2.0.2 >Reporter: Jingyun Tian >Assignee: Allan Yang >Priority: Major > Fix For: 3.0.0, 2.1.2, 2.0.4 > > Attachments: HBASE-21413.branch-2.1.001.patch, Screenshot from > 2018-10-31 18-11-02.png, Screenshot from 2018-10-31 18-11-11.png > > > After I restart whole cluster, there is a splitting directory still exists on > hdfs. Then I found there is only an empty meta wal file in it. I'll dig into > this later. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21413) Empty meta log doesn't get split when restart whole cluster
[ https://issues.apache.org/jira/browse/HBASE-21413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allan Yang updated HBASE-21413: --- Fix Version/s: 2.0.4 2.1.2 3.0.0 Affects Version/s: 2.1.1 2.0.2 Status: Patch Available (was: Open) > Empty meta log doesn't get split when restart whole cluster > --- > > Key: HBASE-21413 > URL: https://issues.apache.org/jira/browse/HBASE-21413 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.2, 2.1.1 >Reporter: Jingyun Tian >Assignee: Allan Yang >Priority: Major > Fix For: 3.0.0, 2.1.2, 2.0.4 > > Attachments: HBASE-21413.branch-2.1.001.patch, Screenshot from > 2018-10-31 18-11-02.png, Screenshot from 2018-10-31 18-11-11.png > > > After I restart whole cluster, there is a splitting directory still exists on > hdfs. Then I found there is only an empty meta wal file in it. I'll dig into > this later. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21518) TestMasterFailoverWithProcedures is flaky
[
https://issues.apache.org/jira/browse/HBASE-21518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Somogyi updated HBASE-21518:
--
Resolution: Fixed
Fix Version/s: 2.1.2
2.2.0
3.0.0
Status: Resolved (was: Patch Available)
> TestMasterFailoverWithProcedures is flaky
> -
>
> Key: HBASE-21518
> URL: https://issues.apache.org/jira/browse/HBASE-21518
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.2.0, 2.0.3, 2.1.2
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.2
>
> Attachments: HBASE-21518-v1.patch, output.txt
>
>
> TestMasterFailoverWithProcedures test is failing frequently, times out. I
> faced this failure on 2.0.3RC0 vote and it also appears on multiple flaky
> dashboards.
> branch-2:
> [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Flaky-Tests/job/branch-2/2007/]
> branch-2.1:
> [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Flaky-Tests/job/branch-2.1/2002/]
> branch-2.0:
> [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Flaky-Tests/job/branch-2.0/1988/]
>
> {noformat}
> [INFO] Running
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures
> [ERROR] Tests run: 4, Failures: 0, Errors: 2, Skipped: 0, Time elapsed:
> 780.648 s <<< FAILURE! - in
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures
> [ERROR]
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures
> Time elapsed: 749.024 s <<< ERROR!
> org.junit.runners.model.TestTimedOutException: test timed out after 780
> seconds
> at
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures.tearDown(TestMasterFailoverWithProcedures.java:86)
> [ERROR]
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures
> Time elapsed: 749.051 s <<< ERROR!
> java.lang.Exception: Appears to be stuck in thread RS-EventLoopGroup-3-2
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21518) TestMasterFailoverWithProcedures is flaky
[
https://issues.apache.org/jira/browse/HBASE-21518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16704397#comment-16704397
]
Peter Somogyi commented on HBASE-21518:
---
Pushed to brach-2.1, branch-2 and master. [~stack] let me know if you'd like
this on branch-2.0.
> TestMasterFailoverWithProcedures is flaky
> -
>
> Key: HBASE-21518
> URL: https://issues.apache.org/jira/browse/HBASE-21518
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.2.0, 2.0.3, 2.1.2
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Major
> Attachments: HBASE-21518-v1.patch, output.txt
>
>
> TestMasterFailoverWithProcedures test is failing frequently, times out. I
> faced this failure on 2.0.3RC0 vote and it also appears on multiple flaky
> dashboards.
> branch-2:
> [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Flaky-Tests/job/branch-2/2007/]
> branch-2.1:
> [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Flaky-Tests/job/branch-2.1/2002/]
> branch-2.0:
> [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Flaky-Tests/job/branch-2.0/1988/]
>
> {noformat}
> [INFO] Running
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures
> [ERROR] Tests run: 4, Failures: 0, Errors: 2, Skipped: 0, Time elapsed:
> 780.648 s <<< FAILURE! - in
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures
> [ERROR]
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures
> Time elapsed: 749.024 s <<< ERROR!
> org.junit.runners.model.TestTimedOutException: test timed out after 780
> seconds
> at
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures.tearDown(TestMasterFailoverWithProcedures.java:86)
> [ERROR]
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures
> Time elapsed: 749.051 s <<< ERROR!
> java.lang.Exception: Appears to be stuck in thread RS-EventLoopGroup-3-2
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
