[jira] [Updated] (HBASE-21465) Retry on reportRegionStateTransition can lead to unexpected errors
[ https://issues.apache.org/jira/browse/HBASE-21465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21465: -- Resolution: Fixed Status: Resolved (was: Patch Available) HBASE-21534 will take care of the flaky TestAssignmentManager related UTs. > Retry on reportRegionStateTransition can lead to unexpected errors > -- > > Key: HBASE-21465 > URL: https://issues.apache.org/jira/browse/HBASE-21465 > Project: HBase > Issue Type: Sub-task > Components: amv2 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: HBASE-21465-UT.patch, HBASE-21465-v1.patch, > HBASE-21465-v2.patch, HBASE-21465-v3.patch, HBASE-21465-v4.patch, > HBASE-21465.patch > > > It is possible that the reportRegionStateTransition method is succeeded at > master side, but before returning the the region server, the rpc connection > is broken, or the master restarts. So next when the region server try > again,we will find that the state for the region and the TRSP is not correct, > and can lead to a RS abort or something even worse. > We should be able to determine whether a reportRegionStateTransition call is > just a retry and has already been succeeded, and just ignore it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21512) Introduce an AsyncClusterConnection and replace the usage of ClusterConnection
[ https://issues.apache.org/jira/browse/HBASE-21512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704362#comment-16704362 ] Duo Zhang commented on HBASE-21512: --- Let me open a HBASE-21512 to land HBASE-21515. > Introduce an AsyncClusterConnection and replace the usage of ClusterConnection > -- > > Key: HBASE-21512 > URL: https://issues.apache.org/jira/browse/HBASE-21512 > Project: HBase > Issue Type: Umbrella >Reporter: Duo Zhang >Priority: Major > Fix For: 3.0.0 > > > At least for the RSProcedureDispatcher, with CompletableFuture we do not need > to set a delay and use a thread pool any more, which could reduce the > resource usage and also the latency. > Once this is done, I think we can remove the ClusterConnection completely, > and start to rewrite the old sync client based on the async client, which > could reduce the code base a lot for our client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21534) TestAssignmentManager is flakey
[
https://issues.apache.org/jira/browse/HBASE-21534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-21534:
--
Attachment: HBASE-21534.patch
> TestAssignmentManager is flakey
> ---
>
> Key: HBASE-21534
> URL: https://issues.apache.org/jira/browse/HBASE-21534
> Project: HBase
> Issue Type: Task
>Reporter: Duo Zhang
>Priority: Major
> Attachments: HBASE-21534.patch
>
>
> See this in the outout and then the test hang
> {noformat}
> 2018-11-29 20:47:50,061 WARN [MockRSProcedureDispatcher-pool5-t10]
> assignment.AssignmentManager(894): The region server localhost,102,1 is
> already dead, skip reportRegionStateTransition call
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21518) TestMasterFailoverWithProcedures is flaky
[
https://issues.apache.org/jira/browse/HBASE-21518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704343#comment-16704343
]
Sean Busbey commented on HBASE-21518:
-
Yes, any UT that has multiple masters and knocks one over.
> TestMasterFailoverWithProcedures is flaky
> -
>
> Key: HBASE-21518
> URL: https://issues.apache.org/jira/browse/HBASE-21518
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.2.0, 2.0.3, 2.1.2
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Major
> Attachments: HBASE-21518-v1.patch, output.txt
>
>
> TestMasterFailoverWithProcedures test is failing frequently, times out. I
> faced this failure on 2.0.3RC0 vote and it also appears on multiple flaky
> dashboards.
> branch-2:
> [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Flaky-Tests/job/branch-2/2007/]
> branch-2.1:
> [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Flaky-Tests/job/branch-2.1/2002/]
> branch-2.0:
> [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Flaky-Tests/job/branch-2.0/1988/]
>
> {noformat}
> [INFO] Running
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures
> [ERROR] Tests run: 4, Failures: 0, Errors: 2, Skipped: 0, Time elapsed:
> 780.648 s <<< FAILURE! - in
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures
> [ERROR]
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures
> Time elapsed: 749.024 s <<< ERROR!
> org.junit.runners.model.TestTimedOutException: test timed out after 780
> seconds
> at
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures.tearDown(TestMasterFailoverWithProcedures.java:86)
> [ERROR]
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures
> Time elapsed: 749.051 s <<< ERROR!
> java.lang.Exception: Appears to be stuck in thread RS-EventLoopGroup-3-2
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21534) TestAssignmentManager is flakey
[
https://issues.apache.org/jira/browse/HBASE-21534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704339#comment-16704339
]
Duo Zhang commented on HBASE-21534:
---
Look at the log, it seems that we assign meta to the server which is already
marked as dead. This should be a test issue, let me provide a fix.
{noformat}
2018-11-29 20:47:49,751 INFO [PEWorker-63]
procedure.ServerCrashProcedure(131): Start pid=14,
state=RUNNABLE:SERVER_CRASH_START, hasLock=true; ServerCrashProcedure
server=localhost,102,1, splitWal=false, meta=true
2018-11-29 20:47:49,751 DEBUG [PEWorker-63] procedure2.RootProcedureState(153):
Add procedure pid=14, state=RUNNABLE:SERVER_CRASH_SPLIT_META_LOGS,
hasLock=true; ServerCrashProcedure server=localhost,102,1, splitWal=false,
meta=true as the 0th rollback step
2018-11-29 20:47:49,751 DEBUG [PEWorker-63]
procedure.MasterProcedureScheduler(351): Add ServerQueue(localhost,102,1,
xlock=false sharedLock=0 size=0) to run queue because: pid=14,
state=RUNNABLE:SERVER_CRASH_SPLIT_META_LOGS, hasLock=false;
ServerCrashProcedure server=localhost,102,1, splitWal=false, meta=true released
exclusive lock
2018-11-29 20:47:49,751 DEBUG [PEWorker-63]
procedure.MasterProcedureScheduler(351): Add ServerQueue(localhost,102,1,
xlock=false sharedLock=0 size=1) to run queue because: the exclusive lock is
not held by anyone when adding pid=14,
state=RUNNABLE:SERVER_CRASH_SPLIT_META_LOGS, hasLock=false;
ServerCrashProcedure server=localhost,102,1, splitWal=false, meta=true
2018-11-29 20:47:49,751 DEBUG [PEWorker-63]
procedure.MasterProcedureScheduler(361): Remove ServerQueue(localhost,102,1,
xlock=false sharedLock=0 size=0) from run queue because: queue is empty after
polling out pid=14, state=RUNNABLE:SERVER_CRASH_SPLIT_META_LOGS, hasLock=false;
ServerCrashProcedure server=localhost,102,1, splitWal=false, meta=true
2018-11-29 20:47:49,751 DEBUG [PEWorker-63]
procedure.MasterProcedureScheduler(361): Remove ServerQueue(localhost,102,1,
xlock=true (14) sharedLock=0 size=0) from run queue because: pid=14,
state=RUNNABLE:SERVER_CRASH_SPLIT_META_LOGS, hasLock=false;
ServerCrashProcedure server=localhost,102,1, splitWal=false, meta=true held
exclusive lock
2018-11-29 20:47:49,751 DEBUG [PEWorker-63]
procedure.ServerCrashProcedure(207): Splitting meta WALs pid=14,
state=RUNNABLE:SERVER_CRASH_SPLIT_META_LOGS, hasLock=true; ServerCrashProcedure
server=localhost,102,1, splitWal=false, meta=true
2018-11-29 20:47:49,755 INFO [PEWorker-63] master.MasterWalManager(314): Log
dir for server localhost,102,1 does not exist
2018-11-29 20:47:49,755 INFO [PEWorker-63] master.SplitLogManager(465): dead
splitlog workers [localhost,102,1]
2018-11-29 20:47:49,755 INFO [PEWorker-63] master.SplitLogManager(297):
Finished splitting (more than or equal to) 0 bytes in 0 log files in [] in 0ms
2018-11-29 20:47:49,756 DEBUG [PEWorker-63]
procedure.ServerCrashProcedure(213): Done splitting meta WALs pid=14,
state=RUNNABLE:SERVER_CRASH_SPLIT_META_LOGS, hasLock=true; ServerCrashProcedure
server=localhost,102,1, splitWal=false, meta=true
2018-11-29 20:47:49,756 DEBUG [PEWorker-63] procedure2.RootProcedureState(153):
Add procedure pid=14, state=RUNNABLE:SERVER_CRASH_ASSIGN_META, hasLock=true;
ServerCrashProcedure server=localhost,102,1, splitWal=false, meta=true as the
1th rollback step
2018-11-29 20:47:49,756 DEBUG [PEWorker-63]
procedure.MasterProcedureScheduler(351): Add ServerQueue(localhost,102,1,
xlock=false sharedLock=0 size=0) to run queue because: pid=14,
state=RUNNABLE:SERVER_CRASH_ASSIGN_META, hasLock=false; ServerCrashProcedure
server=localhost,102,1, splitWal=false, meta=true released exclusive lock
2018-11-29 20:47:49,756 DEBUG [PEWorker-63]
procedure.MasterProcedureScheduler(351): Add ServerQueue(localhost,102,1,
xlock=false sharedLock=0 size=1) to run queue because: the exclusive lock is
not held by anyone when adding pid=14, state=RUNNABLE:SERVER_CRASH_ASSIGN_META,
hasLock=false; ServerCrashProcedure server=localhost,102,1, splitWal=false,
meta=true
2018-11-29 20:47:49,756 DEBUG [PEWorker-63]
procedure.MasterProcedureScheduler(361): Remove ServerQueue(localhost,102,1,
xlock=false sharedLock=0 size=0) from run queue because: queue is empty after
polling out pid=14, state=RUNNABLE:SERVER_CRASH_ASSIGN_META, hasLock=false;
ServerCrashProcedure server=localhost,102,1, splitWal=false, meta=true
2018-11-29 20:47:49,756 DEBUG [PEWorker-63]
procedure.MasterProcedureScheduler(361): Remove ServerQueue(localhost,102,1,
xlock=true (14) sharedLock=0 size=0) from run queue because: pid=14,
state=RUNNABLE:SERVER_CRASH_ASSIGN_META, hasLock=false; ServerCrashProcedure
server=localhost,102,1, splitWal=false, meta=true held exclusive lock
2018-11-29 20:47:49,758 INFO [PEWorker-63] procedure2.ProcedureExecutor(1680):
Initialized subprocedures=[{pid=15, ppid=14,
state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE,
[jira] [Created] (HBASE-21534) TestAssignmentManager is flakey
Duo Zhang created HBASE-21534:
-
Summary: TestAssignmentManager is flakey
Key: HBASE-21534
URL: https://issues.apache.org/jira/browse/HBASE-21534
Project: HBase
Issue Type: Task
Reporter: Duo Zhang
See this in the outout and then the test hang
{noformat}
2018-11-29 20:47:50,061 WARN [MockRSProcedureDispatcher-pool5-t10]
assignment.AssignmentManager(894): The region server localhost,102,1 is already
dead, skip reportRegionStateTransition call
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21518) TestMasterFailoverWithProcedures is flaky
[
https://issues.apache.org/jira/browse/HBASE-21518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704319#comment-16704319
]
Allan Yang commented on HBASE-21518:
So, other UTs can be flaky because of this too?
> TestMasterFailoverWithProcedures is flaky
> -
>
> Key: HBASE-21518
> URL: https://issues.apache.org/jira/browse/HBASE-21518
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.2.0, 2.0.3, 2.1.2
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Major
> Attachments: HBASE-21518-v1.patch, output.txt
>
>
> TestMasterFailoverWithProcedures test is failing frequently, times out. I
> faced this failure on 2.0.3RC0 vote and it also appears on multiple flaky
> dashboards.
> branch-2:
> [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Flaky-Tests/job/branch-2/2007/]
> branch-2.1:
> [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Flaky-Tests/job/branch-2.1/2002/]
> branch-2.0:
> [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Flaky-Tests/job/branch-2.0/1988/]
>
> {noformat}
> [INFO] Running
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures
> [ERROR] Tests run: 4, Failures: 0, Errors: 2, Skipped: 0, Time elapsed:
> 780.648 s <<< FAILURE! - in
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures
> [ERROR]
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures
> Time elapsed: 749.024 s <<< ERROR!
> org.junit.runners.model.TestTimedOutException: test timed out after 780
> seconds
> at
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures.tearDown(TestMasterFailoverWithProcedures.java:86)
> [ERROR]
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures
> Time elapsed: 749.051 s <<< ERROR!
> java.lang.Exception: Appears to be stuck in thread RS-EventLoopGroup-3-2
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21515) Also initialize an AsyncClusterConnection in HRegionServer
[ https://issues.apache.org/jira/browse/HBASE-21515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21515: -- Attachment: HBASE-21515-v4.patch > Also initialize an AsyncClusterConnection in HRegionServer > -- > > Key: HBASE-21515 > URL: https://issues.apache.org/jira/browse/HBASE-21515 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Attachments: HBASE-21515-v1.patch, HBASE-21515-v2.patch, > HBASE-21515-v3.patch, HBASE-21515-v4.patch, HBASE-21515-v4.patch, > HBASE-21515.patch > > > Plan to do this incrementally, so first we will only introduce the new > AsyncClusterConnection, without deleting the old ClusterConnection. And we > can move the code which uses the ClusterConnection to use > AsyncClusterConnection, through different sub tasks here. And finally we can > remove the ClusterConnection completely. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21083) Introduce a mechanism to bypass the execution of a stuck procedure
[ https://issues.apache.org/jira/browse/HBASE-21083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704316#comment-16704316 ] Allan Yang commented on HBASE-21083: And besides, bypassing a procedure in the middle without fix the inconsistency is dangerous. We don't want to expose this method to user themself. > Introduce a mechanism to bypass the execution of a stuck procedure > -- > > Key: HBASE-21083 > URL: https://issues.apache.org/jira/browse/HBASE-21083 > Project: HBase > Issue Type: Sub-task > Components: amv2 >Affects Versions: 2.1.0, 2.0.1 >Reporter: Allan Yang >Assignee: Allan Yang >Priority: Major > Fix For: 3.0.0, 2.1.1, 2.0.2 > > Attachments: HBASE-21083.branch-2.0.001.patch, > HBASE-21083.branch-2.0.002.patch, HBASE-21083.branch-2.0.003.patch, > HBASE-21083.branch-2.0.003.patch, HBASE-21083.branch-2.0.003.patch, > HBASE-21083.branch-2.1.001.patch > > > Offline discussed with [~stack] and [~Apache9]. We all agreed that we need to > introduce a mechanism to 'force complete' a stuck procedure, so the AMv2 can > continue running. > we still have some unrevealed bugs hiding in our AMv2 and procedureV2 system, > we need something to interfere with stuck procedures before HBCK2 can work. > This is very crucial for a production ready system. > For now, we have little ways to interfere with running procedures. Aborting > them is not a good choice, since some procedures are not abort-able. And some > procedure may have overridden the abort() method, which will ignore the abort > request. > So, here, I will introduce a mechanism to bypass the execution of a stuck > procedure. > Basically, I added a field called 'bypass' to Procedure class. If we set this > field to true, all the logic in execute/rollback will be skipped, letting > this procedure and its ancestors complete normally and releasing the lock > resources at last. > Notice that bypassing a procedure may leave the cluster in a middle state, > e.g. the region not assigned, or some hdfs files left behind. > The Operators need know the side effect of bypassing and recover the > inconsistent state of the cluster themselves, like issuing new procedures to > assign the regions. > A patch will be uploaded and review board will be open. For now, only APIs in > ProcedureExecutor are provided. If anything is fine, I will add it to master > service and add a shell command to bypass a procedure. Or, maybe we can use > dynamically compiled JSPs to execute those APIs as mentioned in HBASE-20679. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21531) race between region report and region move causes master to kill RS
[
https://issues.apache.org/jira/browse/HBASE-21531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704315#comment-16704315
]
Duo Zhang commented on HBASE-21531:
---
Anyway if you think the region server could have incorrect state then the most
safe way is to kill the region server...
But since there will be race which leads to a warning in the code but actually
there is nothing wrong, then I do not think we should try to do any 'fix'...
> race between region report and region move causes master to kill RS
> ---
>
> Key: HBASE-21531
> URL: https://issues.apache.org/jira/browse/HBASE-21531
> Project: HBase
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Priority: Major
>
> In this case the delay between the ack from RS and the region report was
> 1.5s, so I'm not sure what caused the race (network hiccup? unreported retry
> by protobuf transport?) but in any case I don't see anything that prevents
> this from happening in a normal case, with a narrowed time window. Any delay
> (e.g. a GC pause on RS right after the report is built, and ack is sent for
> the close) or retries expands the window.
> Master starts moving the region and the source RS acks by 21:51,206
> {noformat}
> Master:
> 2018-11-21 21:21:49,024 INFO [master/6:17000.Chore.1] master.HMaster:
> balance hri=, source=,17020,1542754626176, destination= 2>,17020,1542863268158
> ...
> Server:
> 2018-11-21 21:21:49,394 INFO [RS_CLOSE_REGION-regionserver/:17020-1]
> handler.UnassignRegionHandler: Close
> ...
> 2018-11-21 21:21:51,095 INFO [RS_CLOSE_REGION-regionserver/:17020-1]
> handler.UnassignRegionHandler: Closed
> {noformat}
> By then the region is removed from onlineRegions, so the master proceeds.
> {noformat}
> 2018-11-21 21:21:51,206 INFO [PEWorker-4] procedure2.ProcedureExecutor:
> Finished subprocedure(s) of pid=667,
> state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED, hasLock=true;
> TransitRegionStateProcedure table=, region=, REOPEN/MOVE;
> resume parent processing.
> 2018-11-21 21:21:51,386 INFO [PEWorker-13] assignment.RegionStateStore:
> pid=667 updating hbase:meta row=, regionState=OPENING,
> regionLocation=,17020,1542863268158
> {noformat}
> There are no obvious errors/delays that I see in RS log, and it doesn't log
> starting to send the report.
> However, at 21:52.853 the report is processed that still contains this region.
> {noformat}
> 2018-11-21 21:21:52,853 WARN
> [RpcServer.default.FPBQ.Fifo.handler=48,queue=3,port=17000]
> assignment.AssignmentManager: Killing ,17020,1542754626176:
> rit=OPENING, location=,17020,1542863268158, table=,
> region= reported OPEN on server=,17020,1542754626176 but
> state has otherwise.
> * ABORTING region server ,17020,1542754626176:
> org.apache.hadoop.hbase.YouAreDeadException: rit=OPENING, location= 2>,17020,1542863268158, table=, region= reported OPEN on
> server=,17020,1542754626176 but state has otherwise.
> {noformat}
> RS shuts down in an orderly manner and it can be seen from the log that this
> region is actually not present (there's no line indicating it's being closed,
> unlike for other regions).
> I think there needs to be some sort of versioning for region operations
> and/or in RS reports to allow master to account for concurrent operations and
> avoid races. Probably per region with either a grace period or an additional
> global version, so that master could avoid killing RS based on stale reports,
> but still kill RS if it did retain an old version of the region state due to
> some bug after acking a new version.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21083) Introduce a mechanism to bypass the execution of a stuck procedure
[ https://issues.apache.org/jira/browse/HBASE-21083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704314#comment-16704314 ] Allan Yang commented on HBASE-21083: [~xucang], It should not be very hard to back port this patch, but this feature is only used by HBCK2 in 2.x, and it does not provide a Admin API(over RPC) or something else, it can't be directly called by shell or HBaseAdmin class. > Introduce a mechanism to bypass the execution of a stuck procedure > -- > > Key: HBASE-21083 > URL: https://issues.apache.org/jira/browse/HBASE-21083 > Project: HBase > Issue Type: Sub-task > Components: amv2 >Affects Versions: 2.1.0, 2.0.1 >Reporter: Allan Yang >Assignee: Allan Yang >Priority: Major > Fix For: 3.0.0, 2.1.1, 2.0.2 > > Attachments: HBASE-21083.branch-2.0.001.patch, > HBASE-21083.branch-2.0.002.patch, HBASE-21083.branch-2.0.003.patch, > HBASE-21083.branch-2.0.003.patch, HBASE-21083.branch-2.0.003.patch, > HBASE-21083.branch-2.1.001.patch > > > Offline discussed with [~stack] and [~Apache9]. We all agreed that we need to > introduce a mechanism to 'force complete' a stuck procedure, so the AMv2 can > continue running. > we still have some unrevealed bugs hiding in our AMv2 and procedureV2 system, > we need something to interfere with stuck procedures before HBCK2 can work. > This is very crucial for a production ready system. > For now, we have little ways to interfere with running procedures. Aborting > them is not a good choice, since some procedures are not abort-able. And some > procedure may have overridden the abort() method, which will ignore the abort > request. > So, here, I will introduce a mechanism to bypass the execution of a stuck > procedure. > Basically, I added a field called 'bypass' to Procedure class. If we set this > field to true, all the logic in execute/rollback will be skipped, letting > this procedure and its ancestors complete normally and releasing the lock > resources at last. > Notice that bypassing a procedure may leave the cluster in a middle state, > e.g. the region not assigned, or some hdfs files left behind. > The Operators need know the side effect of bypassing and recover the > inconsistent state of the cluster themselves, like issuing new procedures to > assign the regions. > A patch will be uploaded and review board will be open. For now, only APIs in > ProcedureExecutor are provided. If anything is fine, I will add it to master > service and add a shell command to bypass a procedure. Or, maybe we can use > dynamically compiled JSPs to execute those APIs as mentioned in HBASE-20679. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21524) Unnecessary DEBUG log in ConnectionImplementation#isTableEnabled
[
https://issues.apache.org/jira/browse/HBASE-21524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704241#comment-16704241
]
Hudson commented on HBASE-21524:
Results for branch branch-2.1
[build #646 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/646/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/646//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/646//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/646//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Unnecessary DEBUG log in ConnectionImplementation#isTableEnabled
>
>
> Key: HBASE-21524
> URL: https://issues.apache.org/jira/browse/HBASE-21524
> Project: HBase
> Issue Type: Improvement
> Components: Client
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.2
>
> Attachments: HBASE-21524.001.branch-2.0.patch
>
>
> Too much crap coming out of isTableEnabled for "normal" situations. Don't
> need a DEBUG message to tell us that the table is available.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-18735) Provide a fast mechanism for shutting down mini cluster
[
https://issues.apache.org/jira/browse/HBASE-18735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704242#comment-16704242
]
Hudson commented on HBASE-18735:
Results for branch branch-2.1
[build #646 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/646/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/646//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/646//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/646//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Provide a fast mechanism for shutting down mini cluster
> ---
>
> Key: HBASE-18735
> URL: https://issues.apache.org/jira/browse/HBASE-18735
> Project: HBase
> Issue Type: Wish
>Reporter: Samarth Jain
>Assignee: Artem Ervits
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.0.3, 2.1.2
>
> Attachments: HBASE-18735.v01.patch, HBASE-18735.v02.patch,
> HBASE-18735.v03.patch
>
>
> The current mechanism of shutting down a mini cluster through
> HBaseTestingUtility.shutDownMiniCluster can take a lot of time when the mini
> cluster almost has a lot of tables. A lot of this time is spent in closing
> all the user regions. It would be nice to have a mechanism where this
> shutdown can happen quickly without having to worry about closing these user
> regions. At the same time, this mechanism would need to make sure that all
> the critical system resources like file handles and network ports are still
> released so that subsequently initialized mini clusters on the same JVM or
> system won't run into resource issues. This would make testing using HBase
> mini clusters much faster and immensely help out test frameworks of dependent
> projects like Phoenix.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-18735) Provide a fast mechanism for shutting down mini cluster
[
https://issues.apache.org/jira/browse/HBASE-18735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704240#comment-16704240
]
Hudson commented on HBASE-18735:
Results for branch branch-2.0
[build #1124 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1124/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1124//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1124//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1124//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Provide a fast mechanism for shutting down mini cluster
> ---
>
> Key: HBASE-18735
> URL: https://issues.apache.org/jira/browse/HBASE-18735
> Project: HBase
> Issue Type: Wish
>Reporter: Samarth Jain
>Assignee: Artem Ervits
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.0.3, 2.1.2
>
> Attachments: HBASE-18735.v01.patch, HBASE-18735.v02.patch,
> HBASE-18735.v03.patch
>
>
> The current mechanism of shutting down a mini cluster through
> HBaseTestingUtility.shutDownMiniCluster can take a lot of time when the mini
> cluster almost has a lot of tables. A lot of this time is spent in closing
> all the user regions. It would be nice to have a mechanism where this
> shutdown can happen quickly without having to worry about closing these user
> regions. At the same time, this mechanism would need to make sure that all
> the critical system resources like file handles and network ports are still
> released so that subsequently initialized mini clusters on the same JVM or
> system won't run into resource issues. This would make testing using HBase
> mini clusters much faster and immensely help out test frameworks of dependent
> projects like Phoenix.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21530) Abort_Procedure should be able to take a list of proc IDs
[ https://issues.apache.org/jira/browse/HBASE-21530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704235#comment-16704235 ] Geoffrey Jacoby commented on HBASE-21530: - I'm using a variant of HBase 1.3. If abort procedure has been deprecated later on, I disagree with that decision. I don't think you should ever have something which: 1. Ties up a finite resource on the cluster 2. Doesn't have a timeout and 3. Can't be canceled. > Abort_Procedure should be able to take a list of proc IDs > - > > Key: HBASE-21530 > URL: https://issues.apache.org/jira/browse/HBASE-21530 > Project: HBase > Issue Type: Improvement >Reporter: Geoffrey Jacoby >Priority: Minor > > As a convenience, it would be helpful if the HBase shell's abort_procedure > call had the option of taking in multiple procedure ids at the same time, > rather than relying on operators to use a loop in an external script. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21524) Unnecessary DEBUG log in ConnectionImplementation#isTableEnabled
[
https://issues.apache.org/jira/browse/HBASE-21524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704239#comment-16704239
]
Hudson commented on HBASE-21524:
Results for branch branch-2.0
[build #1124 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1124/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1124//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1124//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1124//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Unnecessary DEBUG log in ConnectionImplementation#isTableEnabled
>
>
> Key: HBASE-21524
> URL: https://issues.apache.org/jira/browse/HBASE-21524
> Project: HBase
> Issue Type: Improvement
> Components: Client
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.2
>
> Attachments: HBASE-21524.001.branch-2.0.patch
>
>
> Too much crap coming out of isTableEnabled for "normal" situations. Don't
> need a DEBUG message to tell us that the table is available.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21530) Abort_Procedure should be able to take a list of proc IDs
[ https://issues.apache.org/jira/browse/HBASE-21530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704237#comment-16704237 ] Geoffrey Jacoby commented on HBASE-21530: - In addition, even if aborting a running procedure is dangerous, aborting a queued (RUNNABLE) procedure should always be OK. > Abort_Procedure should be able to take a list of proc IDs > - > > Key: HBASE-21530 > URL: https://issues.apache.org/jira/browse/HBASE-21530 > Project: HBase > Issue Type: Improvement >Reporter: Geoffrey Jacoby >Priority: Minor > > As a convenience, it would be helpful if the HBase shell's abort_procedure > call had the option of taking in multiple procedure ids at the same time, > rather than relying on operators to use a loop in an external script. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-18222) Tests fail as lock on table is not released
[ https://issues.apache.org/jira/browse/HBASE-18222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704224#comment-16704224 ] Sakthi commented on HBASE-18222: [~anuphal] Are you still able to repro this? I tried, but couldn't repro. > Tests fail as lock on table is not released > > > Key: HBASE-18222 > URL: https://issues.apache.org/jira/browse/HBASE-18222 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0 > Environment: [INFO] > > [INFO] Detecting the operating system and CPU architecture > [INFO] > > [INFO] os.detected.name: linux > [INFO] os.detected.arch: ppcle_64 > [INFO] os.detected.version: 3.16 > [INFO] os.detected.version.major: 3 > [INFO] os.detected.version.minor: 16 > [INFO] os.detected.release: ubuntu > [INFO] os.detected.release.version: 14.04 > [INFO] os.detected.release.like.ubuntu: true > [INFO] os.detected.release.like.debian: true > [INFO] os.detected.classifier: linux-ppcle_64 >Reporter: Anup Halarnkar >Priority: Major > Fix For: 3.0.0 > > > *All Failed Tests* > 1. > org.apache.hadoop.hbase.coprocessor.TestCoprocessorMetrics.testRegionObserverAfterRegionClosed > _org.apache.hadoop.hbase.exceptions.TimeoutIOException: > java.util.concurrent.TimeoutException: The procedure 14 is still running > at > org.apache.hadoop.hbase.client.HBaseAdmin$ProcedureFuture.waitProcedureResult(HBaseAdmin.java:3418) > at > org.apache.hadoop.hbase.client.HBaseAdmin$ProcedureFuture.get(HBaseAdmin.java:3339) > at org.apache.hadoop.hbase.client.HBaseAdmin.get(HBaseAdmin.java:1962) > at > org.apache.hadoop.hbase.client.HBaseAdmin.disableTable(HBaseAdmin.java:764) > at > org.apache.hadoop.hbase.coprocessor.TestCoprocessorMetrics.testRegionObserverAfterRegionClosed(TestCoprocessorMetrics.java:487) > _ > 2.org.apache.hadoop.hbase.coprocessor.TestCoprocessorMetrics.testMasterObserver > 3.org.apache.hadoop.hbase.coprocessor.TestCoprocessorMetrics.testRegionObserverEndpoint > 4.org.apache.hadoop.hbase.coprocessor.TestCoprocessorMetrics.testRegionObserverMultiRegion > 5.org.apache.hadoop.hbase.coprocessor.TestCoprocessorMetrics.testRegionObserverSingleRegion > 6.org.apache.hadoop.hbase.coprocessor.TestCoprocessorMetrics.testWALObserver > 7.org.apache.hadoop.hbase.coprocessor.TestCoprocessorMetrics.testRegionServerObserver > 8.org.apache.hadoop.hbase.coprocessor.TestCoprocessorMetrics.testRegionObserverMultiTable > *-* > *Stack Traces of all tests from 2 -8 is given below* > org.apache.hadoop.hbase.TableNotDisabledException: > testRegionObserverAfterRegionClosed > at > org.apache.hadoop.hbase.master.HMaster.checkTableModifiable(HMaster.java:2470) > at > org.apache.hadoop.hbase.master.procedure.DeleteTableProcedure.prepareDelete(DeleteTableProcedure.java:241) > at > org.apache.hadoop.hbase.master.procedure.DeleteTableProcedure.executeFromState(DeleteTableProcedure.java:91) > at > org.apache.hadoop.hbase.master.procedure.DeleteTableProcedure.executeFromState(DeleteTableProcedure.java:58) > at > org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:155) > at > org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:843) > at > org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1373) > at > org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1142) > at > org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:78) > at > org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1651) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21515) Also initialize an AsyncClusterConnection in HRegionServer
[
https://issues.apache.org/jira/browse/HBASE-21515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704214#comment-16704214
]
Hadoop QA commented on HBASE-21515:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
12s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 20 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
25s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
51s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
47s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
8s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s{color} | {color:green} The patch passed checkstyle in hbase-common {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
31s{color} | {color:green} hbase-client: The patch generated 0 new + 3
unchanged - 3 fixed = 3 total (was 6) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
10s{color} | {color:green} hbase-server: The patch generated 0 new + 285
unchanged - 2 fixed = 285 total (was 287) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
47s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 18s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
31s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
13s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}130m 32s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
1s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}182m 27s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.client.TestAsyncClusterAdminApi |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21515 |
| JIRA
[jira] [Comment Edited] (HBASE-21083) Introduce a mechanism to bypass the execution of a stuck procedure
[ https://issues.apache.org/jira/browse/HBASE-21083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704195#comment-16704195 ] Xu Cang edited comment on HBASE-21083 at 11/30/18 3:32 AM: --- [~allan163] thanks for response The reason I am asking is, seems for hbase branch-1 there is no reliable way to unblock or bypass stuck Procedures. And this feature is something potentially can be applied to branch-1 to alleviate engineering burden such as manually operating on WAL files. I briefly skimmed your patch and I see most of the code change is related to ProcedureV2, not AMv2. So since branch-1 has ProcedureV2, I wanted to ask how hard to port this high-level logic to branch-1. was (Author: xucang): [~allan163] thanks for response The reason I am asking is, seems for hbase branch-1 there is no reliable way to unblock or bypass stuck Procedures. And this feature is something potentially can be applied to branch-1 to alleviate engineering burden such as manually operating on WAL files. I briefly skimmed your patch and I see most of the code change is related to ProcedureV2, not AMv2. So since branch-1 has ProcedureV2, so I was asking how hard to port this high-level logic to branch-1. > Introduce a mechanism to bypass the execution of a stuck procedure > -- > > Key: HBASE-21083 > URL: https://issues.apache.org/jira/browse/HBASE-21083 > Project: HBase > Issue Type: Sub-task > Components: amv2 >Affects Versions: 2.1.0, 2.0.1 >Reporter: Allan Yang >Assignee: Allan Yang >Priority: Major > Fix For: 3.0.0, 2.1.1, 2.0.2 > > Attachments: HBASE-21083.branch-2.0.001.patch, > HBASE-21083.branch-2.0.002.patch, HBASE-21083.branch-2.0.003.patch, > HBASE-21083.branch-2.0.003.patch, HBASE-21083.branch-2.0.003.patch, > HBASE-21083.branch-2.1.001.patch > > > Offline discussed with [~stack] and [~Apache9]. We all agreed that we need to > introduce a mechanism to 'force complete' a stuck procedure, so the AMv2 can > continue running. > we still have some unrevealed bugs hiding in our AMv2 and procedureV2 system, > we need something to interfere with stuck procedures before HBCK2 can work. > This is very crucial for a production ready system. > For now, we have little ways to interfere with running procedures. Aborting > them is not a good choice, since some procedures are not abort-able. And some > procedure may have overridden the abort() method, which will ignore the abort > request. > So, here, I will introduce a mechanism to bypass the execution of a stuck > procedure. > Basically, I added a field called 'bypass' to Procedure class. If we set this > field to true, all the logic in execute/rollback will be skipped, letting > this procedure and its ancestors complete normally and releasing the lock > resources at last. > Notice that bypassing a procedure may leave the cluster in a middle state, > e.g. the region not assigned, or some hdfs files left behind. > The Operators need know the side effect of bypassing and recover the > inconsistent state of the cluster themselves, like issuing new procedures to > assign the regions. > A patch will be uploaded and review board will be open. For now, only APIs in > ProcedureExecutor are provided. If anything is fine, I will add it to master > service and add a shell command to bypass a procedure. Or, maybe we can use > dynamically compiled JSPs to execute those APIs as mentioned in HBASE-20679. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work started] (HBASE-21453) Convert ReadOnlyZKClient to DEBUG instead of INFO
[
https://issues.apache.org/jira/browse/HBASE-21453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HBASE-21453 started by Sakthi.
--
> Convert ReadOnlyZKClient to DEBUG instead of INFO
> -
>
> Key: HBASE-21453
> URL: https://issues.apache.org/jira/browse/HBASE-21453
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: Sakthi
>Priority: Major
> Attachments: hbase-21453.master.001.patch
>
>
> Running commands in spark-shell, this is what it looks like on each
> invocation:
> {code}
> scala> val count = rdd.count()
> 2018-11-07 21:01:46,026 INFO [Executor task launch worker for task 1]
> zookeeper.ReadOnlyZKClient: Connect 0x18f3d868 to localhost:2181 with session
> timeout=9ms, retries 30, retry interval 1000ms, keepAlive=6ms
> 2018-11-07 21:01:46,027 INFO [ReadOnlyZKClient-localhost:2181@0x18f3d868]
> zookeeper.ZooKeeper: Initiating client connection,
> connectString=localhost:2181 sessionTimeout=9
> watcher=org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$20/1362339879@743dab9f
> 2018-11-07 21:01:46,030 INFO
> [ReadOnlyZKClient-localhost:2181@0x18f3d868-SendThread(localhost:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server
> localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL
> (unknown error)
> 2018-11-07 21:01:46,031 INFO
> [ReadOnlyZKClient-localhost:2181@0x18f3d868-SendThread(localhost:2181)]
> zookeeper.ClientCnxn: Socket connection established to
> localhost/127.0.0.1:2181, initiating session
> 2018-11-07 21:01:46,033 INFO
> [ReadOnlyZKClient-localhost:2181@0x18f3d868-SendThread(localhost:2181)]
> zookeeper.ClientCnxn: Session establishment complete on server
> localhost/127.0.0.1:2181, sessionid = 0x166f1b283080005, negotiated timeout =
> 4
> 2018-11-07 21:01:46,035 INFO [Executor task launch worker for task 1]
> mapreduce.TableInputFormatBase: Input split length: 0 bytes.
> [Stage 1:> (0 + 1) /
> 1]2018-11-07 21:01:48,074 INFO [Executor task launch worker for task 1]
> zookeeper.ReadOnlyZKClient: Close zookeeper connection 0x18f3d868 to
> localhost:2181
> 2018-11-07 21:01:48,075 INFO [ReadOnlyZKClient-localhost:2181@0x18f3d868]
> zookeeper.ZooKeeper: Session: 0x166f1b283080005 closed
> 2018-11-07 21:01:48,076 INFO [ReadOnlyZKClient
> -localhost:2181@0x18f3d868-EventThread] zookeeper.ClientCnxn: EventThread
> shut down for session: 0x166f1b283080005
> count: Long = 10
> {code}
> Let me shut down the ReadOnlyZKClient log level.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21453) Convert ReadOnlyZKClient to DEBUG instead of INFO
[
https://issues.apache.org/jira/browse/HBASE-21453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sakthi updated HBASE-21453:
---
Attachment: hbase-21453.master.001.patch
> Convert ReadOnlyZKClient to DEBUG instead of INFO
> -
>
> Key: HBASE-21453
> URL: https://issues.apache.org/jira/browse/HBASE-21453
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Priority: Major
> Attachments: hbase-21453.master.001.patch
>
>
> Running commands in spark-shell, this is what it looks like on each
> invocation:
> {code}
> scala> val count = rdd.count()
> 2018-11-07 21:01:46,026 INFO [Executor task launch worker for task 1]
> zookeeper.ReadOnlyZKClient: Connect 0x18f3d868 to localhost:2181 with session
> timeout=9ms, retries 30, retry interval 1000ms, keepAlive=6ms
> 2018-11-07 21:01:46,027 INFO [ReadOnlyZKClient-localhost:2181@0x18f3d868]
> zookeeper.ZooKeeper: Initiating client connection,
> connectString=localhost:2181 sessionTimeout=9
> watcher=org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$20/1362339879@743dab9f
> 2018-11-07 21:01:46,030 INFO
> [ReadOnlyZKClient-localhost:2181@0x18f3d868-SendThread(localhost:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server
> localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL
> (unknown error)
> 2018-11-07 21:01:46,031 INFO
> [ReadOnlyZKClient-localhost:2181@0x18f3d868-SendThread(localhost:2181)]
> zookeeper.ClientCnxn: Socket connection established to
> localhost/127.0.0.1:2181, initiating session
> 2018-11-07 21:01:46,033 INFO
> [ReadOnlyZKClient-localhost:2181@0x18f3d868-SendThread(localhost:2181)]
> zookeeper.ClientCnxn: Session establishment complete on server
> localhost/127.0.0.1:2181, sessionid = 0x166f1b283080005, negotiated timeout =
> 4
> 2018-11-07 21:01:46,035 INFO [Executor task launch worker for task 1]
> mapreduce.TableInputFormatBase: Input split length: 0 bytes.
> [Stage 1:> (0 + 1) /
> 1]2018-11-07 21:01:48,074 INFO [Executor task launch worker for task 1]
> zookeeper.ReadOnlyZKClient: Close zookeeper connection 0x18f3d868 to
> localhost:2181
> 2018-11-07 21:01:48,075 INFO [ReadOnlyZKClient-localhost:2181@0x18f3d868]
> zookeeper.ZooKeeper: Session: 0x166f1b283080005 closed
> 2018-11-07 21:01:48,076 INFO [ReadOnlyZKClient
> -localhost:2181@0x18f3d868-EventThread] zookeeper.ClientCnxn: EventThread
> shut down for session: 0x166f1b283080005
> count: Long = 10
> {code}
> Let me shut down the ReadOnlyZKClient log level.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21453) Convert ReadOnlyZKClient to DEBUG instead of INFO
[
https://issues.apache.org/jira/browse/HBASE-21453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704208#comment-16704208
]
Sakthi commented on HBASE-21453:
Converted the info level logs in ReadOnlyZKClient to debug level.
> Convert ReadOnlyZKClient to DEBUG instead of INFO
> -
>
> Key: HBASE-21453
> URL: https://issues.apache.org/jira/browse/HBASE-21453
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Priority: Major
> Attachments: hbase-21453.master.001.patch
>
>
> Running commands in spark-shell, this is what it looks like on each
> invocation:
> {code}
> scala> val count = rdd.count()
> 2018-11-07 21:01:46,026 INFO [Executor task launch worker for task 1]
> zookeeper.ReadOnlyZKClient: Connect 0x18f3d868 to localhost:2181 with session
> timeout=9ms, retries 30, retry interval 1000ms, keepAlive=6ms
> 2018-11-07 21:01:46,027 INFO [ReadOnlyZKClient-localhost:2181@0x18f3d868]
> zookeeper.ZooKeeper: Initiating client connection,
> connectString=localhost:2181 sessionTimeout=9
> watcher=org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$20/1362339879@743dab9f
> 2018-11-07 21:01:46,030 INFO
> [ReadOnlyZKClient-localhost:2181@0x18f3d868-SendThread(localhost:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server
> localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL
> (unknown error)
> 2018-11-07 21:01:46,031 INFO
> [ReadOnlyZKClient-localhost:2181@0x18f3d868-SendThread(localhost:2181)]
> zookeeper.ClientCnxn: Socket connection established to
> localhost/127.0.0.1:2181, initiating session
> 2018-11-07 21:01:46,033 INFO
> [ReadOnlyZKClient-localhost:2181@0x18f3d868-SendThread(localhost:2181)]
> zookeeper.ClientCnxn: Session establishment complete on server
> localhost/127.0.0.1:2181, sessionid = 0x166f1b283080005, negotiated timeout =
> 4
> 2018-11-07 21:01:46,035 INFO [Executor task launch worker for task 1]
> mapreduce.TableInputFormatBase: Input split length: 0 bytes.
> [Stage 1:> (0 + 1) /
> 1]2018-11-07 21:01:48,074 INFO [Executor task launch worker for task 1]
> zookeeper.ReadOnlyZKClient: Close zookeeper connection 0x18f3d868 to
> localhost:2181
> 2018-11-07 21:01:48,075 INFO [ReadOnlyZKClient-localhost:2181@0x18f3d868]
> zookeeper.ZooKeeper: Session: 0x166f1b283080005 closed
> 2018-11-07 21:01:48,076 INFO [ReadOnlyZKClient
> -localhost:2181@0x18f3d868-EventThread] zookeeper.ClientCnxn: EventThread
> shut down for session: 0x166f1b283080005
> count: Long = 10
> {code}
> Let me shut down the ReadOnlyZKClient log level.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HBASE-21453) Convert ReadOnlyZKClient to DEBUG instead of INFO
[
https://issues.apache.org/jira/browse/HBASE-21453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sakthi reassigned HBASE-21453:
--
Assignee: Sakthi
> Convert ReadOnlyZKClient to DEBUG instead of INFO
> -
>
> Key: HBASE-21453
> URL: https://issues.apache.org/jira/browse/HBASE-21453
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: Sakthi
>Priority: Major
> Attachments: hbase-21453.master.001.patch
>
>
> Running commands in spark-shell, this is what it looks like on each
> invocation:
> {code}
> scala> val count = rdd.count()
> 2018-11-07 21:01:46,026 INFO [Executor task launch worker for task 1]
> zookeeper.ReadOnlyZKClient: Connect 0x18f3d868 to localhost:2181 with session
> timeout=9ms, retries 30, retry interval 1000ms, keepAlive=6ms
> 2018-11-07 21:01:46,027 INFO [ReadOnlyZKClient-localhost:2181@0x18f3d868]
> zookeeper.ZooKeeper: Initiating client connection,
> connectString=localhost:2181 sessionTimeout=9
> watcher=org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$20/1362339879@743dab9f
> 2018-11-07 21:01:46,030 INFO
> [ReadOnlyZKClient-localhost:2181@0x18f3d868-SendThread(localhost:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server
> localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL
> (unknown error)
> 2018-11-07 21:01:46,031 INFO
> [ReadOnlyZKClient-localhost:2181@0x18f3d868-SendThread(localhost:2181)]
> zookeeper.ClientCnxn: Socket connection established to
> localhost/127.0.0.1:2181, initiating session
> 2018-11-07 21:01:46,033 INFO
> [ReadOnlyZKClient-localhost:2181@0x18f3d868-SendThread(localhost:2181)]
> zookeeper.ClientCnxn: Session establishment complete on server
> localhost/127.0.0.1:2181, sessionid = 0x166f1b283080005, negotiated timeout =
> 4
> 2018-11-07 21:01:46,035 INFO [Executor task launch worker for task 1]
> mapreduce.TableInputFormatBase: Input split length: 0 bytes.
> [Stage 1:> (0 + 1) /
> 1]2018-11-07 21:01:48,074 INFO [Executor task launch worker for task 1]
> zookeeper.ReadOnlyZKClient: Close zookeeper connection 0x18f3d868 to
> localhost:2181
> 2018-11-07 21:01:48,075 INFO [ReadOnlyZKClient-localhost:2181@0x18f3d868]
> zookeeper.ZooKeeper: Session: 0x166f1b283080005 closed
> 2018-11-07 21:01:48,076 INFO [ReadOnlyZKClient
> -localhost:2181@0x18f3d868-EventThread] zookeeper.ClientCnxn: EventThread
> shut down for session: 0x166f1b283080005
> count: Long = 10
> {code}
> Let me shut down the ReadOnlyZKClient log level.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21083) Introduce a mechanism to bypass the execution of a stuck procedure
[ https://issues.apache.org/jira/browse/HBASE-21083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704195#comment-16704195 ] Xu Cang commented on HBASE-21083: - [~allan163] thanks for response The reason I am asking is, seems for hbase branch-1 there is no reliable way to unblock or bypass stuck Procedures. And this feature is something potentially can be applied to branch-1 to alleviate engineering burden such as manually operating on WAL files. I briefly skimmed your patch and I see most of the code change is related to ProcedureV2, not AMv2. So since branch-1 has ProcedureV2, so I was asking how hard to port this high-level logic to branch-1. > Introduce a mechanism to bypass the execution of a stuck procedure > -- > > Key: HBASE-21083 > URL: https://issues.apache.org/jira/browse/HBASE-21083 > Project: HBase > Issue Type: Sub-task > Components: amv2 >Affects Versions: 2.1.0, 2.0.1 >Reporter: Allan Yang >Assignee: Allan Yang >Priority: Major > Fix For: 3.0.0, 2.1.1, 2.0.2 > > Attachments: HBASE-21083.branch-2.0.001.patch, > HBASE-21083.branch-2.0.002.patch, HBASE-21083.branch-2.0.003.patch, > HBASE-21083.branch-2.0.003.patch, HBASE-21083.branch-2.0.003.patch, > HBASE-21083.branch-2.1.001.patch > > > Offline discussed with [~stack] and [~Apache9]. We all agreed that we need to > introduce a mechanism to 'force complete' a stuck procedure, so the AMv2 can > continue running. > we still have some unrevealed bugs hiding in our AMv2 and procedureV2 system, > we need something to interfere with stuck procedures before HBCK2 can work. > This is very crucial for a production ready system. > For now, we have little ways to interfere with running procedures. Aborting > them is not a good choice, since some procedures are not abort-able. And some > procedure may have overridden the abort() method, which will ignore the abort > request. > So, here, I will introduce a mechanism to bypass the execution of a stuck > procedure. > Basically, I added a field called 'bypass' to Procedure class. If we set this > field to true, all the logic in execute/rollback will be skipped, letting > this procedure and its ancestors complete normally and releasing the lock > resources at last. > Notice that bypassing a procedure may leave the cluster in a middle state, > e.g. the region not assigned, or some hdfs files left behind. > The Operators need know the side effect of bypassing and recover the > inconsistent state of the cluster themselves, like issuing new procedures to > assign the regions. > A patch will be uploaded and review board will be open. For now, only APIs in > ProcedureExecutor are provided. If anything is fine, I will add it to master > service and add a shell command to bypass a procedure. Or, maybe we can use > dynamically compiled JSPs to execute those APIs as mentioned in HBASE-20679. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21481) [acl] Superuser's permissions should not be granted or revoked by any non-su global admin
[
https://issues.apache.org/jira/browse/HBASE-21481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704187#comment-16704187
]
Reid Chan commented on HBASE-21481:
---
ping [~dbist13], [~psomogyi], WDYT.
> [acl] Superuser's permissions should not be granted or revoked by any non-su
> global admin
> -
>
> Key: HBASE-21481
> URL: https://issues.apache.org/jira/browse/HBASE-21481
> Project: HBase
> Issue Type: Improvement
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Major
> Labels: ACL, security-issue
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21481.master.001.patch,
> HBASE-21481.master.002.patch, HBASE-21481.master.003.patch,
> HBASE-21481.master.004.patch, HBASE-21481.master.005.patch,
> HBASE-21481.master.006.patch, HBASE-21481.master.007.patch,
> HBASE-21481.master.008.patch
>
>
> Superusers are {{hbase.superuser}} listed in configuration and plus the one
> who start master process, these two may be overlap.
> A superuser must be a global admin, but a global admin may not be a
> superuser, possibly granted afterwards.
> For now, an non-su global admin with a Global.ADMIN permission can grant or
> revoke any superuser's permission, accidentally or deliberately.
> The purpose of this issue is to ban this action.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21481) [acl] Superuser's permissions should not be granted or revoked by any non-su global admin
[
https://issues.apache.org/jira/browse/HBASE-21481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704185#comment-16704185
]
Reid Chan commented on HBASE-21481:
---
Failed test passed locally, not related i think.
{code}
[INFO] ---
[INFO] T E S T S
[INFO] ---
[INFO] Running org.apache.hadoop.hbase.client.TestRestoreSnapshotFromClientClone
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 81.502 s
- in org.apache.hadoop.hbase.client.TestRestoreSnapshotFromClientClone
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0
{code}
> [acl] Superuser's permissions should not be granted or revoked by any non-su
> global admin
> -
>
> Key: HBASE-21481
> URL: https://issues.apache.org/jira/browse/HBASE-21481
> Project: HBase
> Issue Type: Improvement
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Major
> Labels: ACL, security-issue
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21481.master.001.patch,
> HBASE-21481.master.002.patch, HBASE-21481.master.003.patch,
> HBASE-21481.master.004.patch, HBASE-21481.master.005.patch,
> HBASE-21481.master.006.patch, HBASE-21481.master.007.patch,
> HBASE-21481.master.008.patch
>
>
> Superusers are {{hbase.superuser}} listed in configuration and plus the one
> who start master process, these two may be overlap.
> A superuser must be a global admin, but a global admin may not be a
> superuser, possibly granted afterwards.
> For now, an non-su global admin with a Global.ADMIN permission can grant or
> revoke any superuser's permission, accidentally or deliberately.
> The purpose of this issue is to ban this action.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21083) Introduce a mechanism to bypass the execution of a stuck procedure
[ https://issues.apache.org/jira/browse/HBASE-21083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704171#comment-16704171 ] Allan Yang commented on HBASE-21083: [~xucang] this feature is mostly used by HBCK2? how are preparing to use it in 1.x? > Introduce a mechanism to bypass the execution of a stuck procedure > -- > > Key: HBASE-21083 > URL: https://issues.apache.org/jira/browse/HBASE-21083 > Project: HBase > Issue Type: Sub-task > Components: amv2 >Affects Versions: 2.1.0, 2.0.1 >Reporter: Allan Yang >Assignee: Allan Yang >Priority: Major > Fix For: 3.0.0, 2.1.1, 2.0.2 > > Attachments: HBASE-21083.branch-2.0.001.patch, > HBASE-21083.branch-2.0.002.patch, HBASE-21083.branch-2.0.003.patch, > HBASE-21083.branch-2.0.003.patch, HBASE-21083.branch-2.0.003.patch, > HBASE-21083.branch-2.1.001.patch > > > Offline discussed with [~stack] and [~Apache9]. We all agreed that we need to > introduce a mechanism to 'force complete' a stuck procedure, so the AMv2 can > continue running. > we still have some unrevealed bugs hiding in our AMv2 and procedureV2 system, > we need something to interfere with stuck procedures before HBCK2 can work. > This is very crucial for a production ready system. > For now, we have little ways to interfere with running procedures. Aborting > them is not a good choice, since some procedures are not abort-able. And some > procedure may have overridden the abort() method, which will ignore the abort > request. > So, here, I will introduce a mechanism to bypass the execution of a stuck > procedure. > Basically, I added a field called 'bypass' to Procedure class. If we set this > field to true, all the logic in execute/rollback will be skipped, letting > this procedure and its ancestors complete normally and releasing the lock > resources at last. > Notice that bypassing a procedure may leave the cluster in a middle state, > e.g. the region not assigned, or some hdfs files left behind. > The Operators need know the side effect of bypassing and recover the > inconsistent state of the cluster themselves, like issuing new procedures to > assign the regions. > A patch will be uploaded and review board will be open. For now, only APIs in > ProcedureExecutor are provided. If anything is fine, I will add it to master > service and add a shell command to bypass a procedure. Or, maybe we can use > dynamically compiled JSPs to execute those APIs as mentioned in HBASE-20679. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21504) If enable FIFOCompactionPolicy, a compaction may write a "empty" hfile whose maxTimeStamp is long max. This kind of hfile will never be archived.
[ https://issues.apache.org/jira/browse/HBASE-21504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704179#comment-16704179 ] Hudson commented on HBASE-21504: SUCCESS: Integrated in Jenkins build HBase-1.2-IT #1187 (See [https://builds.apache.org/job/HBase-1.2-IT/1187/]) HBASE-21504 If enable FIFOCompactionPolicy, a compaction may write a (openinx: rev f49dbae937379888653e9417a314b0b9ead28d82) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/compactions/TestFIFOCompactionPolicy.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/FIFOCompactionPolicy.java > If enable FIFOCompactionPolicy, a compaction may write a "empty" hfile whose > maxTimeStamp is long max. This kind of hfile will never be archived. > - > > Key: HBASE-21504 > URL: https://issues.apache.org/jira/browse/HBASE-21504 > Project: HBase > Issue Type: Bug > Components: Compaction >Affects Versions: 2.1.0 >Reporter: xuming >Assignee: Zheng Hu >Priority: Critical > Fix For: 3.0.0, 1.5.0, 1.3.3, 2.2.0, 1.4.9, 2.1.2, 2.0.4 > > Attachments: 1.patch, HBASE-21504.v1.patch > > > When i use FIFOCompactionPolicy, and if all hfiles(>1) are TTL expired in a > region, once we do a compaction on the region, the compaction policy will > select the latest hfile to do compaction.But beacuse the latest hfile is > already TTL expired, compactor only write a "empty" hfile(whose entry counter > is 0 and maxTimeStamp is Long.MAX_VALUE) finally. Because maxTimeStamp is > long max, so the "empty" hfile will never be TTL expired, more seriously we > can not archive it by FIFOCompactionPolicy forever. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-21083) Introduce a mechanism to bypass the execution of a stuck procedure
[ https://issues.apache.org/jira/browse/HBASE-21083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704171#comment-16704171 ] Allan Yang edited comment on HBASE-21083 at 11/30/18 2:40 AM: -- [~xucang] this feature is mostly used by HBCK2. how are preparing to use it in 1.x? was (Author: allan163): [~xucang] this feature is mostly used by HBCK2? how are preparing to use it in 1.x? > Introduce a mechanism to bypass the execution of a stuck procedure > -- > > Key: HBASE-21083 > URL: https://issues.apache.org/jira/browse/HBASE-21083 > Project: HBase > Issue Type: Sub-task > Components: amv2 >Affects Versions: 2.1.0, 2.0.1 >Reporter: Allan Yang >Assignee: Allan Yang >Priority: Major > Fix For: 3.0.0, 2.1.1, 2.0.2 > > Attachments: HBASE-21083.branch-2.0.001.patch, > HBASE-21083.branch-2.0.002.patch, HBASE-21083.branch-2.0.003.patch, > HBASE-21083.branch-2.0.003.patch, HBASE-21083.branch-2.0.003.patch, > HBASE-21083.branch-2.1.001.patch > > > Offline discussed with [~stack] and [~Apache9]. We all agreed that we need to > introduce a mechanism to 'force complete' a stuck procedure, so the AMv2 can > continue running. > we still have some unrevealed bugs hiding in our AMv2 and procedureV2 system, > we need something to interfere with stuck procedures before HBCK2 can work. > This is very crucial for a production ready system. > For now, we have little ways to interfere with running procedures. Aborting > them is not a good choice, since some procedures are not abort-able. And some > procedure may have overridden the abort() method, which will ignore the abort > request. > So, here, I will introduce a mechanism to bypass the execution of a stuck > procedure. > Basically, I added a field called 'bypass' to Procedure class. If we set this > field to true, all the logic in execute/rollback will be skipped, letting > this procedure and its ancestors complete normally and releasing the lock > resources at last. > Notice that bypassing a procedure may leave the cluster in a middle state, > e.g. the region not assigned, or some hdfs files left behind. > The Operators need know the side effect of bypassing and recover the > inconsistent state of the cluster themselves, like issuing new procedures to > assign the regions. > A patch will be uploaded and review board will be open. For now, only APIs in > ProcedureExecutor are provided. If anything is fine, I will add it to master > service and add a shell command to bypass a procedure. Or, maybe we can use > dynamically compiled JSPs to execute those APIs as mentioned in HBASE-20679. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-21083) Introduce a mechanism to bypass the execution of a stuck procedure
[ https://issues.apache.org/jira/browse/HBASE-21083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704171#comment-16704171 ] Allan Yang edited comment on HBASE-21083 at 11/30/18 2:40 AM: -- [~xucang] this feature is mostly used by HBCK2. how are you preparing to use it in 1.x? was (Author: allan163): [~xucang] this feature is mostly used by HBCK2. how are preparing to use it in 1.x? > Introduce a mechanism to bypass the execution of a stuck procedure > -- > > Key: HBASE-21083 > URL: https://issues.apache.org/jira/browse/HBASE-21083 > Project: HBase > Issue Type: Sub-task > Components: amv2 >Affects Versions: 2.1.0, 2.0.1 >Reporter: Allan Yang >Assignee: Allan Yang >Priority: Major > Fix For: 3.0.0, 2.1.1, 2.0.2 > > Attachments: HBASE-21083.branch-2.0.001.patch, > HBASE-21083.branch-2.0.002.patch, HBASE-21083.branch-2.0.003.patch, > HBASE-21083.branch-2.0.003.patch, HBASE-21083.branch-2.0.003.patch, > HBASE-21083.branch-2.1.001.patch > > > Offline discussed with [~stack] and [~Apache9]. We all agreed that we need to > introduce a mechanism to 'force complete' a stuck procedure, so the AMv2 can > continue running. > we still have some unrevealed bugs hiding in our AMv2 and procedureV2 system, > we need something to interfere with stuck procedures before HBCK2 can work. > This is very crucial for a production ready system. > For now, we have little ways to interfere with running procedures. Aborting > them is not a good choice, since some procedures are not abort-able. And some > procedure may have overridden the abort() method, which will ignore the abort > request. > So, here, I will introduce a mechanism to bypass the execution of a stuck > procedure. > Basically, I added a field called 'bypass' to Procedure class. If we set this > field to true, all the logic in execute/rollback will be skipped, letting > this procedure and its ancestors complete normally and releasing the lock > resources at last. > Notice that bypassing a procedure may leave the cluster in a middle state, > e.g. the region not assigned, or some hdfs files left behind. > The Operators need know the side effect of bypassing and recover the > inconsistent state of the cluster themselves, like issuing new procedures to > assign the regions. > A patch will be uploaded and review board will be open. For now, only APIs in > ProcedureExecutor are provided. If anything is fine, I will add it to master > service and add a shell command to bypass a procedure. Or, maybe we can use > dynamically compiled JSPs to execute those APIs as mentioned in HBASE-20679. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21504) If enable FIFOCompactionPolicy, a compaction may write a "empty" hfile whose maxTimeStamp is long max. This kind of hfile will never be archived.
[ https://issues.apache.org/jira/browse/HBASE-21504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Hu updated HBASE-21504: - Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) > If enable FIFOCompactionPolicy, a compaction may write a "empty" hfile whose > maxTimeStamp is long max. This kind of hfile will never be archived. > - > > Key: HBASE-21504 > URL: https://issues.apache.org/jira/browse/HBASE-21504 > Project: HBase > Issue Type: Bug > Components: Compaction >Affects Versions: 2.1.0 >Reporter: xuming >Assignee: Zheng Hu >Priority: Critical > Fix For: 3.0.0, 1.5.0, 1.3.3, 2.2.0, 1.4.9, 2.1.2, 2.0.4 > > Attachments: 1.patch, HBASE-21504.v1.patch > > > When i use FIFOCompactionPolicy, and if all hfiles(>1) are TTL expired in a > region, once we do a compaction on the region, the compaction policy will > select the latest hfile to do compaction.But beacuse the latest hfile is > already TTL expired, compactor only write a "empty" hfile(whose entry counter > is 0 and maxTimeStamp is Long.MAX_VALUE) finally. Because maxTimeStamp is > long max, so the "empty" hfile will never be TTL expired, more seriously we > can not archive it by FIFOCompactionPolicy forever. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21504) If enable FIFOCompactionPolicy, a compaction may write a "empty" hfile whose maxTimeStamp is long max. This kind of hfile will never be archived.
[ https://issues.apache.org/jira/browse/HBASE-21504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704167#comment-16704167 ] Hudson commented on HBASE-21504: SUCCESS: Integrated in Jenkins build HBase-1.3-IT #506 (See [https://builds.apache.org/job/HBase-1.3-IT/506/]) HBASE-21504 If enable FIFOCompactionPolicy, a compaction may write a (openinx: rev 57222ff0aa66047fc405c488f86a27b622beea64) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/FIFOCompactionPolicy.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/compactions/TestFIFOCompactionPolicy.java > If enable FIFOCompactionPolicy, a compaction may write a "empty" hfile whose > maxTimeStamp is long max. This kind of hfile will never be archived. > - > > Key: HBASE-21504 > URL: https://issues.apache.org/jira/browse/HBASE-21504 > Project: HBase > Issue Type: Bug > Components: Compaction >Affects Versions: 2.1.0 >Reporter: xuming >Assignee: Zheng Hu >Priority: Critical > Fix For: 3.0.0, 1.5.0, 1.3.3, 2.2.0, 1.4.9, 2.1.2, 2.0.4 > > Attachments: 1.patch, HBASE-21504.v1.patch > > > When i use FIFOCompactionPolicy, and if all hfiles(>1) are TTL expired in a > region, once we do a compaction on the region, the compaction policy will > select the latest hfile to do compaction.But beacuse the latest hfile is > already TTL expired, compactor only write a "empty" hfile(whose entry counter > is 0 and maxTimeStamp is Long.MAX_VALUE) finally. Because maxTimeStamp is > long max, so the "empty" hfile will never be TTL expired, more seriously we > can not archive it by FIFOCompactionPolicy forever. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21530) Abort_Procedure should be able to take a list of proc IDs
[ https://issues.apache.org/jira/browse/HBASE-21530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704164#comment-16704164 ] Allan Yang commented on HBASE-21530: What version of HBase are you using?abort procedure should deprecated in HBase2.0+, this feature is problematic, most of the procedure is not abortable. > Abort_Procedure should be able to take a list of proc IDs > - > > Key: HBASE-21530 > URL: https://issues.apache.org/jira/browse/HBASE-21530 > Project: HBase > Issue Type: Improvement >Reporter: Geoffrey Jacoby >Priority: Minor > > As a convenience, it would be helpful if the HBase shell's abort_procedure > call had the option of taking in multiple procedure ids at the same time, > rather than relying on operators to use a loop in an external script. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21504) If enable FIFOCompactionPolicy, a compaction may write a "empty" hfile whose maxTimeStamp is long max. This kind of hfile will never be archived.
[ https://issues.apache.org/jira/browse/HBASE-21504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704160#comment-16704160 ] Zheng Hu commented on HBASE-21504: -- Pushed to branch-2.x & branch-1.4 & branch-1.3 branch-1.2 & branch-1. Thanks [~allan163] for reviewing. > If enable FIFOCompactionPolicy, a compaction may write a "empty" hfile whose > maxTimeStamp is long max. This kind of hfile will never be archived. > - > > Key: HBASE-21504 > URL: https://issues.apache.org/jira/browse/HBASE-21504 > Project: HBase > Issue Type: Bug > Components: Compaction >Affects Versions: 2.1.0 >Reporter: xuming >Assignee: Zheng Hu >Priority: Critical > Fix For: 3.0.0, 1.5.0, 1.3.3, 2.2.0, 1.4.9, 2.1.2, 2.0.4 > > Attachments: 1.patch, HBASE-21504.v1.patch > > > When i use FIFOCompactionPolicy, and if all hfiles(>1) are TTL expired in a > region, once we do a compaction on the region, the compaction policy will > select the latest hfile to do compaction.But beacuse the latest hfile is > already TTL expired, compactor only write a "empty" hfile(whose entry counter > is 0 and maxTimeStamp is Long.MAX_VALUE) finally. Because maxTimeStamp is > long max, so the "empty" hfile will never be TTL expired, more seriously we > can not archive it by FIFOCompactionPolicy forever. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-21533) Add a section in our ref guide for the folding(removal) of hbase:namespace table
Duo Zhang created HBASE-21533: - Summary: Add a section in our ref guide for the folding(removal) of hbase:namespace table Key: HBASE-21533 URL: https://issues.apache.org/jira/browse/HBASE-21533 Project: HBase Issue Type: Task Components: documentation Reporter: Duo Zhang Fix For: 3.0.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-21532) HFileCorruptionChecker#checkHFile does not enumerate the whole file
Andrew Purtell created HBASE-21532: -- Summary: HFileCorruptionChecker#checkHFile does not enumerate the whole file Key: HBASE-21532 URL: https://issues.apache.org/jira/browse/HBASE-21532 Project: HBase Issue Type: Bug Components: hbck, hbck2 Reporter: Andrew Purtell HFileCorruptionChecker#checkHFile can potentially miss some cases of file corruption because it does not enumerate all the data blocks of the file. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21464) Splitting blocked with meta NSRE during split transaction
[
https://issues.apache.org/jira/browse/HBASE-21464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704102#comment-16704102
]
Andrew Purtell commented on HBASE-21464:
[~lhofhansl] Per [~allan163]'s suggestion no changes were made in that regard
after all. The code isn't further away from the place where it was before. Same
number of assumptions.
> Splitting blocked with meta NSRE during split transaction
> -
>
> Key: HBASE-21464
> URL: https://issues.apache.org/jira/browse/HBASE-21464
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.5.0, 1.4.3, 1.4.4, 1.4.5, 1.4.6, 1.4.8, 1.4.7
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Blocker
> Fix For: 1.5.0, 1.4.9
>
> Attachments: HBASE-21464-branch-1.patch, HBASE-21464-branch-1.patch
>
>
> Splitting is blocked during split transaction. The split worker is trying to
> update meta but isn't able to relocate it after NSRE:
> {noformat}
> 2018-11-09 17:50:45,277 INFO
> [regionserver/ip-172-31-5-92.us-west-2.compute.internal/172.31.5.92:8120-splits-1541785709434]
> client.RpcRetryingCaller: Call exception, tries=13, retries=350,
> started=88590 ms ago, cancelled=false,
> msg=org.apache.hadoop.hbase.NotServingRegionException: Region hbase:meta,,1
> is not online on ip-172-31-13-83.us-west-2.compute.internal,8120,1541785618832
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:3088)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.getRegion(RSRpcServices.java:1271)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:2198)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:36617)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2396)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:297)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:277)row
> 'test,,1541785709452.5ba6596f0050c2dab969d152829227c6.44' on table
> 'hbase:meta' at region=hbase:meta,1.1588230740,
> hostname=ip-172-31-15-225.us-west-2.compute.internal,8120,1541785640586,
> seqNum=0{noformat}
> Clients, in this case YCSB, are hung with part of the keyspace missing:
> {noformat}
> 2018-11-09 17:51:06,033 DEBUG [hconnection-0x5739e567-shared--pool1-t165]
> client.ConnectionManager$HConnectionImplementation: locateRegionInMeta
> parentTable=hbase:meta, metaLocation=, attempt=14 of 35 failed; retrying
> after sleep of 20158 because: No server address listed in hbase:meta for
> region
> test,user307326104267982763,1541785754600.ef90030b05cb02305b75e9bfbc3ee081.
> containing row user3301635648728421323{noformat}
> Balancing cannot run indefinitely because the split transaction is stuck
> {noformat}
> 2018-11-09 17:49:55,478 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=8100] master.HMaster:
> Not running balancer because 3 region(s) in transition:
> [{ef90030b05cb02305b75e9bfbc3ee081 state=SPLITTING_NEW, ts=1541785754606,
> server=ip-172-31-5-92.us-west-2.compute.internal,8120,1541785626417},
> {5ba6596f0050c2dab969d152829227c6 state=SPLITTING, ts=1541785754606,
> server=ip-172-31-5-92.us-west-2.compute{noformat}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21464) Splitting blocked with meta NSRE during split transaction
[
https://issues.apache.org/jira/browse/HBASE-21464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-21464:
---
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Pushed to branch-1.4 and branch-1
> Splitting blocked with meta NSRE during split transaction
> -
>
> Key: HBASE-21464
> URL: https://issues.apache.org/jira/browse/HBASE-21464
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.5.0, 1.4.3, 1.4.4, 1.4.5, 1.4.6, 1.4.8, 1.4.7
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Blocker
> Fix For: 1.5.0, 1.4.9
>
> Attachments: HBASE-21464-branch-1.patch, HBASE-21464-branch-1.patch
>
>
> Splitting is blocked during split transaction. The split worker is trying to
> update meta but isn't able to relocate it after NSRE:
> {noformat}
> 2018-11-09 17:50:45,277 INFO
> [regionserver/ip-172-31-5-92.us-west-2.compute.internal/172.31.5.92:8120-splits-1541785709434]
> client.RpcRetryingCaller: Call exception, tries=13, retries=350,
> started=88590 ms ago, cancelled=false,
> msg=org.apache.hadoop.hbase.NotServingRegionException: Region hbase:meta,,1
> is not online on ip-172-31-13-83.us-west-2.compute.internal,8120,1541785618832
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:3088)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.getRegion(RSRpcServices.java:1271)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:2198)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:36617)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2396)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:297)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:277)row
> 'test,,1541785709452.5ba6596f0050c2dab969d152829227c6.44' on table
> 'hbase:meta' at region=hbase:meta,1.1588230740,
> hostname=ip-172-31-15-225.us-west-2.compute.internal,8120,1541785640586,
> seqNum=0{noformat}
> Clients, in this case YCSB, are hung with part of the keyspace missing:
> {noformat}
> 2018-11-09 17:51:06,033 DEBUG [hconnection-0x5739e567-shared--pool1-t165]
> client.ConnectionManager$HConnectionImplementation: locateRegionInMeta
> parentTable=hbase:meta, metaLocation=, attempt=14 of 35 failed; retrying
> after sleep of 20158 because: No server address listed in hbase:meta for
> region
> test,user307326104267982763,1541785754600.ef90030b05cb02305b75e9bfbc3ee081.
> containing row user3301635648728421323{noformat}
> Balancing cannot run indefinitely because the split transaction is stuck
> {noformat}
> 2018-11-09 17:49:55,478 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=8100] master.HMaster:
> Not running balancer because 3 region(s) in transition:
> [{ef90030b05cb02305b75e9bfbc3ee081 state=SPLITTING_NEW, ts=1541785754606,
> server=ip-172-31-5-92.us-west-2.compute.internal,8120,1541785626417},
> {5ba6596f0050c2dab969d152829227c6 state=SPLITTING, ts=1541785754606,
> server=ip-172-31-5-92.us-west-2.compute{noformat}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21531) race between region report and region move causes master to kill RS
[
https://issues.apache.org/jira/browse/HBASE-21531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704078#comment-16704078
]
Sergey Shelukhin commented on HBASE-21531:
--
It's the snapshot of 3.0 before HBASE-21421. That fix looks a little bit
suspect to me... what if the state of RS is actually invalid and it's serving
some region it shouldn't be serving? Master should at least try to close the
region again.
> race between region report and region move causes master to kill RS
> ---
>
> Key: HBASE-21531
> URL: https://issues.apache.org/jira/browse/HBASE-21531
> Project: HBase
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Priority: Major
>
> In this case the delay between the ack from RS and the region report was
> 1.5s, so I'm not sure what caused the race (network hiccup? unreported retry
> by protobuf transport?) but in any case I don't see anything that prevents
> this from happening in a normal case, with a narrowed time window. Any delay
> (e.g. a GC pause on RS right after the report is built, and ack is sent for
> the close) or retries expands the window.
> Master starts moving the region and the source RS acks by 21:51,206
> {noformat}
> Master:
> 2018-11-21 21:21:49,024 INFO [master/6:17000.Chore.1] master.HMaster:
> balance hri=, source=,17020,1542754626176, destination= 2>,17020,1542863268158
> ...
> Server:
> 2018-11-21 21:21:49,394 INFO [RS_CLOSE_REGION-regionserver/:17020-1]
> handler.UnassignRegionHandler: Close
> ...
> 2018-11-21 21:21:51,095 INFO [RS_CLOSE_REGION-regionserver/:17020-1]
> handler.UnassignRegionHandler: Closed
> {noformat}
> By then the region is removed from onlineRegions, so the master proceeds.
> {noformat}
> 2018-11-21 21:21:51,206 INFO [PEWorker-4] procedure2.ProcedureExecutor:
> Finished subprocedure(s) of pid=667,
> state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED, hasLock=true;
> TransitRegionStateProcedure table=, region=, REOPEN/MOVE;
> resume parent processing.
> 2018-11-21 21:21:51,386 INFO [PEWorker-13] assignment.RegionStateStore:
> pid=667 updating hbase:meta row=, regionState=OPENING,
> regionLocation=,17020,1542863268158
> {noformat}
> There are no obvious errors/delays that I see in RS log, and it doesn't log
> starting to send the report.
> However, at 21:52.853 the report is processed that still contains this region.
> {noformat}
> 2018-11-21 21:21:52,853 WARN
> [RpcServer.default.FPBQ.Fifo.handler=48,queue=3,port=17000]
> assignment.AssignmentManager: Killing ,17020,1542754626176:
> rit=OPENING, location=,17020,1542863268158, table=,
> region= reported OPEN on server=,17020,1542754626176 but
> state has otherwise.
> * ABORTING region server ,17020,1542754626176:
> org.apache.hadoop.hbase.YouAreDeadException: rit=OPENING, location= 2>,17020,1542863268158, table=, region= reported OPEN on
> server=,17020,1542754626176 but state has otherwise.
> {noformat}
> RS shuts down in an orderly manner and it can be seen from the log that this
> region is actually not present (there's no line indicating it's being closed,
> unlike for other regions).
> I think there needs to be some sort of versioning for region operations
> and/or in RS reports to allow master to account for concurrent operations and
> avoid races. Probably per region with either a grace period or an additional
> global version, so that master could avoid killing RS based on stale reports,
> but still kill RS if it did retain an old version of the region state due to
> some bug after acking a new version.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21464) Splitting blocked with meta NSRE during split transaction
[
https://issues.apache.org/jira/browse/HBASE-21464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704076#comment-16704076
]
Lars Hofhansl commented on HBASE-21464:
---
+1
(as a small comment: Will this remove us even further from a splittable meta by
baking in more assumptions about a single region meta?)
> Splitting blocked with meta NSRE during split transaction
> -
>
> Key: HBASE-21464
> URL: https://issues.apache.org/jira/browse/HBASE-21464
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.5.0, 1.4.3, 1.4.4, 1.4.5, 1.4.6, 1.4.8, 1.4.7
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Blocker
> Fix For: 1.5.0, 1.4.9
>
> Attachments: HBASE-21464-branch-1.patch, HBASE-21464-branch-1.patch
>
>
> Splitting is blocked during split transaction. The split worker is trying to
> update meta but isn't able to relocate it after NSRE:
> {noformat}
> 2018-11-09 17:50:45,277 INFO
> [regionserver/ip-172-31-5-92.us-west-2.compute.internal/172.31.5.92:8120-splits-1541785709434]
> client.RpcRetryingCaller: Call exception, tries=13, retries=350,
> started=88590 ms ago, cancelled=false,
> msg=org.apache.hadoop.hbase.NotServingRegionException: Region hbase:meta,,1
> is not online on ip-172-31-13-83.us-west-2.compute.internal,8120,1541785618832
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:3088)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.getRegion(RSRpcServices.java:1271)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:2198)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:36617)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2396)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:297)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:277)row
> 'test,,1541785709452.5ba6596f0050c2dab969d152829227c6.44' on table
> 'hbase:meta' at region=hbase:meta,1.1588230740,
> hostname=ip-172-31-15-225.us-west-2.compute.internal,8120,1541785640586,
> seqNum=0{noformat}
> Clients, in this case YCSB, are hung with part of the keyspace missing:
> {noformat}
> 2018-11-09 17:51:06,033 DEBUG [hconnection-0x5739e567-shared--pool1-t165]
> client.ConnectionManager$HConnectionImplementation: locateRegionInMeta
> parentTable=hbase:meta, metaLocation=, attempt=14 of 35 failed; retrying
> after sleep of 20158 because: No server address listed in hbase:meta for
> region
> test,user307326104267982763,1541785754600.ef90030b05cb02305b75e9bfbc3ee081.
> containing row user3301635648728421323{noformat}
> Balancing cannot run indefinitely because the split transaction is stuck
> {noformat}
> 2018-11-09 17:49:55,478 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=8100] master.HMaster:
> Not running balancer because 3 region(s) in transition:
> [{ef90030b05cb02305b75e9bfbc3ee081 state=SPLITTING_NEW, ts=1541785754606,
> server=ip-172-31-5-92.us-west-2.compute.internal,8120,1541785626417},
> {5ba6596f0050c2dab969d152829227c6 state=SPLITTING, ts=1541785754606,
> server=ip-172-31-5-92.us-west-2.compute{noformat}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21531) race between region report and region move causes master to kill RS
[
https://issues.apache.org/jira/browse/HBASE-21531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704071#comment-16704071
]
Duo Zhang commented on HBASE-21531:
---
What is the HBase version?
I think HBASE-21421 is the solution, where we just remove the killing of RS in
reportOnlineRegions...
> race between region report and region move causes master to kill RS
> ---
>
> Key: HBASE-21531
> URL: https://issues.apache.org/jira/browse/HBASE-21531
> Project: HBase
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Priority: Major
>
> In this case the delay between the ack from RS and the region report was
> 1.5s, so I'm not sure what caused the race (network hiccup? unreported retry
> by protobuf transport?) but in any case I don't see anything that prevents
> this from happening in a normal case, with a narrowed time window. Any delay
> (e.g. a GC pause on RS right after the report is built, and ack is sent for
> the close) or retries expands the window.
> Master starts moving the region and the source RS acks by 21:51,206
> {noformat}
> Master:
> 2018-11-21 21:21:49,024 INFO [master/6:17000.Chore.1] master.HMaster:
> balance hri=, source=,17020,1542754626176, destination= 2>,17020,1542863268158
> ...
> Server:
> 2018-11-21 21:21:49,394 INFO [RS_CLOSE_REGION-regionserver/:17020-1]
> handler.UnassignRegionHandler: Close
> ...
> 2018-11-21 21:21:51,095 INFO [RS_CLOSE_REGION-regionserver/:17020-1]
> handler.UnassignRegionHandler: Closed
> {noformat}
> By then the region is removed from onlineRegions, so the master proceeds.
> {noformat}
> 2018-11-21 21:21:51,206 INFO [PEWorker-4] procedure2.ProcedureExecutor:
> Finished subprocedure(s) of pid=667,
> state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED, hasLock=true;
> TransitRegionStateProcedure table=, region=, REOPEN/MOVE;
> resume parent processing.
> 2018-11-21 21:21:51,386 INFO [PEWorker-13] assignment.RegionStateStore:
> pid=667 updating hbase:meta row=, regionState=OPENING,
> regionLocation=,17020,1542863268158
> {noformat}
> There are no obvious errors/delays that I see in RS log, and it doesn't log
> starting to send the report.
> However, at 21:52.853 the report is processed that still contains this region.
> {noformat}
> 2018-11-21 21:21:52,853 WARN
> [RpcServer.default.FPBQ.Fifo.handler=48,queue=3,port=17000]
> assignment.AssignmentManager: Killing ,17020,1542754626176:
> rit=OPENING, location=,17020,1542863268158, table=,
> region= reported OPEN on server=,17020,1542754626176 but
> state has otherwise.
> * ABORTING region server ,17020,1542754626176:
> org.apache.hadoop.hbase.YouAreDeadException: rit=OPENING, location= 2>,17020,1542863268158, table=, region= reported OPEN on
> server=,17020,1542754626176 but state has otherwise.
> {noformat}
> RS shuts down in an orderly manner and it can be seen from the log that this
> region is actually not present (there's no line indicating it's being closed,
> unlike for other regions).
> I think there needs to be some sort of versioning for region operations
> and/or in RS reports to allow master to account for concurrent operations and
> avoid races. Probably per region with either a grace period or an additional
> global version, so that master could avoid killing RS based on stale reports,
> but still kill RS if it did retain an old version of the region state due to
> some bug after acking a new version.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-17914) Create a new reader instead of cloning a new StoreFile when compaction
[ https://issues.apache.org/jira/browse/HBASE-17914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704064#comment-16704064 ] Duo Zhang commented on HBASE-17914: --- The preStoreFileReaderOpen method has already been marked as Deprecated, which means this method has already exposed too many internal states and we want to stop doing this in the future. For now imaybe it is fine to add these parameters as this method is only suppose to be used by Phoenix , but could you please find another way to implement the feature? The StoreFileReader is IA.Private so the methods and behavior will not be stable... Thanks. > Create a new reader instead of cloning a new StoreFile when compaction > -- > > Key: HBASE-17914 > URL: https://issues.apache.org/jira/browse/HBASE-17914 > Project: HBase > Issue Type: Sub-task > Components: Compaction, regionserver >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.0.0 > > Attachments: HBASE-17914-v1.patch, HBASE-17914-v1.patch, > HBASE-17914-v2.patch, HBASE-17914-v3.patch, HBASE-17914.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21515) Also initialize an AsyncClusterConnection in HRegionServer
[ https://issues.apache.org/jira/browse/HBASE-21515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21515: -- Attachment: HBASE-21515-v4.patch > Also initialize an AsyncClusterConnection in HRegionServer > -- > > Key: HBASE-21515 > URL: https://issues.apache.org/jira/browse/HBASE-21515 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Attachments: HBASE-21515-v1.patch, HBASE-21515-v2.patch, > HBASE-21515-v3.patch, HBASE-21515-v4.patch, HBASE-21515.patch > > > Plan to do this incrementally, so first we will only introduce the new > AsyncClusterConnection, without deleting the old ClusterConnection. And we > can move the code which uses the ClusterConnection to use > AsyncClusterConnection, through different sub tasks here. And finally we can > remove the ClusterConnection completely. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21531) race between region report and region move causes master to kill RS
[
https://issues.apache.org/jira/browse/HBASE-21531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Shelukhin updated HBASE-21531:
-
Description:
In this case the delay between the ack from RS and the region report was 1.5s,
so I'm not sure what caused the race (network hiccup? unreported retry by
protobuf transport?) but in any case I don't see anything that prevents this
from happening in a normal case, with a narrowed time window. Any delay (e.g. a
GC pause on RS right after the report is built, and ack is sent for the close)
or retries expands the window.
Master starts moving the region and the source RS acks by 21:51,206
{noformat}
Master:
2018-11-21 21:21:49,024 INFO [master/6:17000.Chore.1] master.HMaster: balance
hri=, source=,17020,1542754626176, destination=,17020,1542863268158
...
Server:
2018-11-21 21:21:49,394 INFO [RS_CLOSE_REGION-regionserver/:17020-1]
handler.UnassignRegionHandler: Close
...
2018-11-21 21:21:51,095 INFO [RS_CLOSE_REGION-regionserver/:17020-1]
handler.UnassignRegionHandler: Closed
{noformat}
By then the region is removed from onlineRegions, so the master proceeds.
{noformat}
2018-11-21 21:21:51,206 INFO [PEWorker-4] procedure2.ProcedureExecutor:
Finished subprocedure(s) of pid=667,
state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED, hasLock=true;
TransitRegionStateProcedure table=, region=, REOPEN/MOVE;
resume parent processing.
2018-11-21 21:21:51,386 INFO [PEWorker-13] assignment.RegionStateStore:
pid=667 updating hbase:meta row=, regionState=OPENING,
regionLocation=,17020,1542863268158
{noformat}
There are no obvious errors/delays that I see in RS log, and it doesn't log
starting to send the report.
However, at 21:52.853 the report is processed that still contains this region.
{noformat}
2018-11-21 21:21:52,853 WARN
[RpcServer.default.FPBQ.Fifo.handler=48,queue=3,port=17000]
assignment.AssignmentManager: Killing ,17020,1542754626176: rit=OPENING,
location=,17020,1542863268158, table=, region=
reported OPEN on server=,17020,1542754626176 but state has otherwise.
* ABORTING region server ,17020,1542754626176:
org.apache.hadoop.hbase.YouAreDeadException: rit=OPENING, location=,17020,1542863268158, table=, region= reported OPEN on
server=,17020,1542754626176 but state has otherwise.
{noformat}
RS shuts down in an orderly manner and it can be seen from the log that this
region is actually not present (there's no line indicating it's being closed,
unlike for other regions).
I think there needs to be some sort of versioning for region operations and/or
in RS reports to allow master to account for concurrent operations and avoid
races. Probably per region with either a grace period or an additional global
version, so that master could avoid killing RS based on stale reports, but
still kill RS if it did retain an old version of the region state due to some
bug after acking a new version.
was:
In this case the delay between the ack from RS and the region report was 1.5s,
so I'm not sure what caused the race (network hiccup? unreported retry by
protobuf transport?) but in any case I don't see anything that prevents this
from happening in a normal case, with a narrowed time window. Any delay (e.g. a
GC pause on RS right after the report is built, and ack is sent for the close)
or retries expands the window.
Master starts moving the region and the source RS acks by 21:51,206
{noformat}
Master:
2018-11-21 21:21:49,024 INFO [master/6:17000.Chore.1] master.HMaster: balance
hri=, source=,17020,1542754626176, destination=,17020,1542863268158
...
Server:
2018-11-21 21:21:49,394 INFO [RS_CLOSE_REGION-regionserver/:17020-1]
handler.UnassignRegionHandler: Close
...
2018-11-21 21:21:51,095 INFO [RS_CLOSE_REGION-regionserver/:17020-1]
handler.UnassignRegionHandler: Closed
{noformat}
By then the region is removed from onlineRegions, so the master proceeds.
{noformat}
2018-11-21 21:21:51,206 INFO [PEWorker-4] procedure2.ProcedureExecutor:
Finished subprocedure(s) of pid=667,
state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED, hasLock=true;
TransitRegionStateProcedure table=dummy_table, region=,
REOPEN/MOVE; resume parent processing.
2018-11-21 21:21:51,386 INFO [PEWorker-13] assignment.RegionStateStore:
pid=667 updating hbase:meta row=, regionState=OPENING,
regionLocation=,17020,1542863268158
{noformat}
There are no obvious errors/delays that I see in RS log, and it doesn't log
starting to send the report.
However, at 21:52.853 the report is processed that still contains this region.
{noformat}
2018-11-21 21:21:52,853 WARN
[RpcServer.default.FPBQ.Fifo.handler=48,queue=3,port=17000]
assignment.AssignmentManager: Killing ,17020,1542754626176: rit=OPENING,
location=,17020,1542863268158, table=dummy_table, region=
reported OPEN on server=,17020,1542754626176 but state has otherwise.
* ABORTING region server ,17020,1542754626176:
[jira] [Created] (HBASE-21531) race between region report and region move causes master to kill RS
Sergey Shelukhin created HBASE-21531:
Summary: race between region report and region move causes master
to kill RS
Key: HBASE-21531
URL: https://issues.apache.org/jira/browse/HBASE-21531
Project: HBase
Issue Type: Bug
Reporter: Sergey Shelukhin
In this case the delay between the ack from RS and the region report was 1.5s,
so I'm not sure what caused the race (network hiccup? unreported retry by
protobuf transport?) but in any case I don't see anything that prevents this
from happening in a normal case, with a narrowed time window. Any delay (e.g. a
GC pause on RS right after the report is built, and ack is sent for the close)
or retries expands the window.
Master starts moving the region and the source RS acks by 21:51,206
{noformat}
Master:
2018-11-21 21:21:49,024 INFO [master/6:17000.Chore.1] master.HMaster: balance
hri=, source=,17020,1542754626176, destination=,17020,1542863268158
...
Server:
2018-11-21 21:21:49,394 INFO [RS_CLOSE_REGION-regionserver/:17020-1]
handler.UnassignRegionHandler: Close
...
2018-11-21 21:21:51,095 INFO [RS_CLOSE_REGION-regionserver/:17020-1]
handler.UnassignRegionHandler: Closed
{noformat}
By then the region is removed from onlineRegions, so the master proceeds.
{noformat}
2018-11-21 21:21:51,206 INFO [PEWorker-4] procedure2.ProcedureExecutor:
Finished subprocedure(s) of pid=667,
state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED, hasLock=true;
TransitRegionStateProcedure table=dummy_table, region=,
REOPEN/MOVE; resume parent processing.
2018-11-21 21:21:51,386 INFO [PEWorker-13] assignment.RegionStateStore:
pid=667 updating hbase:meta row=, regionState=OPENING,
regionLocation=,17020,1542863268158
{noformat}
There are no obvious errors/delays that I see in RS log, and it doesn't log
starting to send the report.
However, at 21:52.853 the report is processed that still contains this region.
{noformat}
2018-11-21 21:21:52,853 WARN
[RpcServer.default.FPBQ.Fifo.handler=48,queue=3,port=17000]
assignment.AssignmentManager: Killing ,17020,1542754626176: rit=OPENING,
location=,17020,1542863268158, table=dummy_table, region=
reported OPEN on server=,17020,1542754626176 but state has otherwise.
* ABORTING region server ,17020,1542754626176:
org.apache.hadoop.hbase.YouAreDeadException: rit=OPENING, location=,17020,1542863268158, table=dummy_table, region= reported OPEN
on server=,17020,1542754626176 but state has otherwise.
{noformat}
RS shuts down in an orderly manner and it can be seen from the log that this
region is actually not present (there's no line indicating it's being closed,
unlike for other regions).
I think there needs to be some sort of versioning in RS reports to allow master
to account for concurrent operations and avoid races.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21531) race between region report and region move causes master to kill RS
[
https://issues.apache.org/jira/browse/HBASE-21531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Shelukhin updated HBASE-21531:
-
Description:
In this case the delay between the ack from RS and the region report was 1.5s,
so I'm not sure what caused the race (network hiccup? unreported retry by
protobuf transport?) but in any case I don't see anything that prevents this
from happening in a normal case, with a narrowed time window. Any delay (e.g. a
GC pause on RS right after the report is built, and ack is sent for the close)
or retries expands the window.
Master starts moving the region and the source RS acks by 21:51,206
{noformat}
Master:
2018-11-21 21:21:49,024 INFO [master/6:17000.Chore.1] master.HMaster: balance
hri=, source=,17020,1542754626176, destination=,17020,1542863268158
...
Server:
2018-11-21 21:21:49,394 INFO [RS_CLOSE_REGION-regionserver/:17020-1]
handler.UnassignRegionHandler: Close
...
2018-11-21 21:21:51,095 INFO [RS_CLOSE_REGION-regionserver/:17020-1]
handler.UnassignRegionHandler: Closed
{noformat}
By then the region is removed from onlineRegions, so the master proceeds.
{noformat}
2018-11-21 21:21:51,206 INFO [PEWorker-4] procedure2.ProcedureExecutor:
Finished subprocedure(s) of pid=667,
state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED, hasLock=true;
TransitRegionStateProcedure table=dummy_table, region=,
REOPEN/MOVE; resume parent processing.
2018-11-21 21:21:51,386 INFO [PEWorker-13] assignment.RegionStateStore:
pid=667 updating hbase:meta row=, regionState=OPENING,
regionLocation=,17020,1542863268158
{noformat}
There are no obvious errors/delays that I see in RS log, and it doesn't log
starting to send the report.
However, at 21:52.853 the report is processed that still contains this region.
{noformat}
2018-11-21 21:21:52,853 WARN
[RpcServer.default.FPBQ.Fifo.handler=48,queue=3,port=17000]
assignment.AssignmentManager: Killing ,17020,1542754626176: rit=OPENING,
location=,17020,1542863268158, table=dummy_table, region=
reported OPEN on server=,17020,1542754626176 but state has otherwise.
* ABORTING region server ,17020,1542754626176:
org.apache.hadoop.hbase.YouAreDeadException: rit=OPENING, location=,17020,1542863268158, table=dummy_table, region= reported OPEN
on server=,17020,1542754626176 but state has otherwise.
{noformat}
RS shuts down in an orderly manner and it can be seen from the log that this
region is actually not present (there's no line indicating it's being closed,
unlike for other regions).
I think there needs to be some sort of versioning for region operations and/or
in RS reports to allow master to account for concurrent operations and avoid
races. Probably per region with either a grace period or an additional global
version, so that master could avoid killing RS based on stale reports, but
still kill RS if it did retain an old version of the region state due to some
bug after acking a new version.
was:
In this case the delay between the ack from RS and the region report was 1.5s,
so I'm not sure what caused the race (network hiccup? unreported retry by
protobuf transport?) but in any case I don't see anything that prevents this
from happening in a normal case, with a narrowed time window. Any delay (e.g. a
GC pause on RS right after the report is built, and ack is sent for the close)
or retries expands the window.
Master starts moving the region and the source RS acks by 21:51,206
{noformat}
Master:
2018-11-21 21:21:49,024 INFO [master/6:17000.Chore.1] master.HMaster: balance
hri=, source=,17020,1542754626176, destination=,17020,1542863268158
...
Server:
2018-11-21 21:21:49,394 INFO [RS_CLOSE_REGION-regionserver/:17020-1]
handler.UnassignRegionHandler: Close
...
2018-11-21 21:21:51,095 INFO [RS_CLOSE_REGION-regionserver/:17020-1]
handler.UnassignRegionHandler: Closed
{noformat}
By then the region is removed from onlineRegions, so the master proceeds.
{noformat}
2018-11-21 21:21:51,206 INFO [PEWorker-4] procedure2.ProcedureExecutor:
Finished subprocedure(s) of pid=667,
state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED, hasLock=true;
TransitRegionStateProcedure table=dummy_table, region=,
REOPEN/MOVE; resume parent processing.
2018-11-21 21:21:51,386 INFO [PEWorker-13] assignment.RegionStateStore:
pid=667 updating hbase:meta row=, regionState=OPENING,
regionLocation=,17020,1542863268158
{noformat}
There are no obvious errors/delays that I see in RS log, and it doesn't log
starting to send the report.
However, at 21:52.853 the report is processed that still contains this region.
{noformat}
2018-11-21 21:21:52,853 WARN
[RpcServer.default.FPBQ.Fifo.handler=48,queue=3,port=17000]
assignment.AssignmentManager: Killing ,17020,1542754626176: rit=OPENING,
location=,17020,1542863268158, table=dummy_table, region=
reported OPEN on server=,17020,1542754626176 but state has otherwise.
* ABORTING region
[jira] [Updated] (HBASE-21531) race between region report and region move causes master to kill RS
[
https://issues.apache.org/jira/browse/HBASE-21531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Shelukhin updated HBASE-21531:
-
Description:
In this case the delay between the ack from RS and the region report was 1.5s,
so I'm not sure what caused the race (network hiccup? unreported retry by
protobuf transport?) but in any case I don't see anything that prevents this
from happening in a normal case, with a narrowed time window. Any delay (e.g. a
GC pause on RS right after the report is built, and ack is sent for the close)
or retries expands the window.
Master starts moving the region and the source RS acks by 21:51,206
{noformat}
Master:
2018-11-21 21:21:49,024 INFO [master/6:17000.Chore.1] master.HMaster: balance
hri=, source=,17020,1542754626176, destination=,17020,1542863268158
...
Server:
2018-11-21 21:21:49,394 INFO [RS_CLOSE_REGION-regionserver/:17020-1]
handler.UnassignRegionHandler: Close
...
2018-11-21 21:21:51,095 INFO [RS_CLOSE_REGION-regionserver/:17020-1]
handler.UnassignRegionHandler: Closed
{noformat}
By then the region is removed from onlineRegions, so the master proceeds.
{noformat}
2018-11-21 21:21:51,206 INFO [PEWorker-4] procedure2.ProcedureExecutor:
Finished subprocedure(s) of pid=667,
state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED, hasLock=true;
TransitRegionStateProcedure table=dummy_table, region=,
REOPEN/MOVE; resume parent processing.
2018-11-21 21:21:51,386 INFO [PEWorker-13] assignment.RegionStateStore:
pid=667 updating hbase:meta row=, regionState=OPENING,
regionLocation=,17020,1542863268158
{noformat}
There are no obvious errors/delays that I see in RS log, and it doesn't log
starting to send the report.
However, at 21:52.853 the report is processed that still contains this region.
{noformat}
2018-11-21 21:21:52,853 WARN
[RpcServer.default.FPBQ.Fifo.handler=48,queue=3,port=17000]
assignment.AssignmentManager: Killing ,17020,1542754626176: rit=OPENING,
location=,17020,1542863268158, table=dummy_table, region=
reported OPEN on server=,17020,1542754626176 but state has otherwise.
* ABORTING region server ,17020,1542754626176:
org.apache.hadoop.hbase.YouAreDeadException: rit=OPENING, location=,17020,1542863268158, table=dummy_table, region= reported OPEN
on server=,17020,1542754626176 but state has otherwise.
{noformat}
RS shuts down in an orderly manner and it can be seen from the log that this
region is actually not present (there's no line indicating it's being closed,
unlike for other regions).
I think there needs to be some sort of versioning for region operations and/or
in RS reports to allow master to account for concurrent operations and avoid
races. Probably per region with either a grace period or an additional global
version, so that master could avoid killing RS based on stale reports, but
still kill RS if it did retain an old version of the region state due to some
bug after acking a new version state.
was:
In this case the delay between the ack from RS and the region report was 1.5s,
so I'm not sure what caused the race (network hiccup? unreported retry by
protobuf transport?) but in any case I don't see anything that prevents this
from happening in a normal case, with a narrowed time window. Any delay (e.g. a
GC pause on RS right after the report is built, and ack is sent for the close)
or retries expands the window.
Master starts moving the region and the source RS acks by 21:51,206
{noformat}
Master:
2018-11-21 21:21:49,024 INFO [master/6:17000.Chore.1] master.HMaster: balance
hri=, source=,17020,1542754626176, destination=,17020,1542863268158
...
Server:
2018-11-21 21:21:49,394 INFO [RS_CLOSE_REGION-regionserver/:17020-1]
handler.UnassignRegionHandler: Close
...
2018-11-21 21:21:51,095 INFO [RS_CLOSE_REGION-regionserver/:17020-1]
handler.UnassignRegionHandler: Closed
{noformat}
By then the region is removed from onlineRegions, so the master proceeds.
{noformat}
2018-11-21 21:21:51,206 INFO [PEWorker-4] procedure2.ProcedureExecutor:
Finished subprocedure(s) of pid=667,
state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED, hasLock=true;
TransitRegionStateProcedure table=dummy_table, region=,
REOPEN/MOVE; resume parent processing.
2018-11-21 21:21:51,386 INFO [PEWorker-13] assignment.RegionStateStore:
pid=667 updating hbase:meta row=, regionState=OPENING,
regionLocation=,17020,1542863268158
{noformat}
There are no obvious errors/delays that I see in RS log, and it doesn't log
starting to send the report.
However, at 21:52.853 the report is processed that still contains this region.
{noformat}
2018-11-21 21:21:52,853 WARN
[RpcServer.default.FPBQ.Fifo.handler=48,queue=3,port=17000]
assignment.AssignmentManager: Killing ,17020,1542754626176: rit=OPENING,
location=,17020,1542863268158, table=dummy_table, region=
reported OPEN on server=,17020,1542754626176 but state has otherwise.
* ABORTING
[jira] [Commented] (HBASE-21524) Unnecessary DEBUG log in ConnectionImplementation#isTableEnabled
[
https://issues.apache.org/jira/browse/HBASE-21524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704018#comment-16704018
]
Hudson commented on HBASE-21524:
Results for branch branch-2
[build #1531 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1531/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1531//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1531//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1531//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Unnecessary DEBUG log in ConnectionImplementation#isTableEnabled
>
>
> Key: HBASE-21524
> URL: https://issues.apache.org/jira/browse/HBASE-21524
> Project: HBase
> Issue Type: Improvement
> Components: Client
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.2
>
> Attachments: HBASE-21524.001.branch-2.0.patch
>
>
> Too much crap coming out of isTableEnabled for "normal" situations. Don't
> need a DEBUG message to tell us that the table is available.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-21530) Abort_Procedure should be able to take a list of proc IDs
Geoffrey Jacoby created HBASE-21530: --- Summary: Abort_Procedure should be able to take a list of proc IDs Key: HBASE-21530 URL: https://issues.apache.org/jira/browse/HBASE-21530 Project: HBase Issue Type: Improvement Reporter: Geoffrey Jacoby As a convenience, it would be helpful if the HBase shell's abort_procedure call had the option of taking in multiple procedure ids at the same time, rather than relying on operators to use a loop in an external script. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-21529) Abort_Procedure in HBase shell fails when all handlers are taken up by Procedures
Geoffrey Jacoby created HBASE-21529: --- Summary: Abort_Procedure in HBase shell fails when all handlers are taken up by Procedures Key: HBASE-21529 URL: https://issues.apache.org/jira/browse/HBASE-21529 Project: HBase Issue Type: Bug Components: proc-v2 Affects Versions: 1.4.8, 1.3.2 Reporter: Geoffrey Jacoby If all RPC handlers are taken up by stuck procedures, the operator will not be able to abort them, because the abort procedure call will try to use the same (exhausted) pool of RPC handlers. The abort procedure call should use a dedicated pool of RPC handlers. This pool can probably be small, but it needs to be separate. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18735) Provide a fast mechanism for shutting down mini cluster
[ https://issues.apache.org/jira/browse/HBASE-18735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-18735: --- Fix Version/s: 2.1.2 2.0.3 2.2.0 3.0.0 > Provide a fast mechanism for shutting down mini cluster > --- > > Key: HBASE-18735 > URL: https://issues.apache.org/jira/browse/HBASE-18735 > Project: HBase > Issue Type: Wish >Reporter: Samarth Jain >Assignee: Artem Ervits >Priority: Major > Fix For: 3.0.0, 2.2.0, 2.0.3, 2.1.2 > > Attachments: HBASE-18735.v01.patch, HBASE-18735.v02.patch, > HBASE-18735.v03.patch > > > The current mechanism of shutting down a mini cluster through > HBaseTestingUtility.shutDownMiniCluster can take a lot of time when the mini > cluster almost has a lot of tables. A lot of this time is spent in closing > all the user regions. It would be nice to have a mechanism where this > shutdown can happen quickly without having to worry about closing these user > regions. At the same time, this mechanism would need to make sure that all > the critical system resources like file handles and network ports are still > released so that subsequently initialized mini clusters on the same JVM or > system won't run into resource issues. This would make testing using HBase > mini clusters much faster and immensely help out test frameworks of dependent > projects like Phoenix. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18735) Provide a fast mechanism for shutting down mini cluster
[ https://issues.apache.org/jira/browse/HBASE-18735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-18735: --- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Pushing this one to keep Artem chugging along. Seems low risk to the rest of the population. Thanks for the patch, Artem! > Provide a fast mechanism for shutting down mini cluster > --- > > Key: HBASE-18735 > URL: https://issues.apache.org/jira/browse/HBASE-18735 > Project: HBase > Issue Type: Wish >Reporter: Samarth Jain >Assignee: Artem Ervits >Priority: Major > Fix For: 3.0.0, 2.2.0, 2.0.3, 2.1.2 > > Attachments: HBASE-18735.v01.patch, HBASE-18735.v02.patch, > HBASE-18735.v03.patch > > > The current mechanism of shutting down a mini cluster through > HBaseTestingUtility.shutDownMiniCluster can take a lot of time when the mini > cluster almost has a lot of tables. A lot of this time is spent in closing > all the user regions. It would be nice to have a mechanism where this > shutdown can happen quickly without having to worry about closing these user > regions. At the same time, this mechanism would need to make sure that all > the critical system resources like file handles and network ports are still > released so that subsequently initialized mini clusters on the same JVM or > system won't run into resource issues. This would make testing using HBase > mini clusters much faster and immensely help out test frameworks of dependent > projects like Phoenix. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-18735) Provide a fast mechanism for shutting down mini cluster
[ https://issues.apache.org/jira/browse/HBASE-18735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703850#comment-16703850 ] Josh Elser commented on HBASE-18735: Sounds good enough to me. I don't see an obvious reason why your change would have any direct impact on TestMultiColumnScanner. Let's try this on master and 2.x? Would say we can put it on Master only, but I don't think we have a corresponding Phoenix build for HBase 3 yet. > Provide a fast mechanism for shutting down mini cluster > --- > > Key: HBASE-18735 > URL: https://issues.apache.org/jira/browse/HBASE-18735 > Project: HBase > Issue Type: Wish >Reporter: Samarth Jain >Assignee: Artem Ervits >Priority: Major > Attachments: HBASE-18735.v01.patch, HBASE-18735.v02.patch, > HBASE-18735.v03.patch > > > The current mechanism of shutting down a mini cluster through > HBaseTestingUtility.shutDownMiniCluster can take a lot of time when the mini > cluster almost has a lot of tables. A lot of this time is spent in closing > all the user regions. It would be nice to have a mechanism where this > shutdown can happen quickly without having to worry about closing these user > regions. At the same time, this mechanism would need to make sure that all > the critical system resources like file handles and network ports are still > released so that subsequently initialized mini clusters on the same JVM or > system won't run into resource issues. This would make testing using HBase > mini clusters much faster and immensely help out test frameworks of dependent > projects like Phoenix. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-21083) Introduce a mechanism to bypass the execution of a stuck procedure
[ https://issues.apache.org/jira/browse/HBASE-21083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16702299#comment-16702299 ] Xu Cang edited comment on HBASE-21083 at 11/29/18 9:14 PM: --- [~allan163] Do you see a value backporting this to branch-1? We have seen MasterProcedure stuck in production cluster and have very few ways to safely resolve it. was (Author: xucang): [~allan163] Do you see a value backporting this to branch-1? We have seen MasterProcedure stuck in production cluster and have very few ways to safely resolve it. (I am not very familiar with AMv2 but seems AMv2 is available in branch-1 too? I can see procedure2.Procedure class and so on) Thanks. > Introduce a mechanism to bypass the execution of a stuck procedure > -- > > Key: HBASE-21083 > URL: https://issues.apache.org/jira/browse/HBASE-21083 > Project: HBase > Issue Type: Sub-task > Components: amv2 >Affects Versions: 2.1.0, 2.0.1 >Reporter: Allan Yang >Assignee: Allan Yang >Priority: Major > Fix For: 3.0.0, 2.1.1, 2.0.2 > > Attachments: HBASE-21083.branch-2.0.001.patch, > HBASE-21083.branch-2.0.002.patch, HBASE-21083.branch-2.0.003.patch, > HBASE-21083.branch-2.0.003.patch, HBASE-21083.branch-2.0.003.patch, > HBASE-21083.branch-2.1.001.patch > > > Offline discussed with [~stack] and [~Apache9]. We all agreed that we need to > introduce a mechanism to 'force complete' a stuck procedure, so the AMv2 can > continue running. > we still have some unrevealed bugs hiding in our AMv2 and procedureV2 system, > we need something to interfere with stuck procedures before HBCK2 can work. > This is very crucial for a production ready system. > For now, we have little ways to interfere with running procedures. Aborting > them is not a good choice, since some procedures are not abort-able. And some > procedure may have overridden the abort() method, which will ignore the abort > request. > So, here, I will introduce a mechanism to bypass the execution of a stuck > procedure. > Basically, I added a field called 'bypass' to Procedure class. If we set this > field to true, all the logic in execute/rollback will be skipped, letting > this procedure and its ancestors complete normally and releasing the lock > resources at last. > Notice that bypassing a procedure may leave the cluster in a middle state, > e.g. the region not assigned, or some hdfs files left behind. > The Operators need know the side effect of bypassing and recover the > inconsistent state of the cluster themselves, like issuing new procedures to > assign the regions. > A patch will be uploaded and review board will be open. For now, only APIs in > ProcedureExecutor are provided. If anything is fine, I will add it to master > service and add a shell command to bypass a procedure. Or, maybe we can use > dynamically compiled JSPs to execute those APIs as mentioned in HBASE-20679. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21528) hbase-connectors need scala dependency convergence
[
https://issues.apache.org/jira/browse/HBASE-21528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703747#comment-16703747
]
Peter Somogyi commented on HBASE-21528:
---
Currently there is no precommit check for hbase-connectors and
hbase-operator-tools repos and that's why both Hadoop QA tests failed. The plan
is to add GitHub Pull Request verification because Yetus does not support
multiple repositories connected to a single Jira project.
> hbase-connectors need scala dependency convergence
> --
>
> Key: HBASE-21528
> URL: https://issues.apache.org/jira/browse/HBASE-21528
> Project: HBase
> Issue Type: Task
> Components: hbase-connectors
>Reporter: Artem Ervits
>Assignee: Artem Ervits
>Priority: Minor
> Attachments: HBASE-21528.v01.patch, HBASE-21528.v02.patch
>
>
> this issue reappeared after migration of hbase-spark to hbase-connectors
> repo. Originally addressed in
> https://issues.apache.org/jira/browse/HBASE-20175
> introducing scala-maven-plugin and adding a flag to honor compatible major
> scala version makes warning disappear.
>
> {code:java}
> [WARNING] Expected all dependencies to require Scala version: 2.11.12
> [WARNING] org.apache.hbase.connectors.spark:hbase-spark:1.0.0-SNAPSHOT
> requires scala version: 2.11.12
> [WARNING] org.apache.spark:spark-streaming_2.11:2.4.0 requires scala version:
> 2.11.12
> [WARNING] org.apache.spark:spark-streaming_2.11:2.4.0 requires scala version:
> 2.11.12
> [WARNING] org.scala-lang:scala-reflect:2.11.12 requires scala version: 2.11.12
> [WARNING] org.apache.spark:spark-tags_2.11:2.4.0 requires scala version:
> 2.11.12
> [WARNING] com.twitter:chill_2.11:0.9.3 requires scala version: 2.11.12
> [WARNING] org.scala-lang.modules:scala-parser-combinators_2.11:1.1.0 requires
> scala version: 2.11.12
> [WARNING] org.json4s:json4s-core_2.11:3.2.11 requires scala version: 2.11.0
> [WARNING] Multiple versions of scala libraries detected!
> [WARNING] warning: there were 17 deprecation warnings; re-run with
> -deprecation for details
> [WARNING] one warning found
> [WARNING] Expected all dependencies to require Scala version: 2.11.12
> [WARNING] org.apache.hbase.connectors.spark:hbase-spark:1.0.0-SNAPSHOT
> requires scala version: 2.11.12
> [WARNING] org.apache.spark:spark-streaming_2.11:2.4.0 requires scala version:
> 2.11.12
> [WARNING] org.apache.spark:spark-streaming_2.11:2.4.0 requires scala version:
> 2.11.12
> [WARNING] org.scala-lang:scala-reflect:2.11.12 requires scala version: 2.11.12
> [WARNING] org.apache.spark:spark-tags_2.11:2.4.0 requires scala version:
> 2.11.12
> [WARNING] com.twitter:chill_2.11:0.9.3 requires scala version: 2.11.12
> [WARNING] org.scala-lang.modules:scala-parser-combinators_2.11:1.1.0 requires
> scala version: 2.11.12
> [WARNING] org.json4s:json4s-core_2.11:3.2.11 requires scala version: 2.11.0
> [WARNING] Multiple versions of scala libraries detected!{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-21528) hbase-connectors need scala dependency convergence
[
https://issues.apache.org/jira/browse/HBASE-21528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703731#comment-16703731
]
Artem Ervits edited comment on HBASE-21528 at 11/29/18 8:08 PM:
[~stack] btw, had to generate the patch with git diff --no-prefix as there's no
dev-tools/make_patch.sh script. Uploaded v.02 patch in case I fat-fingered it
first time.
was (Author: dbist13):
[~stack] btw, had to generate the patch with git diff --no-prefix as there's no
yetus integration yet. Uploaded v.02 patch in case I fat-fingered it first time.
> hbase-connectors need scala dependency convergence
> --
>
> Key: HBASE-21528
> URL: https://issues.apache.org/jira/browse/HBASE-21528
> Project: HBase
> Issue Type: Task
> Components: hbase-connectors
>Reporter: Artem Ervits
>Assignee: Artem Ervits
>Priority: Minor
> Attachments: HBASE-21528.v01.patch, HBASE-21528.v02.patch
>
>
> this issue reappeared after migration of hbase-spark to hbase-connectors
> repo. Originally addressed in
> https://issues.apache.org/jira/browse/HBASE-20175
> introducing scala-maven-plugin and adding a flag to honor compatible major
> scala version makes warning disappear.
>
> {code:java}
> [WARNING] Expected all dependencies to require Scala version: 2.11.12
> [WARNING] org.apache.hbase.connectors.spark:hbase-spark:1.0.0-SNAPSHOT
> requires scala version: 2.11.12
> [WARNING] org.apache.spark:spark-streaming_2.11:2.4.0 requires scala version:
> 2.11.12
> [WARNING] org.apache.spark:spark-streaming_2.11:2.4.0 requires scala version:
> 2.11.12
> [WARNING] org.scala-lang:scala-reflect:2.11.12 requires scala version: 2.11.12
> [WARNING] org.apache.spark:spark-tags_2.11:2.4.0 requires scala version:
> 2.11.12
> [WARNING] com.twitter:chill_2.11:0.9.3 requires scala version: 2.11.12
> [WARNING] org.scala-lang.modules:scala-parser-combinators_2.11:1.1.0 requires
> scala version: 2.11.12
> [WARNING] org.json4s:json4s-core_2.11:3.2.11 requires scala version: 2.11.0
> [WARNING] Multiple versions of scala libraries detected!
> [WARNING] warning: there were 17 deprecation warnings; re-run with
> -deprecation for details
> [WARNING] one warning found
> [WARNING] Expected all dependencies to require Scala version: 2.11.12
> [WARNING] org.apache.hbase.connectors.spark:hbase-spark:1.0.0-SNAPSHOT
> requires scala version: 2.11.12
> [WARNING] org.apache.spark:spark-streaming_2.11:2.4.0 requires scala version:
> 2.11.12
> [WARNING] org.apache.spark:spark-streaming_2.11:2.4.0 requires scala version:
> 2.11.12
> [WARNING] org.scala-lang:scala-reflect:2.11.12 requires scala version: 2.11.12
> [WARNING] org.apache.spark:spark-tags_2.11:2.4.0 requires scala version:
> 2.11.12
> [WARNING] com.twitter:chill_2.11:0.9.3 requires scala version: 2.11.12
> [WARNING] org.scala-lang.modules:scala-parser-combinators_2.11:1.1.0 requires
> scala version: 2.11.12
> [WARNING] org.json4s:json4s-core_2.11:3.2.11 requires scala version: 2.11.0
> [WARNING] Multiple versions of scala libraries detected!{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21528) hbase-connectors need scala dependency convergence
[
https://issues.apache.org/jira/browse/HBASE-21528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703731#comment-16703731
]
Artem Ervits commented on HBASE-21528:
--
[~stack] btw, had to generate the patch with git diff --no-prefix as there's no
yetus integration yet. Uploaded v.02 patch in case I fat-fingered it first time.
> hbase-connectors need scala dependency convergence
> --
>
> Key: HBASE-21528
> URL: https://issues.apache.org/jira/browse/HBASE-21528
> Project: HBase
> Issue Type: Task
> Components: hbase-connectors
>Reporter: Artem Ervits
>Assignee: Artem Ervits
>Priority: Minor
> Attachments: HBASE-21528.v01.patch, HBASE-21528.v02.patch
>
>
> this issue reappeared after migration of hbase-spark to hbase-connectors
> repo. Originally addressed in
> https://issues.apache.org/jira/browse/HBASE-20175
> introducing scala-maven-plugin and adding a flag to honor compatible major
> scala version makes warning disappear.
>
> {code:java}
> [WARNING] Expected all dependencies to require Scala version: 2.11.12
> [WARNING] org.apache.hbase.connectors.spark:hbase-spark:1.0.0-SNAPSHOT
> requires scala version: 2.11.12
> [WARNING] org.apache.spark:spark-streaming_2.11:2.4.0 requires scala version:
> 2.11.12
> [WARNING] org.apache.spark:spark-streaming_2.11:2.4.0 requires scala version:
> 2.11.12
> [WARNING] org.scala-lang:scala-reflect:2.11.12 requires scala version: 2.11.12
> [WARNING] org.apache.spark:spark-tags_2.11:2.4.0 requires scala version:
> 2.11.12
> [WARNING] com.twitter:chill_2.11:0.9.3 requires scala version: 2.11.12
> [WARNING] org.scala-lang.modules:scala-parser-combinators_2.11:1.1.0 requires
> scala version: 2.11.12
> [WARNING] org.json4s:json4s-core_2.11:3.2.11 requires scala version: 2.11.0
> [WARNING] Multiple versions of scala libraries detected!
> [WARNING] warning: there were 17 deprecation warnings; re-run with
> -deprecation for details
> [WARNING] one warning found
> [WARNING] Expected all dependencies to require Scala version: 2.11.12
> [WARNING] org.apache.hbase.connectors.spark:hbase-spark:1.0.0-SNAPSHOT
> requires scala version: 2.11.12
> [WARNING] org.apache.spark:spark-streaming_2.11:2.4.0 requires scala version:
> 2.11.12
> [WARNING] org.apache.spark:spark-streaming_2.11:2.4.0 requires scala version:
> 2.11.12
> [WARNING] org.scala-lang:scala-reflect:2.11.12 requires scala version: 2.11.12
> [WARNING] org.apache.spark:spark-tags_2.11:2.4.0 requires scala version:
> 2.11.12
> [WARNING] com.twitter:chill_2.11:0.9.3 requires scala version: 2.11.12
> [WARNING] org.scala-lang.modules:scala-parser-combinators_2.11:1.1.0 requires
> scala version: 2.11.12
> [WARNING] org.json4s:json4s-core_2.11:3.2.11 requires scala version: 2.11.0
> [WARNING] Multiple versions of scala libraries detected!{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21518) TestMasterFailoverWithProcedures is flaky
[
https://issues.apache.org/jira/browse/HBASE-21518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703730#comment-16703730
]
Peter Somogyi commented on HBASE-21518:
---
[~stack]: Shall I commit it back until branch-2.0?
> TestMasterFailoverWithProcedures is flaky
> -
>
> Key: HBASE-21518
> URL: https://issues.apache.org/jira/browse/HBASE-21518
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.2.0, 2.0.3, 2.1.2
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Major
> Attachments: HBASE-21518-v1.patch, output.txt
>
>
> TestMasterFailoverWithProcedures test is failing frequently, times out. I
> faced this failure on 2.0.3RC0 vote and it also appears on multiple flaky
> dashboards.
> branch-2:
> [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Flaky-Tests/job/branch-2/2007/]
> branch-2.1:
> [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Flaky-Tests/job/branch-2.1/2002/]
> branch-2.0:
> [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Flaky-Tests/job/branch-2.0/1988/]
>
> {noformat}
> [INFO] Running
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures
> [ERROR] Tests run: 4, Failures: 0, Errors: 2, Skipped: 0, Time elapsed:
> 780.648 s <<< FAILURE! - in
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures
> [ERROR]
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures
> Time elapsed: 749.024 s <<< ERROR!
> org.junit.runners.model.TestTimedOutException: test timed out after 780
> seconds
> at
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures.tearDown(TestMasterFailoverWithProcedures.java:86)
> [ERROR]
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures
> Time elapsed: 749.051 s <<< ERROR!
> java.lang.Exception: Appears to be stuck in thread RS-EventLoopGroup-3-2
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21518) TestMasterFailoverWithProcedures is flaky
[
https://issues.apache.org/jira/browse/HBASE-21518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703728#comment-16703728
]
Hadoop QA commented on HBASE-21518:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
15s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
11s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
9m 15s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}134m
14s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
19s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}174m 12s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21518 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12950045/HBASE-21518-v1.patch |
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 345b56ce27bf 4.4.0-139-generic #165~14.04.1-Ubuntu SMP Wed Oct
31 10:55:11 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / fdddc47e77 |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/15152/testReport/ |
| Max. process+thread count | 4421 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output |
[jira] [Commented] (HBASE-21528) hbase-connectors need scala dependency convergence
[
https://issues.apache.org/jira/browse/HBASE-21528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703716#comment-16703716
]
Hadoop QA commented on HBASE-21528:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 4s{color}
| {color:red} HBASE-21528 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.8.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21528 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12950063/HBASE-21528.v02.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/15154/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> hbase-connectors need scala dependency convergence
> --
>
> Key: HBASE-21528
> URL: https://issues.apache.org/jira/browse/HBASE-21528
> Project: HBase
> Issue Type: Task
> Components: hbase-connectors
>Reporter: Artem Ervits
>Assignee: Artem Ervits
>Priority: Minor
> Attachments: HBASE-21528.v01.patch, HBASE-21528.v02.patch
>
>
> this issue reappeared after migration of hbase-spark to hbase-connectors
> repo. Originally addressed in
> https://issues.apache.org/jira/browse/HBASE-20175
> introducing scala-maven-plugin and adding a flag to honor compatible major
> scala version makes warning disappear.
>
> {code:java}
> [WARNING] Expected all dependencies to require Scala version: 2.11.12
> [WARNING] org.apache.hbase.connectors.spark:hbase-spark:1.0.0-SNAPSHOT
> requires scala version: 2.11.12
> [WARNING] org.apache.spark:spark-streaming_2.11:2.4.0 requires scala version:
> 2.11.12
> [WARNING] org.apache.spark:spark-streaming_2.11:2.4.0 requires scala version:
> 2.11.12
> [WARNING] org.scala-lang:scala-reflect:2.11.12 requires scala version: 2.11.12
> [WARNING] org.apache.spark:spark-tags_2.11:2.4.0 requires scala version:
> 2.11.12
> [WARNING] com.twitter:chill_2.11:0.9.3 requires scala version: 2.11.12
> [WARNING] org.scala-lang.modules:scala-parser-combinators_2.11:1.1.0 requires
> scala version: 2.11.12
> [WARNING] org.json4s:json4s-core_2.11:3.2.11 requires scala version: 2.11.0
> [WARNING] Multiple versions of scala libraries detected!
> [WARNING] warning: there were 17 deprecation warnings; re-run with
> -deprecation for details
> [WARNING] one warning found
> [WARNING] Expected all dependencies to require Scala version: 2.11.12
> [WARNING] org.apache.hbase.connectors.spark:hbase-spark:1.0.0-SNAPSHOT
> requires scala version: 2.11.12
> [WARNING] org.apache.spark:spark-streaming_2.11:2.4.0 requires scala version:
> 2.11.12
> [WARNING] org.apache.spark:spark-streaming_2.11:2.4.0 requires scala version:
> 2.11.12
> [WARNING] org.scala-lang:scala-reflect:2.11.12 requires scala version: 2.11.12
> [WARNING] org.apache.spark:spark-tags_2.11:2.4.0 requires scala version:
> 2.11.12
> [WARNING] com.twitter:chill_2.11:0.9.3 requires scala version: 2.11.12
> [WARNING] org.scala-lang.modules:scala-parser-combinators_2.11:1.1.0 requires
> scala version: 2.11.12
> [WARNING] org.json4s:json4s-core_2.11:3.2.11 requires scala version: 2.11.0
> [WARNING] Multiple versions of scala libraries detected!{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21528) hbase-connectors need scala dependency convergence
[
https://issues.apache.org/jira/browse/HBASE-21528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Artem Ervits updated HBASE-21528:
-
Attachment: HBASE-21528.v02.patch
> hbase-connectors need scala dependency convergence
> --
>
> Key: HBASE-21528
> URL: https://issues.apache.org/jira/browse/HBASE-21528
> Project: HBase
> Issue Type: Task
> Components: hbase-connectors
>Reporter: Artem Ervits
>Assignee: Artem Ervits
>Priority: Minor
> Attachments: HBASE-21528.v01.patch, HBASE-21528.v02.patch
>
>
> this issue reappeared after migration of hbase-spark to hbase-connectors
> repo. Originally addressed in
> https://issues.apache.org/jira/browse/HBASE-20175
> introducing scala-maven-plugin and adding a flag to honor compatible major
> scala version makes warning disappear.
>
> {code:java}
> [WARNING] Expected all dependencies to require Scala version: 2.11.12
> [WARNING] org.apache.hbase.connectors.spark:hbase-spark:1.0.0-SNAPSHOT
> requires scala version: 2.11.12
> [WARNING] org.apache.spark:spark-streaming_2.11:2.4.0 requires scala version:
> 2.11.12
> [WARNING] org.apache.spark:spark-streaming_2.11:2.4.0 requires scala version:
> 2.11.12
> [WARNING] org.scala-lang:scala-reflect:2.11.12 requires scala version: 2.11.12
> [WARNING] org.apache.spark:spark-tags_2.11:2.4.0 requires scala version:
> 2.11.12
> [WARNING] com.twitter:chill_2.11:0.9.3 requires scala version: 2.11.12
> [WARNING] org.scala-lang.modules:scala-parser-combinators_2.11:1.1.0 requires
> scala version: 2.11.12
> [WARNING] org.json4s:json4s-core_2.11:3.2.11 requires scala version: 2.11.0
> [WARNING] Multiple versions of scala libraries detected!
> [WARNING] warning: there were 17 deprecation warnings; re-run with
> -deprecation for details
> [WARNING] one warning found
> [WARNING] Expected all dependencies to require Scala version: 2.11.12
> [WARNING] org.apache.hbase.connectors.spark:hbase-spark:1.0.0-SNAPSHOT
> requires scala version: 2.11.12
> [WARNING] org.apache.spark:spark-streaming_2.11:2.4.0 requires scala version:
> 2.11.12
> [WARNING] org.apache.spark:spark-streaming_2.11:2.4.0 requires scala version:
> 2.11.12
> [WARNING] org.scala-lang:scala-reflect:2.11.12 requires scala version: 2.11.12
> [WARNING] org.apache.spark:spark-tags_2.11:2.4.0 requires scala version:
> 2.11.12
> [WARNING] com.twitter:chill_2.11:0.9.3 requires scala version: 2.11.12
> [WARNING] org.scala-lang.modules:scala-parser-combinators_2.11:1.1.0 requires
> scala version: 2.11.12
> [WARNING] org.json4s:json4s-core_2.11:3.2.11 requires scala version: 2.11.0
> [WARNING] Multiple versions of scala libraries detected!{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21528) hbase-connectors need scala dependency convergence
[
https://issues.apache.org/jira/browse/HBASE-21528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703688#comment-16703688
]
Hadoop QA commented on HBASE-21528:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
11s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 4s{color}
| {color:red} HBASE-21528 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.8.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21528 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12950062/HBASE-21528.v01.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/15153/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> hbase-connectors need scala dependency convergence
> --
>
> Key: HBASE-21528
> URL: https://issues.apache.org/jira/browse/HBASE-21528
> Project: HBase
> Issue Type: Task
> Components: hbase-connectors
>Reporter: Artem Ervits
>Assignee: Artem Ervits
>Priority: Minor
> Attachments: HBASE-21528.v01.patch
>
>
> this issue reappeared after migration of hbase-spark to hbase-connectors
> repo. Originally addressed in
> https://issues.apache.org/jira/browse/HBASE-20175
> introducing scala-maven-plugin and adding a flag to honor compatible major
> scala version makes warning disappear.
>
> {code:java}
> [WARNING] Expected all dependencies to require Scala version: 2.11.12
> [WARNING] org.apache.hbase.connectors.spark:hbase-spark:1.0.0-SNAPSHOT
> requires scala version: 2.11.12
> [WARNING] org.apache.spark:spark-streaming_2.11:2.4.0 requires scala version:
> 2.11.12
> [WARNING] org.apache.spark:spark-streaming_2.11:2.4.0 requires scala version:
> 2.11.12
> [WARNING] org.scala-lang:scala-reflect:2.11.12 requires scala version: 2.11.12
> [WARNING] org.apache.spark:spark-tags_2.11:2.4.0 requires scala version:
> 2.11.12
> [WARNING] com.twitter:chill_2.11:0.9.3 requires scala version: 2.11.12
> [WARNING] org.scala-lang.modules:scala-parser-combinators_2.11:1.1.0 requires
> scala version: 2.11.12
> [WARNING] org.json4s:json4s-core_2.11:3.2.11 requires scala version: 2.11.0
> [WARNING] Multiple versions of scala libraries detected!
> [WARNING] warning: there were 17 deprecation warnings; re-run with
> -deprecation for details
> [WARNING] one warning found
> [WARNING] Expected all dependencies to require Scala version: 2.11.12
> [WARNING] org.apache.hbase.connectors.spark:hbase-spark:1.0.0-SNAPSHOT
> requires scala version: 2.11.12
> [WARNING] org.apache.spark:spark-streaming_2.11:2.4.0 requires scala version:
> 2.11.12
> [WARNING] org.apache.spark:spark-streaming_2.11:2.4.0 requires scala version:
> 2.11.12
> [WARNING] org.scala-lang:scala-reflect:2.11.12 requires scala version: 2.11.12
> [WARNING] org.apache.spark:spark-tags_2.11:2.4.0 requires scala version:
> 2.11.12
> [WARNING] com.twitter:chill_2.11:0.9.3 requires scala version: 2.11.12
> [WARNING] org.scala-lang.modules:scala-parser-combinators_2.11:1.1.0 requires
> scala version: 2.11.12
> [WARNING] org.json4s:json4s-core_2.11:3.2.11 requires scala version: 2.11.0
> [WARNING] Multiple versions of scala libraries detected!{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21528) hbase-connectors need scala dependency convergence
[
https://issues.apache.org/jira/browse/HBASE-21528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Artem Ervits updated HBASE-21528:
-
Description:
this issue reappeared after migration of hbase-spark to hbase-connectors repo.
Originally addressed in https://issues.apache.org/jira/browse/HBASE-20175
introducing scala-maven-plugin and adding a flag to honor compatible major
scala version makes warning disappear.
{code:java}
[WARNING] Expected all dependencies to require Scala version: 2.11.12
[WARNING] org.apache.hbase.connectors.spark:hbase-spark:1.0.0-SNAPSHOT requires
scala version: 2.11.12
[WARNING] org.apache.spark:spark-streaming_2.11:2.4.0 requires scala version:
2.11.12
[WARNING] org.apache.spark:spark-streaming_2.11:2.4.0 requires scala version:
2.11.12
[WARNING] org.scala-lang:scala-reflect:2.11.12 requires scala version: 2.11.12
[WARNING] org.apache.spark:spark-tags_2.11:2.4.0 requires scala version: 2.11.12
[WARNING] com.twitter:chill_2.11:0.9.3 requires scala version: 2.11.12
[WARNING] org.scala-lang.modules:scala-parser-combinators_2.11:1.1.0 requires
scala version: 2.11.12
[WARNING] org.json4s:json4s-core_2.11:3.2.11 requires scala version: 2.11.0
[WARNING] Multiple versions of scala libraries detected!
[WARNING] warning: there were 17 deprecation warnings; re-run with -deprecation
for details
[WARNING] one warning found
[WARNING] Expected all dependencies to require Scala version: 2.11.12
[WARNING] org.apache.hbase.connectors.spark:hbase-spark:1.0.0-SNAPSHOT requires
scala version: 2.11.12
[WARNING] org.apache.spark:spark-streaming_2.11:2.4.0 requires scala version:
2.11.12
[WARNING] org.apache.spark:spark-streaming_2.11:2.4.0 requires scala version:
2.11.12
[WARNING] org.scala-lang:scala-reflect:2.11.12 requires scala version: 2.11.12
[WARNING] org.apache.spark:spark-tags_2.11:2.4.0 requires scala version: 2.11.12
[WARNING] com.twitter:chill_2.11:0.9.3 requires scala version: 2.11.12
[WARNING] org.scala-lang.modules:scala-parser-combinators_2.11:1.1.0 requires
scala version: 2.11.12
[WARNING] org.json4s:json4s-core_2.11:3.2.11 requires scala version: 2.11.0
[WARNING] Multiple versions of scala libraries detected!{code}
was:
this issue reappeared after migration of hbase-spark to hbase-connectors repo.
Originally addressed in https://issues.apache.org/jira/browse/HBASE-20175
introducing scala-maven-plugin and adding a flag to honor compatible major
scala version makes warning disappear.
> hbase-connectors need scala dependency convergence
> --
>
> Key: HBASE-21528
> URL: https://issues.apache.org/jira/browse/HBASE-21528
> Project: HBase
> Issue Type: Task
> Components: hbase-connectors
>Reporter: Artem Ervits
>Assignee: Artem Ervits
>Priority: Minor
> Attachments: HBASE-21528.v01.patch
>
>
> this issue reappeared after migration of hbase-spark to hbase-connectors
> repo. Originally addressed in
> https://issues.apache.org/jira/browse/HBASE-20175
> introducing scala-maven-plugin and adding a flag to honor compatible major
> scala version makes warning disappear.
>
> {code:java}
> [WARNING] Expected all dependencies to require Scala version: 2.11.12
> [WARNING] org.apache.hbase.connectors.spark:hbase-spark:1.0.0-SNAPSHOT
> requires scala version: 2.11.12
> [WARNING] org.apache.spark:spark-streaming_2.11:2.4.0 requires scala version:
> 2.11.12
> [WARNING] org.apache.spark:spark-streaming_2.11:2.4.0 requires scala version:
> 2.11.12
> [WARNING] org.scala-lang:scala-reflect:2.11.12 requires scala version: 2.11.12
> [WARNING] org.apache.spark:spark-tags_2.11:2.4.0 requires scala version:
> 2.11.12
> [WARNING] com.twitter:chill_2.11:0.9.3 requires scala version: 2.11.12
> [WARNING] org.scala-lang.modules:scala-parser-combinators_2.11:1.1.0 requires
> scala version: 2.11.12
> [WARNING] org.json4s:json4s-core_2.11:3.2.11 requires scala version: 2.11.0
> [WARNING] Multiple versions of scala libraries detected!
> [WARNING] warning: there were 17 deprecation warnings; re-run with
> -deprecation for details
> [WARNING] one warning found
> [WARNING] Expected all dependencies to require Scala version: 2.11.12
> [WARNING] org.apache.hbase.connectors.spark:hbase-spark:1.0.0-SNAPSHOT
> requires scala version: 2.11.12
> [WARNING] org.apache.spark:spark-streaming_2.11:2.4.0 requires scala version:
> 2.11.12
> [WARNING] org.apache.spark:spark-streaming_2.11:2.4.0 requires scala version:
> 2.11.12
> [WARNING] org.scala-lang:scala-reflect:2.11.12 requires scala version: 2.11.12
> [WARNING] org.apache.spark:spark-tags_2.11:2.4.0 requires scala version:
> 2.11.12
> [WARNING] com.twitter:chill_2.11:0.9.3 requires scala version: 2.11.12
> [WARNING] org.scala-lang.modules:scala-parser-combinators_2.11:1.1.0 requires
> scala version: 2.11.12
> [WARNING]
[jira] [Updated] (HBASE-21528) hbase-connectors need scala dependency convergence
[ https://issues.apache.org/jira/browse/HBASE-21528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artem Ervits updated HBASE-21528: - Attachment: HBASE-21528.v01.patch > hbase-connectors need scala dependency convergence > -- > > Key: HBASE-21528 > URL: https://issues.apache.org/jira/browse/HBASE-21528 > Project: HBase > Issue Type: Task > Components: hbase-connectors >Reporter: Artem Ervits >Assignee: Artem Ervits >Priority: Minor > Attachments: HBASE-21528.v01.patch > > > this issue reappeared after migration of hbase-spark to hbase-connectors > repo. Originally addressed in > https://issues.apache.org/jira/browse/HBASE-20175 > introducing scala-maven-plugin and adding a flag to honor compatible major > scala version makes warning disappear. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21528) hbase-connectors need scala dependency convergence
[ https://issues.apache.org/jira/browse/HBASE-21528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artem Ervits updated HBASE-21528: - Status: Patch Available (was: Open) > hbase-connectors need scala dependency convergence > -- > > Key: HBASE-21528 > URL: https://issues.apache.org/jira/browse/HBASE-21528 > Project: HBase > Issue Type: Task > Components: hbase-connectors >Reporter: Artem Ervits >Assignee: Artem Ervits >Priority: Minor > Attachments: HBASE-21528.v01.patch > > > this issue reappeared after migration of hbase-spark to hbase-connectors > repo. Originally addressed in > https://issues.apache.org/jira/browse/HBASE-20175 > introducing scala-maven-plugin and adding a flag to honor compatible major > scala version makes warning disappear. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21528) hbase-connectors need scala dependency convergence
[ https://issues.apache.org/jira/browse/HBASE-21528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artem Ervits updated HBASE-21528: - Summary: hbase-connectors need scala dependency convergence (was: hbase-connectors need scala library convergence) > hbase-connectors need scala dependency convergence > -- > > Key: HBASE-21528 > URL: https://issues.apache.org/jira/browse/HBASE-21528 > Project: HBase > Issue Type: Task > Components: hbase-connectors >Reporter: Artem Ervits >Assignee: Artem Ervits >Priority: Minor > > this issue reappeared after migration of hbase-spark to hbase-connectors > repo. Originally addressed in > https://issues.apache.org/jira/browse/HBASE-20175 > introducing scala-maven-plugin and adding a flag to honor compatible major > scala version makes warning disappear. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-21528) hbase-connectors need scala library convergence
Artem Ervits created HBASE-21528: Summary: hbase-connectors need scala library convergence Key: HBASE-21528 URL: https://issues.apache.org/jira/browse/HBASE-21528 Project: HBase Issue Type: Task Components: hbase-connectors Reporter: Artem Ervits Assignee: Artem Ervits this issue reappeared after migration of hbase-spark to hbase-connectors repo. Originally addressed in https://issues.apache.org/jira/browse/HBASE-20175 introducing scala-maven-plugin and adding a flag to honor compatible major scala version makes warning disappear. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21522) meta replicas appear to cause master restart to kill regionservers
[
https://issues.apache.org/jira/browse/HBASE-21522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703665#comment-16703665
]
Sergey Shelukhin commented on HBASE-21522:
--
Meta loaded event is set at the end of loadMeta, in joinCluster, that is in
turn called in finishActiveMasterInitialization before assigning meta replicas.
So, there's a (significant) window where a region server report will go into
else path (i.e. not "else if !isMetaLoaded()") and trigger this condition.
> meta replicas appear to cause master restart to kill regionservers
> --
>
> Key: HBASE-21522
> URL: https://issues.apache.org/jira/browse/HBASE-21522
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Sergey Shelukhin
>Priority: Major
>
> On master restart, AM.start adds FIRST_META_REGIONINFO to regionStates; that
> has replica ID of 0. Before the meta is loaded,
> AssignmentManager.checkOnlineRegionsReportForMeta is called for RS reports,
> and that also only checks for 0th replica of meta and loads it once
> discovered.
> Once the meta is loaded, RS reports are processed normally; however nobody
> appears to add meta replicas to regionStates.
> So, when an RS hosting one reports in, it gets killed:
> {noformat}
> * ABORTING region server :
> org.apache.hadoop.hbase.YouAreDeadException: Not online: hbase:meta,,1_0001
> * ABORTING region server :
> org.apache.hadoop.hbase.YouAreDeadException: Not online: hbase:meta,,1_0002
> {noformat}
> This exception is thrown when regionStates has no record for the region.
> RS in question shut down in an orderly manner and they do have the
> corresponding regions, that master then assigns to someone else in a few
> minutes.
> Still, this seems less than ideal.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (HBASE-21527) [2.0] Unnecessary DEBUG log in ConnectionImplementation#isTableEnabled
[ https://issues.apache.org/jira/browse/HBASE-21527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser resolved HBASE-21527. Resolution: Fixed Hadoop Flags: Reviewed Pushed to branch-2.0. Thanks for the input, Stack! > [2.0] Unnecessary DEBUG log in ConnectionImplementation#isTableEnabled > -- > > Key: HBASE-21527 > URL: https://issues.apache.org/jira/browse/HBASE-21527 > Project: HBase > Issue Type: Improvement > Components: Client >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Major > Fix For: 2.0.3 > > Attachments: HBASE-21524.001.branch-2.0.patch > > > Too much crap coming out of isTableEnabled for "normal" situations. Don't > need a DEBUG message to tell us that the table is available. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-18735) Provide a fast mechanism for shutting down mini cluster
[
https://issues.apache.org/jira/browse/HBASE-18735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703634#comment-16703634
]
Artem Ervits edited comment on HBASE-18735 at 11/29/18 6:39 PM:
The
{code:java}
org.apache.hadoop.hbase.regionserver.TestMultiColumnScanner{code}
test passes locally, albeit runs for 575 seconds on my Mac and unrelated to my
patch.
was (Author: dbist13):
The {org.apache.hadoop.hbase.regionserver.TestMultiColumnScanner} test passes
locally, albeit runs for 575 seconds on my Mac and unrelated to my patch.
> Provide a fast mechanism for shutting down mini cluster
> ---
>
> Key: HBASE-18735
> URL: https://issues.apache.org/jira/browse/HBASE-18735
> Project: HBase
> Issue Type: Wish
>Reporter: Samarth Jain
>Assignee: Artem Ervits
>Priority: Major
> Attachments: HBASE-18735.v01.patch, HBASE-18735.v02.patch,
> HBASE-18735.v03.patch
>
>
> The current mechanism of shutting down a mini cluster through
> HBaseTestingUtility.shutDownMiniCluster can take a lot of time when the mini
> cluster almost has a lot of tables. A lot of this time is spent in closing
> all the user regions. It would be nice to have a mechanism where this
> shutdown can happen quickly without having to worry about closing these user
> regions. At the same time, this mechanism would need to make sure that all
> the critical system resources like file handles and network ports are still
> released so that subsequently initialized mini clusters on the same JVM or
> system won't run into resource issues. This would make testing using HBase
> mini clusters much faster and immensely help out test frameworks of dependent
> projects like Phoenix.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-18735) Provide a fast mechanism for shutting down mini cluster
[
https://issues.apache.org/jira/browse/HBASE-18735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703634#comment-16703634
]
Artem Ervits edited comment on HBASE-18735 at 11/29/18 6:38 PM:
The {org.apache.hadoop.hbase.regionserver.TestMultiColumnScanner} test passes
locally, albeit runs for 575 seconds on my Mac and unrelated to my patch.
was (Author: dbist13):
The [org.apache.hadoop.hbase.regionserver.TestMultiColumnScanner] test passes
locally, albeit runs for 575 seconds on my Mac and unrelated to my patch.
> Provide a fast mechanism for shutting down mini cluster
> ---
>
> Key: HBASE-18735
> URL: https://issues.apache.org/jira/browse/HBASE-18735
> Project: HBase
> Issue Type: Wish
>Reporter: Samarth Jain
>Assignee: Artem Ervits
>Priority: Major
> Attachments: HBASE-18735.v01.patch, HBASE-18735.v02.patch,
> HBASE-18735.v03.patch
>
>
> The current mechanism of shutting down a mini cluster through
> HBaseTestingUtility.shutDownMiniCluster can take a lot of time when the mini
> cluster almost has a lot of tables. A lot of this time is spent in closing
> all the user regions. It would be nice to have a mechanism where this
> shutdown can happen quickly without having to worry about closing these user
> regions. At the same time, this mechanism would need to make sure that all
> the critical system resources like file handles and network ports are still
> released so that subsequently initialized mini clusters on the same JVM or
> system won't run into resource issues. This would make testing using HBase
> mini clusters much faster and immensely help out test frameworks of dependent
> projects like Phoenix.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-18735) Provide a fast mechanism for shutting down mini cluster
[ https://issues.apache.org/jira/browse/HBASE-18735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703634#comment-16703634 ] Artem Ervits commented on HBASE-18735: -- The [org.apache.hadoop.hbase.regionserver.TestMultiColumnScanner] test passes locally, albeit runs for 575 seconds on my Mac and unrelated to my patch. > Provide a fast mechanism for shutting down mini cluster > --- > > Key: HBASE-18735 > URL: https://issues.apache.org/jira/browse/HBASE-18735 > Project: HBase > Issue Type: Wish >Reporter: Samarth Jain >Assignee: Artem Ervits >Priority: Major > Attachments: HBASE-18735.v01.patch, HBASE-18735.v02.patch, > HBASE-18735.v03.patch > > > The current mechanism of shutting down a mini cluster through > HBaseTestingUtility.shutDownMiniCluster can take a lot of time when the mini > cluster almost has a lot of tables. A lot of this time is spent in closing > all the user regions. It would be nice to have a mechanism where this > shutdown can happen quickly without having to worry about closing these user > regions. At the same time, this mechanism would need to make sure that all > the critical system resources like file handles and network ports are still > released so that subsequently initialized mini clusters on the same JVM or > system won't run into resource issues. This would make testing using HBase > mini clusters much faster and immensely help out test frameworks of dependent > projects like Phoenix. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21464) Splitting blocked with meta NSRE during split transaction
[
https://issues.apache.org/jira/browse/HBASE-21464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703602#comment-16703602
]
Andrew Purtell commented on HBASE-21464:
Thanks for the review [~allan163]. Will check the latest with ITBLL and commit
if good.
> Splitting blocked with meta NSRE during split transaction
> -
>
> Key: HBASE-21464
> URL: https://issues.apache.org/jira/browse/HBASE-21464
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.5.0, 1.4.3, 1.4.4, 1.4.5, 1.4.6, 1.4.8, 1.4.7
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
>Priority: Blocker
> Fix For: 1.5.0, 1.4.9
>
> Attachments: HBASE-21464-branch-1.patch, HBASE-21464-branch-1.patch
>
>
> Splitting is blocked during split transaction. The split worker is trying to
> update meta but isn't able to relocate it after NSRE:
> {noformat}
> 2018-11-09 17:50:45,277 INFO
> [regionserver/ip-172-31-5-92.us-west-2.compute.internal/172.31.5.92:8120-splits-1541785709434]
> client.RpcRetryingCaller: Call exception, tries=13, retries=350,
> started=88590 ms ago, cancelled=false,
> msg=org.apache.hadoop.hbase.NotServingRegionException: Region hbase:meta,,1
> is not online on ip-172-31-13-83.us-west-2.compute.internal,8120,1541785618832
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:3088)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.getRegion(RSRpcServices.java:1271)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:2198)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:36617)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2396)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:297)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:277)row
> 'test,,1541785709452.5ba6596f0050c2dab969d152829227c6.44' on table
> 'hbase:meta' at region=hbase:meta,1.1588230740,
> hostname=ip-172-31-15-225.us-west-2.compute.internal,8120,1541785640586,
> seqNum=0{noformat}
> Clients, in this case YCSB, are hung with part of the keyspace missing:
> {noformat}
> 2018-11-09 17:51:06,033 DEBUG [hconnection-0x5739e567-shared--pool1-t165]
> client.ConnectionManager$HConnectionImplementation: locateRegionInMeta
> parentTable=hbase:meta, metaLocation=, attempt=14 of 35 failed; retrying
> after sleep of 20158 because: No server address listed in hbase:meta for
> region
> test,user307326104267982763,1541785754600.ef90030b05cb02305b75e9bfbc3ee081.
> containing row user3301635648728421323{noformat}
> Balancing cannot run indefinitely because the split transaction is stuck
> {noformat}
> 2018-11-09 17:49:55,478 DEBUG
> [RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=8100] master.HMaster:
> Not running balancer because 3 region(s) in transition:
> [{ef90030b05cb02305b75e9bfbc3ee081 state=SPLITTING_NEW, ts=1541785754606,
> server=ip-172-31-5-92.us-west-2.compute.internal,8120,1541785626417},
> {5ba6596f0050c2dab969d152829227c6 state=SPLITTING, ts=1541785754606,
> server=ip-172-31-5-92.us-west-2.compute{noformat}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21518) TestMasterFailoverWithProcedures is flaky
[
https://issues.apache.org/jira/browse/HBASE-21518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703571#comment-16703571
]
Sean Busbey commented on HBASE-21518:
-
+1
> TestMasterFailoverWithProcedures is flaky
> -
>
> Key: HBASE-21518
> URL: https://issues.apache.org/jira/browse/HBASE-21518
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.2.0, 2.0.3, 2.1.2
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Major
> Attachments: HBASE-21518-v1.patch, output.txt
>
>
> TestMasterFailoverWithProcedures test is failing frequently, times out. I
> faced this failure on 2.0.3RC0 vote and it also appears on multiple flaky
> dashboards.
> branch-2:
> [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Flaky-Tests/job/branch-2/2007/]
> branch-2.1:
> [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Flaky-Tests/job/branch-2.1/2002/]
> branch-2.0:
> [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Flaky-Tests/job/branch-2.0/1988/]
>
> {noformat}
> [INFO] Running
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures
> [ERROR] Tests run: 4, Failures: 0, Errors: 2, Skipped: 0, Time elapsed:
> 780.648 s <<< FAILURE! - in
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures
> [ERROR]
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures
> Time elapsed: 749.024 s <<< ERROR!
> org.junit.runners.model.TestTimedOutException: test timed out after 780
> seconds
> at
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures.tearDown(TestMasterFailoverWithProcedures.java:86)
> [ERROR]
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures
> Time elapsed: 749.051 s <<< ERROR!
> java.lang.Exception: Appears to be stuck in thread RS-EventLoopGroup-3-2
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21521) Expose master startup status via JMX and web UI
[ https://issues.apache.org/jira/browse/HBASE-21521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-21521: --- Description: Add an internal API to the master for tracking startup progress. Expose this information via JMX. Modify the master to bring the web UI up sooner. Will require tweaks to various views to prevent attempts to retrieve state before the master fully up (or else expect NPEs). Currently, before the master has fully initialized an attempt to use the web UI will return a 500 error code and display an error page. Finally, update the web UI to display startup progress, like HDFS-4249. Filing this for branch-1. Need to check what if anything is available or improved in branch-2 and master. was: Add an internal API to the master for tracking startup progress. Expose this information via JMX. Modify the master to bring the web UI up sooner. Will require tweaks to various views to prevent attempts to retrieve state before the master fully up (or else expect NPEs). Currently, before the master has fully initialized an attempt to use the web UI will return a 500 error code and display an error page. Finally, update the web UI to display startup progress, like HDFS-4249. (Example: [^HDFS-4249-2.png)] Filing this for branch-1. Need to check what if anything is available or improved in branch-2 and master. > Expose master startup status via JMX and web UI > --- > > Key: HBASE-21521 > URL: https://issues.apache.org/jira/browse/HBASE-21521 > Project: HBase > Issue Type: Improvement > Components: master, UI >Reporter: Andrew Purtell >Priority: Major > > Add an internal API to the master for tracking startup progress. Expose this > information via JMX. > Modify the master to bring the web UI up sooner. Will require tweaks to > various views to prevent attempts to retrieve state before the master fully > up (or else expect NPEs). Currently, before the master has fully initialized > an attempt to use the web UI will return a 500 error code and display an > error page. > Finally, update the web UI to display startup progress, like HDFS-4249. > Filing this for branch-1. Need to check what if anything is available or > improved in branch-2 and master. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-18735) Provide a fast mechanism for shutting down mini cluster
[
https://issues.apache.org/jira/browse/HBASE-18735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703531#comment-16703531
]
Hadoop QA commented on HBASE-18735:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
39s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
23s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
34s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
15s{color} | {color:green} hbase-server: The patch generated 0 new + 228
unchanged - 4 fixed = 228 total (was 232) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
18s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
9m 49s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}137m 51s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
23s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}179m 14s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.regionserver.TestMultiColumnScanner |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-18735 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12950018/HBASE-18735.v03.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux db5673e13012 4.4.0-139-generic #165~14.04.1-Ubuntu SMP Wed Oct
31 10:55:11 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / f1f2b5a038 |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC3 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/15150/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/15150/testReport/ |
| Max. process+thread count | 4668 (vs.
[jira] [Commented] (HBASE-21527) [2.0] Unnecessary DEBUG log in ConnectionImplementation#isTableEnabled
[ https://issues.apache.org/jira/browse/HBASE-21527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703526#comment-16703526 ] stack commented on HBASE-21527: --- Thanks [~elserj] +1 for branch-2.0. Will include in next RC if one... else 2.0.4. > [2.0] Unnecessary DEBUG log in ConnectionImplementation#isTableEnabled > -- > > Key: HBASE-21527 > URL: https://issues.apache.org/jira/browse/HBASE-21527 > Project: HBase > Issue Type: Improvement > Components: Client >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Major > Fix For: 2.0.3 > > Attachments: HBASE-21524.001.branch-2.0.patch > > > Too much crap coming out of isTableEnabled for "normal" situations. Don't > need a DEBUG message to tell us that the table is available. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21518) TestMasterFailoverWithProcedures is flaky
[
https://issues.apache.org/jira/browse/HBASE-21518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703523#comment-16703523
]
Peter Somogyi commented on HBASE-21518:
---
With the help of [~busbey] we figured out that both masters were considered as
active masters. This patch adds an extra check whether master is already
stopped so a previously killed master cannot claim it is still active.
> TestMasterFailoverWithProcedures is flaky
> -
>
> Key: HBASE-21518
> URL: https://issues.apache.org/jira/browse/HBASE-21518
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.2.0, 2.0.3, 2.1.2
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Major
> Attachments: HBASE-21518-v1.patch, output.txt
>
>
> TestMasterFailoverWithProcedures test is failing frequently, times out. I
> faced this failure on 2.0.3RC0 vote and it also appears on multiple flaky
> dashboards.
> branch-2:
> [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Flaky-Tests/job/branch-2/2007/]
> branch-2.1:
> [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Flaky-Tests/job/branch-2.1/2002/]
> branch-2.0:
> [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Flaky-Tests/job/branch-2.0/1988/]
>
> {noformat}
> [INFO] Running
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures
> [ERROR] Tests run: 4, Failures: 0, Errors: 2, Skipped: 0, Time elapsed:
> 780.648 s <<< FAILURE! - in
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures
> [ERROR]
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures
> Time elapsed: 749.024 s <<< ERROR!
> org.junit.runners.model.TestTimedOutException: test timed out after 780
> seconds
> at
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures.tearDown(TestMasterFailoverWithProcedures.java:86)
> [ERROR]
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures
> Time elapsed: 749.051 s <<< ERROR!
> java.lang.Exception: Appears to be stuck in thread RS-EventLoopGroup-3-2
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21518) TestMasterFailoverWithProcedures is flaky
[
https://issues.apache.org/jira/browse/HBASE-21518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Somogyi updated HBASE-21518:
--
Status: Patch Available (was: In Progress)
> TestMasterFailoverWithProcedures is flaky
> -
>
> Key: HBASE-21518
> URL: https://issues.apache.org/jira/browse/HBASE-21518
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.2.0, 2.0.3, 2.1.2
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Major
> Attachments: HBASE-21518-v1.patch, output.txt
>
>
> TestMasterFailoverWithProcedures test is failing frequently, times out. I
> faced this failure on 2.0.3RC0 vote and it also appears on multiple flaky
> dashboards.
> branch-2:
> [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Flaky-Tests/job/branch-2/2007/]
> branch-2.1:
> [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Flaky-Tests/job/branch-2.1/2002/]
> branch-2.0:
> [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Flaky-Tests/job/branch-2.0/1988/]
>
> {noformat}
> [INFO] Running
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures
> [ERROR] Tests run: 4, Failures: 0, Errors: 2, Skipped: 0, Time elapsed:
> 780.648 s <<< FAILURE! - in
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures
> [ERROR]
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures
> Time elapsed: 749.024 s <<< ERROR!
> org.junit.runners.model.TestTimedOutException: test timed out after 780
> seconds
> at
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures.tearDown(TestMasterFailoverWithProcedures.java:86)
> [ERROR]
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures
> Time elapsed: 749.051 s <<< ERROR!
> java.lang.Exception: Appears to be stuck in thread RS-EventLoopGroup-3-2
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21518) TestMasterFailoverWithProcedures is flaky
[
https://issues.apache.org/jira/browse/HBASE-21518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Somogyi updated HBASE-21518:
--
Attachment: HBASE-21518-v1.patch
> TestMasterFailoverWithProcedures is flaky
> -
>
> Key: HBASE-21518
> URL: https://issues.apache.org/jira/browse/HBASE-21518
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.2.0, 2.0.3, 2.1.2
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Major
> Attachments: HBASE-21518-v1.patch, output.txt
>
>
> TestMasterFailoverWithProcedures test is failing frequently, times out. I
> faced this failure on 2.0.3RC0 vote and it also appears on multiple flaky
> dashboards.
> branch-2:
> [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Flaky-Tests/job/branch-2/2007/]
> branch-2.1:
> [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Flaky-Tests/job/branch-2.1/2002/]
> branch-2.0:
> [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Flaky-Tests/job/branch-2.0/1988/]
>
> {noformat}
> [INFO] Running
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures
> [ERROR] Tests run: 4, Failures: 0, Errors: 2, Skipped: 0, Time elapsed:
> 780.648 s <<< FAILURE! - in
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures
> [ERROR]
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures
> Time elapsed: 749.024 s <<< ERROR!
> org.junit.runners.model.TestTimedOutException: test timed out after 780
> seconds
> at
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures.tearDown(TestMasterFailoverWithProcedures.java:86)
> [ERROR]
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures
> Time elapsed: 749.051 s <<< ERROR!
> java.lang.Exception: Appears to be stuck in thread RS-EventLoopGroup-3-2
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-18735) Provide a fast mechanism for shutting down mini cluster
[ https://issues.apache.org/jira/browse/HBASE-18735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703508#comment-16703508 ] Artem Ervits commented on HBASE-18735: -- [~jamestaylor] it's hard to say as I imagine the proportion of improvement is tied to number of tables and rows ingested. I originally tested with 1, 10 and 100 tables. Improvement was 1 second faster for 1 table, 5 seconds or so for 10 tables and 50 seconds for 100 tables. On 100 tables, improvement is 12%. If you want to identify a few tests in Phoenix to test this on, I will be happy to take this upon myself, let's do this in a Phoenix Jira, feel free to assign to me? > Provide a fast mechanism for shutting down mini cluster > --- > > Key: HBASE-18735 > URL: https://issues.apache.org/jira/browse/HBASE-18735 > Project: HBase > Issue Type: Wish >Reporter: Samarth Jain >Assignee: Artem Ervits >Priority: Major > Attachments: HBASE-18735.v01.patch, HBASE-18735.v02.patch, > HBASE-18735.v03.patch > > > The current mechanism of shutting down a mini cluster through > HBaseTestingUtility.shutDownMiniCluster can take a lot of time when the mini > cluster almost has a lot of tables. A lot of this time is spent in closing > all the user regions. It would be nice to have a mechanism where this > shutdown can happen quickly without having to worry about closing these user > regions. At the same time, this mechanism would need to make sure that all > the critical system resources like file handles and network ports are still > released so that subsequently initialized mini clusters on the same JVM or > system won't run into resource issues. This would make testing using HBase > mini clusters much faster and immensely help out test frameworks of dependent > projects like Phoenix. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21524) Unnecessary DEBUG log in ConnectionImplementation#isTableEnabled
[ https://issues.apache.org/jira/browse/HBASE-21524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-21524: --- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Thanks for the review, Stack. Created HBASE-21527 for 2.0 backport. > Unnecessary DEBUG log in ConnectionImplementation#isTableEnabled > > > Key: HBASE-21524 > URL: https://issues.apache.org/jira/browse/HBASE-21524 > Project: HBase > Issue Type: Improvement > Components: Client >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Major > Fix For: 3.0.0, 2.2.0, 2.1.2 > > Attachments: HBASE-21524.001.branch-2.0.patch > > > Too much crap coming out of isTableEnabled for "normal" situations. Don't > need a DEBUG message to tell us that the table is available. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21524) Unnecessary DEBUG log in ConnectionImplementation#isTableEnabled
[ https://issues.apache.org/jira/browse/HBASE-21524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-21524: --- Fix Version/s: (was: 2.0.3) 3.0.0 > Unnecessary DEBUG log in ConnectionImplementation#isTableEnabled > > > Key: HBASE-21524 > URL: https://issues.apache.org/jira/browse/HBASE-21524 > Project: HBase > Issue Type: Improvement > Components: Client >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Major > Fix For: 3.0.0, 2.2.0, 2.1.2 > > Attachments: HBASE-21524.001.branch-2.0.patch > > > Too much crap coming out of isTableEnabled for "normal" situations. Don't > need a DEBUG message to tell us that the table is available. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21527) [2.0] Unnecessary DEBUG log in ConnectionImplementation#isTableEnabled
[ https://issues.apache.org/jira/browse/HBASE-21527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703495#comment-16703495 ] Josh Elser commented on HBASE-21527: FYI [~stack], spun this out for 2.0. > [2.0] Unnecessary DEBUG log in ConnectionImplementation#isTableEnabled > -- > > Key: HBASE-21527 > URL: https://issues.apache.org/jira/browse/HBASE-21527 > Project: HBase > Issue Type: Improvement > Components: Client >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Major > Fix For: 2.0.3 > > Attachments: HBASE-21524.001.branch-2.0.patch > > > Too much crap coming out of isTableEnabled for "normal" situations. Don't > need a DEBUG message to tell us that the table is available. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21527) [2.0] Unnecessary DEBUG log in ConnectionImplementation#isTableEnabled
[ https://issues.apache.org/jira/browse/HBASE-21527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-21527: --- Fix Version/s: (was: 2.1.2) (was: 2.2.0) > [2.0] Unnecessary DEBUG log in ConnectionImplementation#isTableEnabled > -- > > Key: HBASE-21527 > URL: https://issues.apache.org/jira/browse/HBASE-21527 > Project: HBase > Issue Type: Improvement > Components: Client >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Major > Fix For: 2.0.3 > > Attachments: HBASE-21524.001.branch-2.0.patch > > > Too much crap coming out of isTableEnabled for "normal" situations. Don't > need a DEBUG message to tell us that the table is available. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-21527) [2.0] Unnecessary DEBUG log in ConnectionImplementation#isTableEnabled
Josh Elser created HBASE-21527: -- Summary: [2.0] Unnecessary DEBUG log in ConnectionImplementation#isTableEnabled Key: HBASE-21527 URL: https://issues.apache.org/jira/browse/HBASE-21527 Project: HBase Issue Type: Improvement Components: Client Reporter: Josh Elser Assignee: Josh Elser Fix For: 2.2.0, 2.0.3, 2.1.2 Attachments: HBASE-21524.001.branch-2.0.patch Too much crap coming out of isTableEnabled for "normal" situations. Don't need a DEBUG message to tell us that the table is available. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21523) Chatty DEBUG logging in Master log
[
https://issues.apache.org/jira/browse/HBASE-21523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser updated HBASE-21523:
---
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Pushed to master. Thanks, Peter!
> Chatty DEBUG logging in Master log
> --
>
> Key: HBASE-21523
> URL: https://issues.apache.org/jira/browse/HBASE-21523
> Project: HBase
> Issue Type: Improvement
> Components: backuprestore
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-21523.001.patch, HBASE-21523.002.patch
>
>
> Noticed this in testing, a lot of spam in the Master log at DEBUG.
> {noformat}
> 2018-11-27 01:12:28,322 DEBUG [ForkJoinPool-1-worker-6] impl.BackupMetaTable:
> Backup table backup:system exists and available
> 2018-11-27 01:12:28,326 DEBUG [ForkJoinPool-1-worker-6] impl.BackupMetaTable:
> Backup table backup:system_bulk exists and available
> 2018-11-27 01:12:28,341 DEBUG [ForkJoinPool-1-worker-6] impl.BackupMetaTable:
> Backup table backup:system exists and available
> 2018-11-27 01:12:28,345 DEBUG [ForkJoinPool-1-worker-6] impl.BackupMetaTable:
> Backup table backup:system_bulk exists and available
> {noformat}
> FYI [~vrodionov]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21523) Chatty DEBUG logging in Master log
[
https://issues.apache.org/jira/browse/HBASE-21523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703470#comment-16703470
]
Hadoop QA commented on HBASE-21523:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
38s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
55s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
10m 31s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 12m
45s{color} | {color:green} hbase-backup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 48m 18s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21523 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12950026/HBASE-21523.002.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 048dc562518d 4.4.0-139-generic #165~14.04.1-Ubuntu SMP Wed Oct
31 10:55:11 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
|
| git revision | master / f1f2b5a038 |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/15151/testReport/ |
| Max. process+thread count | 4125 (vs. ulimit of 1) |
| modules | C: hbase-backup U: hbase-backup |
| Console output |
[jira] [Commented] (HBASE-21523) Chatty DEBUG logging in Master log
[
https://issues.apache.org/jira/browse/HBASE-21523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703472#comment-16703472
]
Josh Elser commented on HBASE-21523:
Thanks much, Peter!
> Chatty DEBUG logging in Master log
> --
>
> Key: HBASE-21523
> URL: https://issues.apache.org/jira/browse/HBASE-21523
> Project: HBase
> Issue Type: Improvement
> Components: backuprestore
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-21523.001.patch, HBASE-21523.002.patch
>
>
> Noticed this in testing, a lot of spam in the Master log at DEBUG.
> {noformat}
> 2018-11-27 01:12:28,322 DEBUG [ForkJoinPool-1-worker-6] impl.BackupMetaTable:
> Backup table backup:system exists and available
> 2018-11-27 01:12:28,326 DEBUG [ForkJoinPool-1-worker-6] impl.BackupMetaTable:
> Backup table backup:system_bulk exists and available
> 2018-11-27 01:12:28,341 DEBUG [ForkJoinPool-1-worker-6] impl.BackupMetaTable:
> Backup table backup:system exists and available
> 2018-11-27 01:12:28,345 DEBUG [ForkJoinPool-1-worker-6] impl.BackupMetaTable:
> Backup table backup:system_bulk exists and available
> {noformat}
> FYI [~vrodionov]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-18735) Provide a fast mechanism for shutting down mini cluster
[ https://issues.apache.org/jira/browse/HBASE-18735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703468#comment-16703468 ] James Taylor commented on HBASE-18735: -- Thanks, [~dbist]. This is definitely useful. You mentioned that it’s 40-50 seconds faster. What’s the percentage improvement? If you change Phoenix to use this new mechanism, are the test runs faster? In Phoenix, to prevent the long shutdown time, we currently keep the mini cluster up across test suites and asynchronously drop tables to prevent OOM errors (since we create so many of them). If startup/shutdown time is fast, we could instead do that. > Provide a fast mechanism for shutting down mini cluster > --- > > Key: HBASE-18735 > URL: https://issues.apache.org/jira/browse/HBASE-18735 > Project: HBase > Issue Type: Wish >Reporter: Samarth Jain >Assignee: Artem Ervits >Priority: Major > Attachments: HBASE-18735.v01.patch, HBASE-18735.v02.patch, > HBASE-18735.v03.patch > > > The current mechanism of shutting down a mini cluster through > HBaseTestingUtility.shutDownMiniCluster can take a lot of time when the mini > cluster almost has a lot of tables. A lot of this time is spent in closing > all the user regions. It would be nice to have a mechanism where this > shutdown can happen quickly without having to worry about closing these user > regions. At the same time, this mechanism would need to make sure that all > the critical system resources like file handles and network ports are still > released so that subsequently initialized mini clusters on the same JVM or > system won't run into resource issues. This would make testing using HBase > mini clusters much faster and immensely help out test frameworks of dependent > projects like Phoenix. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21523) Chatty DEBUG logging in Master log
[
https://issues.apache.org/jira/browse/HBASE-21523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703469#comment-16703469
]
Peter Somogyi commented on HBASE-21523:
---
+1 on v2.
> Chatty DEBUG logging in Master log
> --
>
> Key: HBASE-21523
> URL: https://issues.apache.org/jira/browse/HBASE-21523
> Project: HBase
> Issue Type: Improvement
> Components: backuprestore
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-21523.001.patch, HBASE-21523.002.patch
>
>
> Noticed this in testing, a lot of spam in the Master log at DEBUG.
> {noformat}
> 2018-11-27 01:12:28,322 DEBUG [ForkJoinPool-1-worker-6] impl.BackupMetaTable:
> Backup table backup:system exists and available
> 2018-11-27 01:12:28,326 DEBUG [ForkJoinPool-1-worker-6] impl.BackupMetaTable:
> Backup table backup:system_bulk exists and available
> 2018-11-27 01:12:28,341 DEBUG [ForkJoinPool-1-worker-6] impl.BackupMetaTable:
> Backup table backup:system exists and available
> 2018-11-27 01:12:28,345 DEBUG [ForkJoinPool-1-worker-6] impl.BackupMetaTable:
> Backup table backup:system_bulk exists and available
> {noformat}
> FYI [~vrodionov]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21300) Fix the wrong reference file path when restoring snapshots for tables with MOB columns
[
https://issues.apache.org/jira/browse/HBASE-21300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703446#comment-16703446
]
Hudson commented on HBASE-21300:
Results for branch branch-2.1
[build #645 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/645/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/645//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/645//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/645//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Fix the wrong reference file path when restoring snapshots for tables with
> MOB columns
> --
>
> Key: HBASE-21300
> URL: https://issues.apache.org/jira/browse/HBASE-21300
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0, 2.2.0
>Reporter: Yi Mei
>Assignee: Yi Mei
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.0.3, 2.1.2
>
> Attachments: HBASE-21300.branch-2.0.001.patch,
> HBASE-21300.branch-2.1.001.patch, HBASE-21300.master.001.patch,
> HBASE-21300.master.002.patch, HBASE-21300.master.003.patch,
> HBASE-21300.master.004.patch, HBASE-21300.master.005.patch,
> HBASE-21300.master.006.patch, HBASE-21300.master.007.patch,
> HBASE-21300.master.008.patch
>
>
> When restoring snapshots for tables with MOB columns, the reference files for
> mob region are created under hbase root dir, rather than restore dir.
> Some of the mob reference file paths are as follows:
> {quote}hdfs:/7ae0d109-3ca4-d0e7-7250-62ed234ab247/mobdir/data/ns_testMob/testMob
> hdfs:/7ae0d109-3ca4-d0e7-7250-62ed234ab247/mobdir/data/ns_testMob/testMob/057a856eb65753c6e6bdb168ba58a0b2
> hdfs:/7ae0d109-3ca4-d0e7-7250-62ed234ab247/mobdir/data/ns_testMob/testMob/057a856eb65753c6e6bdb168ba58a0b2/A
> hdfs:/7ae0d109-3ca4-d0e7-7250-62ed234ab247/mobdir/data/ns_testMob/testMob/057a856eb65753c6e6bdb168ba58a0b2/A/d41d8cd98f00b204e9800998ecf8427e201810120fc8e2446f174598a7280a81b1134cee
> hdfs:/7ae0d109-3ca4-d0e7-7250-62ed234ab247/mobdir/data/ns_testMob/testMob/057a856eb65753c6e6bdb168ba58a0b2/A/ns_testMob=testMob=057a856eb65753c6e6bdb168ba58a0b2-d41d8cd98f00b204e9800998ecf8427e201810120fc8e2446f174598a7280a81b1134cee
> {quote}
> The restore dir files are as follows:
> {quote}hdfs://hbase/.tmpdir-to-restore-snapshot/856e06fa-e018-4e95-9647-2cfbd5161e7e
> hdfs://hbase/.tmpdir-to-restore-snapshot/856e06fa-e018-4e95-9647-2cfbd5161e7e/data
> hdfs://hbase/.tmpdir-to-restore-snapshot/856e06fa-e018-4e95-9647-2cfbd5161e7e/data/ns_testMob
> hdfs://hbase/.tmpdir-to-restore-snapshot/856e06fa-e018-4e95-9647-2cfbd5161e7e/data/ns_testMob/testMob
> hdfs://hbase/.tmpdir-to-restore-snapshot/856e06fa-e018-4e95-9647-2cfbd5161e7e/data/ns_testMob/testMob/ecdf66f0d8c09a816faf37336ad262e1
> hdfs://hbase/.tmpdir-to-restore-snapshot/856e06fa-e018-4e95-9647-2cfbd5161e7e/data/ns_testMob/testMob/ecdf66f0d8c09a816faf37336ad262e1/.regioninfo
> hdfs://hbase/.tmpdir-to-restore-snapshot/856e06fa-e018-4e95-9647-2cfbd5161e7e/data/ns_testMob/testMob/ecdf66f0d8c09a816faf37336ad262e1/A
> hdfs://hbase/.tmpdir-to-restore-snapshot/856e06fa-e018-4e95-9647-2cfbd5161e7e/data/ns_testMob/testMob/ecdf66f0d8c09a816faf37336ad262e1/A/ns_testMob=testMob=ecdf66f0d8c09a816faf37336ad262e1-7208172df03b46518370643aa28ffd05
> {quote}
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21300) Fix the wrong reference file path when restoring snapshots for tables with MOB columns
[
https://issues.apache.org/jira/browse/HBASE-21300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703400#comment-16703400
]
Hudson commented on HBASE-21300:
Results for branch branch-2.0
[build #1123 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1123/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1123//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1123//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1123//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Fix the wrong reference file path when restoring snapshots for tables with
> MOB columns
> --
>
> Key: HBASE-21300
> URL: https://issues.apache.org/jira/browse/HBASE-21300
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0, 2.2.0
>Reporter: Yi Mei
>Assignee: Yi Mei
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.0.3, 2.1.2
>
> Attachments: HBASE-21300.branch-2.0.001.patch,
> HBASE-21300.branch-2.1.001.patch, HBASE-21300.master.001.patch,
> HBASE-21300.master.002.patch, HBASE-21300.master.003.patch,
> HBASE-21300.master.004.patch, HBASE-21300.master.005.patch,
> HBASE-21300.master.006.patch, HBASE-21300.master.007.patch,
> HBASE-21300.master.008.patch
>
>
> When restoring snapshots for tables with MOB columns, the reference files for
> mob region are created under hbase root dir, rather than restore dir.
> Some of the mob reference file paths are as follows:
> {quote}hdfs:/7ae0d109-3ca4-d0e7-7250-62ed234ab247/mobdir/data/ns_testMob/testMob
> hdfs:/7ae0d109-3ca4-d0e7-7250-62ed234ab247/mobdir/data/ns_testMob/testMob/057a856eb65753c6e6bdb168ba58a0b2
> hdfs:/7ae0d109-3ca4-d0e7-7250-62ed234ab247/mobdir/data/ns_testMob/testMob/057a856eb65753c6e6bdb168ba58a0b2/A
> hdfs:/7ae0d109-3ca4-d0e7-7250-62ed234ab247/mobdir/data/ns_testMob/testMob/057a856eb65753c6e6bdb168ba58a0b2/A/d41d8cd98f00b204e9800998ecf8427e201810120fc8e2446f174598a7280a81b1134cee
> hdfs:/7ae0d109-3ca4-d0e7-7250-62ed234ab247/mobdir/data/ns_testMob/testMob/057a856eb65753c6e6bdb168ba58a0b2/A/ns_testMob=testMob=057a856eb65753c6e6bdb168ba58a0b2-d41d8cd98f00b204e9800998ecf8427e201810120fc8e2446f174598a7280a81b1134cee
> {quote}
> The restore dir files are as follows:
> {quote}hdfs://hbase/.tmpdir-to-restore-snapshot/856e06fa-e018-4e95-9647-2cfbd5161e7e
> hdfs://hbase/.tmpdir-to-restore-snapshot/856e06fa-e018-4e95-9647-2cfbd5161e7e/data
> hdfs://hbase/.tmpdir-to-restore-snapshot/856e06fa-e018-4e95-9647-2cfbd5161e7e/data/ns_testMob
> hdfs://hbase/.tmpdir-to-restore-snapshot/856e06fa-e018-4e95-9647-2cfbd5161e7e/data/ns_testMob/testMob
> hdfs://hbase/.tmpdir-to-restore-snapshot/856e06fa-e018-4e95-9647-2cfbd5161e7e/data/ns_testMob/testMob/ecdf66f0d8c09a816faf37336ad262e1
> hdfs://hbase/.tmpdir-to-restore-snapshot/856e06fa-e018-4e95-9647-2cfbd5161e7e/data/ns_testMob/testMob/ecdf66f0d8c09a816faf37336ad262e1/.regioninfo
> hdfs://hbase/.tmpdir-to-restore-snapshot/856e06fa-e018-4e95-9647-2cfbd5161e7e/data/ns_testMob/testMob/ecdf66f0d8c09a816faf37336ad262e1/A
> hdfs://hbase/.tmpdir-to-restore-snapshot/856e06fa-e018-4e95-9647-2cfbd5161e7e/data/ns_testMob/testMob/ecdf66f0d8c09a816faf37336ad262e1/A/ns_testMob=testMob=ecdf66f0d8c09a816faf37336ad262e1-7208172df03b46518370643aa28ffd05
> {quote}
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21524) Unnecessary DEBUG log in ConnectionImplementation#isTableEnabled
[ https://issues.apache.org/jira/browse/HBASE-21524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703401#comment-16703401 ] Josh Elser commented on HBASE-21524: Thanks [~stack]. What would you like me to do for branch-2.0 with your 2.0.3 VOTE thread out? > Unnecessary DEBUG log in ConnectionImplementation#isTableEnabled > > > Key: HBASE-21524 > URL: https://issues.apache.org/jira/browse/HBASE-21524 > Project: HBase > Issue Type: Improvement > Components: Client >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Major > Fix For: 2.2.0, 2.0.3, 2.1.2 > > Attachments: HBASE-21524.001.branch-2.0.patch > > > Too much crap coming out of isTableEnabled for "normal" situations. Don't > need a DEBUG message to tell us that the table is available. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21523) Chatty DEBUG logging in Master log
[
https://issues.apache.org/jira/browse/HBASE-21523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser updated HBASE-21523:
---
Attachment: HBASE-21523.002.patch
> Chatty DEBUG logging in Master log
> --
>
> Key: HBASE-21523
> URL: https://issues.apache.org/jira/browse/HBASE-21523
> Project: HBase
> Issue Type: Improvement
> Components: backuprestore
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-21523.001.patch, HBASE-21523.002.patch
>
>
> Noticed this in testing, a lot of spam in the Master log at DEBUG.
> {noformat}
> 2018-11-27 01:12:28,322 DEBUG [ForkJoinPool-1-worker-6] impl.BackupMetaTable:
> Backup table backup:system exists and available
> 2018-11-27 01:12:28,326 DEBUG [ForkJoinPool-1-worker-6] impl.BackupMetaTable:
> Backup table backup:system_bulk exists and available
> 2018-11-27 01:12:28,341 DEBUG [ForkJoinPool-1-worker-6] impl.BackupMetaTable:
> Backup table backup:system exists and available
> 2018-11-27 01:12:28,345 DEBUG [ForkJoinPool-1-worker-6] impl.BackupMetaTable:
> Backup table backup:system_bulk exists and available
> {noformat}
> FYI [~vrodionov]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21523) Chatty DEBUG logging in Master log
[
https://issues.apache.org/jira/browse/HBASE-21523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703351#comment-16703351
]
Josh Elser commented on HBASE-21523:
bq. Please use parameterized logging and remove LOG.isDebugEnabled().
Sure. Didnt' do it the first time because it was still using the old logger (I
thought?). Let me dbl check and fix the checkstyle too.
> Chatty DEBUG logging in Master log
> --
>
> Key: HBASE-21523
> URL: https://issues.apache.org/jira/browse/HBASE-21523
> Project: HBase
> Issue Type: Improvement
> Components: backuprestore
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-21523.001.patch
>
>
> Noticed this in testing, a lot of spam in the Master log at DEBUG.
> {noformat}
> 2018-11-27 01:12:28,322 DEBUG [ForkJoinPool-1-worker-6] impl.BackupMetaTable:
> Backup table backup:system exists and available
> 2018-11-27 01:12:28,326 DEBUG [ForkJoinPool-1-worker-6] impl.BackupMetaTable:
> Backup table backup:system_bulk exists and available
> 2018-11-27 01:12:28,341 DEBUG [ForkJoinPool-1-worker-6] impl.BackupMetaTable:
> Backup table backup:system exists and available
> 2018-11-27 01:12:28,345 DEBUG [ForkJoinPool-1-worker-6] impl.BackupMetaTable:
> Backup table backup:system_bulk exists and available
> {noformat}
> FYI [~vrodionov]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21518) TestMasterFailoverWithProcedures is flaky
[
https://issues.apache.org/jira/browse/HBASE-21518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703332#comment-16703332
]
Peter Somogyi commented on HBASE-21518:
---
ServerManager#expireServer checks if cluster is shutting down and in this case
does not create ServerCrashProcedure for dead servers. This AtomicBoolean
variable is set to true when ServerManager#shutdownCluster method is called,
however, there are 2 ServerManager instances and on expireServer a different
one is checked.
I added some debug logs with hashcodes where you can see that clusterShutdown
was set to true (hash=1980707837) but later on during shutdown the variable
contains false (hash=416244779) that's why ServerCrashProcedure is created
which hangs since there are no Master.
{noformat}
2018-11-29 15:43:21,948 INFO [Thread-81] master.ServerManager(160):
ServerManager initialized. clusterShutdown false, thread 210, hash 1980707837
2018-11-29 15:43:23,929 INFO [Thread-81] master.ServerManager(913): Called
isClusterShutdown. clusterShutdown=false, thread 210, hash=1980707837
2018-11-29 15:43:23,929 INFO [Thread-81] master.ServerManager(913): Called
isClusterShutdown. clusterShutdown=false, thread 210, hash=1980707837
2018-11-29 15:43:29,732 INFO [Thread-80] master.ServerManager(160):
ServerManager initialized. clusterShutdown false, thread 209, hash 416244779
2018-11-29 15:43:29,820 INFO [Thread-80] master.ServerManager(913): Called
isClusterShutdown. clusterShutdown=false, thread 209, hash=416244779
2018-11-29 15:43:29,820 INFO [Thread-80] master.ServerManager(913): Called
isClusterShutdown. clusterShutdown=false, thread 209, hash=416244779
2018-11-29 15:43:29,937 INFO [Time-limited test] master.ServerManager(904): Set
clusterShutdown to true, thread 14, hash 1980707837
2018-11-29 15:43:30,985 INFO [RegionServerTracker-0] master.ServerManager(913):
Called isClusterShutdown. clusterShutdown=false, thread 461, hash=416244779
2018-11-29 15:43:30,986 INFO [RegionServerTracker-0] master.ServerManager(913):
Called isClusterShutdown. clusterShutdown=false, thread 461, hash=416244779
2018-11-29 15:48:29,851 INFO [master/172.30.65.195:0.Chore.1]
master.ServerManager(913): Called isClusterShutdown. clusterShutdown=false,
thread 417, hash=416244779
2018-11-29 15:48:29,852 INFO [master/172.30.65.195:0.Chore.1]
master.ServerManager(913): Called isClusterShutdown. clusterShutdown=false,
thread 417, hash=416244779
2018-11-29 15:48:29,852 INFO [master/172.30.65.195:0.Chore.1]
master.ServerManager(913): Called isClusterShutdown. clusterShutdown=false,
thread 417, hash=416244779
2018-11-29 15:53:32,277 INFO [master/172.30.65.195:0.Chore.1]
master.ServerManager(913): Called isClusterShutdown. clusterShutdown=false,
thread 417, hash=416244779
2018-11-29 15:53:32,277 INFO [master/172.30.65.195:0.Chore.1]
master.ServerManager(913): Called isClusterShutdown. clusterShutdown=false,
thread 417, hash=416244779
2018-11-29 15:53:32,277 INFO [master/172.30.65.195:0.Chore.1]
master.ServerManager(913): Called isClusterShutdown. clusterShutdown=false,
thread 417, hash=416244779{noformat}
> TestMasterFailoverWithProcedures is flaky
> -
>
> Key: HBASE-21518
> URL: https://issues.apache.org/jira/browse/HBASE-21518
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.2.0, 2.0.3, 2.1.2
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Major
> Attachments: output.txt
>
>
> TestMasterFailoverWithProcedures test is failing frequently, times out. I
> faced this failure on 2.0.3RC0 vote and it also appears on multiple flaky
> dashboards.
> branch-2:
> [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Flaky-Tests/job/branch-2/2007/]
> branch-2.1:
> [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Flaky-Tests/job/branch-2.1/2002/]
> branch-2.0:
> [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Flaky-Tests/job/branch-2.0/1988/]
>
> {noformat}
> [INFO] Running
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures
> [ERROR] Tests run: 4, Failures: 0, Errors: 2, Skipped: 0, Time elapsed:
> 780.648 s <<< FAILURE! - in
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures
> [ERROR]
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures
> Time elapsed: 749.024 s <<< ERROR!
> org.junit.runners.model.TestTimedOutException: test timed out after 780
> seconds
> at
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures.tearDown(TestMasterFailoverWithProcedures.java:86)
> [ERROR]
> org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures
> Time elapsed: 749.051 s <<< ERROR!
> java.lang.Exception: Appears to be stuck in thread RS-EventLoopGroup-3-2
> {noformat}
--
This message was
[jira] [Commented] (HBASE-21401) Sanity check when constructing the KeyValue
[
https://issues.apache.org/jira/browse/HBASE-21401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703289#comment-16703289
]
Hadoop QA commented on HBASE-21401:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
24s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
51s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
43s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
16s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
24s{color} | {color:red} hbase-common: The patch generated 21 new + 148
unchanged - 1 fixed = 169 total (was 149) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
49s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 41s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
47s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 2m 34s{color}
| {color:red} hbase-common in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}248m 19s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
41s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}292m 52s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.TestKeyValue |
| | hadoop.hbase.master.TestAssignmentManagerMetrics |
| | hadoop.hbase.io.hfile.TestCacheOnWrite |
| | hadoop.hbase.io.encoding.TestDataBlockEncoders |
| | hadoop.hbase.io.hfile.TestHFileSeek |
| | hadoop.hbase.client.TestAdmin1 |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21401 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12949984/HBASE-21401.v6.patch |
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux ab114c9e23ba 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
[jira] [Commented] (HBASE-21515) Also initialize an AsyncClusterConnection in HRegionServer
[
https://issues.apache.org/jira/browse/HBASE-21515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703269#comment-16703269
]
Hadoop QA commented on HBASE-21515:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
12s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 20 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
53s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
51s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
9s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 2m
25s{color} | {color:red} root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
32s{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 32s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s{color} | {color:green} The patch passed checkstyle in hbase-common {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
31s{color} | {color:red} hbase-client: The patch generated 3 new + 3 unchanged
- 3 fixed = 6 total (was 6) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
10s{color} | {color:green} hbase-server: The patch generated 0 new + 285
unchanged - 2 fixed = 285 total (was 287) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} shadedjars {color} | {color:red} 2m
50s{color} | {color:red} patch has 16 errors when building our shaded
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 1m
34s{color} | {color:red} The patch causes 16 errors with Hadoop v2.7.4. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 3m
10s{color} | {color:red} The patch causes 16 errors with Hadoop v3.0.0. {color}
|
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
28s{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
45s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
14s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 29s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
28s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 41m 44s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce
