[jira] [Updated] (HBASE-25219) Backport HBASE-24369 "Provide more information about merged child regions in Hbck Overlaps section, which cannot be fixed immediately" to branch-2.2
[ https://issues.apache.org/jira/browse/HBASE-25219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi updated HBASE-25219: -- Fix Version/s: 2.2.7 Resolution: Fixed Status: Resolved (was: Patch Available) > Backport HBASE-24369 "Provide more information about merged child regions in > Hbck Overlaps section, which cannot be fixed immediately" to branch-2.2 > > > Key: HBASE-25219 > URL: https://issues.apache.org/jira/browse/HBASE-25219 > Project: HBase > Issue Type: Sub-task >Reporter: Peter Somogyi >Assignee: Peter Somogyi >Priority: Major > Fix For: 2.2.7 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] petersomogyi merged pull request #2589: HBASE-25219 Backport HBASE-24369 Provide more information about merged child regions in Hbck Overlaps section, which cannot be fixed immediatel
petersomogyi merged pull request #2589: URL: https://github.com/apache/hbase/pull/2589 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] infraio commented on a change in pull request #2579: HBASE-24999 Master manages ReplicationServers
infraio commented on a change in pull request #2579:
URL: https://github.com/apache/hbase/pull/2579#discussion_r512413846
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterRpcServices.java
##
@@ -3402,4 +3406,33 @@ public ListReplicationSinkServersResponse
listReplicationSinkServers(
}
return builder.build();
}
+
+ @Override
+ public RegionServerReportResponse replicationServerReport(RpcController
controller,
+ RegionServerReportRequest request) throws ServiceException {
+try {
+ master.checkServiceStarted();
+ int versionNumber = 0;
+ String version = "0.0.0";
+ VersionInfo versionInfo = VersionInfoUtil.getCurrentClientVersionInfo();
+ if (versionInfo != null) {
+version = versionInfo.getVersion();
+versionNumber = VersionInfoUtil.getVersionNumber(versionInfo);
+ }
+ ClusterStatusProtos.ServerLoad sl = request.getLoad();
+ ServerName serverName = ProtobufUtil.toServerName(request.getServer());
+ ServerMetrics oldMetrics =
master.getReplicationServerManager().getServerMetrics(serverName);
+ ServerMetrics newMetrics =
+ ServerMetricsBuilder.toServerMetrics(serverName, versionNumber,
version, sl);
+ master.getReplicationServerManager().serverReport(serverName,
newMetrics);
+ if (sl != null && master.metricsMaster != null) {
+// Up our metrics.
+master.metricsMaster.incrementRequests(sl.getTotalNumberOfRequests()
Review comment:
Got it.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2579: HBASE-24999 Master manages ReplicationServers
Apache-HBase commented on pull request #2579: URL: https://github.com/apache/hbase/pull/2579#issuecomment-716974126 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 28s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | prototool | 0m 0s | prototool was not available. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ HBASE-24666 Compile Tests _ | | +0 :ok: | mvndep | 0m 29s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 24s | HBASE-24666 passed | | +1 :green_heart: | checkstyle | 1m 18s | HBASE-24666 passed | | -1 :x: | spotbugs | 1m 58s | hbase-server in HBASE-24666 has 3 extant spotbugs warnings. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 23s | the patch passed | | +1 :green_heart: | checkstyle | 0m 11s | The patch passed checkstyle in hbase-protocol-shaded | | +1 :green_heart: | checkstyle | 1m 8s | hbase-server: The patch generated 0 new + 147 unchanged - 3 fixed = 147 total (was 150) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 16m 55s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | hbaseprotoc | 1m 38s | the patch passed | | +1 :green_heart: | spotbugs | 3m 51s | hbase-protocol-shaded in the patch passed. | | +1 :green_heart: | spotbugs | 2m 7s | hbase-server generated 0 new + 0 unchanged - 3 fixed = 0 total (was 3) | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 25s | The patch does not generate ASF License warnings. | | | | 48m 39s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2579/5/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2579 | | Optional Tests | dupname asflicense cc hbaseprotoc prototool spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 1ce4f90fc5a4 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | HBASE-24666 / 1a64f1a0db | | spotbugs | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2579/5/artifact/yetus-general-check/output/branch-spotbugs-hbase-server-warnings.html | | Max. process+thread count | 94 (vs. ulimit of 3) | | modules | C: hbase-protocol-shaded hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2579/5/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Reidddddd commented on a change in pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close
Reidd commented on a change in pull request #2574:
URL: https://github.com/apache/hbase/pull/2574#discussion_r512404932
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
##
@@ -4569,13 +4665,29 @@ private void doMiniBatchMutate(BatchOperation
batchOp) throws IOException {
// We should record the timestamp only after we have acquired the
rowLock,
// otherwise, newer puts/deletes/increment/append are not guaranteed to
have a newer
// timestamp
+
+ // Check for thread interrupt status in case we have been signaled from
+ // #interruptRegionOperation.
+ checkInterrupt();
+
long now = EnvironmentEdgeManager.currentTime();
batchOp.prepareMiniBatchOperations(miniBatchOp, now, acquiredRowLocks);
// STEP 3. Build WAL edit
+
+ // Check for thread interrupt status in case we have been signaled from
+ // #interruptRegionOperation.
+ checkInterrupt();
+
List> walEdits =
batchOp.buildWALEdits(miniBatchOp);
// STEP 4. Append the WALEdits to WAL and sync.
+
+ // Check for thread interrupt status in case we have been signaled from
+ // #interruptRegionOperation. This is the last place we can do it
"safely" before
+ // WAL appends.
+ checkInterrupt();
+
Review comment:
Could we only call checkInterrupt() **once** before or after WAL synced,
instead of a check after every step.
Personally I incline to check after WAL synced.
It also makes sense to call before WAL synced if considering the sync issue
which occasionally happen.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] ddupg commented on a change in pull request #2579: HBASE-24999 Master manages ReplicationServers
ddupg commented on a change in pull request #2579:
URL: https://github.com/apache/hbase/pull/2579#discussion_r512399178
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterRpcServices.java
##
@@ -3402,4 +3406,33 @@ public ListReplicationSinkServersResponse
listReplicationSinkServers(
}
return builder.build();
}
+
+ @Override
+ public RegionServerReportResponse replicationServerReport(RpcController
controller,
+ RegionServerReportRequest request) throws ServiceException {
+try {
+ master.checkServiceStarted();
+ int versionNumber = 0;
+ String version = "0.0.0";
+ VersionInfo versionInfo = VersionInfoUtil.getCurrentClientVersionInfo();
+ if (versionInfo != null) {
+version = versionInfo.getVersion();
+versionNumber = VersionInfoUtil.getVersionNumber(versionInfo);
+ }
+ ClusterStatusProtos.ServerLoad sl = request.getLoad();
+ ServerName serverName = ProtobufUtil.toServerName(request.getServer());
+ ServerMetrics oldMetrics =
master.getReplicationServerManager().getServerMetrics(serverName);
+ ServerMetrics newMetrics =
+ ServerMetricsBuilder.toServerMetrics(serverName, versionNumber,
version, sl);
+ master.getReplicationServerManager().serverReport(serverName,
newMetrics);
+ if (sl != null && master.metricsMaster != null) {
+// Up our metrics.
+master.metricsMaster.incrementRequests(sl.getTotalNumberOfRequests()
Review comment:
RegionServer increments request count metric in all RPC method,
including `replicateWALEntry`, so the request count metric should mean more
than just reading and writing metric?
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/HReplicationServer.java
##
@@ -388,4 +446,152 @@ protected ReplicationServerRpcServices
createRpcServices() throws IOException {
protected boolean setAbortRequested() {
return abortRequested.compareAndSet(false, true);
}
+
+ private void tryReplicationServerReport(long reportStartTime, long
reportEndTime)
+ throws IOException {
+ReplicationServerStatusService.BlockingInterface rss = rssStub;
+if (rss == null) {
+ createReplicationServerStatusStub(true);
+ rss = rssStub;
+ if (rss == null) {
+return;
+ }
+}
+ClusterStatusProtos.ServerLoad sl = buildServerLoad(reportStartTime,
reportEndTime);
+try {
+ RegionServerReportRequest.Builder request = RegionServerReportRequest
+ .newBuilder();
+ request.setServer(ProtobufUtil.toServerName(this.serverName));
+ request.setLoad(sl);
+ rss.replicationServerReport(null, request.build());
+} catch (ServiceException se) {
+ IOException ioe = ProtobufUtil.getRemoteException(se);
+ if (ioe instanceof YouAreDeadException) {
+// This will be caught and handled as a fatal error in run()
+throw ioe;
+ }
+ if (rssStub == rss) {
+rssStub = null;
+ }
+ // Couldn't connect to the master, get location from zk and reconnect
+ // Method blocks until new master is found or we are stopped
+ createReplicationServerStatusStub(true);
+}
+ }
+
+ private ClusterStatusProtos.ServerLoad buildServerLoad(long reportStartTime,
long reportEndTime) {
+long usedMemory = -1L;
+long maxMemory = -1L;
+final MemoryUsage usage = MemorySizeUtil.safeGetHeapMemoryUsage();
+if (usage != null) {
+ usedMemory = usage.getUsed();
+ maxMemory = usage.getMax();
+}
+
+ClusterStatusProtos.ServerLoad.Builder serverLoad =
ClusterStatusProtos.ServerLoad.newBuilder();
+serverLoad.setTotalNumberOfRequests(rpcServices.requestCount.sum());
+serverLoad.setUsedHeapMB((int) (usedMemory / 1024 / 1024));
+serverLoad.setMaxHeapMB((int) (maxMemory / 1024 / 1024));
+
+serverLoad.setReportStartTime(reportStartTime);
+serverLoad.setReportEndTime(reportEndTime);
+
+// for the replicationLoad purpose. Only need to get from one
executorService
+// either source or sink will get the same info
+ReplicationSinkService sinks = getReplicationSinkService();
+
+if (sinks != null) {
+ // always refresh first to get the latest value
+ ReplicationLoad rLoad = sinks.refreshAndGetReplicationLoad();
+ if (rLoad != null) {
+serverLoad.setReplLoadSink(rLoad.getReplicationLoadSink());
+ }
+}
+return serverLoad.build();
+ }
+
+ /**
+ * Get the current master from ZooKeeper and open the RPC connection to it.
To get a fresh
+ * connection, the current rssStub must be null. Method will block until a
master is available.
+ * You can break from this block by requesting the server stop.
+ * @param refresh If true then master address will be read from ZK,
otherwise use cached data
+ * @return master + port, or null if server has been stopped
+ */
+ private synchronized ServerName
[GitHub] [hbase] Apache-HBase commented on pull request #2584: HBASE-25126 Add load balance logic in hbase-client to distribute read…
Apache-HBase commented on pull request #2584: URL: https://github.com/apache/hbase/pull/2584#issuecomment-716954120 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 7s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 47s | master passed | | +1 :green_heart: | compile | 1m 44s | master passed | | +1 :green_heart: | shadedjars | 7m 9s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 17s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 49s | the patch passed | | +1 :green_heart: | compile | 1m 45s | the patch passed | | +1 :green_heart: | javac | 1m 45s | the patch passed | | +1 :green_heart: | shadedjars | 7m 15s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 21s | hbase-client generated 1 new + 3 unchanged - 0 fixed = 4 total (was 3) | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 29s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 1m 8s | hbase-client in the patch passed. | | -1 :x: | unit | 204m 42s | hbase-server in the patch failed. | | | | 238m 56s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2584 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 0b2475984663 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 164cc5a3dc | | Default Java | 1.8.0_232 | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/3/artifact/yetus-jdk8-hadoop3-check/output/diff-javadoc-javadoc-hbase-client.txt | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/3/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/3/testReport/ | | Max. process+thread count | 3696 (vs. ulimit of 3) | | modules | C: hbase-common hbase-client hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/3/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] apurtell commented on pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close
apurtell commented on pull request #2574: URL: https://github.com/apache/hbase/pull/2574#issuecomment-716933862 The benefit of interrupting the thread is Java's thread sleeps, and futures, etc. will do the right thing and check this status themselves. However that is a mixed bag on the write path. @bharathv This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Joseph295 commented on pull request #2568: HBASE-25205 Corrupted hfiles append timestamp every time the region i…
Joseph295 commented on pull request #2568: URL: https://github.com/apache/hbase/pull/2568#issuecomment-716932347 > Can we change the jira title/description, as this really deals with recovered edits, not hfiles? > > And just giving a second thought on this issue: here we log only once the corruption is first detected, then mark it and never log it again, silently ignoring it. I'm wondering if we should rather stay verbose, in this case, given it may imply data loss. I checked the code, and find it is not essential for recovered edits as every time we begin to deal with recover edits, we will clear the files first. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Joseph295 commented on pull request #2568: HBASE-25205 Corrupted hfiles append timestamp every time the region i…
Joseph295 commented on pull request #2568: URL: https://github.com/apache/hbase/pull/2568#issuecomment-716931810 > Can we change the jira title/description, as this really deals with recovered edits, not hfiles? > > And just giving a second thought on this issue: here we log only once the corruption is first detected, then mark it and never log it again, silently ignoring it. I'm wondering if we should rather stay verbose, in this case, given it may imply data loss. Do you have other concerns? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2584: HBASE-25126 Add load balance logic in hbase-client to distribute read…
Apache-HBase commented on pull request #2584: URL: https://github.com/apache/hbase/pull/2584#issuecomment-716931695 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 27s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 16s | master passed | | +1 :green_heart: | compile | 1m 58s | master passed | | +1 :green_heart: | shadedjars | 6m 44s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 32s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 2s | the patch passed | | +1 :green_heart: | compile | 1m 59s | the patch passed | | +1 :green_heart: | javac | 1m 59s | the patch passed | | +1 :green_heart: | shadedjars | 6m 36s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 32s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 45s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 1m 5s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 130m 1s | hbase-server in the patch passed. | | | | 165m 7s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2584 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 8984ff93fc9c 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 164cc5a3dc | | Default Java | 2020-01-14 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/3/testReport/ | | Max. process+thread count | 4087 (vs. ulimit of 3) | | modules | C: hbase-common hbase-client hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/3/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Joseph295 commented on a change in pull request #2568: HBASE-25205 Corrupted hfiles append timestamp every time the region i…
Joseph295 commented on a change in pull request #2568:
URL: https://github.com/apache/hbase/pull/2568#discussion_r512373857
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
##
@@ -5301,7 +5301,11 @@ private long replayRecoveredEditsForPaths(long
minSeqIdForTheRegion, FileSystem
long maxSeqId;
String fileName = edits.getName();
- maxSeqId = Math.abs(Long.parseLong(fileName));
+ if (isSkipErrorEnabled() &&
fileName.endsWith(HConstants.HREGION_EDITS_CORRUPTED_SUFFIX)) {
Review comment:
> recovered.edits have same problem? or problem only for
SplitWALToHFile? Can we have UT for this?
I checked the code, recovered.edits will be cleared at the beginning, so we
don't need UT now?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Joseph295 commented on a change in pull request #2568: HBASE-25205 Corrupted hfiles append timestamp every time the region i…
Joseph295 commented on a change in pull request #2568:
URL: https://github.com/apache/hbase/pull/2568#discussion_r512373857
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
##
@@ -5301,7 +5301,11 @@ private long replayRecoveredEditsForPaths(long
minSeqIdForTheRegion, FileSystem
long maxSeqId;
String fileName = edits.getName();
- maxSeqId = Math.abs(Long.parseLong(fileName));
+ if (isSkipErrorEnabled() &&
fileName.endsWith(HConstants.HREGION_EDITS_CORRUPTED_SUFFIX)) {
Review comment:
> recovered.edits have same problem? or problem only for
SplitWALToHFile? Can we have UT for this?
I checked the code, recovered.edits will clear the files at the beginning,
so we don't need UT now?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Joseph295 commented on a change in pull request #2568: HBASE-25205 Corrupted hfiles append timestamp every time the region i…
Joseph295 commented on a change in pull request #2568:
URL: https://github.com/apache/hbase/pull/2568#discussion_r512373549
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
##
@@ -5324,22 +5328,25 @@ private long replayRecoveredEditsForPaths(long
minSeqIdForTheRegion, FileSystem
}
private void handleException(FileSystem fs, Path edits, IOException e)
throws IOException {
-boolean skipErrors =
conf.getBoolean(HConstants.HREGION_EDITS_REPLAY_SKIP_ERRORS,
- conf.getBoolean("hbase.skip.errors",
HConstants.DEFAULT_HREGION_EDITS_REPLAY_SKIP_ERRORS));
-if (conf.get("hbase.skip.errors") != null) {
- LOG.warn("The property 'hbase.skip.errors' has been deprecated. Please
use "
- + HConstants.HREGION_EDITS_REPLAY_SKIP_ERRORS + " instead.");
-}
-if (skipErrors) {
+if (isSkipErrorEnabled()) {
Review comment:
Sorry sir, I don't catch you. You mean set skip error config to true?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] infraio commented on a change in pull request #2579: HBASE-24999 Master manages ReplicationServers
infraio commented on a change in pull request #2579:
URL: https://github.com/apache/hbase/pull/2579#discussion_r512361785
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/HReplicationServer.java
##
@@ -388,4 +446,152 @@ protected ReplicationServerRpcServices
createRpcServices() throws IOException {
protected boolean setAbortRequested() {
return abortRequested.compareAndSet(false, true);
}
+
+ private void tryReplicationServerReport(long reportStartTime, long
reportEndTime)
+ throws IOException {
+ReplicationServerStatusService.BlockingInterface rss = rssStub;
+if (rss == null) {
+ createReplicationServerStatusStub(true);
+ rss = rssStub;
+ if (rss == null) {
+return;
+ }
+}
+ClusterStatusProtos.ServerLoad sl = buildServerLoad(reportStartTime,
reportEndTime);
+try {
+ RegionServerReportRequest.Builder request = RegionServerReportRequest
+ .newBuilder();
+ request.setServer(ProtobufUtil.toServerName(this.serverName));
+ request.setLoad(sl);
+ rss.replicationServerReport(null, request.build());
+} catch (ServiceException se) {
+ IOException ioe = ProtobufUtil.getRemoteException(se);
+ if (ioe instanceof YouAreDeadException) {
+// This will be caught and handled as a fatal error in run()
+throw ioe;
+ }
+ if (rssStub == rss) {
+rssStub = null;
+ }
+ // Couldn't connect to the master, get location from zk and reconnect
+ // Method blocks until new master is found or we are stopped
+ createReplicationServerStatusStub(true);
+}
+ }
+
+ private ClusterStatusProtos.ServerLoad buildServerLoad(long reportStartTime,
long reportEndTime) {
+long usedMemory = -1L;
+long maxMemory = -1L;
+final MemoryUsage usage = MemorySizeUtil.safeGetHeapMemoryUsage();
+if (usage != null) {
+ usedMemory = usage.getUsed();
+ maxMemory = usage.getMax();
+}
+
+ClusterStatusProtos.ServerLoad.Builder serverLoad =
ClusterStatusProtos.ServerLoad.newBuilder();
+serverLoad.setTotalNumberOfRequests(rpcServices.requestCount.sum());
+serverLoad.setUsedHeapMB((int) (usedMemory / 1024 / 1024));
+serverLoad.setMaxHeapMB((int) (maxMemory / 1024 / 1024));
+
+serverLoad.setReportStartTime(reportStartTime);
+serverLoad.setReportEndTime(reportEndTime);
+
+// for the replicationLoad purpose. Only need to get from one
executorService
+// either source or sink will get the same info
+ReplicationSinkService sinks = getReplicationSinkService();
+
+if (sinks != null) {
+ // always refresh first to get the latest value
+ ReplicationLoad rLoad = sinks.refreshAndGetReplicationLoad();
+ if (rLoad != null) {
+serverLoad.setReplLoadSink(rLoad.getReplicationLoadSink());
+ }
+}
+return serverLoad.build();
+ }
+
+ /**
+ * Get the current master from ZooKeeper and open the RPC connection to it.
To get a fresh
+ * connection, the current rssStub must be null. Method will block until a
master is available.
+ * You can break from this block by requesting the server stop.
+ * @param refresh If true then master address will be read from ZK,
otherwise use cached data
+ * @return master + port, or null if server has been stopped
+ */
+ private synchronized ServerName createReplicationServerStatusStub(boolean
refresh) {
Review comment:
The return ServerName never used?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] infraio commented on a change in pull request #2579: HBASE-24999 Master manages ReplicationServers
infraio commented on a change in pull request #2579:
URL: https://github.com/apache/hbase/pull/2579#discussion_r512355359
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterRpcServices.java
##
@@ -3402,4 +3406,33 @@ public ListReplicationSinkServersResponse
listReplicationSinkServers(
}
return builder.build();
}
+
+ @Override
+ public RegionServerReportResponse replicationServerReport(RpcController
controller,
+ RegionServerReportRequest request) throws ServiceException {
+try {
+ master.checkServiceStarted();
+ int versionNumber = 0;
+ String version = "0.0.0";
+ VersionInfo versionInfo = VersionInfoUtil.getCurrentClientVersionInfo();
+ if (versionInfo != null) {
+version = versionInfo.getVersion();
+versionNumber = VersionInfoUtil.getVersionNumber(versionInfo);
+ }
+ ClusterStatusProtos.ServerLoad sl = request.getLoad();
+ ServerName serverName = ProtobufUtil.toServerName(request.getServer());
+ ServerMetrics oldMetrics =
master.getReplicationServerManager().getServerMetrics(serverName);
+ ServerMetrics newMetrics =
+ ServerMetricsBuilder.toServerMetrics(serverName, versionNumber,
version, sl);
+ master.getReplicationServerManager().serverReport(serverName,
newMetrics);
+ if (sl != null && master.metricsMaster != null) {
+// Up our metrics.
+master.metricsMaster.incrementRequests(sl.getTotalNumberOfRequests()
Review comment:
One new question: ReplicationServer should not have request count
metric? Because this metric should mean that read metric or write metric?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] bharathv commented on pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close
bharathv commented on pull request #2574: URL: https://github.com/apache/hbase/pull/2574#issuecomment-716907446 > The original motivation for interrupting RPCs in flight was to address the case where we get stuck closing the region, so we can be less aggressive and wait until we actually seem to be stuck That makes sense to me. In most cases (which are short lived operations), the lock holder should gracefully release the lock in the time window and no point force-interrupting them. My concern on the loop was readability of code flow and I see your point of repeated interrupts incase the interrupt is swallowed and reset by the layers underneath. Another way to address both these issues is to have a boolean state in the HRegion (abortOperations or some such) that is set only after the graceful time window has passed. Instead of polling isInterrupted(), the threads now check for this boolean flag. Not a big deal I guess but just wanted to point out incase you haven't already considered this approach. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] infraio edited a comment on pull request #2579: HBASE-24999 Master manages ReplicationServers
infraio edited a comment on pull request #2579: URL: https://github.com/apache/hbase/pull/2579#issuecomment-716902255 > Thanks @infraio for reviewing. I have fixed as comments. > > > Add master UI to show ReplicationServer? > > Do it in this PR? I was going to do it in a new PR? Ok. Let's do it in follow-on PR. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] infraio commented on pull request #2579: HBASE-24999 Master manages ReplicationServers
infraio commented on pull request #2579: URL: https://github.com/apache/hbase/pull/2579#issuecomment-716902255 > Thanks @infraio for reviewing. I have fixed as comments. > > > Add master UI to show ReplicationServer? > > Do it in this PR? I was going to do it in a new PR? Ok. Can be done in follow-on PR. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2575: HBASE-25212 [branch-2] Optionally abort requests in progress after deciding a region should close
Apache-HBase commented on pull request #2575: URL: https://github.com/apache/hbase/pull/2575#issuecomment-716901251 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 57s | Docker mode activated. | | -0 :warning: | yetus | 0m 8s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 19s | branch-2 passed | | +1 :green_heart: | compile | 1m 18s | branch-2 passed | | +1 :green_heart: | shadedjars | 7m 53s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 44s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 30s | the patch passed | | +1 :green_heart: | compile | 1m 15s | the patch passed | | +1 :green_heart: | javac | 1m 15s | the patch passed | | +1 :green_heart: | shadedjars | 7m 30s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 44s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 209m 21s | hbase-server in the patch passed. | | | | 243m 56s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2575/6/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2575 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 9cae75f19c3a 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 70631e901f | | Default Java | 2020-01-14 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2575/6/testReport/ | | Max. process+thread count | 2500 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2575/6/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2584: HBASE-25126 Add load balance logic in hbase-client to distribute read…
Apache-HBase commented on pull request #2584: URL: https://github.com/apache/hbase/pull/2584#issuecomment-716896021 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 19s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 49s | master passed | | +1 :green_heart: | checkstyle | 2m 1s | master passed | | +1 :green_heart: | spotbugs | 4m 9s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 50s | the patch passed | | -0 :warning: | checkstyle | 0m 24s | hbase-common: The patch generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) | | -0 :warning: | checkstyle | 0m 26s | hbase-client: The patch generated 1 new + 2 unchanged - 0 fixed = 3 total (was 2) | | -0 :warning: | checkstyle | 1m 10s | hbase-server: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 19m 10s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 4m 16s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 31s | The patch does not generate ASF License warnings. | | | | 49m 57s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/3/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2584 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 39e1032d98bf 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 164cc5a3dc | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/3/artifact/yetus-general-check/output/diff-checkstyle-hbase-common.txt | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/3/artifact/yetus-general-check/output/diff-checkstyle-hbase-client.txt | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/3/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 84 (vs. ulimit of 3) | | modules | C: hbase-common hbase-client hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/3/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25189) [Metrics] Add checkAndPut and checkAndDelete latency metrics at table level
[
https://issues.apache.org/jira/browse/HBASE-25189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221039#comment-17221039
]
Hudson commented on HBASE-25189:
Results for branch branch-2.2
[build #107 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/107/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/107//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/107//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/107//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 2. [see log for
details|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/107//artifact/output-integration/hadoop-2.log].
(note that this means we didn't run on Hadoop 3)
> [Metrics] Add checkAndPut and checkAndDelete latency metrics at table level
> ---
>
> Key: HBASE-25189
> URL: https://issues.apache.org/jira/browse/HBASE-25189
> Project: HBase
> Issue Type: Improvement
> Components: metrics
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Minor
> Fix For: 3.0.0-alpha-1, 2.3.3, 1.7.0, 2.2.7
>
>
> Add *checkAndPut* and *checkAndDelete* table latency metrics.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2589: HBASE-25219 Backport HBASE-24369 Provide more information about merged child regions in Hbck Overlaps section, which cannot be fixed imme
Apache-HBase commented on pull request #2589: URL: https://github.com/apache/hbase/pull/2589#issuecomment-716884250 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 34s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ branch-2.2 Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 23s | branch-2.2 passed | | +1 :green_heart: | compile | 0m 58s | branch-2.2 passed | | +1 :green_heart: | checkstyle | 1m 19s | branch-2.2 passed | | +1 :green_heart: | shadedjars | 4m 9s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 38s | branch-2.2 passed | | +0 :ok: | spotbugs | 3m 11s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 3m 8s | branch-2.2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 53s | the patch passed | | +1 :green_heart: | compile | 0m 55s | the patch passed | | +1 :green_heart: | javac | 0m 55s | the patch passed | | +1 :green_heart: | checkstyle | 1m 17s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 4m 5s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 25m 22s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 2.10.0 or 3.1.2 3.2.1. | | +1 :green_heart: | javadoc | 0m 34s | the patch passed | | +1 :green_heart: | findbugs | 3m 26s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 128m 16s | hbase-server in the patch passed. | | +1 :green_heart: | asflicense | 0m 35s | The patch does not generate ASF License warnings. | | | | 190m 1s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2589/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2589 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux fcc4ebe308f0 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-2589/out/precommit/personality/provided.sh | | git revision | branch-2.2 / 84cf1186d0 | | Default Java | 1.8.0_181 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2589/1/testReport/ | | Max. process+thread count | 3236 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2589/1/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2576: HBASE-25212 [branch-1] Optionally abort requests in progress after deciding a region should close
Apache-HBase commented on pull request #2576: URL: https://github.com/apache/hbase/pull/2576#issuecomment-716884000 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 13m 32s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 7 new or modified test files. | ||| _ branch-1 Compile Tests _ | | +1 :green_heart: | mvninstall | 10m 7s | branch-1 passed | | +1 :green_heart: | compile | 0m 44s | branch-1 passed with JDK v1.8.0_262 | | +1 :green_heart: | compile | 0m 47s | branch-1 passed with JDK v1.7.0_272 | | +1 :green_heart: | checkstyle | 2m 7s | branch-1 passed | | +1 :green_heart: | shadedjars | 3m 17s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 48s | branch-1 passed with JDK v1.8.0_262 | | +1 :green_heart: | javadoc | 0m 46s | branch-1 passed with JDK v1.7.0_272 | | +0 :ok: | spotbugs | 3m 49s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 3m 43s | branch-1 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 40s | the patch passed | | +1 :green_heart: | compile | 0m 57s | the patch passed with JDK v1.8.0_262 | | +1 :green_heart: | javac | 0m 57s | the patch passed | | +1 :green_heart: | compile | 1m 1s | the patch passed with JDK v1.7.0_272 | | +1 :green_heart: | javac | 1m 1s | the patch passed | | +1 :green_heart: | checkstyle | 2m 27s | hbase-server: The patch generated 0 new + 812 unchanged - 18 fixed = 812 total (was 830) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 3m 40s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 5m 49s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. | | +1 :green_heart: | javadoc | 0m 34s | the patch passed with JDK v1.8.0_262 | | +1 :green_heart: | javadoc | 0m 47s | the patch passed with JDK v1.7.0_272 | | +1 :green_heart: | findbugs | 3m 25s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 129m 23s | hbase-server in the patch passed. | | +1 :green_heart: | asflicense | 0m 35s | The patch does not generate ASF License warnings. | | | | 191m 20s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2576/5/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2576 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 5cdf62c42e3c 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-agent/workspace/Base-PreCommit-GitHub-PR_PR-2576/out/precommit/personality/provided.sh | | git revision | branch-1 / b30d1d1 | | Default Java | 1.7.0_272 | | Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:1.8.0_262 /usr/lib/jvm/zulu-7-amd64:1.7.0_272 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2576/5/testReport/ | | Max. process+thread count | 3979 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2576/5/console | | versions | git=1.9.1 maven=3.0.5 findbugs=3.0.1 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2575: HBASE-25212 [branch-2] Optionally abort requests in progress after deciding a region should close
Apache-HBase commented on pull request #2575: URL: https://github.com/apache/hbase/pull/2575#issuecomment-716881489 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 34s | Docker mode activated. | | -0 :warning: | yetus | 0m 6s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 21s | branch-2 passed | | +1 :green_heart: | compile | 0m 56s | branch-2 passed | | +1 :green_heart: | shadedjars | 5m 56s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 37s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 20s | the patch passed | | +1 :green_heart: | compile | 0m 55s | the patch passed | | +1 :green_heart: | javac | 0m 55s | the patch passed | | +1 :green_heart: | shadedjars | 5m 56s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 34s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 159m 31s | hbase-server in the patch passed. | | | | 183m 59s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2575/6/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2575 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 100e2a2b771e 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 70631e901f | | Default Java | 1.8.0_232 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2575/6/testReport/ | | Max. process+thread count | 3758 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2575/6/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close
Apache-HBase commented on pull request #2574: URL: https://github.com/apache/hbase/pull/2574#issuecomment-716880573 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 34s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 28s | master passed | | +1 :green_heart: | compile | 1m 8s | master passed | | +1 :green_heart: | shadedjars | 6m 53s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 41s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 9s | the patch passed | | +1 :green_heart: | compile | 1m 8s | the patch passed | | +1 :green_heart: | javac | 1m 8s | the patch passed | | +1 :green_heart: | shadedjars | 8m 3s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 52s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 151m 7s | hbase-server in the patch passed. | | | | 181m 20s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2574/5/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2574 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 9d68d384721b 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 164cc5a3dc | | Default Java | 2020-01-14 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2574/5/testReport/ | | Max. process+thread count | 3831 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2574/5/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2584: HBASE-25126 Add load balance logic in hbase-client to distribute read…
Apache-HBase commented on pull request #2584: URL: https://github.com/apache/hbase/pull/2584#issuecomment-716878459 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 6s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 5s | master passed | | +1 :green_heart: | compile | 1m 45s | master passed | | +1 :green_heart: | shadedjars | 7m 10s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 19s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 49s | the patch passed | | +1 :green_heart: | compile | 1m 45s | the patch passed | | +1 :green_heart: | javac | 1m 45s | the patch passed | | +1 :green_heart: | shadedjars | 7m 7s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 21s | hbase-client generated 1 new + 3 unchanged - 0 fixed = 4 total (was 3) | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 30s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 1m 8s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 203m 4s | hbase-server in the patch passed. | | | | 237m 52s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2584 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 9909eeb7958b 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 164cc5a3dc | | Default Java | 1.8.0_232 | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/2/artifact/yetus-jdk8-hadoop3-check/output/diff-javadoc-javadoc-hbase-client.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/2/testReport/ | | Max. process+thread count | 3387 (vs. ulimit of 3) | | modules | C: hbase-common hbase-client hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close
Apache-HBase commented on pull request #2574: URL: https://github.com/apache/hbase/pull/2574#issuecomment-716875344 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 32s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 51s | master passed | | +1 :green_heart: | compile | 0m 56s | master passed | | +1 :green_heart: | shadedjars | 6m 39s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 41s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 28s | the patch passed | | +1 :green_heart: | compile | 0m 56s | the patch passed | | +1 :green_heart: | javac | 0m 56s | the patch passed | | +1 :green_heart: | shadedjars | 6m 33s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 38s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 140m 45s | hbase-server in the patch passed. | | | | 167m 11s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2574/5/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2574 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 433b58bb770f 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 164cc5a3dc | | Default Java | 1.8.0_232 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2574/5/testReport/ | | Max. process+thread count | 4825 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2574/5/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-24972) Wait for connection attempt to succeed before performing operations on ZK
[
https://issues.apache.org/jira/browse/HBASE-24972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221018#comment-17221018
]
Prathyusha commented on HBASE-24972:
Thanks [~sandeep.guggilam]
> Wait for connection attempt to succeed before performing operations on ZK
> -
>
> Key: HBASE-24972
> URL: https://issues.apache.org/jira/browse/HBASE-24972
> Project: HBase
> Issue Type: Bug
>Reporter: Sandeep Guggilam
>Assignee: Prathyusha
>Priority: Minor
>
> {color:#1d1c1d}Creating the connection with ZK is asynchronous and notified
> via the passed in watcher about the successful connection event. When we
> attempt any operations, we try to create a connection and then perform a
> read/write
> ({color}{color:#1d1c1d}[https://github.com/apache/hbase/blob/979edfe72046b2075adcc869c65ae820e6f3ec2d/hbase-zookeeper/src/main/java/org/apache/hadoop/hbase/zookeeper/RecoverableZooKeeper.java#L323]{color}{color:#1d1c1d})
> without really waiting for the notification event
> ([https://github.com/apache/hbase/blob/979edfe72046b2075adcc869c65ae820e6f3ec2d/hbase-zookeeper/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKWatcher.java#L582)]{color}
>
> {color:#1d1c1d}It is possible we get ConnectionLoss errors when we perform
> operations on ZK without waiting for the connection attempt to succeed{color}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HBASE-24972) Wait for connection attempt to succeed before performing operations on ZK
[
https://issues.apache.org/jira/browse/HBASE-24972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prathyusha reassigned HBASE-24972:
--
Assignee: Prathyusha
> Wait for connection attempt to succeed before performing operations on ZK
> -
>
> Key: HBASE-24972
> URL: https://issues.apache.org/jira/browse/HBASE-24972
> Project: HBase
> Issue Type: Bug
>Reporter: Sandeep Guggilam
>Assignee: Prathyusha
>Priority: Minor
>
> {color:#1d1c1d}Creating the connection with ZK is asynchronous and notified
> via the passed in watcher about the successful connection event. When we
> attempt any operations, we try to create a connection and then perform a
> read/write
> ({color}{color:#1d1c1d}[https://github.com/apache/hbase/blob/979edfe72046b2075adcc869c65ae820e6f3ec2d/hbase-zookeeper/src/main/java/org/apache/hadoop/hbase/zookeeper/RecoverableZooKeeper.java#L323]{color}{color:#1d1c1d})
> without really waiting for the notification event
> ([https://github.com/apache/hbase/blob/979edfe72046b2075adcc869c65ae820e6f3ec2d/hbase-zookeeper/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKWatcher.java#L582)]{color}
>
> {color:#1d1c1d}It is possible we get ConnectionLoss errors when we perform
> operations on ZK without waiting for the connection attempt to succeed{color}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2584: HBASE-25126 Add load balance logic in hbase-client to distribute read…
Apache-HBase commented on pull request #2584: URL: https://github.com/apache/hbase/pull/2584#issuecomment-716859524 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 4s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 25s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 11s | master passed | | +1 :green_heart: | compile | 1m 59s | master passed | | +1 :green_heart: | shadedjars | 6m 39s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 32s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 59s | the patch passed | | +1 :green_heart: | compile | 1m 59s | the patch passed | | +1 :green_heart: | javac | 1m 59s | the patch passed | | +1 :green_heart: | shadedjars | 6m 35s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 27s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 32s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 1m 4s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 149m 51s | hbase-server in the patch passed. | | | | 185m 8s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2584 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 54383e8ed4df 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 164cc5a3dc | | Default Java | 2020-01-14 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/2/testReport/ | | Max. process+thread count | 3843 (vs. ulimit of 3) | | modules | C: hbase-common hbase-client hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2578: HBASE-24768 Clear cached service kerberos ticket in case of SASL fail…
Apache-HBase commented on pull request #2578: URL: https://github.com/apache/hbase/pull/2578#issuecomment-716843830 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 11m 52s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ branch-1 Compile Tests _ | | +0 :ok: | mvndep | 2m 20s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 8m 0s | branch-1 passed | | +1 :green_heart: | compile | 0m 39s | branch-1 passed with JDK v1.8.0_262 | | +1 :green_heart: | compile | 0m 46s | branch-1 passed with JDK v1.7.0_272 | | +1 :green_heart: | checkstyle | 1m 14s | branch-1 passed | | +1 :green_heart: | shadedjars | 3m 17s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 46s | branch-1 passed with JDK v1.8.0_262 | | +1 :green_heart: | javadoc | 0m 49s | branch-1 passed with JDK v1.7.0_272 | | +0 :ok: | spotbugs | 1m 5s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 2m 49s | branch-1 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 8s | the patch passed | | +1 :green_heart: | compile | 0m 37s | the patch passed with JDK v1.8.0_262 | | +1 :green_heart: | javac | 0m 37s | the patch passed | | +1 :green_heart: | compile | 0m 47s | the patch passed with JDK v1.7.0_272 | | +1 :green_heart: | javac | 0m 47s | the patch passed | | +1 :green_heart: | checkstyle | 1m 6s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 3m 4s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 4m 54s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. | | +1 :green_heart: | javadoc | 0m 37s | the patch passed with JDK v1.8.0_262 | | +1 :green_heart: | javadoc | 0m 52s | the patch passed with JDK v1.7.0_272 | | +1 :green_heart: | findbugs | 2m 58s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 2m 35s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 2m 41s | hbase-client in the patch passed. | | +1 :green_heart: | asflicense | 0m 24s | The patch does not generate ASF License warnings. | | | | 59m 22s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2578/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2578 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 7fd1a91d0c07 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-2578/out/precommit/personality/provided.sh | | git revision | branch-1 / b30d1d1 | | Default Java | 1.7.0_272 | | Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:1.8.0_262 /usr/lib/jvm/zulu-7-amd64:1.7.0_272 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2578/2/testReport/ | | Max. process+thread count | 138 (vs. ulimit of 1) | | modules | C: hbase-common hbase-client U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2578/2/console | | versions | git=1.9.1 maven=3.0.5 findbugs=3.0.1 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close
Apache-HBase commented on pull request #2574: URL: https://github.com/apache/hbase/pull/2574#issuecomment-716828893 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 7m 41s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 58s | master passed | | +1 :green_heart: | checkstyle | 1m 17s | master passed | | +1 :green_heart: | spotbugs | 2m 8s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 46s | the patch passed | | +1 :green_heart: | checkstyle | 1m 17s | hbase-server: The patch generated 0 new + 409 unchanged - 2 fixed = 409 total (was 411) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 18m 42s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 2m 16s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 11s | The patch does not generate ASF License warnings. | | | | 49m 1s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2574/5/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2574 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux fe23a278b4de 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 164cc5a3dc | | Max. process+thread count | 84 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2574/5/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2575: HBASE-25212 [branch-2] Optionally abort requests in progress after deciding a region should close
Apache-HBase commented on pull request #2575: URL: https://github.com/apache/hbase/pull/2575#issuecomment-716822076 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 34s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 32s | branch-2 passed | | +1 :green_heart: | checkstyle | 1m 15s | branch-2 passed | | +1 :green_heart: | spotbugs | 1m 59s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 13s | the patch passed | | -0 :warning: | checkstyle | 1m 15s | hbase-server: The patch generated 1 new + 479 unchanged - 2 fixed = 480 total (was 481) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 11m 26s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 2m 46s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 13s | The patch does not generate ASF License warnings. | | | | 35m 6s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2575/6/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2575 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux cc1e2a68e233 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 70631e901f | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2575/6/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 94 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2575/6/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] sguggilam commented on pull request #2578: HBASE-24768 Clear cached service kerberos ticket in case of SASL fail…
sguggilam commented on pull request #2578: URL: https://github.com/apache/hbase/pull/2578#issuecomment-716815878 @apurtell Incorporated the comments, please review This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-25219) Backport HBASE-24369 "Provide more information about merged child regions in Hbck Overlaps section, which cannot be fixed immediately" to branch-2.2
[ https://issues.apache.org/jira/browse/HBASE-25219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi updated HBASE-25219: -- Status: Patch Available (was: Open) > Backport HBASE-24369 "Provide more information about merged child regions in > Hbck Overlaps section, which cannot be fixed immediately" to branch-2.2 > > > Key: HBASE-25219 > URL: https://issues.apache.org/jira/browse/HBASE-25219 > Project: HBase > Issue Type: Sub-task >Reporter: Peter Somogyi >Assignee: Peter Somogyi >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] petersomogyi opened a new pull request #2589: HBASE-25219 Backport HBASE-24369 Provide more information about merged child regions in Hbck Overlaps section, which cannot be fixed imme
petersomogyi opened a new pull request #2589: URL: https://github.com/apache/hbase/pull/2589 Signed-off-by: Viraj Jasani Signed-off-by: Jan Hentschel Signed-off-by: Clara Xiong Signed-off-by: stack This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] apurtell commented on pull request #2576: HBASE-25212 [branch-1] Optionally abort requests in progress after deciding a region should close
apurtell commented on pull request #2576: URL: https://github.com/apache/hbase/pull/2576#issuecomment-716802337 Rebase. Sync with updates to master branch patch. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] apurtell commented on pull request #2575: HBASE-25212 [branch-2] Optionally abort requests in progress after deciding a region should close
apurtell commented on pull request #2575: URL: https://github.com/apache/hbase/pull/2575#issuecomment-716802279 Rebase. Sync with updates to master branch patch. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] apurtell commented on pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close
apurtell commented on pull request #2574: URL: https://github.com/apache/hbase/pull/2574#issuecomment-716802100 Rebased. Address review feedback. Address my two comments above. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2584: HBASE-25126 Add load balance logic in hbase-client to distribute read…
Apache-HBase commented on pull request #2584: URL: https://github.com/apache/hbase/pull/2584#issuecomment-716799487 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 11s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 29s | master passed | | +1 :green_heart: | checkstyle | 2m 13s | master passed | | +1 :green_heart: | spotbugs | 4m 18s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 4s | the patch passed | | -0 :warning: | checkstyle | 0m 25s | hbase-common: The patch generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) | | -0 :warning: | checkstyle | 0m 28s | hbase-client: The patch generated 1 new + 2 unchanged - 0 fixed = 3 total (was 2) | | -0 :warning: | checkstyle | 1m 18s | hbase-server: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 20m 20s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 4m 14s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 32s | The patch does not generate ASF License warnings. | | | | 53m 36s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2584 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux ae977589cea9 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 164cc5a3dc | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/2/artifact/yetus-general-check/output/diff-checkstyle-hbase-common.txt | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/2/artifact/yetus-general-check/output/diff-checkstyle-hbase-client.txt | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/2/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 84 (vs. ulimit of 3) | | modules | C: hbase-common hbase-client hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2584/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (HBASE-25220) Backport HBASE-24246 "Miscellaneous hbck2 fixMeta bulk merge fixes: better logging around merges/overlap-fixing, 'HBCK Report' overlap listing, and configuration" to br
[ https://issues.apache.org/jira/browse/HBASE-25220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi resolved HBASE-25220. --- Fix Version/s: 2.2.7 Resolution: Fixed > Backport HBASE-24246 "Miscellaneous hbck2 fixMeta bulk merge fixes: better > logging around merges/overlap-fixing, 'HBCK Report' overlap listing, and > configuration" to branch-2.2 > > > Key: HBASE-25220 > URL: https://issues.apache.org/jira/browse/HBASE-25220 > Project: HBase > Issue Type: Sub-task >Reporter: Peter Somogyi >Assignee: Peter Somogyi >Priority: Major > Fix For: 2.2.7 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] petersomogyi merged pull request #2588: HBASE-25220 Backport HBASE-24246 Miscellaneous hbck2 fixMeta bulk merge fixes: better logging around merges/overlap-fixing, 'HBCK Report' overl
petersomogyi merged pull request #2588: URL: https://github.com/apache/hbase/pull/2588 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] huaxiangsun commented on pull request #2584: HBASE-25126 Add load balance logic in hbase-client to distribute read…
huaxiangsun commented on pull request #2584: URL: https://github.com/apache/hbase/pull/2584#issuecomment-716774965 Uploaded a new patch. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] petersomogyi commented on a change in pull request #2588: HBASE-25220 Backport HBASE-24246 Miscellaneous hbck2 fixMeta bulk merge fixes: better logging around merges/overlap-fixing, '
petersomogyi commented on a change in pull request #2588:
URL: https://github.com/apache/hbase/pull/2588#discussion_r512216019
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/MergeTableRegionsProcedure.java
##
@@ -135,8 +134,8 @@ private static void checkRegionsToMerge(MasterProcedureEnv
env, final RegionInfo
throw new MergeRegionException(msg);
}
if (!force && !ri.isAdjacent(previous) && !ri.isOverlap(previous)) {
- String msg = "Unable to merge non-adjacent or non-overlapping
regions " +
- previous.getShortNameToLog() + ", " + ri.getShortNameToLog() + "
when force=false";
+ String msg = "Unable to merge non-adjacent or non-overlapping
regions '" +
+ previous.getShortNameToLog() + "', '" + ri.getShortNameToLog() +
"' when force=false";
Review comment:
This is a clean cherry-pick of your commit to master, the `'` character
was part of the original commit. It was not pushed to branch-2.2 at that
time I found this as a nice improvement.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] saintstack commented on a change in pull request #2588: HBASE-25220 Backport HBASE-24246 Miscellaneous hbck2 fixMeta bulk merge fixes: better logging around merges/overlap-fixing, 'HB
saintstack commented on a change in pull request #2588:
URL: https://github.com/apache/hbase/pull/2588#discussion_r512196004
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/MetaFixer.java
##
@@ -247,6 +247,10 @@ void fixOverlaps(CatalogJanitor.Report report) throws
IOException {
if (regionInfoWithlargestEndKey != null) {
if (!isOverlap(regionInfoWithlargestEndKey, pair) ||
currentMergeSet.size() >= maxMergeCount) {
+ // Log when we cut-off-merge because we hit the configured maximum
merge limit.
+ if (currentMergeSet.size() >= maxMergeCount) {
+LOG.warn("Ran into maximum-at-a-time merges limit={}",
maxMergeCount);
Review comment:
Good
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/MergeTableRegionsProcedure.java
##
@@ -135,8 +134,8 @@ private static void checkRegionsToMerge(MasterProcedureEnv
env, final RegionInfo
throw new MergeRegionException(msg);
}
if (!force && !ri.isAdjacent(previous) && !ri.isOverlap(previous)) {
- String msg = "Unable to merge non-adjacent or non-overlapping
regions " +
- previous.getShortNameToLog() + ", " + ri.getShortNameToLog() + "
when force=false";
+ String msg = "Unable to merge non-adjacent or non-overlapping
regions '" +
+ previous.getShortNameToLog() + "', '" + ri.getShortNameToLog() +
"' when force=false";
Review comment:
nit: we need the ' characters? we don't usually do this?
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/MergeTableRegionsProcedure.java
##
@@ -479,16 +478,20 @@ private boolean prepareMergeRegion(final
MasterProcedureEnv env) throws IOExcept
for (RegionInfo ri: this.regionsToMerge) {
if (!catalogJanitor.cleanMergeQualifier(ri)) {
String msg = "Skip merging " +
RegionInfo.getShortNameToLog(regionsToMerge) +
-", because parent " + RegionInfo.getShortNameToLog(ri) + " has a
merge qualifier";
+", because a parent, " + RegionInfo.getShortNameToLog(ri) + ", has
a merge qualifier " +
+ "(if a 'merge column' in parent, it was recently merged but still
has outstanding " +
+ "references to its parents that must be cleared before it can
participate in merge -- " +
+ "major compact it to hurry clearing of its references)";
Review comment:
Good
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] huaxiangsun commented on pull request #2584: HBASE-25126 Add load balance logic in hbase-client to distribute read…
huaxiangsun commented on pull request #2584: URL: https://github.com/apache/hbase/pull/2584#issuecomment-716751088 Changing the newly added test to Parameterized one, will load a new patch shortly. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2588: HBASE-25220 Backport HBASE-24246 Miscellaneous hbck2 fixMeta bulk merge fixes: better logging around merges/overlap-fixing, 'HBCK Report'
Apache-HBase commented on pull request #2588: URL: https://github.com/apache/hbase/pull/2588#issuecomment-716744178 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 4m 5s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ branch-2.2 Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 30s | branch-2.2 passed | | +1 :green_heart: | compile | 1m 28s | branch-2.2 passed | | +1 :green_heart: | checkstyle | 2m 6s | branch-2.2 passed | | +1 :green_heart: | shadedjars | 4m 20s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 59s | branch-2.2 passed | | +0 :ok: | spotbugs | 3m 35s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 4m 49s | branch-2.2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 13s | the patch passed | | +1 :green_heart: | compile | 1m 26s | the patch passed | | +1 :green_heart: | javac | 1m 26s | the patch passed | | +1 :green_heart: | checkstyle | 0m 37s | hbase-client: The patch generated 0 new + 22 unchanged - 1 fixed = 22 total (was 23) | | +1 :green_heart: | checkstyle | 1m 27s | The patch passed checkstyle in hbase-server | | +1 :green_heart: | whitespace | 0m 1s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 4m 24s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 28m 7s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 2.10.0 or 3.1.2 3.2.1. | | +1 :green_heart: | javadoc | 0m 58s | the patch passed | | +1 :green_heart: | findbugs | 5m 13s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 3m 36s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 174m 20s | hbase-server in the patch passed. | | +1 :green_heart: | asflicense | 0m 48s | The patch does not generate ASF License warnings. | | | | 252m 53s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2588/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2588 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 05f425750648 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-agent/workspace/Base-PreCommit-GitHub-PR_PR-2588/out/precommit/personality/provided.sh | | git revision | branch-2.2 / 110b71a62b | | Default Java | 1.8.0_181 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2588/1/testReport/ | | Max. process+thread count | 2581 (vs. ulimit of 12500) | | modules | C: hbase-client hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2588/1/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] apurtell commented on a change in pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close
apurtell commented on a change in pull request #2574:
URL: https://github.com/apache/hbase/pull/2574#discussion_r512176656
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionInterrupt.java
##
@@ -0,0 +1,356 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hbase.regionserver;
+
+import static org.junit.Assert.assertTrue;
+
+import java.io.IOException;
+import java.io.InterruptedIOException;
+import java.util.Optional;
+import java.util.concurrent.atomic.AtomicBoolean;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hbase.Cell;
+import org.apache.hadoop.hbase.HBaseClassTestRule;
+import org.apache.hadoop.hbase.HBaseTestingUtility;
+import org.apache.hadoop.hbase.HConstants;
+import org.apache.hadoop.hbase.NotServingRegionException;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.TableNameTestRule;
+import org.apache.hadoop.hbase.Waiter;
+import org.apache.hadoop.hbase.client.Admin;
+import org.apache.hadoop.hbase.client.Append;
+import org.apache.hadoop.hbase.client.BufferedMutator;
+import org.apache.hadoop.hbase.client.ColumnFamilyDescriptorBuilder;
+import org.apache.hadoop.hbase.client.Delete;
+import org.apache.hadoop.hbase.client.Durability;
+import org.apache.hadoop.hbase.client.Increment;
+import org.apache.hadoop.hbase.client.Put;
+import org.apache.hadoop.hbase.client.RegionInfo;
+import org.apache.hadoop.hbase.client.Result;
+import org.apache.hadoop.hbase.client.ResultScanner;
+import org.apache.hadoop.hbase.client.Scan;
+import org.apache.hadoop.hbase.client.Table;
+import org.apache.hadoop.hbase.client.TableDescriptor;
+import org.apache.hadoop.hbase.client.TableDescriptorBuilder;
+import org.apache.hadoop.hbase.coprocessor.ObserverContext;
+import org.apache.hadoop.hbase.coprocessor.RegionCoprocessor;
+import org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment;
+import org.apache.hadoop.hbase.coprocessor.RegionObserver;
+import org.apache.hadoop.hbase.exceptions.DeserializationException;
+import org.apache.hadoop.hbase.filter.FilterBase;
+import org.apache.hadoop.hbase.testclassification.LargeTests;
+import org.apache.hadoop.hbase.testclassification.RegionServerTests;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.hadoop.hbase.wal.WAL;
+import org.apache.hadoop.hbase.wal.WALEdit;
+import org.junit.After;
+import org.junit.Before;
+import org.junit.BeforeClass;
+import org.junit.ClassRule;
+import org.junit.Rule;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+@Category({RegionServerTests.class, LargeTests.class})
+public class TestRegionInterrupt {
+
+ @ClassRule
+ public static final HBaseClassTestRule CLASS_RULE =
+HBaseClassTestRule.forClass(TestRegionInterrupt.class);
+
+ private static HBaseTestingUtility TEST_UTIL = new HBaseTestingUtility();
+ private static final Logger LOG =
LoggerFactory.getLogger(TestRegionInterrupt.class);
+
+ static final int SLEEP_TIME = 10 * 1000;
+ static final byte[] FAMILY = Bytes.toBytes("info");
+
+ @Rule
+ public TableNameTestRule name = new TableNameTestRule();
+
+ @BeforeClass
+ public static void setUpBeforeClass() throws Exception {
+Configuration conf = TEST_UTIL.getConfiguration();
+conf.setClass(HConstants.REGION_IMPL, InterruptInterceptingHRegion.class,
Region.class);
+conf.setBoolean(HRegion.CLOSE_WAIT_ABORT, true);
Review comment:
Done
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] virajjasani commented on a change in pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close
virajjasani commented on a change in pull request #2574:
URL: https://github.com/apache/hbase/pull/2574#discussion_r512174196
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
##
@@ -7364,4 +7366,157 @@ protected HStoreForTesting(final HRegion region,
return super.doCompaction(cr, filesToCompact, user, compactionStartTime,
newFiles);
}
}
+
+ @Test
+ public void testCloseNoInterrupt() throws Exception {
+byte[] cf1 = Bytes.toBytes("CF1");
+byte[][] families = { cf1 };
+
+Configuration conf = new Configuration(CONF);
+// Disable close thread interrupt and server abort behavior
+conf.setBoolean(HRegion.CLOSE_WAIT_ABORT, false);
+region = initHRegion(tableName, method, conf, families);
+
+final CountDownLatch latch = new CountDownLatch(1);
+final AtomicBoolean holderInterrupted = new AtomicBoolean();
+Thread holder = new Thread(new Runnable() {
+ @Override
+ public void run() {
+try {
+ LOG.info("Starting region operation holder");
+ region.startRegionOperation(Operation.SCAN);
+ latch.countDown();
+ try {
+Thread.sleep(10*1000);
+ } catch (InterruptedException e) {
+LOG.info("Interrupted");
+holderInterrupted.set(true);
Review comment:
Oh yes, nvm, current approach is better.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (HBASE-25189) [Metrics] Add checkAndPut and checkAndDelete latency metrics at table level
[
https://issues.apache.org/jira/browse/HBASE-25189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17220897#comment-17220897
]
Hudson commented on HBASE-25189:
Results for branch branch-2
[build #84 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/84/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/84/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/84/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/84/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/84/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> [Metrics] Add checkAndPut and checkAndDelete latency metrics at table level
> ---
>
> Key: HBASE-25189
> URL: https://issues.apache.org/jira/browse/HBASE-25189
> Project: HBase
> Issue Type: Improvement
> Components: metrics
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Minor
> Fix For: 3.0.0-alpha-1, 2.3.3, 1.7.0, 2.2.7
>
>
> Add *checkAndPut* and *checkAndDelete* table latency metrics.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24972) Wait for connection attempt to succeed before performing operations on ZK
[
https://issues.apache.org/jira/browse/HBASE-24972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17220891#comment-17220891
]
Sandeep Guggilam commented on HBASE-24972:
--
[~pratg] Sure , feel free to pick this one
> Wait for connection attempt to succeed before performing operations on ZK
> -
>
> Key: HBASE-24972
> URL: https://issues.apache.org/jira/browse/HBASE-24972
> Project: HBase
> Issue Type: Bug
>Reporter: Sandeep Guggilam
>Priority: Minor
>
> {color:#1d1c1d}Creating the connection with ZK is asynchronous and notified
> via the passed in watcher about the successful connection event. When we
> attempt any operations, we try to create a connection and then perform a
> read/write
> ({color}{color:#1d1c1d}[https://github.com/apache/hbase/blob/979edfe72046b2075adcc869c65ae820e6f3ec2d/hbase-zookeeper/src/main/java/org/apache/hadoop/hbase/zookeeper/RecoverableZooKeeper.java#L323]{color}{color:#1d1c1d})
> without really waiting for the notification event
> ([https://github.com/apache/hbase/blob/979edfe72046b2075adcc869c65ae820e6f3ec2d/hbase-zookeeper/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKWatcher.java#L582)]{color}
>
> {color:#1d1c1d}It is possible we get ConnectionLoss errors when we perform
> operations on ZK without waiting for the connection attempt to succeed{color}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] apurtell commented on a change in pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close
apurtell commented on a change in pull request #2574:
URL: https://github.com/apache/hbase/pull/2574#discussion_r512168388
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionInterrupt.java
##
@@ -0,0 +1,356 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hbase.regionserver;
+
+import static org.junit.Assert.assertTrue;
+
+import java.io.IOException;
+import java.io.InterruptedIOException;
+import java.util.Optional;
+import java.util.concurrent.atomic.AtomicBoolean;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hbase.Cell;
+import org.apache.hadoop.hbase.HBaseClassTestRule;
+import org.apache.hadoop.hbase.HBaseTestingUtility;
+import org.apache.hadoop.hbase.HConstants;
+import org.apache.hadoop.hbase.NotServingRegionException;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.TableNameTestRule;
+import org.apache.hadoop.hbase.Waiter;
+import org.apache.hadoop.hbase.client.Admin;
+import org.apache.hadoop.hbase.client.Append;
+import org.apache.hadoop.hbase.client.BufferedMutator;
+import org.apache.hadoop.hbase.client.ColumnFamilyDescriptorBuilder;
+import org.apache.hadoop.hbase.client.Delete;
+import org.apache.hadoop.hbase.client.Durability;
+import org.apache.hadoop.hbase.client.Increment;
+import org.apache.hadoop.hbase.client.Put;
+import org.apache.hadoop.hbase.client.RegionInfo;
+import org.apache.hadoop.hbase.client.Result;
+import org.apache.hadoop.hbase.client.ResultScanner;
+import org.apache.hadoop.hbase.client.Scan;
+import org.apache.hadoop.hbase.client.Table;
+import org.apache.hadoop.hbase.client.TableDescriptor;
+import org.apache.hadoop.hbase.client.TableDescriptorBuilder;
+import org.apache.hadoop.hbase.coprocessor.ObserverContext;
+import org.apache.hadoop.hbase.coprocessor.RegionCoprocessor;
+import org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment;
+import org.apache.hadoop.hbase.coprocessor.RegionObserver;
+import org.apache.hadoop.hbase.exceptions.DeserializationException;
+import org.apache.hadoop.hbase.filter.FilterBase;
+import org.apache.hadoop.hbase.testclassification.LargeTests;
+import org.apache.hadoop.hbase.testclassification.RegionServerTests;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.hadoop.hbase.wal.WAL;
+import org.apache.hadoop.hbase.wal.WALEdit;
+import org.junit.After;
+import org.junit.Before;
+import org.junit.BeforeClass;
+import org.junit.ClassRule;
+import org.junit.Rule;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+@Category({RegionServerTests.class, LargeTests.class})
+public class TestRegionInterrupt {
+
+ @ClassRule
+ public static final HBaseClassTestRule CLASS_RULE =
+HBaseClassTestRule.forClass(TestRegionInterrupt.class);
+
+ private static HBaseTestingUtility TEST_UTIL = new HBaseTestingUtility();
+ private static final Logger LOG =
LoggerFactory.getLogger(TestRegionInterrupt.class);
+
+ static final int SLEEP_TIME = 10 * 1000;
+ static final byte[] FAMILY = Bytes.toBytes("info");
+
+ @Rule
+ public TableNameTestRule name = new TableNameTestRule();
+
+ @BeforeClass
+ public static void setUpBeforeClass() throws Exception {
+Configuration conf = TEST_UTIL.getConfiguration();
+conf.setClass(HConstants.REGION_IMPL, InterruptInterceptingHRegion.class,
Region.class);
+conf.setBoolean(HRegion.CLOSE_WAIT_ABORT, true);
+conf.setInt(HConstants.HBASE_CLIENT_RETRIES_NUMBER, 1);
+ }
+
+ @Before
+ public void setUp() throws Exception {
+TEST_UTIL.startMiniCluster();
+ }
+
+ @After
+ public void tearDown() throws Exception {
+TEST_UTIL.shutdownMiniCluster();
+ }
+
+ @Test(timeout=12)
Review comment:
Ok
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[GitHub] [hbase] apurtell commented on a change in pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close
apurtell commented on a change in pull request #2574:
URL: https://github.com/apache/hbase/pull/2574#discussion_r512165464
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
##
@@ -1679,22 +1688,82 @@ public void setTimeoutForWriteLock(long
timeoutForWriteLock) {
}
}
-if (timeoutForWriteLock == null
-|| timeoutForWriteLock == Long.MAX_VALUE) {
- // block waiting for the lock for closing
- lock.writeLock().lock(); // FindBugs: Complains
UL_UNRELEASED_LOCK_EXCEPTION_PATH but seems fine
+// Set the closing flag
+// From this point new arrivals at the region lock will get NSRE.
+
+this.closing.set(true);
+LOG.info("Closing region {}", this);
+
+// Acquire the close lock
+
+// The configuration parameter CLOSE_WAIT_ABORT is overloaded to enable
both
+// the new regionserver abort condition and interrupts for running
requests.
+// If CLOSE_WAIT_ABORT is not enabled there is no change from earlier
behavior,
+// we will not attempt to interrupt threads servicing requests nor crash
out
+// the regionserver if something remains stubborn.
+
+boolean canAbort = conf.getBoolean(CLOSE_WAIT_ABORT,
DEFAULT_CLOSE_WAIT_ABORT);
+boolean useTimedWait = false;
+if (timeoutForWriteLock == null || timeoutForWriteLock == Long.MAX_VALUE) {
+ if (canAbort) {
+timeoutForWriteLock = conf.getLong(CLOSE_WAIT_TIME,
DEFAULT_CLOSE_WAIT_TIME);
+useTimedWait = true;
+ }
} else {
+ // convert legacy use of timeoutForWriteLock in seconds to new use in
millis
+ timeoutForWriteLock = TimeUnit.SECONDS.toMillis(timeoutForWriteLock);
+ useTimedWait = true;
+}
+if (LOG.isDebugEnabled()) {
+ LOG.debug((useTimedWait ? "Time limited wait" : "Waiting") + " for close
lock on " + this);
Review comment:
Ok
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] apurtell edited a comment on pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close
apurtell edited a comment on pull request #2574: URL: https://github.com/apache/hbase/pull/2574#issuecomment-716726888 Also I think upon reflection if interrupted while waiting on this lock to close the region we shouldn't abort the RS, just throw an IO exception like in the timeout case where lock acquisition fails. Why would we be interrupted during close? Seems likely to be a case like the server is shutting down, not normal operation, so aborting upon interrupt while waiting on this lock would not provide much value. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] apurtell commented on pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close
apurtell commented on pull request #2574: URL: https://github.com/apache/hbase/pull/2574#issuecomment-716726888 Also I think upon reflection if interrupted while waiting on this lock to close the region we shouldn't abort the RS, just throw an IO exception like in the timeout case where lock acquisition fails. Why would we be interrupted during close? Seems likely to be a case like the server is shutting down, not normal operation, so aborting upon interrupt would not provide much value. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] apurtell edited a comment on pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close
apurtell edited a comment on pull request #2574:
URL: https://github.com/apache/hbase/pull/2574#issuecomment-716699603
I have found one test where interrupt by default causes a repeatable
problem.
TestSyncReplicationActive
[ERROR] TestSyncReplicationActive.testActive:99
Expected: a string containing "only marker edit is allowed"
but: was "Failed after attempts=1, exceptions:
2020-10-25T02:46:59.367Z, java.io.InterruptedIOException:
java.io.InterruptedIOException
at
org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.convertInterruptedExceptionToIOException(AbstractFSWAL.java:878)
at
org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.blockOnSync(AbstractFSWAL.java:866)
at
org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.sync(AsyncFSWAL.java:710)
at
org.apache.hadoop.hbase.regionserver.HRegion.sync(HRegion.java:9031)
at
org.apache.hadoop.hbase.regionserver.HRegion.doWALAppend(HRegion.java:8624)
at
org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutate(HRegion.java:4674)
at
org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:4594)
at
org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:4522)
at
org.apache.hadoop.hbase.regionserver.HRegion.doBatchMutate(HRegion.java:4992)
at
org.apache.hadoop.hbase.regionserver.HRegion.doBatchMutate(HRegion.java:4987)
at
org.apache.hadoop.hbase.regionserver.HRegion.doBatchMutate(HRegion.java:4983)
at
org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:3302)
at
org.apache.hadoop.hbase.regionserver.RSRpcServices.put(RSRpcServices.java:3031)
at
org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2994)
at
org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:45251)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:397)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:133)
at
org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:338)
at
org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:318)
Caused by: java.lang.InterruptedException
at java.lang.Object.wait(Native Method)
at
org.apache.hadoop.hbase.regionserver.wal.SyncFuture.get(SyncFuture.java:142)
at
org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.blockOnSync(AbstractFSWAL.java:855)
... 17 more
Thinking about what to do here it occurred to me we do not need to be so
greedy about interrupting handlers by default. The original motivation for
interrupting RPCs in flight was to address the case where we get stuck closing
the region, so we can be less aggressive and wait until we actually seem to be
stuck. In an early version of this patch the tryLock was attempted in a loop
that would wait the entire configured wait interval before triggering the
abort. Rightfully so @bharathv provided feedback the loop wasn't providing any
advantage, especially considering we crash the RS if interrupted, but I think
we should bring this back to do this:
remainingWaitTime = ;
waitInterval = ;
do {
start = EnvironmentEdgeManager.getCurrentTime();
acquired = tryLock(Math.min(remainingWaitTime, waitInterval),
TimeUnit.MILLISECONDS);
end = EnvironmentEdgeManager.getCurrentTime();
totalWaitTime += end - start;
remainingWaitTime -= end - start;
if (!acquired) {
interruptRegionOperations();
}
} while (!acquired && remainingWaitTime > 0);
This will cause us to begin interrupting region lock holders only if we have
already waited for some significant fraction of the total wait interval. This
also has the benefit (IMHO) of potentially issuing more than one interrupt to a
handler if the earlier interrupt was somehow swallowed by code we don't
control, like a Hadoop library, or in the HDFS client.
We do want to interrupt even things like WAL append if the handler is
holding us up closing the region, but upon reflection I do not believe we
should be so aggressive to proactively issue interrupts immediately when we
want to close.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] apurtell commented on pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close
apurtell commented on pull request #2574:
URL: https://github.com/apache/hbase/pull/2574#issuecomment-716699603
I have found one test where interrupt by default causes a repeatable
problem.
TestSyncReplicationActive
[ERROR] TestSyncReplicationActive.testActive:99
Expected: a string containing "only marker edit is allowed"
but: was "Failed after attempts=1, exceptions:
2020-10-25T02:46:59.367Z, java.io.InterruptedIOException:
java.io.InterruptedIOException
at
org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.convertInterruptedExceptionToIOException(AbstractFSWAL.java:878)
at
org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.blockOnSync(AbstractFSWAL.java:866)
at
org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.sync(AsyncFSWAL.java:710)
at
org.apache.hadoop.hbase.regionserver.HRegion.sync(HRegion.java:9031)
at
org.apache.hadoop.hbase.regionserver.HRegion.doWALAppend(HRegion.java:8624)
at
org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutate(HRegion.java:4674)
at
org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:4594)
at
org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:4522)
at
org.apache.hadoop.hbase.regionserver.HRegion.doBatchMutate(HRegion.java:4992)
at
org.apache.hadoop.hbase.regionserver.HRegion.doBatchMutate(HRegion.java:4987)
at
org.apache.hadoop.hbase.regionserver.HRegion.doBatchMutate(HRegion.java:4983)
at
org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:3302)
at
org.apache.hadoop.hbase.regionserver.RSRpcServices.put(RSRpcServices.java:3031)
at
org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2994)
at
org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:45251)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:397)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:133)
at
org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:338)
at
org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:318)
Caused by: java.lang.InterruptedException
at java.lang.Object.wait(Native Method)
at
org.apache.hadoop.hbase.regionserver.wal.SyncFuture.get(SyncFuture.java:142)
at
org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.blockOnSync(AbstractFSWAL.java:855)
... 17 more
Thinking about what to do here it occurred to me we do not need to be so
greedy about interrupting handlers by default. The original motivation for
interrupting RPCs in flight was to address the case where we get stuck closing
the region, so we can be less aggressive and wait until we actually seem to be
stuck. In an early version of this patch the tryLock was attempted in a loop
that would wait the entire configured wait interval before triggering the
abort. Rightfully so @bharathv provided feedback the loop wasn't providing any
advantage, especially considering we crash the RS if interrupted, but I think
we should bring this back to do this:
waitTime =
do {
start = EnvironmentEdgeManager.getCurrentTime();
acquired = tryLock(waitTime, TimeUnit.MILLISECONDS);
end = EnvironmentEdgeManager.getCurrentTime();
totalWaitTime += end - start;
waitTime -= end - start;
if (!acquired) {
interruptRegionOperations();
}
} while (!acquired && waitTime > 0);
This will cause us to begin interrupting region lock holders only if we have
already waited for some significant fraction of the total wait interval. This
also has the benefit (IMHO) of potentially issuing more than one interrupt to a
handler if the earlier interrupt was somehow swallowed by code we don't
control, like a Hadoop library, or in the HDFS client.
We do want to interrupt even things like WAL append if the handler is
holding us up closing the region, but upon reflection I do not believe we
should be so aggressive to proactively issue interrupts immediately when we
want to close.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] apurtell commented on a change in pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close
apurtell commented on a change in pull request #2574:
URL: https://github.com/apache/hbase/pull/2574#discussion_r512090958
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionInterrupt.java
##
@@ -0,0 +1,356 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hbase.regionserver;
+
+import static org.junit.Assert.assertTrue;
+
+import java.io.IOException;
+import java.io.InterruptedIOException;
+import java.util.Optional;
+import java.util.concurrent.atomic.AtomicBoolean;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hbase.Cell;
+import org.apache.hadoop.hbase.HBaseClassTestRule;
+import org.apache.hadoop.hbase.HBaseTestingUtility;
+import org.apache.hadoop.hbase.HConstants;
+import org.apache.hadoop.hbase.NotServingRegionException;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.TableNameTestRule;
+import org.apache.hadoop.hbase.Waiter;
+import org.apache.hadoop.hbase.client.Admin;
+import org.apache.hadoop.hbase.client.Append;
+import org.apache.hadoop.hbase.client.BufferedMutator;
+import org.apache.hadoop.hbase.client.ColumnFamilyDescriptorBuilder;
+import org.apache.hadoop.hbase.client.Delete;
+import org.apache.hadoop.hbase.client.Durability;
+import org.apache.hadoop.hbase.client.Increment;
+import org.apache.hadoop.hbase.client.Put;
+import org.apache.hadoop.hbase.client.RegionInfo;
+import org.apache.hadoop.hbase.client.Result;
+import org.apache.hadoop.hbase.client.ResultScanner;
+import org.apache.hadoop.hbase.client.Scan;
+import org.apache.hadoop.hbase.client.Table;
+import org.apache.hadoop.hbase.client.TableDescriptor;
+import org.apache.hadoop.hbase.client.TableDescriptorBuilder;
+import org.apache.hadoop.hbase.coprocessor.ObserverContext;
+import org.apache.hadoop.hbase.coprocessor.RegionCoprocessor;
+import org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment;
+import org.apache.hadoop.hbase.coprocessor.RegionObserver;
+import org.apache.hadoop.hbase.exceptions.DeserializationException;
+import org.apache.hadoop.hbase.filter.FilterBase;
+import org.apache.hadoop.hbase.testclassification.LargeTests;
+import org.apache.hadoop.hbase.testclassification.RegionServerTests;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.hadoop.hbase.wal.WAL;
+import org.apache.hadoop.hbase.wal.WALEdit;
+import org.junit.After;
+import org.junit.Before;
+import org.junit.BeforeClass;
+import org.junit.ClassRule;
+import org.junit.Rule;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+@Category({RegionServerTests.class, LargeTests.class})
+public class TestRegionInterrupt {
+
+ @ClassRule
+ public static final HBaseClassTestRule CLASS_RULE =
+HBaseClassTestRule.forClass(TestRegionInterrupt.class);
+
+ private static HBaseTestingUtility TEST_UTIL = new HBaseTestingUtility();
+ private static final Logger LOG =
LoggerFactory.getLogger(TestRegionInterrupt.class);
+
+ static final int SLEEP_TIME = 10 * 1000;
+ static final byte[] FAMILY = Bytes.toBytes("info");
+
+ @Rule
+ public TableNameTestRule name = new TableNameTestRule();
+
+ @BeforeClass
+ public static void setUpBeforeClass() throws Exception {
+Configuration conf = TEST_UTIL.getConfiguration();
+conf.setClass(HConstants.REGION_IMPL, InterruptInterceptingHRegion.class,
Region.class);
+conf.setBoolean(HRegion.CLOSE_WAIT_ABORT, true);
Review comment:
Let me restate this as what I think you were getting at: "Can we set a
shorter abort interval so this test test will not run for a long time if
interrupts do not work as expected?"
Sure, ok.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] apurtell commented on a change in pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close
apurtell commented on a change in pull request #2574:
URL: https://github.com/apache/hbase/pull/2574#discussion_r512090958
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionInterrupt.java
##
@@ -0,0 +1,356 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hbase.regionserver;
+
+import static org.junit.Assert.assertTrue;
+
+import java.io.IOException;
+import java.io.InterruptedIOException;
+import java.util.Optional;
+import java.util.concurrent.atomic.AtomicBoolean;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hbase.Cell;
+import org.apache.hadoop.hbase.HBaseClassTestRule;
+import org.apache.hadoop.hbase.HBaseTestingUtility;
+import org.apache.hadoop.hbase.HConstants;
+import org.apache.hadoop.hbase.NotServingRegionException;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.TableNameTestRule;
+import org.apache.hadoop.hbase.Waiter;
+import org.apache.hadoop.hbase.client.Admin;
+import org.apache.hadoop.hbase.client.Append;
+import org.apache.hadoop.hbase.client.BufferedMutator;
+import org.apache.hadoop.hbase.client.ColumnFamilyDescriptorBuilder;
+import org.apache.hadoop.hbase.client.Delete;
+import org.apache.hadoop.hbase.client.Durability;
+import org.apache.hadoop.hbase.client.Increment;
+import org.apache.hadoop.hbase.client.Put;
+import org.apache.hadoop.hbase.client.RegionInfo;
+import org.apache.hadoop.hbase.client.Result;
+import org.apache.hadoop.hbase.client.ResultScanner;
+import org.apache.hadoop.hbase.client.Scan;
+import org.apache.hadoop.hbase.client.Table;
+import org.apache.hadoop.hbase.client.TableDescriptor;
+import org.apache.hadoop.hbase.client.TableDescriptorBuilder;
+import org.apache.hadoop.hbase.coprocessor.ObserverContext;
+import org.apache.hadoop.hbase.coprocessor.RegionCoprocessor;
+import org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment;
+import org.apache.hadoop.hbase.coprocessor.RegionObserver;
+import org.apache.hadoop.hbase.exceptions.DeserializationException;
+import org.apache.hadoop.hbase.filter.FilterBase;
+import org.apache.hadoop.hbase.testclassification.LargeTests;
+import org.apache.hadoop.hbase.testclassification.RegionServerTests;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.hadoop.hbase.wal.WAL;
+import org.apache.hadoop.hbase.wal.WALEdit;
+import org.junit.After;
+import org.junit.Before;
+import org.junit.BeforeClass;
+import org.junit.ClassRule;
+import org.junit.Rule;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+@Category({RegionServerTests.class, LargeTests.class})
+public class TestRegionInterrupt {
+
+ @ClassRule
+ public static final HBaseClassTestRule CLASS_RULE =
+HBaseClassTestRule.forClass(TestRegionInterrupt.class);
+
+ private static HBaseTestingUtility TEST_UTIL = new HBaseTestingUtility();
+ private static final Logger LOG =
LoggerFactory.getLogger(TestRegionInterrupt.class);
+
+ static final int SLEEP_TIME = 10 * 1000;
+ static final byte[] FAMILY = Bytes.toBytes("info");
+
+ @Rule
+ public TableNameTestRule name = new TableNameTestRule();
+
+ @BeforeClass
+ public static void setUpBeforeClass() throws Exception {
+Configuration conf = TEST_UTIL.getConfiguration();
+conf.setClass(HConstants.REGION_IMPL, InterruptInterceptingHRegion.class,
Region.class);
+conf.setBoolean(HRegion.CLOSE_WAIT_ABORT, true);
Review comment:
Let me restate this as what I think you were getting at: "Can we set a
shorter abort interval so this test will not run for a long time if interrupts
do not work as expected?"
Sure, ok.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] apurtell commented on a change in pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close
apurtell commented on a change in pull request #2574:
URL: https://github.com/apache/hbase/pull/2574#discussion_r512090958
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionInterrupt.java
##
@@ -0,0 +1,356 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hbase.regionserver;
+
+import static org.junit.Assert.assertTrue;
+
+import java.io.IOException;
+import java.io.InterruptedIOException;
+import java.util.Optional;
+import java.util.concurrent.atomic.AtomicBoolean;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hbase.Cell;
+import org.apache.hadoop.hbase.HBaseClassTestRule;
+import org.apache.hadoop.hbase.HBaseTestingUtility;
+import org.apache.hadoop.hbase.HConstants;
+import org.apache.hadoop.hbase.NotServingRegionException;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.TableNameTestRule;
+import org.apache.hadoop.hbase.Waiter;
+import org.apache.hadoop.hbase.client.Admin;
+import org.apache.hadoop.hbase.client.Append;
+import org.apache.hadoop.hbase.client.BufferedMutator;
+import org.apache.hadoop.hbase.client.ColumnFamilyDescriptorBuilder;
+import org.apache.hadoop.hbase.client.Delete;
+import org.apache.hadoop.hbase.client.Durability;
+import org.apache.hadoop.hbase.client.Increment;
+import org.apache.hadoop.hbase.client.Put;
+import org.apache.hadoop.hbase.client.RegionInfo;
+import org.apache.hadoop.hbase.client.Result;
+import org.apache.hadoop.hbase.client.ResultScanner;
+import org.apache.hadoop.hbase.client.Scan;
+import org.apache.hadoop.hbase.client.Table;
+import org.apache.hadoop.hbase.client.TableDescriptor;
+import org.apache.hadoop.hbase.client.TableDescriptorBuilder;
+import org.apache.hadoop.hbase.coprocessor.ObserverContext;
+import org.apache.hadoop.hbase.coprocessor.RegionCoprocessor;
+import org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment;
+import org.apache.hadoop.hbase.coprocessor.RegionObserver;
+import org.apache.hadoop.hbase.exceptions.DeserializationException;
+import org.apache.hadoop.hbase.filter.FilterBase;
+import org.apache.hadoop.hbase.testclassification.LargeTests;
+import org.apache.hadoop.hbase.testclassification.RegionServerTests;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.hadoop.hbase.wal.WAL;
+import org.apache.hadoop.hbase.wal.WALEdit;
+import org.junit.After;
+import org.junit.Before;
+import org.junit.BeforeClass;
+import org.junit.ClassRule;
+import org.junit.Rule;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+@Category({RegionServerTests.class, LargeTests.class})
+public class TestRegionInterrupt {
+
+ @ClassRule
+ public static final HBaseClassTestRule CLASS_RULE =
+HBaseClassTestRule.forClass(TestRegionInterrupt.class);
+
+ private static HBaseTestingUtility TEST_UTIL = new HBaseTestingUtility();
+ private static final Logger LOG =
LoggerFactory.getLogger(TestRegionInterrupt.class);
+
+ static final int SLEEP_TIME = 10 * 1000;
+ static final byte[] FAMILY = Bytes.toBytes("info");
+
+ @Rule
+ public TableNameTestRule name = new TableNameTestRule();
+
+ @BeforeClass
+ public static void setUpBeforeClass() throws Exception {
+Configuration conf = TEST_UTIL.getConfiguration();
+conf.setClass(HConstants.REGION_IMPL, InterruptInterceptingHRegion.class,
Region.class);
+conf.setBoolean(HRegion.CLOSE_WAIT_ABORT, true);
Review comment:
Let me restate this as what I think you were getting at: "Can we set
this shorter so it doesn't cause the test to run for a long time and/or timeout
if interrupts do not work as expected?"
Sure, ok.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] apurtell commented on a change in pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close
apurtell commented on a change in pull request #2574:
URL: https://github.com/apache/hbase/pull/2574#discussion_r512089577
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
##
@@ -688,14 +688,17 @@ void sawNoSuchFamily() {
// Last flush time for each Store. Useful when we are flushing for each
column
private final ConcurrentMap lastStoreFlushTimeMap = new
ConcurrentHashMap<>();
- final RegionServerServices rsServices;
+ protected RegionServerServices rsServices;
Review comment:
In the HRegion tests I mock RSS with mockito, to test if the HRegion
code calls RSS#abort as it should. We don't need to make a real cluster test
that aborts a server (and adds a lot of time to the unit test suite) just to
confirm this. So, it needs to be nonfinal for mocking, but it does not need to
be exposed as a settable thing, because nowhere else in the code is there a use
for that.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] apurtell commented on a change in pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close
apurtell commented on a change in pull request #2574:
URL: https://github.com/apache/hbase/pull/2574#discussion_r512087407
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
##
@@ -9000,6 +9129,49 @@ public long getReadPoint() {
return getReadPoint(IsolationLevel.READ_COMMITTED);
}
+ /**
+ * Interrupt any region options that have acquired the region lock via
+ * {@link
#startRegionOperation(org.apache.hadoop.hbase.regionserver.Region.Operation)},
+ * or {@link #startBulkRegionOperation(boolean)}.
+ */
+ private void interruptRegionOperations() {
+for (Thread t: regionLockHolders) {
+ t.interrupt();
+}
+ }
+
+ /**
+ * Check thread interrupt status and throw an exception if interrupted.
+ * @throws NotServingRegionException if region is closing
+ * @throws InterruptedIOException if interrupted but region is not closing
+ */
+ // Package scope for tests
+ void checkInterrupt() throws NotServingRegionException,
InterruptedIOException {
+if (Thread.interrupted()) {
+ if (this.closing.get()) {
+throw new NotServingRegionException(
+ getRegionInfo().getRegionNameAsString() + " is closing");
+ }
+ throw new InterruptedIOException();
+}
+ }
+
+ /**
+ * Throw the correct exception upon interrupt
+ * @param t cause
+ * @throws NotServingRegionException if region is closing
+ * @throws InterruptedIOException in all cases except if region is closing
Review comment:
Old stuff, will update
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] apurtell commented on a change in pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close
apurtell commented on a change in pull request #2574:
URL: https://github.com/apache/hbase/pull/2574#discussion_r512087013
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
##
@@ -8793,7 +8921,7 @@ public void closeRegionOperation(Operation operation)
throws IOException {
* @throws InterruptedIOException if interrupted while waiting for a lock
*/
private void startBulkRegionOperation(boolean writeLockNeeded)
- throws NotServingRegionException, RegionTooBusyException,
InterruptedIOException {
+ throws NotServingRegionException, IOException {
Review comment:
Ok
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] apurtell commented on a change in pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close
apurtell commented on a change in pull request #2574:
URL: https://github.com/apache/hbase/pull/2574#discussion_r512083604
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
##
@@ -7364,4 +7366,157 @@ protected HStoreForTesting(final HRegion region,
return super.doCompaction(cr, filesToCompact, user, compactionStartTime,
newFiles);
}
}
+
+ @Test
+ public void testCloseNoInterrupt() throws Exception {
+byte[] cf1 = Bytes.toBytes("CF1");
+byte[][] families = { cf1 };
+
+Configuration conf = new Configuration(CONF);
+// Disable close thread interrupt and server abort behavior
+conf.setBoolean(HRegion.CLOSE_WAIT_ABORT, false);
+region = initHRegion(tableName, method, conf, families);
+
+final CountDownLatch latch = new CountDownLatch(1);
+final AtomicBoolean holderInterrupted = new AtomicBoolean();
+Thread holder = new Thread(new Runnable() {
+ @Override
+ public void run() {
+try {
+ LOG.info("Starting region operation holder");
+ region.startRegionOperation(Operation.SCAN);
+ latch.countDown();
+ try {
+Thread.sleep(10*1000);
+ } catch (InterruptedException e) {
+LOG.info("Interrupted");
+holderInterrupted.set(true);
Review comment:
If you throw an assertion inside a Thread runnable it won't unwind to
the JUnit test method that forked the thread, it will be caught by
Thread.UncaughtExceptionHandler, which is not what I think we want. So I set
the boolean and then test it in the unit test method where an assertion will
propagate the way I want. Maybe JUnit installs its own UEH, didn't test, but
the way I have done it makes more sense to me.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] apurtell commented on a change in pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close
apurtell commented on a change in pull request #2574:
URL: https://github.com/apache/hbase/pull/2574#discussion_r512084799
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
##
@@ -1679,22 +1688,82 @@ public void setTimeoutForWriteLock(long
timeoutForWriteLock) {
}
}
-if (timeoutForWriteLock == null
-|| timeoutForWriteLock == Long.MAX_VALUE) {
- // block waiting for the lock for closing
- lock.writeLock().lock(); // FindBugs: Complains
UL_UNRELEASED_LOCK_EXCEPTION_PATH but seems fine
+// Set the closing flag
+// From this point new arrivals at the region lock will get NSRE.
+
+this.closing.set(true);
+LOG.info("Closing region {}", this);
+
+// Acquire the close lock
+
+// The configuration parameter CLOSE_WAIT_ABORT is overloaded to enable
both
+// the new regionserver abort condition and interrupts for running
requests.
+// If CLOSE_WAIT_ABORT is not enabled there is no change from earlier
behavior,
+// we will not attempt to interrupt threads servicing requests nor crash
out
+// the regionserver if something remains stubborn.
+
+boolean canAbort = conf.getBoolean(CLOSE_WAIT_ABORT,
DEFAULT_CLOSE_WAIT_ABORT);
+boolean useTimedWait = false;
+if (timeoutForWriteLock == null || timeoutForWriteLock == Long.MAX_VALUE) {
+ if (canAbort) {
+timeoutForWriteLock = conf.getLong(CLOSE_WAIT_TIME,
DEFAULT_CLOSE_WAIT_TIME);
+useTimedWait = true;
+ }
} else {
+ // convert legacy use of timeoutForWriteLock in seconds to new use in
millis
+ timeoutForWriteLock = TimeUnit.SECONDS.toMillis(timeoutForWriteLock);
Review comment:
Ok
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] apurtell commented on a change in pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close
apurtell commented on a change in pull request #2574:
URL: https://github.com/apache/hbase/pull/2574#discussion_r512083604
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
##
@@ -7364,4 +7366,157 @@ protected HStoreForTesting(final HRegion region,
return super.doCompaction(cr, filesToCompact, user, compactionStartTime,
newFiles);
}
}
+
+ @Test
+ public void testCloseNoInterrupt() throws Exception {
+byte[] cf1 = Bytes.toBytes("CF1");
+byte[][] families = { cf1 };
+
+Configuration conf = new Configuration(CONF);
+// Disable close thread interrupt and server abort behavior
+conf.setBoolean(HRegion.CLOSE_WAIT_ABORT, false);
+region = initHRegion(tableName, method, conf, families);
+
+final CountDownLatch latch = new CountDownLatch(1);
+final AtomicBoolean holderInterrupted = new AtomicBoolean();
+Thread holder = new Thread(new Runnable() {
+ @Override
+ public void run() {
+try {
+ LOG.info("Starting region operation holder");
+ region.startRegionOperation(Operation.SCAN);
+ latch.countDown();
+ try {
+Thread.sleep(10*1000);
+ } catch (InterruptedException e) {
+LOG.info("Interrupted");
+holderInterrupted.set(true);
Review comment:
If you throw an assertion inside a Thread runnable it won't unwind to
the Junit test method, it will be caught by Thread.UncaughtExceptionHandler,
which is not what I think we want. So I set the boolean and then test it in the
unit test method where an assertion will propagate the way I want. Maybe JUnit
installs its own UEH, didn't test, but the way I have done it makes more sense
to me.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] apurtell commented on a change in pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close
apurtell commented on a change in pull request #2574:
URL: https://github.com/apache/hbase/pull/2574#discussion_r512084523
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
##
@@ -7364,4 +7366,157 @@ protected HStoreForTesting(final HRegion region,
return super.doCompaction(cr, filesToCompact, user, compactionStartTime,
newFiles);
}
}
+
+ @Test
+ public void testCloseNoInterrupt() throws Exception {
+byte[] cf1 = Bytes.toBytes("CF1");
+byte[][] families = { cf1 };
+
+Configuration conf = new Configuration(CONF);
+// Disable close thread interrupt and server abort behavior
+conf.setBoolean(HRegion.CLOSE_WAIT_ABORT, false);
+region = initHRegion(tableName, method, conf, families);
+
+final CountDownLatch latch = new CountDownLatch(1);
+final AtomicBoolean holderInterrupted = new AtomicBoolean();
+Thread holder = new Thread(new Runnable() {
+ @Override
+ public void run() {
+try {
+ LOG.info("Starting region operation holder");
+ region.startRegionOperation(Operation.SCAN);
+ latch.countDown();
+ try {
+Thread.sleep(10*1000);
+ } catch (InterruptedException e) {
+LOG.info("Interrupted");
+holderInterrupted.set(true);
+ }
+} catch (Exception e) {
+ throw new RuntimeException(e);
+} finally {
+ try {
+region.closeRegionOperation();
+ } catch (IOException e) {
+ }
+ LOG.info("Stopped region operation holder");
+}
+ }
+});
+
+holder.start();
+latch.await();
+region.close();
+holder.join();
+region = null;
+
+assertFalse("Region lock holder should not have been interrupted",
holderInterrupted.get());
+ }
+
+ @Test
+ public void testCloseInterrupt() throws Exception {
+byte[] cf1 = Bytes.toBytes("CF1");
+byte[][] families = { cf1 };
+
+Configuration conf = new Configuration(CONF);
+// Enable close thread interrupt and server abort behavior
+conf.setBoolean(HRegion.CLOSE_WAIT_ABORT, true);
+region = initHRegion(tableName, method, conf, families);
+
+final CountDownLatch latch = new CountDownLatch(1);
+final AtomicBoolean holderInterrupted = new AtomicBoolean();
+Thread holder = new Thread(new Runnable() {
+ @Override
+ public void run() {
+try {
+ LOG.info("Starting region operation holder");
+ region.startRegionOperation(Operation.SCAN);
+ latch.countDown();
+ try {
+Thread.sleep(10*1000);
+ } catch (InterruptedException e) {
+LOG.info("Interrupted");
+holderInterrupted.set(true);
+ }
+} catch (Exception e) {
+ throw new RuntimeException(e);
+} finally {
+ try {
+region.closeRegionOperation();
+ } catch (IOException e) {
+ }
+ LOG.info("Stopped region operation holder");
+}
+ }
+});
+
+holder.start();
+latch.await();
+region.close();
+holder.join();
+region = null;
+
+assertTrue("Region lock holder was not interrupted",
holderInterrupted.get());
+ }
+
+ @Test
+ public void testCloseAbort() throws Exception {
+byte[] cf1 = Bytes.toBytes("CF1");
+byte[][] families = { cf1 };
+
+Configuration conf = new Configuration(CONF);
+// Enable close thread interrupt and server abort behavior
+// Set the close lock acquisition wait time to 5 seconds
+conf.setBoolean(HRegion.CLOSE_WAIT_ABORT, true);
+conf.setInt(HRegion.CLOSE_WAIT_TIME, 5*1000);
+region = initHRegion(tableName, method, conf, families);
+RegionServerServices rsServices = mock(RegionServerServices.class);
+region.rsServices = rsServices;
+
+final CountDownLatch latch = new CountDownLatch(1);
+Thread holder = new Thread(new Runnable() {
+ @Override
+ public void run() {
+try {
+ LOG.info("Starting region operation holder");
+ region.startRegionOperation(Operation.SCAN);
+ latch.countDown();
+ // Hold the lock for 10 seconds no matter how many times we are
interrupted
+ int timeRemaining = 10 * 1000;
+ while (timeRemaining > 0) {
+long start = EnvironmentEdgeManager.currentTime();
Review comment:
We always use EnvironmentEdgeManager, especially in tests, per our
coding conventions.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] apurtell commented on a change in pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close
apurtell commented on a change in pull request #2574:
URL: https://github.com/apache/hbase/pull/2574#discussion_r512083604
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
##
@@ -7364,4 +7366,157 @@ protected HStoreForTesting(final HRegion region,
return super.doCompaction(cr, filesToCompact, user, compactionStartTime,
newFiles);
}
}
+
+ @Test
+ public void testCloseNoInterrupt() throws Exception {
+byte[] cf1 = Bytes.toBytes("CF1");
+byte[][] families = { cf1 };
+
+Configuration conf = new Configuration(CONF);
+// Disable close thread interrupt and server abort behavior
+conf.setBoolean(HRegion.CLOSE_WAIT_ABORT, false);
+region = initHRegion(tableName, method, conf, families);
+
+final CountDownLatch latch = new CountDownLatch(1);
+final AtomicBoolean holderInterrupted = new AtomicBoolean();
+Thread holder = new Thread(new Runnable() {
+ @Override
+ public void run() {
+try {
+ LOG.info("Starting region operation holder");
+ region.startRegionOperation(Operation.SCAN);
+ latch.countDown();
+ try {
+Thread.sleep(10*1000);
+ } catch (InterruptedException e) {
+LOG.info("Interrupted");
+holderInterrupted.set(true);
Review comment:
If you throw an assertion inside a Thread runnable it won't unwind to
the Junit test method, it will be caught by Thread.UncaughtExceptionHandler,
which is not what I think we want. So I set the boolean and then test it in the
unit test method where an assertion will propagate the way I want.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] virajjasani commented on a change in pull request #2574: HBASE-25212 Optionally abort requests in progress after deciding a region should close
virajjasani commented on a change in pull request #2574:
URL: https://github.com/apache/hbase/pull/2574#discussion_r512006608
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
##
@@ -7364,4 +7366,157 @@ protected HStoreForTesting(final HRegion region,
return super.doCompaction(cr, filesToCompact, user, compactionStartTime,
newFiles);
}
}
+
+ @Test
+ public void testCloseNoInterrupt() throws Exception {
+byte[] cf1 = Bytes.toBytes("CF1");
+byte[][] families = { cf1 };
+
+Configuration conf = new Configuration(CONF);
+// Disable close thread interrupt and server abort behavior
+conf.setBoolean(HRegion.CLOSE_WAIT_ABORT, false);
+region = initHRegion(tableName, method, conf, families);
+
+final CountDownLatch latch = new CountDownLatch(1);
+final AtomicBoolean holderInterrupted = new AtomicBoolean();
+Thread holder = new Thread(new Runnable() {
+ @Override
+ public void run() {
+try {
+ LOG.info("Starting region operation holder");
+ region.startRegionOperation(Operation.SCAN);
+ latch.countDown();
+ try {
+Thread.sleep(10*1000);
+ } catch (InterruptedException e) {
+LOG.info("Interrupted");
+holderInterrupted.set(true);
+ }
+} catch (Exception e) {
+ throw new RuntimeException(e);
+} finally {
+ try {
+region.closeRegionOperation();
+ } catch (IOException e) {
+ }
+ LOG.info("Stopped region operation holder");
+}
+ }
+});
+
+holder.start();
+latch.await();
+region.close();
+holder.join();
+region = null;
+
+assertFalse("Region lock holder should not have been interrupted",
holderInterrupted.get());
+ }
+
+ @Test
+ public void testCloseInterrupt() throws Exception {
+byte[] cf1 = Bytes.toBytes("CF1");
+byte[][] families = { cf1 };
+
+Configuration conf = new Configuration(CONF);
+// Enable close thread interrupt and server abort behavior
+conf.setBoolean(HRegion.CLOSE_WAIT_ABORT, true);
+region = initHRegion(tableName, method, conf, families);
+
+final CountDownLatch latch = new CountDownLatch(1);
+final AtomicBoolean holderInterrupted = new AtomicBoolean();
+Thread holder = new Thread(new Runnable() {
+ @Override
+ public void run() {
+try {
+ LOG.info("Starting region operation holder");
+ region.startRegionOperation(Operation.SCAN);
+ latch.countDown();
+ try {
+Thread.sleep(10*1000);
+ } catch (InterruptedException e) {
+LOG.info("Interrupted");
+holderInterrupted.set(true);
+ }
+} catch (Exception e) {
+ throw new RuntimeException(e);
+} finally {
+ try {
+region.closeRegionOperation();
+ } catch (IOException e) {
+ }
+ LOG.info("Stopped region operation holder");
+}
+ }
+});
+
+holder.start();
+latch.await();
+region.close();
+holder.join();
+region = null;
+
+assertTrue("Region lock holder was not interrupted",
holderInterrupted.get());
+ }
+
+ @Test
+ public void testCloseAbort() throws Exception {
+byte[] cf1 = Bytes.toBytes("CF1");
+byte[][] families = { cf1 };
+
+Configuration conf = new Configuration(CONF);
+// Enable close thread interrupt and server abort behavior
+// Set the close lock acquisition wait time to 5 seconds
+conf.setBoolean(HRegion.CLOSE_WAIT_ABORT, true);
+conf.setInt(HRegion.CLOSE_WAIT_TIME, 5*1000);
+region = initHRegion(tableName, method, conf, families);
+RegionServerServices rsServices = mock(RegionServerServices.class);
+region.rsServices = rsServices;
+
+final CountDownLatch latch = new CountDownLatch(1);
+Thread holder = new Thread(new Runnable() {
+ @Override
+ public void run() {
+try {
+ LOG.info("Starting region operation holder");
+ region.startRegionOperation(Operation.SCAN);
+ latch.countDown();
+ // Hold the lock for 10 seconds no matter how many times we are
interrupted
+ int timeRemaining = 10 * 1000;
+ while (timeRemaining > 0) {
+long start = EnvironmentEdgeManager.currentTime();
Review comment:
nit: if the only usage is to find diff b/ `end` and `start`, directly
using `System.currentTimeMillis()` might be preferred option?
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
##
@@ -7364,4 +7366,157 @@ protected HStoreForTesting(final HRegion region,
return super.doCompaction(cr, filesToCompact, user, compactionStartTime,
newFiles);
}
}
+
+ @Test
+ public void testCloseNoInterrupt() throws Exception {
+byte[] cf1 = Bytes.toBytes("CF1");
+
[jira] [Commented] (HBASE-24015) Coverage for Assign and Unassign of Regions on RegionServer on failure
[
https://issues.apache.org/jira/browse/HBASE-24015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17220761#comment-17220761
]
Hudson commented on HBASE-24015:
Results for branch branch-2.3
[build #90 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/90/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/90/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/90/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/90/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/90/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Coverage for Assign and Unassign of Regions on RegionServer on failure
> --
>
> Key: HBASE-24015
> URL: https://issues.apache.org/jira/browse/HBASE-24015
> Project: HBase
> Issue Type: Test
> Components: amv2
>Reporter: Michael Stack
>Assignee: Sandeep Pal
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.3, 1.7.0, 2.4.0
>
>
> Looking at 'HBASE-23984 [Flakey Tests] TestMasterAbortAndRSGotKilled fails in
> teardown', and at UnassignRegionHandler, AssignRegionHandler,
> CloseRegionHandler, and the work that is done inline w/ request vs that which
> to the side in executors, we need more coverage and specification of what
> happens around the edges. This coverage would be more to see if holes in our
> handling currently in a unit test case context before we see it out on
> clusters.
> HBASE-23984 addresses holes where UnassignRegionHandler and
> AssignRegionHandler could skip out w/o clearing Regions from the
> RegionServer#regionsInTransitionInRS Map of Regions In Transition if failed
> open or close because the RegionServer is aborting.
> Other holes seem lurking. On exception, we were leaving entries in the
> RegionServer# submittedRegionProcedure map added by HBASE-2204; not the end
> of the world but they should be cleared on error? HBASE-23984 adds clearning
> from submittedRegionProcedure but then procedures even if failed get added to
> the cache of procedures... so if we try to run the procedure again against
> this server it won't be scheduled.
> interesting stuff.
> This issue is about adding tests that fail assign/unassign/close on the
> RegionServer side making sure RS state is left in a good condition on fail.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25189) [Metrics] Add checkAndPut and checkAndDelete latency metrics at table level
[
https://issues.apache.org/jira/browse/HBASE-25189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17220762#comment-17220762
]
Hudson commented on HBASE-25189:
Results for branch branch-2.3
[build #90 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/90/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/90/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/90/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/90/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/90/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> [Metrics] Add checkAndPut and checkAndDelete latency metrics at table level
> ---
>
> Key: HBASE-25189
> URL: https://issues.apache.org/jira/browse/HBASE-25189
> Project: HBase
> Issue Type: Improvement
> Components: metrics
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Minor
> Fix For: 3.0.0-alpha-1, 2.3.3, 1.7.0, 2.2.7
>
>
> Add *checkAndPut* and *checkAndDelete* table latency metrics.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] petersomogyi opened a new pull request #2588: HBASE-25220 Backport HBASE-24246 Miscellaneous hbck2 fixMeta bulk merge fixes: better logging around merges/overlap-fixing, 'HBCK Report'
petersomogyi opened a new pull request #2588: URL: https://github.com/apache/hbase/pull/2588 Includes addendum hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java Fix weird brackets around each region name when logging. hbase-server/src/main/java/org/apache/hadoop/hbase/master/MetaFixer.java Log when we hit the max merge limit. Also up limit to 64. hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/MergeTableRegionsProcedure.java Make logs make more sense to operator. hbase-server/src/main/resources/hbase-webapps/master/hbck.jsp Make RegionName show when you mouseover so long names don't mess up display of holes and overlaps. Address Mingliang Liu liuml07 feedback Signed-off-by: Peter Somogyi Signed-off-by: Mingliang Liu Addendum to address minor feedback on text This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (HBASE-25220) Backport HBASE-24246 "Miscellaneous hbck2 fixMeta bulk merge fixes: better logging around merges/overlap-fixing, 'HBCK Report' overlap listing, and configuration" to bra
Peter Somogyi created HBASE-25220: - Summary: Backport HBASE-24246 "Miscellaneous hbck2 fixMeta bulk merge fixes: better logging around merges/overlap-fixing, 'HBCK Report' overlap listing, and configuration" to branch-2.2 Key: HBASE-25220 URL: https://issues.apache.org/jira/browse/HBASE-25220 Project: HBase Issue Type: Sub-task Reporter: Peter Somogyi Assignee: Peter Somogyi -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2579: HBASE-24999 Master manages ReplicationServers
Apache-HBase commented on pull request #2579: URL: https://github.com/apache/hbase/pull/2579#issuecomment-716572121 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 9s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ HBASE-24666 Compile Tests _ | | +0 :ok: | mvndep | 0m 19s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 44s | HBASE-24666 passed | | +1 :green_heart: | compile | 2m 14s | HBASE-24666 passed | | +1 :green_heart: | shadedjars | 8m 18s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 7s | HBASE-24666 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 38s | the patch passed | | +1 :green_heart: | compile | 2m 37s | the patch passed | | +1 :green_heart: | javac | 2m 37s | the patch passed | | +1 :green_heart: | shadedjars | 8m 0s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 41s | hbase-server generated 1 new + 83 unchanged - 0 fixed = 84 total (was 83) | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 2s | hbase-protocol-shaded in the patch passed. | | +1 :green_heart: | unit | 196m 58s | hbase-server in the patch passed. | | | | 235m 19s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2579/4/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2579 | | Optional Tests | unit javac javadoc shadedjars compile | | uname | Linux 3e0525c8a849 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | HBASE-24666 / 1a64f1a0db | | Default Java | 2020-01-14 | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2579/4/artifact/yetus-jdk11-hadoop3-check/output/diff-javadoc-javadoc-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2579/4/testReport/ | | Max. process+thread count | 3270 (vs. ulimit of 3) | | modules | C: hbase-protocol-shaded hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2579/4/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25189) [Metrics] Add checkAndPut and checkAndDelete latency metrics at table level
[
https://issues.apache.org/jira/browse/HBASE-25189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17220698#comment-17220698
]
Hudson commented on HBASE-25189:
Results for branch master
[build #106 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/106/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/106/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/106/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/106/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> [Metrics] Add checkAndPut and checkAndDelete latency metrics at table level
> ---
>
> Key: HBASE-25189
> URL: https://issues.apache.org/jira/browse/HBASE-25189
> Project: HBase
> Issue Type: Improvement
> Components: metrics
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Minor
> Fix For: 3.0.0-alpha-1, 2.3.3, 1.7.0, 2.2.7
>
>
> Add *checkAndPut* and *checkAndDelete* table latency metrics.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25215) TestClientOperationTimeout.testScanTimeout is flaky
[
https://issues.apache.org/jira/browse/HBASE-25215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17220699#comment-17220699
]
Hudson commented on HBASE-25215:
Results for branch master
[build #106 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/106/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/106/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/106/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/106/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> TestClientOperationTimeout.testScanTimeout is flaky
> ---
>
> Key: HBASE-25215
> URL: https://issues.apache.org/jira/browse/HBASE-25215
> Project: HBase
> Issue Type: Bug
> Components: flakies, test
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25194) Do not publish workspace in flaky find job
[
https://issues.apache.org/jira/browse/HBASE-25194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17220682#comment-17220682
]
Hudson commented on HBASE-25194:
Results for branch branch-1.4
[build #14 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1.4/14/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1.4/14//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1.4/14//JDK7_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1.4/14//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
> Do not publish workspace in flaky find job
> --
>
> Key: HBASE-25194
> URL: https://issues.apache.org/jira/browse/HBASE-25194
> Project: HBase
> Issue Type: Sub-task
> Components: jenkins
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.3, 1.7.0, 2.4.0, 1.4.14, 2.2.7
>
>
> As said in the parent issue.
> And I tried to use the publishHTML target on jenkins job configuration page,
> the help message for reportDir is
> {noformat}
> The path to the HTML report directory relative to the workspace.
> {noformat}
> For the reportFiles is
> {noformat}
> The file(s) to provide links inside the report directory
> {noformat}
> I think this clearly means that the plugin will publish all the files under
> reportDir, the reportFiles is just the index page.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25203) Change the reference url to flaky list in our jenkins jobs
[
https://issues.apache.org/jira/browse/HBASE-25203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17220683#comment-17220683
]
Hudson commented on HBASE-25203:
Results for branch branch-1.4
[build #14 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1.4/14/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1.4/14//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1.4/14//JDK7_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1.4/14//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
> Change the reference url to flaky list in our jenkins jobs
> --
>
> Key: HBASE-25203
> URL: https://issues.apache.org/jira/browse/HBASE-25203
> Project: HBase
> Issue Type: Sub-task
> Components: flakies, jenkins
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.3, 1.7.0, 2.4.0, 1.4.14, 2.2.7
>
>
> Nightly
> Flaky tests
> PreCommit
> personality
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25204) Nightly job failed as the name of jdk and maven changed
[
https://issues.apache.org/jira/browse/HBASE-25204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17220684#comment-17220684
]
Hudson commented on HBASE-25204:
Results for branch branch-1.4
[build #14 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1.4/14/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1.4/14//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1.4/14//JDK7_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1.4/14//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
> Nightly job failed as the name of jdk and maven changed
>
>
> Key: HBASE-25204
> URL: https://issues.apache.org/jira/browse/HBASE-25204
> Project: HBase
> Issue Type: Bug
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.3, 1.7.0, 2.4.0, 1.4.14, 2.2.7
>
>
> See
> [https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/85/console]
> [https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/103/console]
>
> org.codehaus.groovy.control.MultipleCompilationErrorsException: startup
> failed: WorkflowScript: 508: Tool type "maven" does not have an install of
> "Maven (latest)" configured - did you mean "maven_latest"? @ line 508, column
> 19. maven 'Maven (latest)' ^ WorkflowScript: 510: Tool type "jdk" does not
> have an install of "JDK 1.8 (latest)" configured - did you mean
> "jdk_1.8_latest"? @ line 510, column 17. jdk "JDK 1.8 (latest)"
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-25219) Backport HBASE-24369 "Provide more information about merged child regions in Hbck Overlaps section, which cannot be fixed immediately" to branch-2.2
Peter Somogyi created HBASE-25219: - Summary: Backport HBASE-24369 "Provide more information about merged child regions in Hbck Overlaps section, which cannot be fixed immediately" to branch-2.2 Key: HBASE-25219 URL: https://issues.apache.org/jira/browse/HBASE-25219 Project: HBase Issue Type: Sub-task Reporter: Peter Somogyi Assignee: Peter Somogyi -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25195) [branch-1] getNumOpenConnections is not effective
[
https://issues.apache.org/jira/browse/HBASE-25195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17220672#comment-17220672
]
Hudson commented on HBASE-25195:
Results for branch branch-1
[build #47 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/47/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/47//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/47//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/47//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
> [branch-1] getNumOpenConnections is not effective
> -
>
> Key: HBASE-25195
> URL: https://issues.apache.org/jira/browse/HBASE-25195
> Project: HBase
> Issue Type: Improvement
> Components: metrics
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Minor
> Fix For: 1.7.0
>
>
> In MetricsHBaseServerWrapperImpl
> {code}
> @Override
> public int getNumOpenConnections() {
> if (!isServerStarted() || this.server.connectionList == null) {
> return 0;
> }
> return server.connectionList.size();
> }
> {code}
> But connectionList is created by Collections.synchronizedList(new
> LinkedList());
> So each connectionList.size() will compete for the mutex, we can easily
> replace it with another parameter *numConnections*, as for metrics, it is
> enough.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25194) Do not publish workspace in flaky find job
[
https://issues.apache.org/jira/browse/HBASE-25194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17220673#comment-17220673
]
Hudson commented on HBASE-25194:
Results for branch branch-1
[build #47 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/47/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/47//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/47//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/47//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
> Do not publish workspace in flaky find job
> --
>
> Key: HBASE-25194
> URL: https://issues.apache.org/jira/browse/HBASE-25194
> Project: HBase
> Issue Type: Sub-task
> Components: jenkins
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.3, 1.7.0, 2.4.0, 1.4.14, 2.2.7
>
>
> As said in the parent issue.
> And I tried to use the publishHTML target on jenkins job configuration page,
> the help message for reportDir is
> {noformat}
> The path to the HTML report directory relative to the workspace.
> {noformat}
> For the reportFiles is
> {noformat}
> The file(s) to provide links inside the report directory
> {noformat}
> I think this clearly means that the plugin will publish all the files under
> reportDir, the reportFiles is just the index page.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25193) Add support for row prefix and type in the WAL Pretty Printer and some minor fixes
[
https://issues.apache.org/jira/browse/HBASE-25193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17220676#comment-17220676
]
Hudson commented on HBASE-25193:
Results for branch branch-1
[build #47 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/47/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/47//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/47//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/47//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
> Add support for row prefix and type in the WAL Pretty Printer and some minor
> fixes
> --
>
> Key: HBASE-25193
> URL: https://issues.apache.org/jira/browse/HBASE-25193
> Project: HBase
> Issue Type: Improvement
> Components: wal
>Reporter: Sandeep Pal
>Assignee: Sandeep Pal
>Priority: Minor
> Fix For: 3.0.0-alpha-1, 2.3.3, 1.7.0, 2.4.0
>
>
> Currently, the WAL Pretty Printer has an option to filter the keys with an
> exact match of row. However, it is super useful sometimes to have a row key
> prefix instead of an exact match.
> The prefix can act as a full match filter as well due to the nature of the
> prefix.
> Secondly, we are not having the cell type in the WAL Pretty Printer in any of
> the branches.
> Lastly, the option rowkey only options prints additional stuff as well.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25203) Change the reference url to flaky list in our jenkins jobs
[
https://issues.apache.org/jira/browse/HBASE-25203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17220674#comment-17220674
]
Hudson commented on HBASE-25203:
Results for branch branch-1
[build #47 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/47/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/47//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/47//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/47//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
> Change the reference url to flaky list in our jenkins jobs
> --
>
> Key: HBASE-25203
> URL: https://issues.apache.org/jira/browse/HBASE-25203
> Project: HBase
> Issue Type: Sub-task
> Components: flakies, jenkins
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.3, 1.7.0, 2.4.0, 1.4.14, 2.2.7
>
>
> Nightly
> Flaky tests
> PreCommit
> personality
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25204) Nightly job failed as the name of jdk and maven changed
[
https://issues.apache.org/jira/browse/HBASE-25204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17220675#comment-17220675
]
Hudson commented on HBASE-25204:
Results for branch branch-1
[build #47 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/47/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/47//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/47//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/47//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
> Nightly job failed as the name of jdk and maven changed
>
>
> Key: HBASE-25204
> URL: https://issues.apache.org/jira/browse/HBASE-25204
> Project: HBase
> Issue Type: Bug
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.3, 1.7.0, 2.4.0, 1.4.14, 2.2.7
>
>
> See
> [https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/85/console]
> [https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/103/console]
>
> org.codehaus.groovy.control.MultipleCompilationErrorsException: startup
> failed: WorkflowScript: 508: Tool type "maven" does not have an install of
> "Maven (latest)" configured - did you mean "maven_latest"? @ line 508, column
> 19. maven 'Maven (latest)' ^ WorkflowScript: 510: Tool type "jdk" does not
> have an install of "JDK 1.8 (latest)" configured - did you mean
> "jdk_1.8_latest"? @ line 510, column 17. jdk "JDK 1.8 (latest)"
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25189) [Metrics] Add checkAndPut and checkAndDelete latency metrics at table level
[
https://issues.apache.org/jira/browse/HBASE-25189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17220677#comment-17220677
]
Hudson commented on HBASE-25189:
Results for branch branch-1
[build #47 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/47/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/47//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/47//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/47//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
> [Metrics] Add checkAndPut and checkAndDelete latency metrics at table level
> ---
>
> Key: HBASE-25189
> URL: https://issues.apache.org/jira/browse/HBASE-25189
> Project: HBase
> Issue Type: Improvement
> Components: metrics
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Minor
> Fix For: 3.0.0-alpha-1, 2.3.3, 1.7.0, 2.2.7
>
>
> Add *checkAndPut* and *checkAndDelete* table latency metrics.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2579: HBASE-24999 Master manages ReplicationServers
Apache-HBase commented on pull request #2579: URL: https://github.com/apache/hbase/pull/2579#issuecomment-716533757 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 28s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ HBASE-24666 Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 26s | HBASE-24666 passed | | +1 :green_heart: | compile | 1m 42s | HBASE-24666 passed | | +1 :green_heart: | shadedjars | 6m 35s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 50s | HBASE-24666 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 25s | the patch passed | | +1 :green_heart: | compile | 1m 45s | the patch passed | | +1 :green_heart: | javac | 1m 45s | the patch passed | | +1 :green_heart: | shadedjars | 6m 34s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 52s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 45s | hbase-protocol-shaded in the patch passed. | | +1 :green_heart: | unit | 141m 42s | hbase-server in the patch passed. | | | | 171m 1s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2579/4/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2579 | | Optional Tests | unit javac javadoc shadedjars compile | | uname | Linux 857effd163cc 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | HBASE-24666 / 1a64f1a0db | | Default Java | 1.8.0_232 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2579/4/testReport/ | | Max. process+thread count | 4564 (vs. ulimit of 3) | | modules | C: hbase-protocol-shaded hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2579/4/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2587: HBASE-25213 Should request Compaction after bulkLoadHFiles is done
Apache-HBase commented on pull request #2587: URL: https://github.com/apache/hbase/pull/2587#issuecomment-716478119 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 28s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 9s | master passed | | +1 :green_heart: | compile | 0m 58s | master passed | | +1 :green_heart: | shadedjars | 7m 6s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 40s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 48s | the patch passed | | +1 :green_heart: | compile | 0m 57s | the patch passed | | +1 :green_heart: | javac | 0m 57s | the patch passed | | +1 :green_heart: | shadedjars | 7m 7s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 35s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 206m 20s | hbase-server in the patch passed. | | | | 236m 0s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2587/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2587 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 653d9f122c90 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 164cc5a3dc | | Default Java | 1.8.0_232 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2587/1/testReport/ | | Max. process+thread count | 3638 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2587/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2579: HBASE-24999 Master manages ReplicationServers
Apache-HBase commented on pull request #2579: URL: https://github.com/apache/hbase/pull/2579#issuecomment-716476519 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 28s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | prototool | 0m 0s | prototool was not available. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ HBASE-24666 Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 40s | HBASE-24666 passed | | +1 :green_heart: | checkstyle | 1m 18s | HBASE-24666 passed | | -1 :x: | spotbugs | 1m 55s | hbase-server in HBASE-24666 has 3 extant spotbugs warnings. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 22s | the patch passed | | +1 :green_heart: | checkstyle | 0m 11s | The patch passed checkstyle in hbase-protocol-shaded | | +1 :green_heart: | checkstyle | 1m 7s | hbase-server: The patch generated 0 new + 147 unchanged - 3 fixed = 147 total (was 150) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 17m 24s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | hbaseprotoc | 1m 40s | the patch passed | | +1 :green_heart: | spotbugs | 3m 40s | hbase-protocol-shaded in the patch passed. | | +1 :green_heart: | spotbugs | 2m 10s | hbase-server generated 0 new + 0 unchanged - 3 fixed = 0 total (was 3) | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 26s | The patch does not generate ASF License warnings. | | | | 49m 13s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2579/4/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2579 | | Optional Tests | dupname asflicense cc hbaseprotoc prototool spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux d554b84d17b2 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | HBASE-24666 / 1a64f1a0db | | spotbugs | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2579/4/artifact/yetus-general-check/output/branch-spotbugs-hbase-server-warnings.html | | Max. process+thread count | 94 (vs. ulimit of 3) | | modules | C: hbase-protocol-shaded hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2579/4/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2585: HBASE-25217 [Metrics] Add metrics for Call in IPC response queue
Apache-HBase commented on pull request #2585: URL: https://github.com/apache/hbase/pull/2585#issuecomment-716474064 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 34s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 1s | The patch does not contain any @author tags. | | -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ branch-1 Compile Tests _ | | +0 :ok: | mvndep | 2m 24s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 8m 1s | branch-1 passed | | +1 :green_heart: | compile | 1m 13s | branch-1 passed with JDK v1.8.0_262 | | +1 :green_heart: | compile | 1m 17s | branch-1 passed with JDK v1.7.0_272 | | +1 :green_heart: | checkstyle | 2m 12s | branch-1 passed | | +1 :green_heart: | shadedjars | 3m 4s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 8s | branch-1 passed with JDK v1.8.0_262 | | +1 :green_heart: | javadoc | 1m 17s | branch-1 passed with JDK v1.7.0_272 | | +0 :ok: | spotbugs | 2m 49s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 4m 13s | branch-1 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 19s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 17s | the patch passed | | +1 :green_heart: | compile | 1m 13s | the patch passed with JDK v1.8.0_262 | | +1 :green_heart: | javac | 1m 13s | the patch passed | | +1 :green_heart: | compile | 1m 21s | the patch passed with JDK v1.7.0_272 | | +1 :green_heart: | javac | 1m 21s | the patch passed | | +1 :green_heart: | checkstyle | 0m 15s | The patch passed checkstyle in hbase-hadoop-compat | | +1 :green_heart: | checkstyle | 0m 15s | hbase-hadoop2-compat: The patch generated 0 new + 2 unchanged - 1 fixed = 2 total (was 3) | | +1 :green_heart: | checkstyle | 1m 33s | The patch passed checkstyle in hbase-server | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 2m 56s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 4m 34s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. | | +1 :green_heart: | javadoc | 0m 59s | the patch passed with JDK v1.8.0_262 | | +1 :green_heart: | javadoc | 1m 12s | the patch passed with JDK v1.7.0_272 | | +1 :green_heart: | findbugs | 4m 25s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 27s | hbase-hadoop-compat in the patch passed. | | +1 :green_heart: | unit | 0m 38s | hbase-hadoop2-compat in the patch passed. | | +1 :green_heart: | unit | 114m 17s | hbase-server in the patch passed. | | +1 :green_heart: | asflicense | 1m 12s | The patch does not generate ASF License warnings. | | | | 167m 32s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2585/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2585 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 43dcdd0efbae 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-agent/workspace/Base-PreCommit-GitHub-PR_PR-2585/out/precommit/personality/provided.sh | | git revision | branch-1 / b30d1d1 | | Default Java | 1.7.0_272 | | Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:1.8.0_262 /usr/lib/jvm/zulu-7-amd64:1.7.0_272 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2585/2/testReport/ | | Max. process+thread count | 3676 (vs. ulimit of 1) | | modules | C: hbase-hadoop-compat hbase-hadoop2-compat hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2585/2/console | | versions | git=1.9.1 maven=3.0.5 findbugs=3.0.1 | | Powered by | Apache Yetus 0.11.1
[GitHub] [hbase] Apache-HBase commented on pull request #2445: HBASE-11288 Splittable Meta (WIP) Draft
Apache-HBase commented on pull request #2445: URL: https://github.com/apache/hbase/pull/2445#issuecomment-716471729 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 6m 30s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +0 :ok: | prototool | 0m 1s | prototool was not available. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 20s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 59s | master passed | | +1 :green_heart: | checkstyle | 2m 32s | master passed | | +1 :green_heart: | spotbugs | 19m 4s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 48s | the patch passed | | -0 :warning: | checkstyle | 2m 36s | root: The patch generated 110 new + 962 unchanged - 5 fixed = 1072 total (was 967) | | -0 :warning: | whitespace | 0m 0s | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply | | +1 :green_heart: | hadoopcheck | 18m 50s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | hbaseprotoc | 8m 6s | the patch passed | | -1 :x: | spotbugs | 1m 21s | hbase-client generated 3 new + 0 unchanged - 0 fixed = 3 total (was 0) | | -1 :x: | spotbugs | 2m 38s | hbase-server generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | | -1 :x: | spotbugs | 11m 10s | root generated 4 new + 0 unchanged - 0 fixed = 4 total (was 0) | ||| _ Other Tests _ | | -1 :x: | asflicense | 1m 25s | The patch generated 1 ASF License warnings. | | | | 100m 10s | | | Reason | Tests | |---:|:--| | FindBugs | module:hbase-client | | | Nullcheck of other at line 833 of value previously dereferenced in org.apache.hadoop.hbase.client.RegionInfo.isOverlap(RegionInfo) At RegionInfo.java:833 of value previously dereferenced in org.apache.hadoop.hbase.client.RegionInfo.isOverlap(RegionInfo) At RegionInfo.java:[line 832] | | | Nullcheck of rhs at line 113 of value previously dereferenced in org.apache.hadoop.hbase.client.RegionInfo.lambda$static$0(RegionInfo, RegionInfo) At RegionInfo.java:113 of value previously dereferenced in org.apache.hadoop.hbase.client.RegionInfo.lambda$static$0(RegionInfo, RegionInfo) At RegionInfo.java:[line 112] | | | Nullcheck of regionA at line 448 of value previously dereferenced in org.apache.hadoop.hbase.client.RegionInfo.areAdjacent(RegionInfo, RegionInfo) At RegionInfo.java:448 of value previously dereferenced in org.apache.hadoop.hbase.client.RegionInfo.areAdjacent(RegionInfo, RegionInfo) At RegionInfo.java:[line 447] | | FindBugs | module:hbase-server | | | Possible null pointer dereference of region in org.apache.hadoop.hbase.regionserver.handler.UnassignRegionHandler.create(HRegionServer, String, long, boolean, ServerName) Dereferenced at UnassignRegionHandler.java:region in org.apache.hadoop.hbase.regionserver.handler.UnassignRegionHandler.create(HRegionServer, String, long, boolean, ServerName) Dereferenced at UnassignRegionHandler.java:[line 149] | | FindBugs | module:root | | | Nullcheck of other at line 833 of value previously dereferenced in org.apache.hadoop.hbase.client.RegionInfo.isOverlap(RegionInfo) At RegionInfo.java:833 of value previously dereferenced in org.apache.hadoop.hbase.client.RegionInfo.isOverlap(RegionInfo) At RegionInfo.java:[line 832] | | | Nullcheck of rhs at line 113 of value previously dereferenced in org.apache.hadoop.hbase.client.RegionInfo.lambda$static$0(RegionInfo, RegionInfo) At RegionInfo.java:113 of value previously dereferenced in org.apache.hadoop.hbase.client.RegionInfo.lambda$static$0(RegionInfo, RegionInfo) At RegionInfo.java:[line 112] | | | Nullcheck of regionA at line 448 of value previously dereferenced in org.apache.hadoop.hbase.client.RegionInfo.areAdjacent(RegionInfo, RegionInfo) At RegionInfo.java:448 of value previously dereferenced in org.apache.hadoop.hbase.client.RegionInfo.areAdjacent(RegionInfo, RegionInfo) At RegionInfo.java:[line 447] | | | Possible null pointer dereference of region in org.apache.hadoop.hbase.regionserver.handler.UnassignRegionHandler.create(HRegionServer, String, long, boolean, ServerName) Dereferenced at UnassignRegionHandler.java:region in
[GitHub] [hbase] ddupg commented on pull request #2579: HBASE-24999 Master manages ReplicationServers
ddupg commented on pull request #2579: URL: https://github.com/apache/hbase/pull/2579#issuecomment-716453008 Thanks @infraio for reviewing. I have fixed as comments. > Add master UI to show ReplicationServer? Do it in this PR? I was going to do it in a new PR? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ddupg commented on a change in pull request #2579: HBASE-24999 Master manages ReplicationServers
ddupg commented on a change in pull request #2579:
URL: https://github.com/apache/hbase/pull/2579#discussion_r511852023
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/HBaseReplicationEndpoint.java
##
@@ -445,7 +418,7 @@ public PeerRegionServerListener(HBaseReplicationEndpoint
endpoint) {
@Override
public synchronized void nodeChildrenChanged(String path) {
- if (path.equals(regionServerListNode)) {
+ if (replicationEndpoint.fetchServersUseZk &&
path.equals(regionServerListNode)) {
Review comment:
OK
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2445: HBASE-11288 Splittable Meta (WIP) Draft
Apache-HBase commented on pull request #2445: URL: https://github.com/apache/hbase/pull/2445#issuecomment-716450300 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 12s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 1s | master passed | | +1 :green_heart: | compile | 2m 35s | master passed | | +1 :green_heart: | shadedjars | 7m 11s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 4m 19s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 45s | the patch passed | | +1 :green_heart: | compile | 2m 37s | the patch passed | | +1 :green_heart: | javac | 2m 37s | the patch passed | | -1 :x: | shadedjars | 5m 38s | patch has 10 errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 19s | hbase-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | | -0 :warning: | javadoc | 2m 11s | root generated 1 new + 32 unchanged - 0 fixed = 33 total (was 32) | ||| _ Other Tests _ | | -1 :x: | unit | 21m 23s | root in the patch failed. | | | | 60m 12s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2445/6/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2445 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 538973ea2b1b 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 164cc5a3dc | | Default Java | 1.8.0_232 | | shadedjars | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2445/6/artifact/yetus-jdk8-hadoop3-check/output/patch-shadedjars.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2445/6/artifact/yetus-jdk8-hadoop3-check/output/diff-javadoc-javadoc-hbase-common.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2445/6/artifact/yetus-jdk8-hadoop3-check/output/diff-javadoc-javadoc-root.txt | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2445/6/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-root.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2445/6/testReport/ | | Max. process+thread count | 908 (vs. ulimit of 3) | | modules | C: hbase-protocol-shaded hbase-common hbase-client hbase-zookeeper hbase-balancer hbase-server hbase-backup . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2445/6/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2445: HBASE-11288 Splittable Meta (WIP) Draft
Apache-HBase commented on pull request #2445: URL: https://github.com/apache/hbase/pull/2445#issuecomment-716446598 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 28s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 21s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 14s | master passed | | +1 :green_heart: | compile | 2m 47s | master passed | | +1 :green_heart: | shadedjars | 6m 38s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 5m 13s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 56s | the patch passed | | +1 :green_heart: | compile | 2m 48s | the patch passed | | +1 :green_heart: | javac | 2m 48s | the patch passed | | -1 :x: | shadedjars | 5m 26s | patch has 10 errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 17s | hbase-balancer generated 3 new + 1 unchanged - 1 fixed = 4 total (was 2) | | -0 :warning: | javadoc | 2m 42s | root generated 3 new + 173 unchanged - 1 fixed = 176 total (was 174) | ||| _ Other Tests _ | | -1 :x: | unit | 12m 20s | root in the patch failed. | | | | 52m 55s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2445/6/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2445 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 62696b6b0efa 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 164cc5a3dc | | Default Java | 2020-01-14 | | shadedjars | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2445/6/artifact/yetus-jdk11-hadoop3-check/output/patch-shadedjars.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2445/6/artifact/yetus-jdk11-hadoop3-check/output/diff-javadoc-javadoc-hbase-balancer.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2445/6/artifact/yetus-jdk11-hadoop3-check/output/diff-javadoc-javadoc-root.txt | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2445/6/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-root.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2445/6/testReport/ | | Max. process+thread count | 1228 (vs. ulimit of 3) | | modules | C: hbase-protocol-shaded hbase-common hbase-client hbase-zookeeper hbase-balancer hbase-server hbase-backup . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2445/6/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2587: HBASE-25213 Should request Compaction after bulkLoadHFiles is done
Apache-HBase commented on pull request #2587: URL: https://github.com/apache/hbase/pull/2587#issuecomment-716441418 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 2s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 20s | master passed | | +1 :green_heart: | compile | 1m 11s | master passed | | +1 :green_heart: | shadedjars | 6m 44s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 44s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 8s | the patch passed | | +1 :green_heart: | compile | 1m 4s | the patch passed | | +1 :green_heart: | javac | 1m 4s | the patch passed | | +1 :green_heart: | shadedjars | 6m 37s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 39s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 138m 48s | hbase-server in the patch failed. | | | | 167m 28s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2587/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2587 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 4c5ca3f0f62f 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 164cc5a3dc | | Default Java | 2020-01-14 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2587/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2587/1/testReport/ | | Max. process+thread count | 3903 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2587/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2579: HBASE-24999 Master manages ReplicationServers
Apache-HBase commented on pull request #2579: URL: https://github.com/apache/hbase/pull/2579#issuecomment-716382359 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 13m 29s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ HBASE-24666 Compile Tests _ | | +0 :ok: | mvndep | 0m 29s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 52s | HBASE-24666 passed | | +1 :green_heart: | compile | 2m 41s | HBASE-24666 passed | | +1 :green_heart: | shadedjars | 8m 52s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 11s | HBASE-24666 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 19s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 24s | the patch passed | | +1 :green_heart: | compile | 2m 20s | the patch passed | | +1 :green_heart: | javac | 2m 20s | the patch passed | | +1 :green_heart: | shadedjars | 9m 48s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 4s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 8s | hbase-protocol-shaded in the patch passed. | | +1 :green_heart: | unit | 198m 59s | hbase-server in the patch passed. | | | | 253m 42s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2579/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2579 | | Optional Tests | unit javac javadoc shadedjars compile | | uname | Linux a7955df059b1 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | HBASE-24666 / 1a64f1a0db | | Default Java | 2020-01-14 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2579/3/testReport/ | | Max. process+thread count | 3707 (vs. ulimit of 3) | | modules | C: hbase-protocol-shaded hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2579/3/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2587: HBASE-25213 Should request Compaction after bulkLoadHFiles is done
Apache-HBase commented on pull request #2587: URL: https://github.com/apache/hbase/pull/2587#issuecomment-716381049 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 39s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 8s | master passed | | +1 :green_heart: | checkstyle | 1m 10s | master passed | | +1 :green_heart: | spotbugs | 2m 33s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 46s | the patch passed | | +1 :green_heart: | checkstyle | 1m 19s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 23m 11s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 2m 43s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 16s | The patch does not generate ASF License warnings. | | | | 49m 45s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2587/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2587 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 05a7c01e0a24 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 164cc5a3dc | | Max. process+thread count | 95 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2587/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-24972) Wait for connection attempt to succeed before performing operations on ZK
[
https://issues.apache.org/jira/browse/HBASE-24972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17220549#comment-17220549
]
Prathyusha commented on HBASE-24972:
[~sandeep.guggilam] If you have not already started working on this, can I pick
this one?
> Wait for connection attempt to succeed before performing operations on ZK
> -
>
> Key: HBASE-24972
> URL: https://issues.apache.org/jira/browse/HBASE-24972
> Project: HBase
> Issue Type: Bug
>Reporter: Sandeep Guggilam
>Priority: Minor
>
> {color:#1d1c1d}Creating the connection with ZK is asynchronous and notified
> via the passed in watcher about the successful connection event. When we
> attempt any operations, we try to create a connection and then perform a
> read/write
> ({color}{color:#1d1c1d}[https://github.com/apache/hbase/blob/979edfe72046b2075adcc869c65ae820e6f3ec2d/hbase-zookeeper/src/main/java/org/apache/hadoop/hbase/zookeeper/RecoverableZooKeeper.java#L323]{color}{color:#1d1c1d})
> without really waiting for the notification event
> ([https://github.com/apache/hbase/blob/979edfe72046b2075adcc869c65ae820e6f3ec2d/hbase-zookeeper/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKWatcher.java#L582)]{color}
>
> {color:#1d1c1d}It is possible we get ConnectionLoss errors when we perform
> operations on ZK without waiting for the connection attempt to succeed{color}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
