[hbase] 02/02: HBASE-26147 Add a dry run mode to the balancer, where moves are calculated but not actually executed
This is an automated email from the ASF dual-hosted git repository.
elserj pushed a commit to branch branch-2
in repository https://gitbox.apache.org/repos/asf/hbase.git
commit e07928df4cf2ef7564808bc988a289ef336af03a
Author: Bryan Beaudreault
AuthorDate: Wed Sep 1 21:55:40 2021 -0400
HBASE-26147 Add a dry run mode to the balancer, where moves are calculated
but not actually executed
Signed-off-by: Duo Zhang
Signed-off-by: Josh Elser
(Re-application of HBASE-26147 with the correct author metadata)
---
.../java/org/apache/hadoop/hbase/client/Admin.java | 28 -
.../org/apache/hadoop/hbase/client/AsyncAdmin.java | 22 +++-
.../hadoop/hbase/client/AsyncHBaseAdmin.java | 4 +-
.../apache/hadoop/hbase/client/BalanceRequest.java | 114 +++
.../hadoop/hbase/client/BalanceResponse.java | 126 +
.../org/apache/hadoop/hbase/client/HBaseAdmin.java | 69 +--
.../hadoop/hbase/client/RawAsyncHBaseAdmin.java| 14 +--
.../hadoop/hbase/shaded/protobuf/ProtobufUtil.java | 32 ++
.../hbase/shaded/protobuf/RequestConverter.java| 10 --
.../src/main/protobuf/Master.proto | 5 +-
.../apache/hadoop/hbase/rsgroup/RSGroupAdmin.java | 14 ++-
.../hadoop/hbase/rsgroup/RSGroupAdminClient.java | 12 +-
.../hadoop/hbase/rsgroup/RSGroupAdminEndpoint.java | 21 +++-
.../hadoop/hbase/rsgroup/RSGroupAdminServer.java | 29 +++--
.../hadoop/hbase/rsgroup/RSGroupProtobufUtil.java | 35 +-
hbase-rsgroup/src/main/protobuf/RSGroupAdmin.proto | 4 +
.../hadoop/hbase/rsgroup/TestRSGroupsBalance.java | 90 ++-
.../hadoop/hbase/rsgroup/TestRSGroupsBase.java | 6 +-
.../hadoop/hbase/rsgroup/TestRSGroupsFallback.java | 4 +-
.../hbase/rsgroup/VerifyingRSGroupAdminClient.java | 6 +-
.../hadoop/hbase/coprocessor/MasterObserver.java | 16 ++-
.../org/apache/hadoop/hbase/master/HMaster.java| 54 +
.../hadoop/hbase/master/MasterCoprocessorHost.java | 18 +--
.../hadoop/hbase/master/MasterRpcServices.java | 3 +-
.../hbase/security/access/AccessController.java| 3 +-
.../apache/hadoop/hbase/TestRegionRebalancing.java | 27 -
.../client/TestAsyncTableGetMultiThreaded.java | 2 +-
.../hadoop/hbase/client/TestMultiParallel.java | 2 +-
.../hbase/client/TestSeparateClientZKCluster.java | 4 +-
.../hbase/coprocessor/TestMasterObserver.java | 13 ++-
.../hbase/master/TestMasterDryRunBalancer.java | 126 +
.../master/procedure/TestProcedurePriority.java| 3 +-
.../security/access/TestAccessController.java | 3 +-
.../access/TestWithDisabledAuthorization.java | 3 +-
hbase-shell/src/main/ruby/hbase/admin.rb | 7 +-
hbase-shell/src/main/ruby/hbase/balancer_utils.rb | 57 ++
hbase-shell/src/main/ruby/hbase/rsgroup_admin.rb | 7 +-
.../main/ruby/shell/commands/balance_rsgroup.rb| 22 +++-
.../src/main/ruby/shell/commands/balancer.rb | 33 +++---
hbase-shell/src/test/ruby/hbase/admin_test.rb | 8 +-
.../src/test/ruby/hbase/balancer_utils_test.rb | 78 +
.../src/test/ruby/shell/rsgroup_shell_test.rb | 2 +
.../hadoop/hbase/thrift2/client/ThriftAdmin.java | 7 ++
43 files changed, 943 insertions(+), 200 deletions(-)
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Admin.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Admin.java
index fb61612..a3a5107 100644
--- a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Admin.java
+++ b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Admin.java
@@ -1251,7 +1251,20 @@ public interface Admin extends Abortable, Closeable {
* @return true if balancer ran, false otherwise.
* @throws IOException if a remote or network exception occurs
*/
- boolean balance() throws IOException;
+ default boolean balance() throws IOException {
+return balance(BalanceRequest.defaultInstance())
+ .isBalancerRan();
+ }
+
+ /**
+ * Invoke the balancer with the given balance request. The BalanceRequest
defines how the
+ * balancer will run. See {@link BalanceRequest} for more details.
+ *
+ * @param request defines how the balancer should run
+ * @return {@link BalanceResponse} with details about the results of the
invocation.
+ * @throws IOException if a remote or network exception occurs
+ */
+ BalanceResponse balance(BalanceRequest request) throws IOException;
/**
* Invoke the balancer. Will run the balancer and if regions to move, it
will
@@ -1262,7 +1275,7 @@ public interface Admin extends Abortable, Closeable {
* @return true if balancer ran, false otherwise.
* @throws IOException if a remote or network exception occurs
* @deprecated Since 2.0.0. Will be removed in 3.0.0.
- * Use {@link #balance(boolean)} instead.
+ * Use {@link #balance(BalanceRequest)} instead.
*/
@Deprecated
[hbase] 01/02: Revert "HBASE-26147 Add a dry run mode to the balancer, where moves are calculated but not actually executed"
This is an automated email from the ASF dual-hosted git repository.
elserj pushed a commit to branch branch-2
in repository https://gitbox.apache.org/repos/asf/hbase.git
commit 1e74f084035850130a71c7e7b7cad3194883078b
Author: Josh Elser
AuthorDate: Wed Sep 1 21:55:13 2021 -0400
Revert "HBASE-26147 Add a dry run mode to the balancer, where moves are
calculated but not actually executed"
This reverts commit 5b58d11c91cbbc2f73e4252088c97df12a49ff87.
I accidentally applied this as myself instead of as Bryan. Reverting it
to apply the patch properly.
---
.../java/org/apache/hadoop/hbase/client/Admin.java | 28 +
.../org/apache/hadoop/hbase/client/AsyncAdmin.java | 22 +---
.../hadoop/hbase/client/AsyncHBaseAdmin.java | 4 +-
.../apache/hadoop/hbase/client/BalanceRequest.java | 114 ---
.../hadoop/hbase/client/BalanceResponse.java | 126 -
.../org/apache/hadoop/hbase/client/HBaseAdmin.java | 69 ++-
.../hadoop/hbase/client/RawAsyncHBaseAdmin.java| 14 +--
.../hadoop/hbase/shaded/protobuf/ProtobufUtil.java | 32 --
.../hbase/shaded/protobuf/RequestConverter.java| 10 ++
.../src/main/protobuf/Master.proto | 5 +-
.../apache/hadoop/hbase/rsgroup/RSGroupAdmin.java | 14 +--
.../hadoop/hbase/rsgroup/RSGroupAdminClient.java | 12 +-
.../hadoop/hbase/rsgroup/RSGroupAdminEndpoint.java | 21 +---
.../hadoop/hbase/rsgroup/RSGroupAdminServer.java | 29 ++---
.../hadoop/hbase/rsgroup/RSGroupProtobufUtil.java | 35 +-
hbase-rsgroup/src/main/protobuf/RSGroupAdmin.proto | 4 -
.../hadoop/hbase/rsgroup/TestRSGroupsBalance.java | 90 +--
.../hadoop/hbase/rsgroup/TestRSGroupsBase.java | 6 +-
.../hadoop/hbase/rsgroup/TestRSGroupsFallback.java | 4 +-
.../hbase/rsgroup/VerifyingRSGroupAdminClient.java | 6 +-
.../hadoop/hbase/coprocessor/MasterObserver.java | 16 +--
.../org/apache/hadoop/hbase/master/HMaster.java| 54 -
.../hadoop/hbase/master/MasterCoprocessorHost.java | 18 ++-
.../hadoop/hbase/master/MasterRpcServices.java | 3 +-
.../hbase/security/access/AccessController.java| 3 +-
.../apache/hadoop/hbase/TestRegionRebalancing.java | 27 +
.../client/TestAsyncTableGetMultiThreaded.java | 2 +-
.../hadoop/hbase/client/TestMultiParallel.java | 2 +-
.../hbase/client/TestSeparateClientZKCluster.java | 4 +-
.../hbase/coprocessor/TestMasterObserver.java | 13 +--
.../hbase/master/TestMasterDryRunBalancer.java | 126 -
.../master/procedure/TestProcedurePriority.java| 3 +-
.../security/access/TestAccessController.java | 3 +-
.../access/TestWithDisabledAuthorization.java | 3 +-
hbase-shell/src/main/ruby/hbase/admin.rb | 7 +-
hbase-shell/src/main/ruby/hbase/balancer_utils.rb | 57 --
hbase-shell/src/main/ruby/hbase/rsgroup_admin.rb | 7 +-
.../main/ruby/shell/commands/balance_rsgroup.rb| 22 +---
.../src/main/ruby/shell/commands/balancer.rb | 33 +++---
hbase-shell/src/test/ruby/hbase/admin_test.rb | 8 +-
.../src/test/ruby/hbase/balancer_utils_test.rb | 78 -
.../src/test/ruby/shell/rsgroup_shell_test.rb | 2 -
.../hadoop/hbase/thrift2/client/ThriftAdmin.java | 7 --
43 files changed, 200 insertions(+), 943 deletions(-)
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Admin.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Admin.java
index a3a5107..fb61612 100644
--- a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Admin.java
+++ b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Admin.java
@@ -1251,20 +1251,7 @@ public interface Admin extends Abortable, Closeable {
* @return true if balancer ran, false otherwise.
* @throws IOException if a remote or network exception occurs
*/
- default boolean balance() throws IOException {
-return balance(BalanceRequest.defaultInstance())
- .isBalancerRan();
- }
-
- /**
- * Invoke the balancer with the given balance request. The BalanceRequest
defines how the
- * balancer will run. See {@link BalanceRequest} for more details.
- *
- * @param request defines how the balancer should run
- * @return {@link BalanceResponse} with details about the results of the
invocation.
- * @throws IOException if a remote or network exception occurs
- */
- BalanceResponse balance(BalanceRequest request) throws IOException;
+ boolean balance() throws IOException;
/**
* Invoke the balancer. Will run the balancer and if regions to move, it
will
@@ -1275,7 +1262,7 @@ public interface Admin extends Abortable, Closeable {
* @return true if balancer ran, false otherwise.
* @throws IOException if a remote or network exception occurs
* @deprecated Since 2.0.0. Will be removed in 3.0.0.
- * Use {@link #balance(BalanceRequest)} instead.
+ * Use {@link
[hbase] branch branch-2 updated (5b58d11 -> e07928d)
This is an automated email from the ASF dual-hosted git repository. elserj pushed a change to branch branch-2 in repository https://gitbox.apache.org/repos/asf/hbase.git. from 5b58d11 HBASE-26147 Add a dry run mode to the balancer, where moves are calculated but not actually executed new 1e74f08 Revert "HBASE-26147 Add a dry run mode to the balancer, where moves are calculated but not actually executed" new e07928d HBASE-26147 Add a dry run mode to the balancer, where moves are calculated but not actually executed The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes:
[hbase] branch master updated: HBASE-26147 Add a dry run mode to the balancer, where moves are calculated but not actually executed
This is an automated email from the ASF dual-hosted git repository.
elserj pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/master by this push:
new a15e94a HBASE-26147 Add a dry run mode to the balancer, where moves
are calculated but not actually executed
a15e94a is described below
commit a15e94a47f06ae665a0c213d092761431e907eac
Author: Bryan Beaudreault
AuthorDate: Tue Jul 27 17:25:59 2021 -0400
HBASE-26147 Add a dry run mode to the balancer, where moves are calculated
but not actually executed
Closes #3630
Signed-off-by: Duo Zhang
Signed-off-by: Josh Elser true if balancer ran, false otherwise.
* @throws IOException if a remote or network exception occurs
*/
- boolean balance() throws IOException;
+ default boolean balance() throws IOException {
+return balance(BalanceRequest.defaultInstance())
+ .isBalancerRan();
+ }
+
+ /**
+ * Invoke the balancer with the given balance request. The BalanceRequest
defines how the
+ * balancer will run. See {@link BalanceRequest} for more details.
+ *
+ * @param request defines how the balancer should run
+ * @return {@link BalanceResponse} with details about the results of the
invocation.
+ * @throws IOException if a remote or network exception occurs
+ */
+ BalanceResponse balance(BalanceRequest request) throws IOException;
/**
* Invoke the balancer. Will run the balancer and if regions to move, it
will
@@ -841,8 +854,17 @@ public interface Admin extends Abortable, Closeable {
* @param force whether we should force balance even if there is region in
transition
* @return true if balancer ran, false otherwise.
* @throws IOException if a remote or network exception occurs
+ * @deprecated Since 2.5.0. Will be removed in 4.0.0.
+ * Use {@link #balance(BalanceRequest)} instead.
*/
- boolean balance(boolean force) throws IOException;
+ @Deprecated
+ default boolean balance(boolean force) throws IOException {
+return balance(
+ BalanceRequest.newBuilder()
+ .setIgnoreRegionsInTransition(force)
+ .build()
+).isBalancerRan();
+ }
/**
* Query the current state of the balancer.
@@ -2494,10 +2516,20 @@ public interface Admin extends Abortable, Closeable {
/**
* Balance regions in the given RegionServer group
* @param groupName the group name
- * @return boolean Whether balance ran or not
+ * @return BalanceResponse details about the balancer run
* @throws IOException if a remote or network exception occurs
*/
- boolean balanceRSGroup(String groupName) throws IOException;
+ default BalanceResponse balanceRSGroup(String groupName) throws IOException {
+return balanceRSGroup(groupName, BalanceRequest.defaultInstance());
+ }
+
+ /**
+ * Balance regions in the given RegionServer group, running based on
+ * the given {@link BalanceRequest}.
+ *
+ * @return BalanceResponse details about the balancer run
+ */
+ BalanceResponse balanceRSGroup(String groupName, BalanceRequest request)
throws IOException;
/**
* Rename rsgroup
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AdminOverAsyncAdmin.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AdminOverAsyncAdmin.java
index 4a62f02..08de979 100644
---
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AdminOverAsyncAdmin.java
+++
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AdminOverAsyncAdmin.java
@@ -374,6 +374,11 @@ class AdminOverAsyncAdmin implements Admin {
return get(admin.balancerSwitch(onOrOff, synchronous));
}
+
+ public BalanceResponse balance(BalanceRequest request) throws IOException {
+return get(admin.balance(request));
+ }
+
@Override
public boolean balance() throws IOException {
return get(admin.balance());
@@ -1006,8 +1011,8 @@ class AdminOverAsyncAdmin implements Admin {
}
@Override
- public boolean balanceRSGroup(String groupName) throws IOException {
-return get(admin.balanceRSGroup(groupName));
+ public BalanceResponse balanceRSGroup(String groupName, BalanceRequest
request) throws IOException {
+return get(admin.balanceRSGroup(groupName, request));
}
@Override
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncAdmin.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncAdmin.java
index a5b1510..c366d3e 100644
--- a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncAdmin.java
+++ b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncAdmin.java
@@ -1287,7 +1287,8 @@ public interface AsyncAdmin {
* {@link CompletableFuture}.
*/
default CompletableFuture balance() {
-return balance(false);
+return balance(BalanceRequest.defaultInstance())
+ .thenApply(BalanceResponse::isBalancerRan);
}
/**
@@ -1297,8 +1298,25
[hbase] branch branch-2 updated: HBASE-26147 Add a dry run mode to the balancer, where moves are calculated but not actually executed
This is an automated email from the ASF dual-hosted git repository.
elserj pushed a commit to branch branch-2
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/branch-2 by this push:
new 5b58d11 HBASE-26147 Add a dry run mode to the balancer, where moves
are calculated but not actually executed
5b58d11 is described below
commit 5b58d11c91cbbc2f73e4252088c97df12a49ff87
Author: Josh Elser
AuthorDate: Wed Sep 1 21:47:53 2021 -0400
HBASE-26147 Add a dry run mode to the balancer, where moves are calculated
but not actually executed
Signed-off-by: Duo Zhang
Signed-off-by: Josh Elser true if balancer ran, false otherwise.
* @throws IOException if a remote or network exception occurs
*/
- boolean balance() throws IOException;
+ default boolean balance() throws IOException {
+return balance(BalanceRequest.defaultInstance())
+ .isBalancerRan();
+ }
+
+ /**
+ * Invoke the balancer with the given balance request. The BalanceRequest
defines how the
+ * balancer will run. See {@link BalanceRequest} for more details.
+ *
+ * @param request defines how the balancer should run
+ * @return {@link BalanceResponse} with details about the results of the
invocation.
+ * @throws IOException if a remote or network exception occurs
+ */
+ BalanceResponse balance(BalanceRequest request) throws IOException;
/**
* Invoke the balancer. Will run the balancer and if regions to move, it
will
@@ -1262,7 +1275,7 @@ public interface Admin extends Abortable, Closeable {
* @return true if balancer ran, false otherwise.
* @throws IOException if a remote or network exception occurs
* @deprecated Since 2.0.0. Will be removed in 3.0.0.
- * Use {@link #balance(boolean)} instead.
+ * Use {@link #balance(BalanceRequest)} instead.
*/
@Deprecated
default boolean balancer(boolean force) throws IOException {
@@ -1277,8 +1290,17 @@ public interface Admin extends Abortable, Closeable {
* @param force whether we should force balance even if there is region in
transition
* @return true if balancer ran, false otherwise.
* @throws IOException if a remote or network exception occurs
+ * @deprecated Since 2.5.0. Will be removed in 4.0.0.
+ * Use {@link #balance(BalanceRequest)} instead.
*/
- boolean balance(boolean force) throws IOException;
+ @Deprecated
+ default boolean balance(boolean force) throws IOException {
+return balance(
+ BalanceRequest.newBuilder()
+ .setIgnoreRegionsInTransition(force)
+ .build()
+).isBalancerRan();
+ }
/**
* Query the current state of the balancer.
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncAdmin.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncAdmin.java
index 6b8fda7..85d5455 100644
--- a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncAdmin.java
+++ b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncAdmin.java
@@ -1257,7 +1257,8 @@ public interface AsyncAdmin {
* {@link CompletableFuture}.
*/
default CompletableFuture balance() {
-return balance(false);
+return balance(BalanceRequest.defaultInstance())
+ .thenApply(BalanceResponse::isBalancerRan);
}
/**
@@ -1267,8 +1268,25 @@ public interface AsyncAdmin {
* @param forcible whether we should force balance even if there is region
in transition.
* @return True if balancer ran, false otherwise. The return value will be
wrapped by a
* {@link CompletableFuture}.
+ * @deprecated Since 2.5.0. Will be removed in 4.0.0.
+ * Use {@link #balance(BalanceRequest)} instead.
+ */
+ default CompletableFuture balance(boolean forcible) {
+return balance(
+ BalanceRequest.newBuilder()
+.setIgnoreRegionsInTransition(forcible)
+.build()
+).thenApply(BalanceResponse::isBalancerRan);
+ }
+
+ /**
+ * Invoke the balancer with the given balance request. The BalanceRequest
defines how the
+ * balancer will run. See {@link BalanceRequest} for more details.
+ *
+ * @param request defines how the balancer should run
+ * @return {@link BalanceResponse} with details about the results of the
invocation.
*/
- CompletableFuture balance(boolean forcible);
+ CompletableFuture balance(BalanceRequest request);
/**
* Query the current state of the balancer.
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncHBaseAdmin.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncHBaseAdmin.java
index c7b9897..db720f3 100644
---
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncHBaseAdmin.java
+++
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncHBaseAdmin.java
@@ -684,8 +684,8 @@ class AsyncHBaseAdmin implements AsyncAdmin {
}
@Override
- public CompletableFuture balance(boolean forcible) {
-return
[hbase] 16/18: HBASE-26139 Backport HBASE-23762 "Add documentation on how to enable and view tracing with OpenTelemetry" to branch-2 (#3629)
This is an automated email from the ASF dual-hosted git repository. taklwu pushed a commit to branch branch-2 in repository https://gitbox.apache.org/repos/asf/hbase.git commit 65b5b9b2a867957232719005740bf30ac988accb Author: Tak Lon (Stephen) Wu AuthorDate: Thu Aug 26 09:36:05 2021 -0700 HBASE-26139 Backport HBASE-23762 "Add documentation on how to enable and view tracing with OpenTelemetry" to branch-2 (#3629) 16/17 commits of HBASE-22120, original commit be4503d9f82f044fbfce21c8a42d0b1684607238 Co-authored-by: Duo Zhang Signed-off-by: Duo Zhang --- src/main/asciidoc/_chapters/tracing.adoc | 57 ++ src/main/asciidoc/_chapters/upgrading.adoc | 4 +-- src/main/asciidoc/book.adoc| 1 + 3 files changed, 60 insertions(+), 2 deletions(-) diff --git a/src/main/asciidoc/_chapters/tracing.adoc b/src/main/asciidoc/_chapters/tracing.adoc new file mode 100644 index 000..cc9f093 --- /dev/null +++ b/src/main/asciidoc/_chapters/tracing.adoc @@ -0,0 +1,57 @@ + +/** + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + + +[[tracing]] += Tracing +:doctype: book +:numbered: +:toc: left +:icons: font +:experimental: + +== Overview + +HBase used to depend on the HTrace project for tracing. After the Apache HTrace project moved to the Attic/retired, we decided to move to https://opentelemetry.io[OpenTelemetry] in https://issues.apache.org/jira/browse/HBASE-22120[HBASE-22120]. + +The basic support for tracing has been done, where we added tracing for async client, rpc, region read/write/scan operation, and WAL. We use opentelemetry-api to implement the tracing support manually by code, as our code base is way too complicated to be instrumented through a java agent. But notice that you still need to attach the opentelemetry java agent to enable tracing. Please see the official site for https://opentelemetry.io/[OpenTelemetry] and the documentation for https://gith [...] + +== Usage + +=== Enable Tracing + +See this section in hbase-env.sh + +[source,shell] + +# Uncomment to enable trace, you can change the options to use other exporters such as jaeger or +# zipkin. See https://github.com/open-telemetry/opentelemetry-java-instrumentation on how to +# configure exporters and other components through system properties. +# export HBASE_TRACE_OPTS="-Dotel.resource.attributes=service.name=HBase -Dotel.traces.exporter=logging otel.metrics.exporter=none" + + +Uncomment this line to enable tracing. The default config is to output the tracing data to log. Please see the documentation for https://github.com/open-telemetry/opentelemetry-java-instrumentation[opentelemetry-java-instrumentation] for more details on how to export tracing data to other tracing system such as OTel collector, jaeger or zipkin, what does the _service.name_ mean, and how to change the sampling rate, etc. + +NOTE: The https://github.com/open-telemetry/opentelemetry-java/blob/v1.0.1/exporters/logging/src/main/java/io/opentelemetry/exporter/logging/LoggingSpanExporter.java[LoggingSpanExporter] uses java.util.logging(jul) for logging tracing data, and the logger is initialized in opentelemetry java agent, which seems to be ahead of our jul to slf4j bridge initialization, so it will always log the tracing data to console. We highly suggest that you use other tracing systems to collect and view t [...] + +=== Performance Impact + +According to the result in https://issues.apache.org/jira/browse/HBASE-25658[HBASE-25658], the performance impact is minimal. Of course the test cluster is not under heavy load, so if you find out that enabling tracing would impact the performance, try to lower the sampling rate. See documentation for configuring https://github.com/open-telemetry/opentelemetry-java/blob/main/sdk-extensions/autoconfigure/README.md#sampler[sampler] for more details. + diff --git a/src/main/asciidoc/_chapters/upgrading.adoc b/src/main/asciidoc/_chapters/upgrading.adoc index cf1568e..d4f9e37 100644 --- a/src/main/asciidoc/_chapters/upgrading.adoc +++ b/src/main/asciidoc/_chapters/upgrading.adoc @@ -634,11 +634,11 @@ The Java client API for HBase has a number of changes
[hbase] 18/18: HBASE-26168 Backport HBASE-25811 "The client integration test is failing after HBASE-22120 merged" into branch-2 (#3634)
This is an automated email from the ASF dual-hosted git repository.
taklwu pushed a commit to branch branch-2
in repository https://gitbox.apache.org/repos/asf/hbase.git
commit 625093f907b97a7f51946862843edbd6a7c57e8d
Author: Tak Lon (Stephen) Wu
AuthorDate: Fri Aug 27 09:09:55 2021 -0700
HBASE-26168 Backport HBASE-25811 "The client integration test is failing
after HBASE-22120 merged" into branch-2 (#3634)
move opentelemetry jars to client-facing-thirdparty
add opentelemetry jars when init map reduce job dependencies
original commit 8d2a0efb7a9c17e58892369b281b4af49a5f0d7e
Co-authored-by: Duo Zhang
Signed-off-by: Duo Zhang
Signed-off-by: Peter Somogyi
---
hbase-assembly/src/main/assembly/client.xml | 8
hbase-assembly/src/main/assembly/hadoop-three-compat.xml | 1 +
.../org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java | 4 +++-
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/hbase-assembly/src/main/assembly/client.xml
b/hbase-assembly/src/main/assembly/client.xml
index bd65cb4..48940b7 100644
--- a/hbase-assembly/src/main/assembly/client.xml
+++ b/hbase-assembly/src/main/assembly/client.xml
@@ -65,6 +65,7 @@
org.slf4j:jcl-over-slf4j
org.slf4j:jul-to-slf4j
org.slf4j:slf4j-log4j12
+ io.opentelemetry.javaagent:*
@@ -153,6 +154,13 @@
org.slf4j:jcl-over-slf4j
org.slf4j:jul-to-slf4j
org.slf4j:slf4j-log4j12
+io.opentelemetry:*
+
+
+
+ lib/trace
+
+io.opentelemetry.javaagent:*
diff --git a/hbase-assembly/src/main/assembly/hadoop-three-compat.xml
b/hbase-assembly/src/main/assembly/hadoop-three-compat.xml
index 9780444..6c670a7 100644
--- a/hbase-assembly/src/main/assembly/hadoop-three-compat.xml
+++ b/hbase-assembly/src/main/assembly/hadoop-three-compat.xml
@@ -213,6 +213,7 @@
org.slf4j:jcl-over-slf4j
org.slf4j:jul-to-slf4j
org.slf4j:slf4j-log4j12
+io.opentelemetry:*
diff --git
a/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java
b/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java
index d89c5b5..eaf1b40 100644
---
a/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java
+++
b/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java
@@ -832,7 +832,9 @@ public class TableMapReduceUtil {
org.apache.zookeeper.ZooKeeper.class, //
zookeeper
com.google.protobuf.Message.class, //
protobuf
com.codahale.metrics.MetricRegistry.class, //
metrics-core
- org.apache.commons.lang3.ArrayUtils.class);//
commons-lang
+ org.apache.commons.lang3.ArrayUtils.class, //
commons-lang
+ io.opentelemetry.api.trace.Span.class, //
opentelemetry-api
+ io.opentelemetry.semconv.trace.attributes.SemanticAttributes.class); //
opentelemetry-semconv
}
/**
[hbase] 14/18: HBASE-26137 Backport HBASE-25732 "Change the command line argument for tracing after upgrading opentelemtry to 1.0.0" to branch-2 (#3624)
This is an automated email from the ASF dual-hosted git repository. taklwu pushed a commit to branch branch-2 in repository https://gitbox.apache.org/repos/asf/hbase.git commit dd1bdc8aff73ec1d01db83ce45e8506ef926bb91 Author: Tak Lon (Stephen) Wu AuthorDate: Wed Aug 25 09:01:19 2021 -0700 HBASE-26137 Backport HBASE-25732 "Change the command line argument for tracing after upgrading opentelemtry to 1.0.0" to branch-2 (#3624) 14/17 commits of HBASE-22120, original commit 8df9bebdd367d52a32b08c18a7cf4f9c2d712071 Co-authored-by: Duo Zhang Signed-off-by: Duo Zhang --- conf/hbase-env.cmd | 5 + conf/hbase-env.sh | 6 +++--- 2 files changed, 8 insertions(+), 3 deletions(-) diff --git a/conf/hbase-env.cmd b/conf/hbase-env.cmd index 1f1c3e3..4beebf6 100644 --- a/conf/hbase-env.cmd +++ b/conf/hbase-env.cmd @@ -81,3 +81,8 @@ set HBASE_OPTS=%HBASE_OPTS% "-XX:+UseConcMarkSweepGC" "-Djava.net.preferIPv4Stac @rem Tell HBase whether it should manage it's own instance of ZooKeeper or not. @rem set HBASE_MANAGES_ZK=true + +@rem Uncomment to enable trace, you can change the options to use other exporters such as jaeger or +@rem zipkin. See https://github.com/open-telemetry/opentelemetry-java-instrumentation on how to +@rem configure exporters and other components through system properties. +@rem set HBASE_TRACE_OPTS="-Dotel.resource.attributes=service.name=HBase -Dotel.traces.exporter=logging otel.metrics.exporter=none" diff --git a/conf/hbase-env.sh b/conf/hbase-env.sh index b3c4baf..d1d9a6f 100644 --- a/conf/hbase-env.sh +++ b/conf/hbase-env.sh @@ -146,6 +146,6 @@ # export HBASE_JSHELL_ARGS="--startup DEFAULT --startup PRINTING --startup hbase_startup.jsh" # Uncomment to enable trace, you can change the options to use other exporters such as jaeger or -# zipkin. See https://github.com/open-telemetry/opentelemetry-java-instrumentation on how to config -# exporters and other components through system properties. -# export HBASE_TRACE_OPTS="-Dotel.config.sampler.probability=0.1 -Dotel.exporter=logging" +# zipkin. See https://github.com/open-telemetry/opentelemetry-java-instrumentation on how to +# configure exporters and other components through system properties. +# export HBASE_TRACE_OPTS="-Dotel.resource.attributes=service.name=HBase -Dotel.traces.exporter=logging otel.metrics.exporter=none"
[hbase] 17/18: HBASE-26140 Backport HBASE-25778 "The tracinig implementation for AsyncConnectionImpl.getHbck is incorrect" to branch-2 (#3631)
This is an automated email from the ASF dual-hosted git repository.
taklwu pushed a commit to branch branch-2
in repository https://gitbox.apache.org/repos/asf/hbase.git
commit ab431fc8a9b106091f703eeea3a4f71de16b83e7
Author: Tak Lon (Stephen) Wu
AuthorDate: Thu Aug 26 11:29:30 2021 -0700
HBASE-26140 Backport HBASE-25778 "The tracinig implementation for
AsyncConnectionImpl.getHbck is incorrect" to branch-2 (#3631)
17/17 commits of HBASE-22120, original commit
f36e1539648bbaee84c626fd54d1605baebf3c5a
Co-authored-by: Duo Zhang
Signed-off-by: Peter Somogyi
---
.../hadoop/hbase/client/AsyncConnectionImpl.java | 37 +++
.../hbase/client/TestAsyncConnectionTracing.java | 112 +
.../org/apache/hadoop/hbase/trace/TraceUtil.java | 14 +++
3 files changed, 145 insertions(+), 18 deletions(-)
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncConnectionImpl.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncConnectionImpl.java
index 98e7825..76b1208 100644
---
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncConnectionImpl.java
+++
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncConnectionImpl.java
@@ -28,7 +28,6 @@ import static
org.apache.hadoop.hbase.client.NonceGenerator.CLIENT_NONCES_ENABLE
import static org.apache.hadoop.hbase.util.FutureUtils.addListener;
import io.opentelemetry.api.trace.Span;
-import io.opentelemetry.context.Scope;
import java.io.IOException;
import java.util.Optional;
import java.util.concurrent.CompletableFuture;
@@ -37,6 +36,7 @@ import java.util.concurrent.ConcurrentMap;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicReference;
+import java.util.function.Supplier;
import org.apache.commons.io.IOUtils;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.AuthUtil;
@@ -370,6 +370,15 @@ class AsyncConnectionImpl implements AsyncConnection {
RETRY_TIMER);
}
+ private Hbck getHbckInternal(ServerName masterServer) {
+Span.current().setAttribute(TraceUtil.SERVER_NAME_KEY,
masterServer.getServerName());
+// we will not create a new connection when creating a new protobuf stub,
and for hbck there
+// will be no performance consideration, so for simplification we will
create a new stub every
+// time instead of caching the stub here.
+return new HBaseHbck(MasterProtos.HbckService.newBlockingStub(
+ rpcClient.createBlockingRpcChannel(masterServer, user, rpcTimeout)),
rpcControllerFactory);
+ }
+
@Override
public CompletableFuture getHbck() {
return TraceUtil.tracedFuture(() -> {
@@ -378,11 +387,7 @@ class AsyncConnectionImpl implements AsyncConnection {
if (error != null) {
future.completeExceptionally(error);
} else {
- try {
-future.complete(getHbck(sn));
- } catch (IOException e) {
-future.completeExceptionally(e);
- }
+ future.complete(getHbckInternal(sn));
}
});
return future;
@@ -390,18 +395,14 @@ class AsyncConnectionImpl implements AsyncConnection {
}
@Override
- public Hbck getHbck(ServerName masterServer) throws IOException {
-Span span = TraceUtil.createSpan("AsyncConnection.getHbck")
- .setAttribute(TraceUtil.SERVER_NAME_KEY, masterServer.getServerName());
-try (Scope scope = span.makeCurrent()) {
- // we will not create a new connection when creating a new protobuf
stub, and for hbck there
- // will be no performance consideration, so for simplification we will
create a new stub every
- // time instead of caching the stub here.
- return new HBaseHbck(
-MasterProtos.HbckService
- .newBlockingStub(rpcClient.createBlockingRpcChannel(masterServer,
user, rpcTimeout)),
-rpcControllerFactory);
-}
+ public Hbck getHbck(ServerName masterServer) {
+return TraceUtil.trace(new Supplier() {
+
+ @Override
+ public Hbck get() {
+return getHbckInternal(masterServer);
+ }
+}, "AsyncConnection.getHbck");
}
@Override
diff --git
a/hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestAsyncConnectionTracing.java
b/hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestAsyncConnectionTracing.java
new file mode 100644
index 000..fec5f6d
--- /dev/null
+++
b/hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestAsyncConnectionTracing.java
@@ -0,0 +1,112 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.
[hbase] 15/18: HBASE-26138 Backport HBASE-25733 "Upgrade opentelemetry to 1.0.1" to branch-2 (#3628)
This is an automated email from the ASF dual-hosted git repository. taklwu pushed a commit to branch branch-2 in repository https://gitbox.apache.org/repos/asf/hbase.git commit a746e19c3be6b69ce3a7f661a6b83cea05215daa Author: Tak Lon (Stephen) Wu AuthorDate: Wed Aug 25 19:28:10 2021 -0700 HBASE-26138 Backport HBASE-25733 "Upgrade opentelemetry to 1.0.1" to branch-2 (#3628) 15/17 commits of HBASE-22120, original commit b71488998970a3353086a34736ed1edab527f673 Co-authored-by: Duo Zhang Signed-off-by: Peter Somogyi --- pom.xml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/pom.xml b/pom.xml index 7e08795..e15611e 100755 --- a/pom.xml +++ b/pom.xml @@ -1483,8 +1483,8 @@ 9.2.13.0 4.13 1.3 -1.0.0 -1.0.0 +1.0.1 +1.0.1 1.2.17 2.28.2
[hbase] 08/18: HBASE-26131 Backport HBASE-25484 "Add trace support for WAL sync" to branch-2 (#3597)
This is an automated email from the ASF dual-hosted git repository.
taklwu pushed a commit to branch branch-2
in repository https://gitbox.apache.org/repos/asf/hbase.git
commit 9cce94a2d05db892570b6a727d8e3fee5a7ac5ed
Author: Tak Lon (Stephen) Wu
AuthorDate: Thu Aug 19 09:15:05 2021 -0700
HBASE-26131 Backport HBASE-25484 "Add trace support for WAL sync" to
branch-2 (#3597)
8/17 commits of HBASE-22120, original commit
2be2c63f0d3917a243b74af9754cbfc805b858d1
Co-authored-by: Duo Zhang
Signed-off-by: Duo Zhang
Signed-off-by: Peter Somogyi
---
.../org/apache/hadoop/hbase/trace/TraceUtil.java | 2 +
.../hbase/regionserver/wal/AbstractFSWAL.java | 77 +++-
.../hadoop/hbase/regionserver/wal/AsyncFSWAL.java | 101 +++--
.../hadoop/hbase/regionserver/wal/FSHLog.java | 38 +---
.../hadoop/hbase/regionserver/TestHRegion.java | 4 +-
.../regionserver/wal/TestAsyncFSWALDurability.java | 8 +-
.../regionserver/wal/TestFSHLogDurability.java | 8 +-
.../org/apache/hadoop/hbase/wal/FaultyFSLog.java | 10 +-
8 files changed, 104 insertions(+), 144 deletions(-)
diff --git
a/hbase-common/src/main/java/org/apache/hadoop/hbase/trace/TraceUtil.java
b/hbase-common/src/main/java/org/apache/hadoop/hbase/trace/TraceUtil.java
index ea16df1..886a4a9 100644
--- a/hbase-common/src/main/java/org/apache/hadoop/hbase/trace/TraceUtil.java
+++ b/hbase-common/src/main/java/org/apache/hadoop/hbase/trace/TraceUtil.java
@@ -62,6 +62,8 @@ public final class TraceUtil {
public static final AttributeKey ROW_LOCK_READ_LOCK_KEY =
AttributeKey.booleanKey("db.hbase.rowlock.readlock");
+ public static final AttributeKey WAL_IMPL =
AttributeKey.stringKey("db.hbase.wal.impl");
+
private TraceUtil() {
}
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/AbstractFSWAL.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/AbstractFSWAL.java
index 5c247d1..c39e10d 100644
---
a/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/AbstractFSWAL.java
+++
b/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/AbstractFSWAL.java
@@ -26,7 +26,6 @@ import static
org.apache.hbase.thirdparty.com.google.common.base.Preconditions.c
import com.lmax.disruptor.RingBuffer;
import io.opentelemetry.api.trace.Span;
-import io.opentelemetry.context.Scope;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.io.InterruptedIOException;
@@ -549,6 +548,35 @@ public abstract class AbstractFSWAL
implements WAL {
return rollWriter(false);
}
+ @Override
+ public final void sync() throws IOException {
+sync(useHsync);
+ }
+
+ @Override
+ public final void sync(long txid) throws IOException {
+sync(txid, useHsync);
+ }
+
+ @Override
+ public final void sync(boolean forceSync) throws IOException {
+TraceUtil.trace(() -> {
+ doSync(forceSync);
+ return null;
+}, () -> createSpan("WAL.sync"));
+ }
+
+ @Override
+ public final void sync(long txid, boolean forceSync) throws IOException {
+TraceUtil.trace(() -> {
+ doSync(txid, forceSync);
+ return null;
+}, () -> createSpan("WAL.sync"));
+ }
+
+ protected abstract void doSync(boolean forceSync) throws IOException;
+
+ protected abstract void doSync(long txid, boolean forceSync) throws
IOException;
/**
* This is a convenience method that computes a new filename with a given
file-number.
* @param filenum to use
@@ -650,7 +678,7 @@ public abstract class AbstractFSWAL
implements WAL {
this.sequenceIdAccounting.findLower(firstWALEntry.getValue().encodedName2HighestSequenceId);
}
if (regions != null) {
- List listForPrint = new ArrayList();
+ List listForPrint = new ArrayList<>();
for (Map.Entry> r : regions.entrySet()) {
StringBuilder families = new StringBuilder();
for (int i = 0; i < r.getValue().size(); i++) {
@@ -768,6 +796,10 @@ public abstract class AbstractFSWAL
implements WAL {
}
}
+ private Span createSpan(String name) {
+return TraceUtil.createSpan(name).setAttribute(TraceUtil.WAL_IMPL,
implClassName);
+ }
+
/**
* Cleans up current writer closing it and then puts in place the passed in
{@code nextWriter}.
*
@@ -785,13 +817,10 @@ public abstract class AbstractFSWAL
implements WAL {
* @throws IOException if there is a problem flushing or closing the
underlying FS
*/
Path replaceWriter(Path oldPath, Path newPath, W nextWriter) throws
IOException {
-Span span =
TraceUtil.getGlobalTracer().spanBuilder("FSHFile.replaceWriter").startSpan();
-try (Scope scope = span.makeCurrent()) {
+return TraceUtil.trace(() -> {
doReplaceWriter(oldPath, newPath, nextWriter);
return newPath;
-} finally {
- span.end();
-}
+}, () -> createSpan("WAL.replaceWriter"));
}
protected
[hbase] 09/18: HBASE-26132 Backport HBASE-25535 "Set span kind to CLIENT in AbstractRpcClient" to branch-2 (#3607)
This is an automated email from the ASF dual-hosted git repository.
taklwu pushed a commit to branch branch-2
in repository https://gitbox.apache.org/repos/asf/hbase.git
commit 8b7450f5d8480e0243978b279eaaba5fb1c4994b
Author: Tak Lon (Stephen) Wu
AuthorDate: Thu Aug 19 20:12:37 2021 -0700
HBASE-26132 Backport HBASE-25535 "Set span kind to CLIENT in
AbstractRpcClient" to branch-2 (#3607)
9/17 commits of HBASE-22120, original commit
bb8c4967f8ce2c89ebaf1ddc5d8a1bf55f1e20d3
Co-authored-by: Duo Zhang
Signed-off-by: Duo Zhang
---
.../apache/hadoop/hbase/ipc/AbstractRpcClient.java | 2 +-
.../org/apache/hadoop/hbase/trace/TraceUtil.java | 7 +++
.../apache/hadoop/hbase/ipc/AbstractTestIPC.java | 23 +-
3 files changed, 26 insertions(+), 6 deletions(-)
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/AbstractRpcClient.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/AbstractRpcClient.java
index fa7dfb1..bb7f4af 100644
---
a/hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/AbstractRpcClient.java
+++
b/hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/AbstractRpcClient.java
@@ -395,7 +395,7 @@ public abstract class AbstractRpcClient implements RpcC
private Call callMethod(final Descriptors.MethodDescriptor md, final
HBaseRpcController hrc,
final Message param, Message returnType, final User ticket, final Address
addr,
final RpcCallback callback) {
-Span span = TraceUtil.createSpan("RpcClient.callMethod")
+Span span = TraceUtil.createClientSpan("RpcClient.callMethod")
.setAttribute(TraceUtil.RPC_SERVICE_KEY, md.getService().getName())
.setAttribute(TraceUtil.RPC_METHOD_KEY, md.getName())
.setAttribute(TraceUtil.REMOTE_HOST_KEY, addr.getHostName())
diff --git
a/hbase-common/src/main/java/org/apache/hadoop/hbase/trace/TraceUtil.java
b/hbase-common/src/main/java/org/apache/hadoop/hbase/trace/TraceUtil.java
index 886a4a9..2a6b6b6 100644
--- a/hbase-common/src/main/java/org/apache/hadoop/hbase/trace/TraceUtil.java
+++ b/hbase-common/src/main/java/org/apache/hadoop/hbase/trace/TraceUtil.java
@@ -106,6 +106,13 @@ public final class TraceUtil {
}
/**
+ * Create a span with {@link Kind#CLIENT}.
+ */
+ public static Span createClientSpan(String name) {
+return createSpan(name, Kind.CLIENT);
+ }
+
+ /**
* Trace an asynchronous operation for a table.
*/
public static CompletableFuture
tracedFuture(Supplier> action,
diff --git
a/hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/AbstractTestIPC.java
b/hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/AbstractTestIPC.java
index ffecb09..c4951b2 100644
---
a/hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/AbstractTestIPC.java
+++
b/hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/AbstractTestIPC.java
@@ -34,6 +34,7 @@ import static org.mockito.Mockito.spy;
import static org.mockito.Mockito.verify;
import static org.mockito.internal.verification.VerificationModeFactory.times;
+import io.opentelemetry.api.trace.Span.Kind;
import io.opentelemetry.api.trace.StatusCode;
import io.opentelemetry.sdk.testing.junit4.OpenTelemetryRule;
import io.opentelemetry.sdk.trace.data.SpanData;
@@ -455,7 +456,8 @@ public abstract class AbstractTestIPC {
return traceRule.getSpans().stream().filter(s ->
s.getName().equals(name)).findFirst().get();
}
- private void assertRpcAttribute(SpanData data, String methodName,
InetSocketAddress addr) {
+ private void assertRpcAttribute(SpanData data, String methodName,
InetSocketAddress addr,
+Kind kind) {
assertEquals(SERVICE.getDescriptorForType().getName(),
data.getAttributes().get(TraceUtil.RPC_SERVICE_KEY));
assertEquals(methodName,
data.getAttributes().get(TraceUtil.RPC_METHOD_KEY));
@@ -463,6 +465,13 @@ public abstract class AbstractTestIPC {
assertEquals(addr.getHostName(),
data.getAttributes().get(TraceUtil.REMOTE_HOST_KEY));
assertEquals(addr.getPort(),
data.getAttributes().get(TraceUtil.REMOTE_PORT_KEY).intValue());
}
+assertEquals(kind, data.getKind());
+ }
+
+ private void assertRemoteSpan() {
+SpanData data = waitSpan("RpcServer.process");
+assertTrue(data.getParentSpanContext().isRemote());
+assertEquals(Kind.SERVER, data.getKind());
}
@Test
@@ -474,8 +483,10 @@ public abstract class AbstractTestIPC {
rpcServer.start();
BlockingInterface stub = newBlockingStub(client,
rpcServer.getListenerAddress());
stub.pause(null, PauseRequestProto.newBuilder().setMs(100).build());
- assertRpcAttribute(waitSpan("RpcClient.callMethod"), "pause",
rpcServer.getListenerAddress());
- assertRpcAttribute(waitSpan("RpcServer.callMethod"), "pause", null);
+ assertRpcAttribute(waitSpan("RpcClient.callMethod"), "pause",
rpcServer.getListenerAddress(),
+Kind.CLIENT);
+
[hbase] 13/18: HBASE-26136 Backport HBASE-25723 "Temporarily remove the trace support for RegionScanner.next" to branch-2 (#3623)
This is an automated email from the ASF dual-hosted git repository.
taklwu pushed a commit to branch branch-2
in repository https://gitbox.apache.org/repos/asf/hbase.git
commit c639d6f353b09c1466e2eb7ec76e4d4e0e42009d
Author: Tak Lon (Stephen) Wu
AuthorDate: Tue Aug 24 17:45:04 2021 -0700
HBASE-26136 Backport HBASE-25723 "Temporarily remove the trace support for
RegionScanner.next" to branch-2 (#3623)
13/17 commits of HBASE-22120, original commit
7f90c2201f6a17d2e2d031505c35ae7c2b1ed7ea
Co-authored-by: Duo Zhang
Signed-off-by: Duo Zhang
---
.../hbase/regionserver/RegionScannerImpl.java | 58 +++---
.../hbase/regionserver/TestHRegionTracing.java | 1 -
2 files changed, 28 insertions(+), 31 deletions(-)
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionScannerImpl.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionScannerImpl.java
index b2e793f..881f423 100644
---
a/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionScannerImpl.java
+++
b/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionScannerImpl.java
@@ -255,39 +255,37 @@ class RegionScannerImpl implements RegionScanner,
Shipper, RpcCallback {
@Override
public boolean nextRaw(List outResults, ScannerContext scannerContext)
throws IOException {
-return TraceUtil.trace(() -> {
- if (storeHeap == null) {
-// scanner is closed
-throw new UnknownScannerException("Scanner was closed");
- }
- boolean moreValues = false;
- if (outResults.isEmpty()) {
-// Usually outResults is empty. This is true when next is called
-// to handle scan or get operation.
-moreValues = nextInternal(outResults, scannerContext);
- } else {
-List tmpList = new ArrayList<>();
-moreValues = nextInternal(tmpList, scannerContext);
-outResults.addAll(tmpList);
- }
+if (storeHeap == null) {
+ // scanner is closed
+ throw new UnknownScannerException("Scanner was closed");
+}
+boolean moreValues = false;
+if (outResults.isEmpty()) {
+ // Usually outResults is empty. This is true when next is called
+ // to handle scan or get operation.
+ moreValues = nextInternal(outResults, scannerContext);
+} else {
+ List tmpList = new ArrayList<>();
+ moreValues = nextInternal(tmpList, scannerContext);
+ outResults.addAll(tmpList);
+}
- region.addReadRequestsCount(1);
- if (region.getMetrics() != null) {
-region.getMetrics().updateReadRequestCount();
- }
+region.addReadRequestsCount(1);
+if (region.getMetrics() != null) {
+ region.getMetrics().updateReadRequestCount();
+}
- // If the size limit was reached it means a partial Result is being
returned. Returning a
- // partial Result means that we should not reset the filters; filters
should only be reset in
- // between rows
- if (!scannerContext.mayHaveMoreCellsInRow()) {
-resetFilters();
- }
+// If the size limit was reached it means a partial Result is being
returned. Returning a
+// partial Result means that we should not reset the filters; filters
should only be reset in
+// between rows
+if (!scannerContext.mayHaveMoreCellsInRow()) {
+ resetFilters();
+}
- if (isFilterDoneInternal()) {
-moreValues = false;
- }
- return moreValues;
-}, () -> region.createRegionSpan("RegionScanner.next"));
+if (isFilterDoneInternal()) {
+ moreValues = false;
+}
+return moreValues;
}
/**
diff --git
a/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionTracing.java
b/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionTracing.java
index 17c3eb4..e5779b4 100644
---
a/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionTracing.java
+++
b/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionTracing.java
@@ -178,7 +178,6 @@ public class TestHRegionTracing {
}
assertSpan("Region.getScanner");
assertSpan("RegionScanner.reseek");
-assertSpan("RegionScanner.next");
assertSpan("RegionScanner.close");
}
}
[hbase] 12/18: HBASE-26135 Backport HBASE-25616 "Upgrade opentelemetry to 1.0.0" to branch-2 (#3620)
This is an automated email from the ASF dual-hosted git repository. taklwu pushed a commit to branch branch-2 in repository https://gitbox.apache.org/repos/asf/hbase.git commit 8f160109cd70466e4c11c8f554c6e258661d3d34 Author: Tak Lon (Stephen) Wu AuthorDate: Tue Aug 24 09:22:37 2021 -0700 HBASE-26135 Backport HBASE-25616 "Upgrade opentelemetry to 1.0.0" to branch-2 (#3620) 12/17 commits of HBASE-22120, original commits 8399293e21127df3ffdcb757242e4cb5964c7e99 Co-authored-by: Duo Zhang Signed-off-by: Duo Zhang Signed-off-by: Peter Somogyi --- pom.xml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/pom.xml b/pom.xml index 6f04e4d..7e08795 100755 --- a/pom.xml +++ b/pom.xml @@ -1483,8 +1483,8 @@ 9.2.13.0 4.13 1.3 -0.17.1 -0.17.0 +1.0.0 +1.0.0 1.2.17 2.28.2
[hbase] 06/18: HBASE-26129 Backport HBASE-25481 "Add host and port attribute when tracing rpc call at client side" to branch-2 (#3589)
This is an automated email from the ASF dual-hosted git repository.
taklwu pushed a commit to branch branch-2
in repository https://gitbox.apache.org/repos/asf/hbase.git
commit 83a27389a025f4d6b9481f4b66a8e6799c84f8b0
Author: Tak Lon (Stephen) Wu
AuthorDate: Tue Aug 17 09:24:10 2021 -0700
HBASE-26129 Backport HBASE-25481 "Add host and port attribute when tracing
rpc call at client side" to branch-2 (#3589)
6/17 commits of HBASE-22120, original commit
ae2c62ffaad5ba4c976b0a79c10a365edf2844fd
Co-authored-by: Duo Zhang
Signed-off-by: Peter Somogyi
---
.../org/apache/hadoop/hbase/ipc/AbstractRpcClient.java | 4 +++-
.../java/org/apache/hadoop/hbase/trace/TraceUtil.java | 4
.../org/apache/hadoop/hbase/ipc/AbstractTestIPC.java| 17 ++---
3 files changed, 17 insertions(+), 8 deletions(-)
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/AbstractRpcClient.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/AbstractRpcClient.java
index b671095..878f9cf 100644
---
a/hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/AbstractRpcClient.java
+++
b/hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/AbstractRpcClient.java
@@ -397,7 +397,9 @@ public abstract class AbstractRpcClient implements RpcC
final RpcCallback callback) {
Span span = TraceUtil.createSpan("RpcClient.callMethod")
.setAttribute(TraceUtil.RPC_SERVICE_KEY, md.getService().getName())
- .setAttribute(TraceUtil.RPC_METHOD_KEY, md.getName());
+ .setAttribute(TraceUtil.RPC_METHOD_KEY, md.getName())
+ .setAttribute(TraceUtil.REMOTE_HOST_KEY, addr.getHostName())
+ .setAttribute(TraceUtil.REMOTE_PORT_KEY, addr.getPort());
try (Scope scope = span.makeCurrent()) {
final MetricsConnection.CallStats cs = MetricsConnection.newCallStats();
cs.setStartTime(EnvironmentEdgeManager.currentTime());
diff --git
a/hbase-common/src/main/java/org/apache/hadoop/hbase/trace/TraceUtil.java
b/hbase-common/src/main/java/org/apache/hadoop/hbase/trace/TraceUtil.java
index d0da071..8eb2399 100644
--- a/hbase-common/src/main/java/org/apache/hadoop/hbase/trace/TraceUtil.java
+++ b/hbase-common/src/main/java/org/apache/hadoop/hbase/trace/TraceUtil.java
@@ -54,6 +54,10 @@ public final class TraceUtil {
public static final AttributeKey SERVER_NAME_KEY =
AttributeKey.stringKey("db.hbase.server.name");
+ public static final AttributeKey REMOTE_HOST_KEY =
SemanticAttributes.NET_PEER_NAME;
+
+ public static final AttributeKey REMOTE_PORT_KEY =
SemanticAttributes.NET_PEER_PORT;
+
private TraceUtil() {
}
diff --git
a/hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/AbstractTestIPC.java
b/hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/AbstractTestIPC.java
index 4aca764..ffecb09 100644
---
a/hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/AbstractTestIPC.java
+++
b/hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/AbstractTestIPC.java
@@ -455,11 +455,14 @@ public abstract class AbstractTestIPC {
return traceRule.getSpans().stream().filter(s ->
s.getName().equals(name)).findFirst().get();
}
- private void assertRpcAttribute(SpanData data, String methodName) {
+ private void assertRpcAttribute(SpanData data, String methodName,
InetSocketAddress addr) {
assertEquals(SERVICE.getDescriptorForType().getName(),
data.getAttributes().get(TraceUtil.RPC_SERVICE_KEY));
-assertEquals(methodName,
- data.getAttributes().get(TraceUtil.RPC_METHOD_KEY));
+assertEquals(methodName,
data.getAttributes().get(TraceUtil.RPC_METHOD_KEY));
+if (addr != null) {
+ assertEquals(addr.getHostName(),
data.getAttributes().get(TraceUtil.REMOTE_HOST_KEY));
+ assertEquals(addr.getPort(),
data.getAttributes().get(TraceUtil.REMOTE_PORT_KEY).intValue());
+}
}
@Test
@@ -471,8 +474,8 @@ public abstract class AbstractTestIPC {
rpcServer.start();
BlockingInterface stub = newBlockingStub(client,
rpcServer.getListenerAddress());
stub.pause(null, PauseRequestProto.newBuilder().setMs(100).build());
- assertRpcAttribute(waitSpan("RpcClient.callMethod"), "pause");
- assertRpcAttribute(waitSpan("RpcServer.callMethod"), "pause");
+ assertRpcAttribute(waitSpan("RpcClient.callMethod"), "pause",
rpcServer.getListenerAddress());
+ assertRpcAttribute(waitSpan("RpcServer.callMethod"), "pause", null);
assertSameTraceId();
for (SpanData data : traceRule.getSpans()) {
assertThat(
@@ -484,8 +487,8 @@ public abstract class AbstractTestIPC {
traceRule.clearSpans();
assertThrows(ServiceException.class,
() -> stub.error(null, EmptyRequestProto.getDefaultInstance()));
- assertRpcAttribute(waitSpan("RpcClient.callMethod"), "error");
- assertRpcAttribute(waitSpan("RpcServer.callMethod"), "error");
+ assertRpcAttribute(waitSpan("RpcClient.callMethod"), "error",
[hbase] 11/18: HBASE-26134 Backport HBASE-25617 "Revisit the span names" to branch-2 (#3618)
This is an automated email from the ASF dual-hosted git repository.
taklwu pushed a commit to branch branch-2
in repository https://gitbox.apache.org/repos/asf/hbase.git
commit a2b8f7a3be38050858594c01b5be7171917c6253
Author: Tak Lon (Stephen) Wu
AuthorDate: Mon Aug 23 21:04:16 2021 -0700
HBASE-26134 Backport HBASE-25617 "Revisit the span names" to branch-2
(#3618)
11/17 commits of HBASE-22120, original commit
8d68f8cd1c8613be1b499eaa99f46806b2743294
Co-authored-by: Duo Zhang
Signed-off-by: Duo Zhang
---
.../main/java/org/apache/hadoop/hbase/client/AsyncConnectionImpl.java | 4 ++--
.../main/java/org/apache/hadoop/hbase/client/AsyncRegionLocator.java | 2 +-
.../main/java/org/apache/hadoop/hbase/client/ConnectionFactory.java | 2 +-
3 files changed, 4 insertions(+), 4 deletions(-)
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncConnectionImpl.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncConnectionImpl.java
index b919ee1..98e7825 100644
---
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncConnectionImpl.java
+++
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncConnectionImpl.java
@@ -386,12 +386,12 @@ class AsyncConnectionImpl implements AsyncConnection {
}
});
return future;
-}, getClass().getName() + ".getHbck");
+}, "AsyncConnection.getHbck");
}
@Override
public Hbck getHbck(ServerName masterServer) throws IOException {
-Span span = TraceUtil.createSpan(getClass().getName() + ".getHbck")
+Span span = TraceUtil.createSpan("AsyncConnection.getHbck")
.setAttribute(TraceUtil.SERVER_NAME_KEY, masterServer.getServerName());
try (Scope scope = span.makeCurrent()) {
// we will not create a new connection when creating a new protobuf
stub, and for hbck there
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRegionLocator.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRegionLocator.java
index 3067785..61086c4 100644
---
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRegionLocator.java
+++
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRegionLocator.java
@@ -97,7 +97,7 @@ class AsyncRegionLocator {
private CompletableFuture
tracedLocationFuture(Supplier> action,
Function> getRegionNames, TableName tableName, String
methodName) {
-Span span = createTableSpan(getClass().getSimpleName() + "." + methodName,
tableName);
+Span span = createTableSpan("AsyncRegionLocator." + methodName, tableName);
try (Scope scope = span.makeCurrent()) {
CompletableFuture future = action.get();
FutureUtils.addListener(future, (resp, error) -> {
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionFactory.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionFactory.java
index 627e8d2..23bd333 100644
---
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionFactory.java
+++
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionFactory.java
@@ -307,6 +307,6 @@ public class ConnectionFactory {
}
});
return future;
-}, ConnectionFactory.class.getSimpleName() + ".createAsyncConnection");
+}, "ConnectionFactory.createAsyncConnection");
}
}
[hbase] 10/18: HBASE-26133 Backport HBASE-25591 "Upgrade opentelemetry to 0.17.1" to branch-2 (#3608)
This is an automated email from the ASF dual-hosted git repository.
taklwu pushed a commit to branch branch-2
in repository https://gitbox.apache.org/repos/asf/hbase.git
commit d0a53e3f297cc7060782f94e02c86c10d6114af7
Author: Tak Lon (Stephen) Wu
AuthorDate: Mon Aug 23 08:52:07 2021 -0700
HBASE-26133 Backport HBASE-25591 "Upgrade opentelemetry to 0.17.1" to
branch-2 (#3608)
10/17 commits of HBASE-22120, original commit
f6ff519dd0c7fe0e3ae3c175eefee27a26a065a4
Co-authored-by: Duo Zhang
Signed-off-by: Duo Zhang
Signed-off-by: Peter Somogyi
---
.../hadoop/hbase/client/TestAsyncTableTracing.java | 6 +++---
hbase-common/pom.xml | 4
.../org/apache/hadoop/hbase/trace/TraceUtil.java | 25 +++---
.../hadoop/hbase/mttr/IntegrationTestMTTR.java | 3 +--
.../hadoop/hbase/ipc/ServerRpcConnection.java | 4 ++--
.../apache/hadoop/hbase/ipc/AbstractTestIPC.java | 14 ++--
.../hbase/regionserver/TestHRegionTracing.java | 15 ++---
pom.xml| 18 +++-
8 files changed, 44 insertions(+), 45 deletions(-)
diff --git
a/hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestAsyncTableTracing.java
b/hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestAsyncTableTracing.java
index 07cdf0e..a3ad9df 100644
---
a/hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestAsyncTableTracing.java
+++
b/hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestAsyncTableTracing.java
@@ -24,7 +24,7 @@ import static org.mockito.ArgumentMatchers.anyLong;
import static org.mockito.Mockito.doAnswer;
import static org.mockito.Mockito.mock;
-import io.opentelemetry.api.trace.Span.Kind;
+import io.opentelemetry.api.trace.SpanKind;
import io.opentelemetry.api.trace.StatusCode;
import io.opentelemetry.sdk.testing.junit4.OpenTelemetryRule;
import io.opentelemetry.sdk.trace.data.SpanData;
@@ -222,7 +222,7 @@ public class TestAsyncTableTracing {
Waiter.waitFor(CONF, 1000,
() -> traceRule.getSpans().stream()
.anyMatch(span -> span.getName().equals("AsyncTable." + methodName) &&
- span.getKind() == Kind.INTERNAL && span.hasEnded()));
+ span.getKind() == SpanKind.INTERNAL && span.hasEnded()));
SpanData data = traceRule.getSpans().stream()
.filter(s -> s.getName().equals("AsyncTable." +
methodName)).findFirst().get();
assertEquals(StatusCode.OK, data.getStatus().getStatusCode());
@@ -409,7 +409,7 @@ public class TestAsyncTableTracing {
Waiter.waitFor(CONF, 1000,
() -> traceRule.getSpans().stream()
.anyMatch(span -> span.getName().equals("AsyncConnection.close") &&
- span.getKind() == Kind.INTERNAL && span.hasEnded()));
+ span.getKind() == SpanKind.INTERNAL && span.hasEnded()));
SpanData data = traceRule.getSpans().stream()
.filter(s ->
s.getName().equals("AsyncConnection.close")).findFirst().get();
assertEquals(StatusCode.OK, data.getStatus().getStatusCode());
diff --git a/hbase-common/pom.xml b/hbase-common/pom.xml
index 62661b8..2212fd1 100644
--- a/hbase-common/pom.xml
+++ b/hbase-common/pom.xml
@@ -196,6 +196,10 @@
opentelemetry-api
+ io.opentelemetry
+ opentelemetry-semconv
+
+
org.apache.commons
commons-crypto
diff --git
a/hbase-common/src/main/java/org/apache/hadoop/hbase/trace/TraceUtil.java
b/hbase-common/src/main/java/org/apache/hadoop/hbase/trace/TraceUtil.java
index 2a6b6b6..43c2e04 100644
--- a/hbase-common/src/main/java/org/apache/hadoop/hbase/trace/TraceUtil.java
+++ b/hbase-common/src/main/java/org/apache/hadoop/hbase/trace/TraceUtil.java
@@ -20,12 +20,12 @@ package org.apache.hadoop.hbase.trace;
import io.opentelemetry.api.GlobalOpenTelemetry;
import io.opentelemetry.api.common.AttributeKey;
import io.opentelemetry.api.trace.Span;
-import io.opentelemetry.api.trace.Span.Kind;
+import io.opentelemetry.api.trace.SpanKind;
import io.opentelemetry.api.trace.StatusCode;
import io.opentelemetry.api.trace.Tracer;
-import io.opentelemetry.api.trace.attributes.SemanticAttributes;
import io.opentelemetry.context.Context;
import io.opentelemetry.context.Scope;
+import io.opentelemetry.semconv.trace.attributes.SemanticAttributes;
import java.io.IOException;
import java.util.List;
import java.util.concurrent.CompletableFuture;
@@ -72,14 +72,14 @@ public final class TraceUtil {
}
/**
- * Create a {@link Kind#INTERNAL} span.
+ * Create a {@link SpanKind#INTERNAL} span.
*/
public static Span createSpan(String name) {
-return createSpan(name, Kind.INTERNAL);
+return createSpan(name, SpanKind.INTERNAL);
}
/**
- * Create a {@link Kind#INTERNAL} span and set table related attributes.
+ * Create a {@link SpanKind#INTERNAL} span and set table related attributes.
*/
public static Span createTableSpan(String
[hbase] 04/18: HBASE-26127 Backport HBASE-23898 "Add trace support for simple apis i… (#3556)
This is an automated email from the ASF dual-hosted git repository.
taklwu pushed a commit to branch branch-2
in repository https://gitbox.apache.org/repos/asf/hbase.git
commit 58b5c007b9aa0ea6687aa116ee8605f33496dc3f
Author: Tak Lon (Stephen) Wu
AuthorDate: Wed Aug 4 18:25:24 2021 -0700
HBASE-26127 Backport HBASE-23898 "Add trace support for simple apis i…
(#3556)
4/17 commits of HBASE-22120, original commit
805b2ae2ad0f6325515d46043ff01e4e2c7a9f59
Co-authored-by: Duo Zhang
Signed-off-by: Duo Zhang
---
hbase-client/pom.xml | 10 +
.../hadoop/hbase/client/AsyncConnection.java | 4 +-
.../hadoop/hbase/client/AsyncConnectionImpl.java | 106 +++---
.../hadoop/hbase/client/AsyncRegionLocator.java| 166 +---
.../org/apache/hadoop/hbase/client/AsyncTable.java | 31 +-
.../hbase/client/AsyncTableRegionLocatorImpl.java | 18 +-
.../hadoop/hbase/client/ConnectionFactory.java | 53 +--
.../hadoop/hbase/client/RawAsyncTableImpl.java | 345 +
.../apache/hadoop/hbase/ipc/AbstractRpcClient.java | 9 +-
.../client/TestAsyncRegionLocatorTracing.java | 157
.../hadoop/hbase/client/TestAsyncTableTracing.java | 417 +
.../org/apache/hadoop/hbase/trace/TraceUtil.java | 134 +++
.../org/apache/hadoop/hbase/ipc/CallRunner.java| 7 +-
.../hadoop/hbase/ipc/ServerRpcConnection.java | 3 +-
.../apache/hadoop/hbase/ipc/AbstractTestIPC.java | 24 +-
pom.xml| 3 +-
16 files changed, 1159 insertions(+), 328 deletions(-)
diff --git a/hbase-client/pom.xml b/hbase-client/pom.xml
index e62a7d1..01a74cc 100644
--- a/hbase-client/pom.xml
+++ b/hbase-client/pom.xml
@@ -157,6 +157,16 @@
joni
+ io.opentelemetry
+ opentelemetry-sdk
+ test
+
+
+ io.opentelemetry
+ opentelemetry-sdk-testing
+ test
+
+
org.slf4j
jcl-over-slf4j
test
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncConnection.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncConnection.java
index 75971ad..d04b5f2 100644
---
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncConnection.java
+++
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncConnection.java
@@ -64,8 +64,8 @@ public interface AsyncConnection extends Closeable {
/**
* Retrieve an {@link AsyncTable} implementation for accessing a table.
*
- * The returned instance will use default configs. Use {@link
#getTableBuilder(TableName)} if
- * you want to customize some configs.
+ * The returned instance will use default configs. Use {@link
#getTableBuilder(TableName)} if you
+ * want to customize some configs.
*
* This method no longer checks table existence. An exception will be thrown
if the table does not
* exist only when the first operation is attempted.
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncConnectionImpl.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncConnectionImpl.java
index 2ed7399..b919ee1 100644
---
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncConnectionImpl.java
+++
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncConnectionImpl.java
@@ -27,6 +27,8 @@ import static
org.apache.hadoop.hbase.client.MetricsConnection.CLIENT_SIDE_METRI
import static
org.apache.hadoop.hbase.client.NonceGenerator.CLIENT_NONCES_ENABLED_KEY;
import static org.apache.hadoop.hbase.util.FutureUtils.addListener;
+import io.opentelemetry.api.trace.Span;
+import io.opentelemetry.context.Scope;
import java.io.IOException;
import java.util.Optional;
import java.util.concurrent.CompletableFuture;
@@ -48,6 +50,7 @@ import org.apache.hadoop.hbase.ipc.RpcClient;
import org.apache.hadoop.hbase.ipc.RpcClientFactory;
import org.apache.hadoop.hbase.ipc.RpcControllerFactory;
import org.apache.hadoop.hbase.security.User;
+import org.apache.hadoop.hbase.trace.TraceUtil;
import org.apache.hadoop.hbase.util.ConcurrentMapUtils;
import org.apache.hadoop.hbase.util.Threads;
import org.apache.hadoop.security.UserGroupInformation;
@@ -153,14 +156,13 @@ class AsyncConnectionImpl implements AsyncConnection {
LOG.warn("{} is true, but {} is not set", STATUS_PUBLISHED,
STATUS_LISTENER_CLASS);
} else {
try {
- listener = new ClusterStatusListener(
-new ClusterStatusListener.DeadServerHandler() {
- @Override
- public void newDead(ServerName sn) {
-locator.clearCache(sn);
-rpcClient.cancelConnections(sn);
- }
-}, conf, listenerClass);
+ listener = new ClusterStatusListener(new
ClusterStatusListener.DeadServerHandler() {
+@Override
+public void newDead(ServerName sn) {
+
[hbase] 07/18: HBASE-26130 Backport HBASE-25455 "Add trace support for HRegion read/… (#3594)
This is an automated email from the ASF dual-hosted git repository.
taklwu pushed a commit to branch branch-2
in repository https://gitbox.apache.org/repos/asf/hbase.git
commit 321dcd4e7746b7660ca0c20c07fbfeed9ca4eb69
Author: Tak Lon (Stephen) Wu
AuthorDate: Tue Aug 17 22:37:41 2021 -0700
HBASE-26130 Backport HBASE-25455 "Add trace support for HRegion read/…
(#3594)
7/17 commits of HBASE-22120, original commit
03e12bfa4ad62ecc6eee6a2c68d431bea2d5c473
Co-authored-by: Duo Zhang
Signed-off-by: Duo Zhang
---
.../hadoop/hbase/client/AsyncRegionLocator.java| 3 +-
.../apache/hadoop/hbase/ipc/AbstractRpcClient.java | 3 +-
.../org/apache/hadoop/hbase/trace/TraceUtil.java | 40 -
.../org/apache/hadoop/hbase/ipc/CallRunner.java| 6 +-
.../org/apache/hadoop/hbase/ipc/ServerCall.java| 4 +-
.../apache/hadoop/hbase/regionserver/HRegion.java | 185 +++-
.../hbase/regionserver/RegionScannerImpl.java | 101 ++-
.../hbase/regionserver/TestAtomicOperation.java| 2 +-
.../hbase/regionserver/TestHRegionTracing.java | 187 +
9 files changed, 393 insertions(+), 138 deletions(-)
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRegionLocator.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRegionLocator.java
index 1d0efcc..3067785 100644
---
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRegionLocator.java
+++
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRegionLocator.java
@@ -102,8 +102,7 @@ class AsyncRegionLocator {
CompletableFuture future = action.get();
FutureUtils.addListener(future, (resp, error) -> {
if (error != null) {
- span.recordException(error);
- span.setStatus(StatusCode.ERROR);
+ TraceUtil.setError(span, error);
} else {
List regionNames = getRegionNames.apply(resp);
if (!regionNames.isEmpty()) {
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/AbstractRpcClient.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/AbstractRpcClient.java
index 878f9cf..fa7dfb1 100644
---
a/hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/AbstractRpcClient.java
+++
b/hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/AbstractRpcClient.java
@@ -424,8 +424,7 @@ public abstract class AbstractRpcClient implements RpcC
onCallFinished(call, hrc, addr, callback);
} finally {
if (hrc.failed()) {
-span.setStatus(StatusCode.ERROR);
-span.recordException(hrc.getFailed());
+TraceUtil.setError(span, hrc.getFailed());
} else {
span.setStatus(StatusCode.OK);
}
diff --git
a/hbase-common/src/main/java/org/apache/hadoop/hbase/trace/TraceUtil.java
b/hbase-common/src/main/java/org/apache/hadoop/hbase/trace/TraceUtil.java
index 8eb2399..ea16df1 100644
--- a/hbase-common/src/main/java/org/apache/hadoop/hbase/trace/TraceUtil.java
+++ b/hbase-common/src/main/java/org/apache/hadoop/hbase/trace/TraceUtil.java
@@ -26,6 +26,7 @@ import io.opentelemetry.api.trace.Tracer;
import io.opentelemetry.api.trace.attributes.SemanticAttributes;
import io.opentelemetry.context.Context;
import io.opentelemetry.context.Scope;
+import java.io.IOException;
import java.util.List;
import java.util.concurrent.CompletableFuture;
import java.util.function.Supplier;
@@ -58,6 +59,9 @@ public final class TraceUtil {
public static final AttributeKey REMOTE_PORT_KEY =
SemanticAttributes.NET_PEER_PORT;
+ public static final AttributeKey ROW_LOCK_READ_LOCK_KEY =
+AttributeKey.booleanKey("db.hbase.rowlock.readlock");
+
private TraceUtil() {
}
@@ -139,14 +143,18 @@ public final class TraceUtil {
}
}
+ public static void setError(Span span, Throwable error) {
+span.recordException(error);
+span.setStatus(StatusCode.ERROR);
+ }
+
/**
* Finish the {@code span} when the given {@code future} is completed.
*/
private static void endSpan(CompletableFuture future, Span span) {
FutureUtils.addListener(future, (resp, error) -> {
if (error != null) {
-span.recordException(error);
-span.setStatus(StatusCode.ERROR);
+setError(span, error);
} else {
span.setStatus(StatusCode.OK);
}
@@ -164,8 +172,32 @@ public final class TraceUtil {
action.run();
span.setStatus(StatusCode.OK);
} catch (Throwable e) {
- span.recordException(e);
- span.setStatus(StatusCode.ERROR);
+ setError(span, e);
+ throw e;
+} finally {
+ span.end();
+}
+ }
+
+ @FunctionalInterface
+ public interface IOExceptionCallable {
+V call() throws IOException;
+ }
+
+ public static T trace(IOExceptionCallable callable, String spanName)
throws IOException {
+return trace(callable,
[hbase] 05/18: HBASE-26128 Backport HBASE-25454 "Add trace support for connection re… (#3561)
This is an automated email from the ASF dual-hosted git repository.
taklwu pushed a commit to branch branch-2
in repository https://gitbox.apache.org/repos/asf/hbase.git
commit 7ecf54f365fadf87f1ab1d90b21f094ce86646c4
Author: Tak Lon (Stephen) Wu
AuthorDate: Thu Aug 12 12:38:53 2021 -0700
HBASE-26128 Backport HBASE-25454 "Add trace support for connection re…
(#3561)
5/17 commits of HBASE-22120, original commit
dcb78bd4bda4a4ae13d863df8aec266031e5bc93 and merged conflicts after rebasing on
HBASE-26150 with commit 63d4970de451bf234f2ddbda949995b1420e525b
Co-authored-by: Duo Zhang
Signed-off-by: Duo Zhang
---
.../client/AbstractRpcBasedConnectionRegistry.java | 52 +-
.../hadoop/hbase/client/ZKConnectionRegistry.java | 36 ---
2 files changed, 53 insertions(+), 35 deletions(-)
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AbstractRpcBasedConnectionRegistry.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AbstractRpcBasedConnectionRegistry.java
index 7d946aa..54138d3 100644
---
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AbstractRpcBasedConnectionRegistry.java
+++
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AbstractRpcBasedConnectionRegistry.java
@@ -17,6 +17,8 @@
*/
package org.apache.hadoop.hbase.client;
+import static org.apache.hadoop.hbase.trace.TraceUtil.trace;
+import static org.apache.hadoop.hbase.trace.TraceUtil.tracedFuture;
import static org.apache.hadoop.hbase.util.FutureUtils.addListener;
import com.google.errorprone.annotations.RestrictedApi;
@@ -237,38 +239,48 @@ abstract class AbstractRpcBasedConnectionRegistry
implements ConnectionRegistry
@Override
public CompletableFuture getMetaRegionLocations() {
-return this
- . call((c, s, d) ->
s.getMetaRegionLocations(c,
-GetMetaRegionLocationsRequest.getDefaultInstance(), d), r ->
r.getMetaLocationsCount() != 0,
+return tracedFuture(
+ () -> this
+. call(
+ (c, s, d) -> s.getMetaRegionLocations(c,
+GetMetaRegionLocationsRequest.getDefaultInstance(), d),
+ r -> r.getMetaLocationsCount() != 0,
"getMetaLocationsCount")
-
.thenApply(AbstractRpcBasedConnectionRegistry::transformMetaRegionLocations);
+
.thenApply(AbstractRpcBasedConnectionRegistry::transformMetaRegionLocations),
+ getClass().getSimpleName() + ".getMetaRegionLocations");
}
@Override
public CompletableFuture getClusterId() {
-return this
- . call(
-(c, s, d) -> s.getClusterId(c,
GetClusterIdRequest.getDefaultInstance(), d),
-GetClusterIdResponse::hasClusterId, "getClusterId()")
- .thenApply(GetClusterIdResponse::getClusterId);
+return tracedFuture(
+ () -> this
+. call(
+ (c, s, d) -> s.getClusterId(c,
GetClusterIdRequest.getDefaultInstance(), d),
+ GetClusterIdResponse::hasClusterId, "getClusterId()")
+.thenApply(GetClusterIdResponse::getClusterId),
+ getClass().getSimpleName() + ".getClusterId");
}
@Override
public CompletableFuture getActiveMaster() {
-return this
- . call(
-(c, s, d) -> s.getActiveMaster(c,
GetActiveMasterRequest.getDefaultInstance(), d),
-GetActiveMasterResponse::hasServerName, "getActiveMaster()")
- .thenApply(resp -> ProtobufUtil.toServerName(resp.getServerName()));
+return tracedFuture(
+ () -> this
+.call(
+ (c, s, d) -> s.getActiveMaster(c,
GetActiveMasterRequest.getDefaultInstance(), d),
+ GetActiveMasterResponse::hasServerName, "getActiveMaster()")
+.thenApply(resp -> ProtobufUtil.toServerName(resp.getServerName())),
+ getClass().getSimpleName() + ".getActiveMaster");
}
@Override
public void close() {
-if (registryEndpointRefresher != null) {
- registryEndpointRefresher.stop();
-}
-if (rpcClient != null) {
- rpcClient.close();
-}
+trace(() -> {
+ if (registryEndpointRefresher != null) {
+registryEndpointRefresher.stop();
+ }
+ if (rpcClient != null) {
+rpcClient.close();
+ }
+}, getClass().getSimpleName() + ".close");
}
}
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ZKConnectionRegistry.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ZKConnectionRegistry.java
index 35107ca..6e94afe 100644
---
a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ZKConnectionRegistry.java
+++
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ZKConnectionRegistry.java
@@ -22,6 +22,7 @@ import static
org.apache.hadoop.hbase.client.RegionInfoBuilder.FIRST_META_REGION
import static
org.apache.hadoop.hbase.client.RegionReplicaUtil.getRegionInfoForDefaultReplica;
import static
org.apache.hadoop.hbase.client.RegionReplicaUtil.getRegionInfoForReplica;
import static
[hbase] 03/18: HBASE-26126 Backport HBASE-25424 "Find a way to config OpenTelemetry tracing without directly depending on opentelemetry-sdk" to branch-2 (#3552)
This is an automated email from the ASF dual-hosted git repository.
taklwu pushed a commit to branch branch-2
in repository https://gitbox.apache.org/repos/asf/hbase.git
commit af460132c09964e1ea2ff858182ef662ee973bc9
Author: Tak Lon (Stephen) Wu
AuthorDate: Tue Aug 3 09:54:37 2021 -0700
HBASE-26126 Backport HBASE-25424 "Find a way to config OpenTelemetry
tracing without directly depending on opentelemetry-sdk" to branch-2 (#3552)
3/17 commits of HBASE-22120, original
commit#57960fa8fa7228d65b1a4adc8e9b5b1a8158824d
Co-authored-by: Duo Zhang
Signed-off-by: Peter Somogyi
---
bin/hbase| 10 ++
conf/hbase-env.sh| 5 +
hbase-assembly/pom.xml | 6 ++
hbase-assembly/src/main/assembly/hadoop-three-compat.xml | 7 +++
pom.xml | 7 +++
5 files changed, 35 insertions(+)
diff --git a/bin/hbase b/bin/hbase
index 496ebe1..5f3b326 100755
--- a/bin/hbase
+++ b/bin/hbase
@@ -487,6 +487,11 @@ add_jdk11_deps_to_classpath() {
done
}
+enable_trace() {
+ agent_jar=$(find lib/trace -type f -name "opentelemetry-javaagent-*")
+ HBASE_OPTS="$HBASE_OPTS -javaagent:$agent_jar $HBASE_TRACE_OPTS"
+}
+
#Add the development env class path stuff
if $in_dev_env; then
add_maven_deps_to_classpath "cached_classpath.txt"
@@ -764,6 +769,11 @@ elif [ "${DEBUG}" = "true" ]; then
echo "JDK11 jars skipped from classpath."
fi
+if [[ -n "${HBASE_TRACE_OPTS}" ]]; then
+ echo "Attach opentelemetry agent to enable trace"
+ enable_trace
+fi
+
# Have JVM dump heap if we run out of memory. Files will be 'launch directory'
# and are named like the following: java_pid21612.hprof. Apparently it doesn't
# 'cost' to have this flag enabled. Its a 1.6 flag only. See:
diff --git a/conf/hbase-env.sh b/conf/hbase-env.sh
index 6cc16d8..b3c4baf 100644
--- a/conf/hbase-env.sh
+++ b/conf/hbase-env.sh
@@ -144,3 +144,8 @@
# Additional argments passed to jshell invocation
# export HBASE_JSHELL_ARGS="--startup DEFAULT --startup PRINTING --startup
hbase_startup.jsh"
+
+# Uncomment to enable trace, you can change the options to use other exporters
such as jaeger or
+# zipkin. See
https://github.com/open-telemetry/opentelemetry-java-instrumentation on how to
config
+# exporters and other components through system properties.
+# export HBASE_TRACE_OPTS="-Dotel.config.sampler.probability=0.1
-Dotel.exporter=logging"
diff --git a/hbase-assembly/pom.xml b/hbase-assembly/pom.xml
index 451e9f3..4ea650d 100644
--- a/hbase-assembly/pom.xml
+++ b/hbase-assembly/pom.xml
@@ -339,6 +339,12 @@
log4j
log4j
+
+
+ io.opentelemetry.javaagent
+ opentelemetry-javaagent
+ all
+
diff --git a/hbase-assembly/src/main/assembly/hadoop-three-compat.xml
b/hbase-assembly/src/main/assembly/hadoop-three-compat.xml
index e5422f3..9780444 100644
--- a/hbase-assembly/src/main/assembly/hadoop-three-compat.xml
+++ b/hbase-assembly/src/main/assembly/hadoop-three-compat.xml
@@ -112,6 +112,7 @@
org.apache.yetus:audience-annotations
org.slf4j:slf4j-api
org.slf4j:slf4j-log4j12
+ io.opentelemetry.javaagent:*
@@ -256,6 +257,12 @@
jakarta.jws:jakarta.jws-api
+
+ lib/trace
+
+io.opentelemetry.javaagent:*
+
+
diff --git a/pom.xml b/pom.xml
index 8cf9b02..95bb712 100755
--- a/pom.xml
+++ b/pom.xml
@@ -1484,6 +1484,7 @@
4.13
1.3
0.13.1
+
0.13.0
1.2.17
2.28.2
@@ -2190,6 +2191,12 @@
${opentelemetry.version}
+io.opentelemetry.javaagent
+opentelemetry-javaagent
+${opentelemetry-instrumentation.version}
+all
+
+
com.lmax
disruptor
${disruptor.version}
[hbase] 01/18: HBASE-26124 Backport HBASE-25373 "Remove HTrace completely in code base and try to make use of OpenTelemetry" to branch-2 (#3529)
This is an automated email from the ASF dual-hosted git repository.
taklwu pushed a commit to branch branch-2
in repository https://gitbox.apache.org/repos/asf/hbase.git
commit 665305cc3b50079cac62d9a55b0c5aefb13d7c67
Author: Tak Lon (Stephen) Wu
AuthorDate: Thu Jul 29 09:15:10 2021 -0700
HBASE-26124 Backport HBASE-25373 "Remove HTrace completely in code base and
try to make use of OpenTelemetry" to branch-2 (#3529)
1/17 commits of HBASE-22120
Signed-off-by: Peter Somogyi
---
.../hadoop/hbase/io/asyncfs/AsyncFSTestBase.java | 3 -
hbase-client/pom.xml | 4 +-
.../hbase/client/AsyncRequestFutureImpl.java | 14 +-
.../client/ResultBoundedCompletionService.java | 4 +-
.../hadoop/hbase/ipc/BlockingRpcConnection.java| 11 +-
.../java/org/apache/hadoop/hbase/ipc/Call.java | 7 +-
hbase-common/pom.xml | 4 +-
.../hbase/trace/HBaseHTraceConfiguration.java | 80 ---
.../hadoop/hbase/trace/SpanReceiverHost.java | 120 -
.../org/apache/hadoop/hbase/trace/TraceUtil.java | 105 +--
hbase-external-blockcache/pom.xml | 4 -
.../hadoop/hbase/io/hfile/MemcachedBlockCache.java | 14 +-
hbase-it/pom.xml | 4 +-
.../IntegrationTestTableMapReduceUtil.java | 1 -
.../hadoop/hbase/mttr/IntegrationTestMTTR.java | 21 +--
.../trace/IntegrationTestSendTraceRequests.java| 135 ++-
hbase-mapreduce/pom.xml| 4 +-
.../hadoop/hbase/mapreduce/TableMapReduceUtil.java | 1 -
.../apache/hadoop/hbase/PerformanceEvaluation.java | 29 ++--
hbase-protocol-shaded/pom.xml | 4 -
hbase-server/pom.xml | 4 +-
.../apache/hadoop/hbase/executor/EventHandler.java | 16 ++-
.../hadoop/hbase/io/hfile/HFileReaderImpl.java | 11 +-
.../org/apache/hadoop/hbase/ipc/CallRunner.java| 31 +++--
.../org/apache/hadoop/hbase/master/HMaster.java| 2 -
.../hadoop/hbase/master/HMasterCommandLine.java| 6 +-
.../apache/hadoop/hbase/regionserver/HRegion.java | 15 ++-
.../hadoop/hbase/regionserver/HRegionServer.java | 9 --
.../regionserver/HRegionServerCommandLine.java | 8 +-
.../hadoop/hbase/regionserver/MemStoreFlusher.java | 14 +-
.../hbase/regionserver/wal/AbstractFSWAL.java | 19 ++-
.../hadoop/hbase/regionserver/wal/AsyncFSWAL.java | 21 +--
.../hadoop/hbase/regionserver/wal/FSHLog.java | 22 +--
.../apache/hadoop/hbase/HBaseTestingUtility.java | 13 +-
.../hadoop/hbase/executor/TestExecutorService.java | 4 +-
.../apache/hadoop/hbase/trace/TestHTraceHooks.java | 134 ---
.../org/apache/hadoop/hbase/trace/TraceTree.java | 148 -
.../hadoop/hbase/wal/WALPerformanceEvaluation.java | 73 +++---
hbase-shaded/hbase-shaded-client/pom.xml | 1 +
hbase-shaded/hbase-shaded-testing-util/pom.xml | 1 +
hbase-shaded/pom.xml | 1 +
hbase-shell/src/main/ruby/shell/commands/trace.rb | 43 +++---
hbase-zookeeper/pom.xml| 4 +
.../hbase/zookeeper/RecoverableZooKeeper.java | 77 ---
pom.xml| 30 -
45 files changed, 362 insertions(+), 914 deletions(-)
diff --git
a/hbase-asyncfs/src/test/java/org/apache/hadoop/hbase/io/asyncfs/AsyncFSTestBase.java
b/hbase-asyncfs/src/test/java/org/apache/hadoop/hbase/io/asyncfs/AsyncFSTestBase.java
index 9b276ac..fc148e8 100644
---
a/hbase-asyncfs/src/test/java/org/apache/hadoop/hbase/io/asyncfs/AsyncFSTestBase.java
+++
b/hbase-asyncfs/src/test/java/org/apache/hadoop/hbase/io/asyncfs/AsyncFSTestBase.java
@@ -19,11 +19,9 @@ package org.apache.hadoop.hbase.io.asyncfs;
import java.io.File;
import java.io.IOException;
-
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.hbase.HBaseCommonTestingUtility;
-import org.apache.hadoop.hbase.trace.TraceUtil;
import org.apache.hadoop.hdfs.MiniDFSCluster;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@@ -104,7 +102,6 @@ public abstract class AsyncFSTestBase {
org.apache.log4j.Logger.getLogger(org.apache.hadoop.metrics2.impl.MetricsSystemImpl.class)
.setLevel(org.apache.log4j.Level.ERROR);
-TraceUtil.initTracer(conf);
CLUSTER = new MiniDFSCluster.Builder(conf).numDataNodes(servers).build();
CLUSTER.waitClusterUp();
}
diff --git a/hbase-client/pom.xml b/hbase-client/pom.xml
index 08e9174..e62a7d1 100644
--- a/hbase-client/pom.xml
+++ b/hbase-client/pom.xml
@@ -145,8 +145,8 @@
zookeeper
- org.apache.htrace
- htrace-core4
+ io.opentelemetry
+ opentelemetry-api
org.jruby.jcodings
diff --git
[hbase] 02/18: HBASE-26125 Backport HBASE-25401 "Add trace support for async call in rpc client" to branch-2 (#3543)
This is an automated email from the ASF dual-hosted git repository.
taklwu pushed a commit to branch branch-2
in repository https://gitbox.apache.org/repos/asf/hbase.git
commit 4dfc59c19e7b380483dc422ec1e503f704430c67
Author: Tak Lon (Stephen) Wu
AuthorDate: Mon Aug 2 11:33:19 2021 -0700
HBASE-26125 Backport HBASE-25401 "Add trace support for async call in rpc
client" to branch-2 (#3543)
2/17 commits of HBASE-22120
Co-authored-by: Duo Zhang
Signed-off-by: Peter Somogyi
---
.../apache/hadoop/hbase/ipc/AbstractRpcClient.java | 75 ---
.../hadoop/hbase/ipc/BlockingRpcConnection.java| 21 +-
.../java/org/apache/hadoop/hbase/ipc/Call.java | 2 +-
.../java/org/apache/hadoop/hbase/ipc/IPCUtil.java | 12 +-
.../hadoop/hbase/ipc/NettyRpcDuplexHandler.java| 8 +-
.../org/apache/hadoop/hbase/trace/TraceUtil.java | 4 +-
.../src/main/protobuf/Tracing.proto| 14 +-
hbase-server/pom.xml | 10 +
.../org/apache/hadoop/hbase/ipc/CallRunner.java| 19 +-
.../org/apache/hadoop/hbase/ipc/ServerCall.java| 14 ++
.../hadoop/hbase/ipc/ServerRpcConnection.java | 222 -
.../apache/hadoop/hbase/ipc/AbstractTestIPC.java | 59 +-
.../org/apache/hadoop/hbase/ipc/TestNettyIPC.java | 1 +
pom.xml| 12 +-
14 files changed, 299 insertions(+), 174 deletions(-)
diff --git
a/hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/AbstractRpcClient.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/AbstractRpcClient.java
index 4bbb729..9117fef 100644
---
a/hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/AbstractRpcClient.java
+++
b/hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/AbstractRpcClient.java
@@ -21,6 +21,9 @@ package org.apache.hadoop.hbase.ipc;
import static org.apache.hadoop.hbase.ipc.IPCUtil.toIOE;
import static org.apache.hadoop.hbase.ipc.IPCUtil.wrapException;
+import io.opentelemetry.api.trace.Span;
+import io.opentelemetry.api.trace.StatusCode;
+import io.opentelemetry.context.Scope;
import java.io.IOException;
import java.net.SocketAddress;
import java.util.Collection;
@@ -38,6 +41,7 @@ import org.apache.hadoop.hbase.codec.KeyValueCodec;
import org.apache.hadoop.hbase.net.Address;
import org.apache.hadoop.hbase.security.User;
import org.apache.hadoop.hbase.security.UserProvider;
+import org.apache.hadoop.hbase.trace.TraceUtil;
import org.apache.hadoop.hbase.util.EnvironmentEdgeManager;
import org.apache.hadoop.hbase.util.PoolMap;
import org.apache.hadoop.hbase.util.Threads;
@@ -365,7 +369,7 @@ public abstract class AbstractRpcClient implements RpcC
protected abstract T createConnection(ConnectionId remoteId) throws
IOException;
private void onCallFinished(Call call, HBaseRpcController hrc, Address addr,
- RpcCallback callback) {
+RpcCallback callback) {
call.callStats.setCallTimeMs(EnvironmentEdgeManager.currentTime() -
call.getStartTime());
if (metrics != null) {
metrics.updateRpc(call.md, call.param, call.callStats);
@@ -388,44 +392,59 @@ public abstract class AbstractRpcClient implements RpcC
}
}
- Call callMethod(final Descriptors.MethodDescriptor md, final
HBaseRpcController hrc,
+ private Call callMethod(final Descriptors.MethodDescriptor md, final
HBaseRpcController hrc,
final Message param, Message returnType, final User ticket,
final Address addr, final RpcCallback callback) {
-final MetricsConnection.CallStats cs = MetricsConnection.newCallStats();
-cs.setStartTime(EnvironmentEdgeManager.currentTime());
-
-if (param instanceof ClientProtos.MultiRequest) {
- ClientProtos.MultiRequest req = (ClientProtos.MultiRequest) param;
- int numActions = 0;
- for (ClientProtos.RegionAction regionAction : req.getRegionActionList())
{
-numActions += regionAction.getActionCount();
- }
+Span span =
TraceUtil.getGlobalTracer().spanBuilder("RpcClient.callMethod." +
md.getFullName())
+ .startSpan();
+try (Scope scope = span.makeCurrent()) {
+ final MetricsConnection.CallStats cs = MetricsConnection.newCallStats();
+ cs.setStartTime(EnvironmentEdgeManager.currentTime());
+
+ if (param instanceof ClientProtos.MultiRequest) {
+ClientProtos.MultiRequest req = (ClientProtos.MultiRequest) param;
+int numActions = 0;
+for (ClientProtos.RegionAction regionAction :
req.getRegionActionList()) {
+ numActions += regionAction.getActionCount();
+}
- cs.setNumActionsPerServer(numActions);
-}
+cs.setNumActionsPerServer(numActions);
+ }
-final AtomicInteger counter = concurrentCounterCache.getUnchecked(addr);
-Call call = new Call(nextCallId(), md, param, hrc.cellScanner(),
returnType,
+ final AtomicInteger counter = concurrentCounterCache.getUnchecked(addr);
+ Call call = new
[hbase] branch branch-2 updated (008ffd2 -> 625093f)
This is an automated email from the ASF dual-hosted git repository. taklwu pushed a change to branch branch-2 in repository https://gitbox.apache.org/repos/asf/hbase.git. from 008ffd2 HBASE-26205 Use specified cluster conf for UserProvider in TableMRUtil#initCredentialsForCluster (#3592) new 665305c HBASE-26124 Backport HBASE-25373 "Remove HTrace completely in code base and try to make use of OpenTelemetry" to branch-2 (#3529) new 4dfc59c HBASE-26125 Backport HBASE-25401 "Add trace support for async call in rpc client" to branch-2 (#3543) new af46013 HBASE-26126 Backport HBASE-25424 "Find a way to config OpenTelemetry tracing without directly depending on opentelemetry-sdk" to branch-2 (#3552) new 58b5c00 HBASE-26127 Backport HBASE-23898 "Add trace support for simple apis i… (#3556) new 7ecf54f HBASE-26128 Backport HBASE-25454 "Add trace support for connection re… (#3561) new 83a2738 HBASE-26129 Backport HBASE-25481 "Add host and port attribute when tracing rpc call at client side" to branch-2 (#3589) new 321dcd4 HBASE-26130 Backport HBASE-25455 "Add trace support for HRegion read/… (#3594) new 9cce94a HBASE-26131 Backport HBASE-25484 "Add trace support for WAL sync" to branch-2 (#3597) new 8b7450f HBASE-26132 Backport HBASE-25535 "Set span kind to CLIENT in AbstractRpcClient" to branch-2 (#3607) new d0a53e3 HBASE-26133 Backport HBASE-25591 "Upgrade opentelemetry to 0.17.1" to branch-2 (#3608) new a2b8f7a HBASE-26134 Backport HBASE-25617 "Revisit the span names" to branch-2 (#3618) new 8f16010 HBASE-26135 Backport HBASE-25616 "Upgrade opentelemetry to 1.0.0" to branch-2 (#3620) new c639d6f HBASE-26136 Backport HBASE-25723 "Temporarily remove the trace support for RegionScanner.next" to branch-2 (#3623) new dd1bdc8 HBASE-26137 Backport HBASE-25732 "Change the command line argument for tracing after upgrading opentelemtry to 1.0.0" to branch-2 (#3624) new a746e19 HBASE-26138 Backport HBASE-25733 "Upgrade opentelemetry to 1.0.1" to branch-2 (#3628) new 65b5b9b HBASE-26139 Backport HBASE-23762 "Add documentation on how to enable and view tracing with OpenTelemetry" to branch-2 (#3629) new ab431fc HBASE-26140 Backport HBASE-25778 "The tracinig implementation for AsyncConnectionImpl.getHbck is incorrect" to branch-2 (#3631) new 625093f HBASE-26168 Backport HBASE-25811 "The client integration test is failing after HBASE-22120 merged" into branch-2 (#3634) The 18 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: bin/hbase | 10 + conf/hbase-env.cmd | 5 + conf/hbase-env.sh | 5 + hbase-assembly/pom.xml | 6 + hbase-assembly/src/main/assembly/client.xml| 8 + .../src/main/assembly/hadoop-three-compat.xml | 8 + .../hadoop/hbase/io/asyncfs/AsyncFSTestBase.java | 3 - hbase-client/pom.xml | 14 +- .../client/AbstractRpcBasedConnectionRegistry.java | 52 ++- .../hadoop/hbase/client/AsyncConnection.java | 4 +- .../hadoop/hbase/client/AsyncConnectionImpl.java | 109 +++--- .../hadoop/hbase/client/AsyncRegionLocator.java| 165 +--- .../hbase/client/AsyncRequestFutureImpl.java | 14 +- .../org/apache/hadoop/hbase/client/AsyncTable.java | 31 +- .../hbase/client/AsyncTableRegionLocatorImpl.java | 18 +- .../hadoop/hbase/client/ConnectionFactory.java | 53 +-- .../hadoop/hbase/client/RawAsyncTableImpl.java | 345 + .../client/ResultBoundedCompletionService.java | 4 +- .../hadoop/hbase/client/ZKConnectionRegistry.java | 36 +- .../apache/hadoop/hbase/ipc/AbstractRpcClient.java | 81 ++-- .../hadoop/hbase/ipc/BlockingRpcConnection.java| 18 +- .../java/org/apache/hadoop/hbase/ipc/Call.java | 9 +- .../java/org/apache/hadoop/hbase/ipc/IPCUtil.java | 12 +- .../hadoop/hbase/ipc/NettyRpcDuplexHandler.java| 8 +- .../hbase/client/TestAsyncConnectionTracing.java | 112 ++ .../client/TestAsyncRegionLocatorTracing.java | 157 .../hadoop/hbase/client/TestAsyncTableTracing.java | 417 + hbase-common/pom.xml | 8 +- .../hbase/trace/HBaseHTraceConfiguration.java | 80 .../hadoop/hbase/trace/SpanReceiverHost.java | 120 -- .../org/apache/hadoop/hbase/trace/TraceUtil.java | 231 hbase-external-blockcache/pom.xml | 4 - .../hadoop/hbase/io/hfile/MemcachedBlockCache.java | 14 +- hbase-it/pom.xml | 4 +- .../IntegrationTestTableMapReduceUtil.java | 1 -
[hbase-site] branch asf-site updated: INFRA-10751 Empty commit
This is an automated email from the ASF dual-hosted git repository. git-site-role pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/hbase-site.git The following commit(s) were added to refs/heads/asf-site by this push: new 724d7b4 INFRA-10751 Empty commit 724d7b4 is described below commit 724d7b494dbdeca8a46e2b9d1d6fd3144b92b5b9 Author: jenkins AuthorDate: Wed Sep 1 20:17:37 2021 + INFRA-10751 Empty commit
[hbase] branch branch-2 updated (dbf43dc -> 008ffd2)
This is an automated email from the ASF dual-hosted git repository. zhangduo pushed a change to branch branch-2 in repository https://gitbox.apache.org/repos/asf/hbase.git. from dbf43dc HBASE-26210 HBase Write should be doomed to hang when cell size exceeds InmemoryFlushSize for CompactingMemStore (#3604) add 008ffd2 HBASE-26205 Use specified cluster conf for UserProvider in TableMRUtil#initCredentialsForCluster (#3592) No new revisions were added by this update. Summary of changes: .../hadoop/hbase/mapreduce/TableMapReduceUtil.java | 2 +- .../hbase/mapreduce/TestTableMapReduceUtil.java| 207 +++-- 2 files changed, 196 insertions(+), 13 deletions(-)
[hbase] branch branch-2.4 updated: HBASE-26205 Use specified cluster conf for UserProvider in TableMRUtil#initCredentialsForCluster (#3592)
This is an automated email from the ASF dual-hosted git repository.
zhangduo pushed a commit to branch branch-2.4
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/branch-2.4 by this push:
new 875f3af HBASE-26205 Use specified cluster conf for UserProvider in
TableMRUtil#initCredentialsForCluster (#3592)
875f3af is described below
commit 875f3afc2ace7cc635dc04303cd50cadde770750
Author: bitterfox
AuthorDate: Wed Sep 1 21:30:41 2021 +0900
HBASE-26205 Use specified cluster conf for UserProvider in
TableMRUtil#initCredentialsForCluster (#3592)
Signed-off-by: Duo Zhang
Reviewed-by: Rushabh Shah
---
.../hadoop/hbase/mapreduce/TableMapReduceUtil.java | 2 +-
.../hbase/mapreduce/TestTableMapReduceUtil.java| 207 +++--
2 files changed, 196 insertions(+), 13 deletions(-)
diff --git
a/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java
b/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java
index fe8031c..b844fa4 100644
---
a/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java
+++
b/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java
@@ -585,7 +585,7 @@ public class TableMapReduceUtil {
*/
public static void initCredentialsForCluster(Job job, Configuration conf)
throws IOException {
-UserProvider userProvider =
UserProvider.instantiate(job.getConfiguration());
+UserProvider userProvider = UserProvider.instantiate(conf);
if (userProvider.isHBaseSecurityEnabled()) {
try {
Connection peerConn = ConnectionFactory.createConnection(conf);
diff --git
a/hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduceUtil.java
b/hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduceUtil.java
index 54415df..ae2450f 100644
---
a/hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduceUtil.java
+++
b/hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduceUtil.java
@@ -17,18 +17,40 @@
*/
package org.apache.hadoop.hbase.mapreduce;
+import static
org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertNull;
+import static org.junit.Assert.assertTrue;
+import java.io.Closeable;
+import java.io.File;
+import java.util.Collection;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.HBaseClassTestRule;
+import org.apache.hadoop.hbase.HBaseTestingUtility;
import org.apache.hadoop.hbase.client.Scan;
+import org.apache.hadoop.hbase.coprocessor.CoprocessorHost;
+import org.apache.hadoop.hbase.security.HBaseKerberosUtils;
+import org.apache.hadoop.hbase.security.access.AccessController;
+import org.apache.hadoop.hbase.security.access.PermissionStorage;
+import org.apache.hadoop.hbase.security.access.SecureTestUtil;
+import
org.apache.hadoop.hbase.security.provider.SaslClientAuthenticationProviders;
+import org.apache.hadoop.hbase.security.token.AuthenticationTokenIdentifier;
+import org.apache.hadoop.hbase.security.token.TokenProvider;
+import org.apache.hadoop.hbase.security.visibility.VisibilityTestUtil;
import org.apache.hadoop.hbase.testclassification.MapReduceTests;
import org.apache.hadoop.hbase.testclassification.MediumTests;
import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.hadoop.hbase.zookeeper.ZKClusterId;
import org.apache.hadoop.io.LongWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Job;
+import org.apache.hadoop.minikdc.MiniKdc;
+import org.apache.hadoop.security.Credentials;
+import org.apache.hadoop.security.UserGroupInformation;
+import org.apache.hadoop.security.token.Token;
+import org.apache.hadoop.security.token.TokenIdentifier;
+import org.junit.After;
import org.junit.ClassRule;
import org.junit.Test;
import org.junit.experimental.categories.Category;
@@ -38,11 +60,17 @@ import org.junit.experimental.categories.Category;
*/
@Category({MapReduceTests.class, MediumTests.class})
public class TestTableMapReduceUtil {
+ private static final String HTTP_PRINCIPAL = "HTTP/localhost";
@ClassRule
public static final HBaseClassTestRule CLASS_RULE =
HBaseClassTestRule.forClass(TestTableMapReduceUtil.class);
+ @After
+ public void after() {
+SaslClientAuthenticationProviders.reset();
+ }
+
/*
* initTableSnapshotMapperJob is tested in {@link
TestTableSnapshotInputFormat} because
* the method depends on an online cluster.
@@ -51,10 +79,11 @@ public class TestTableMapReduceUtil {
@Test
public void testInitTableMapperJob1() throws Exception {
Configuration configuration = new Configuration();
-Job job = new Job(configuration, "tableName");
+Job job = Job.getInstance(configuration,
[hbase] branch branch-2.3 updated: HBASE-26205 Use specified cluster conf for UserProvider in TableMRUtil#initCredentialsForCluster (#3592)
This is an automated email from the ASF dual-hosted git repository.
zhangduo pushed a commit to branch branch-2.3
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/branch-2.3 by this push:
new 6823a3d HBASE-26205 Use specified cluster conf for UserProvider in
TableMRUtil#initCredentialsForCluster (#3592)
6823a3d is described below
commit 6823a3d81fdf0f7ffe7f1bb8d2c72fa2c8dffe56
Author: bitterfox
AuthorDate: Wed Sep 1 21:30:41 2021 +0900
HBASE-26205 Use specified cluster conf for UserProvider in
TableMRUtil#initCredentialsForCluster (#3592)
Signed-off-by: Duo Zhang
Reviewed-by: Rushabh Shah
---
.../hadoop/hbase/mapreduce/TableMapReduceUtil.java | 2 +-
.../hbase/mapreduce/TestTableMapReduceUtil.java| 207 +++--
2 files changed, 196 insertions(+), 13 deletions(-)
diff --git
a/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java
b/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java
index 3326f6d..3d4329d 100644
---
a/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java
+++
b/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java
@@ -585,7 +585,7 @@ public class TableMapReduceUtil {
*/
public static void initCredentialsForCluster(Job job, Configuration conf)
throws IOException {
-UserProvider userProvider =
UserProvider.instantiate(job.getConfiguration());
+UserProvider userProvider = UserProvider.instantiate(conf);
if (userProvider.isHBaseSecurityEnabled()) {
try {
Connection peerConn = ConnectionFactory.createConnection(conf);
diff --git
a/hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduceUtil.java
b/hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduceUtil.java
index 54415df..ae2450f 100644
---
a/hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduceUtil.java
+++
b/hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduceUtil.java
@@ -17,18 +17,40 @@
*/
package org.apache.hadoop.hbase.mapreduce;
+import static
org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertNull;
+import static org.junit.Assert.assertTrue;
+import java.io.Closeable;
+import java.io.File;
+import java.util.Collection;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.HBaseClassTestRule;
+import org.apache.hadoop.hbase.HBaseTestingUtility;
import org.apache.hadoop.hbase.client.Scan;
+import org.apache.hadoop.hbase.coprocessor.CoprocessorHost;
+import org.apache.hadoop.hbase.security.HBaseKerberosUtils;
+import org.apache.hadoop.hbase.security.access.AccessController;
+import org.apache.hadoop.hbase.security.access.PermissionStorage;
+import org.apache.hadoop.hbase.security.access.SecureTestUtil;
+import
org.apache.hadoop.hbase.security.provider.SaslClientAuthenticationProviders;
+import org.apache.hadoop.hbase.security.token.AuthenticationTokenIdentifier;
+import org.apache.hadoop.hbase.security.token.TokenProvider;
+import org.apache.hadoop.hbase.security.visibility.VisibilityTestUtil;
import org.apache.hadoop.hbase.testclassification.MapReduceTests;
import org.apache.hadoop.hbase.testclassification.MediumTests;
import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.hadoop.hbase.zookeeper.ZKClusterId;
import org.apache.hadoop.io.LongWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Job;
+import org.apache.hadoop.minikdc.MiniKdc;
+import org.apache.hadoop.security.Credentials;
+import org.apache.hadoop.security.UserGroupInformation;
+import org.apache.hadoop.security.token.Token;
+import org.apache.hadoop.security.token.TokenIdentifier;
+import org.junit.After;
import org.junit.ClassRule;
import org.junit.Test;
import org.junit.experimental.categories.Category;
@@ -38,11 +60,17 @@ import org.junit.experimental.categories.Category;
*/
@Category({MapReduceTests.class, MediumTests.class})
public class TestTableMapReduceUtil {
+ private static final String HTTP_PRINCIPAL = "HTTP/localhost";
@ClassRule
public static final HBaseClassTestRule CLASS_RULE =
HBaseClassTestRule.forClass(TestTableMapReduceUtil.class);
+ @After
+ public void after() {
+SaslClientAuthenticationProviders.reset();
+ }
+
/*
* initTableSnapshotMapperJob is tested in {@link
TestTableSnapshotInputFormat} because
* the method depends on an online cluster.
@@ -51,10 +79,11 @@ public class TestTableMapReduceUtil {
@Test
public void testInitTableMapperJob1() throws Exception {
Configuration configuration = new Configuration();
-Job job = new Job(configuration, "tableName");
+Job job = Job.getInstance(configuration,
[hbase] branch master updated (345d725 -> f62caa5)
This is an automated email from the ASF dual-hosted git repository. zhangduo pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hbase.git. from 345d725 HBASE-25773 TestSnapshotScannerHDFSAclController.setupBeforeClass is flaky (#3651) add f62caa5 HBASE-26205 Use specified cluster conf for UserProvider in TableMRUtil#initCredentialsForCluster (#3592) No new revisions were added by this update. Summary of changes: .../hadoop/hbase/mapreduce/TableMapReduceUtil.java | 2 +- .../hbase/mapreduce/TestTableMapReduceUtil.java| 208 +++-- 2 files changed, 196 insertions(+), 14 deletions(-)
[hbase] branch master updated (36884c3 -> 345d725)
This is an automated email from the ASF dual-hosted git repository. sunxin pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hbase.git. from 36884c3 HBASE-26210 HBase Write should be doomed to hang when cell size exceeds InmemoryFlushSize for CompactingMemStore (#3604) add 345d725 HBASE-25773 TestSnapshotScannerHDFSAclController.setupBeforeClass is flaky (#3651) No new revisions were added by this update. Summary of changes: .../security/access/TestSnapshotScannerHDFSAclController.java | 11 --- .../access/TestSnapshotScannerHDFSAclController2.java | 11 --- 2 files changed, 8 insertions(+), 14 deletions(-)
[hbase] branch branch-2 updated: HBASE-26210 HBase Write should be doomed to hang when cell size exceeds InmemoryFlushSize for CompactingMemStore (#3604)
This is an automated email from the ASF dual-hosted git repository.
zhangduo pushed a commit to branch branch-2
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/branch-2 by this push:
new dbf43dc HBASE-26210 HBase Write should be doomed to hang when cell
size exceeds InmemoryFlushSize for CompactingMemStore (#3604)
dbf43dc is described below
commit dbf43dcfbd83a673c3844da9fca751e74ea2f805
Author: chenglei
AuthorDate: Wed Sep 1 15:50:51 2021 +0800
HBASE-26210 HBase Write should be doomed to hang when cell size exceeds
InmemoryFlushSize for CompactingMemStore (#3604)
Signed-off-by: Duo Zhang
---
.../hbase/regionserver/CompactingMemStore.java | 92 +++
.../hbase/regionserver/CompactionPipeline.java | 2 +-
.../TestCompactingToCellFlatMapMemStore.java | 6 +-
.../hadoop/hbase/regionserver/TestHStore.java | 267 +++--
4 files changed, 303 insertions(+), 64 deletions(-)
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactingMemStore.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactingMemStore.java
index 973d5a4..5da0de9 100644
---
a/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactingMemStore.java
+++
b/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactingMemStore.java
@@ -208,7 +208,7 @@ public class CompactingMemStore extends AbstractMemStore {
stopCompaction();
// region level lock ensures pushing active to pipeline is done in
isolation
// no concurrent update operations trying to flush the active segment
- pushActiveToPipeline(getActive());
+ pushActiveToPipeline(getActive(), true);
resetTimeOfOldestEdit();
snapshotId = EnvironmentEdgeManager.currentTime();
// in both cases whatever is pushed to snapshot is cleared from the
pipeline
@@ -413,33 +413,61 @@ public class CompactingMemStore extends AbstractMemStore {
}
/**
- * Check whether anything need to be done based on the current active set
size.
- * The method is invoked upon every addition to the active set.
- * For CompactingMemStore, flush the active set to the read-only memory if
it's
- * size is above threshold
+ * Check whether anything need to be done based on the current active set
size. The method is
+ * invoked upon every addition to the active set. For CompactingMemStore,
flush the active set to
+ * the read-only memory if it's size is above threshold
* @param currActive intended segment to update
* @param cellToAdd cell to be added to the segment
* @param memstoreSizing object to accumulate changed size
- * @return true if the cell can be added to the
+ * @return true if the cell can be added to the currActive
*/
- private boolean checkAndAddToActiveSize(MutableSegment currActive, Cell
cellToAdd,
+ protected boolean checkAndAddToActiveSize(MutableSegment currActive, Cell
cellToAdd,
MemStoreSizing memstoreSizing) {
-if (shouldFlushInMemory(currActive, cellToAdd, memstoreSizing)) {
- if (currActive.setInMemoryFlushed()) {
-flushInMemory(currActive);
-if (setInMemoryCompactionFlag()) {
- // The thread is dispatched to do in-memory compaction in the
background
- InMemoryCompactionRunnable runnable = new
InMemoryCompactionRunnable();
- if (LOG.isTraceEnabled()) {
-LOG.trace("Dispatching the MemStore in-memory flush for store " +
store
-.getColumnFamilyName());
- }
- getPool().execute(runnable);
+long cellSize = MutableSegment.getCellLength(cellToAdd);
+boolean successAdd = false;
+while (true) {
+ long segmentDataSize = currActive.getDataSize();
+ if (!inWalReplay && segmentDataSize > inmemoryFlushSize) {
+// when replaying edits from WAL there is no need in in-memory flush
regardless the size
+// otherwise size below flush threshold try to update atomically
+break;
+ }
+ if (currActive.compareAndSetDataSize(segmentDataSize, segmentDataSize +
cellSize)) {
+if (memstoreSizing != null) {
+ memstoreSizing.incMemStoreSize(cellSize, 0, 0, 0);
}
+successAdd = true;
+break;
+ }
+}
+
+if (!inWalReplay && currActive.getDataSize() > inmemoryFlushSize) {
+ // size above flush threshold so we flush in memory
+ this.tryFlushInMemoryAndCompactingAsync(currActive);
+}
+return successAdd;
+ }
+
+ /**
+ * Try to flush the currActive in memory and submit the background
+ * {@link InMemoryCompactionRunnable} to
+ * {@link RegionServicesForStores#getInMemoryCompactionPool()}. Just one
thread can do the actual
+ * flushing in memory.
+ * @param currActive current Active Segment to be flush in memory.
+ */
+ private void tryFlushInMemoryAndCompactingAsync(MutableSegment
[hbase] branch branch-2.4 updated: HBASE-26210 HBase Write should be doomed to hang when cell size exceeds InmemoryFlushSize for CompactingMemStore (#3604)
This is an automated email from the ASF dual-hosted git repository.
zhangduo pushed a commit to branch branch-2.4
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/branch-2.4 by this push:
new cf4c764 HBASE-26210 HBase Write should be doomed to hang when cell
size exceeds InmemoryFlushSize for CompactingMemStore (#3604)
cf4c764 is described below
commit cf4c764d164c277fc663d9f824fe701ae7295473
Author: chenglei
AuthorDate: Wed Sep 1 15:50:51 2021 +0800
HBASE-26210 HBase Write should be doomed to hang when cell size exceeds
InmemoryFlushSize for CompactingMemStore (#3604)
Signed-off-by: Duo Zhang
---
.../hbase/regionserver/CompactingMemStore.java | 92 +++
.../hbase/regionserver/CompactionPipeline.java | 2 +-
.../TestCompactingToCellFlatMapMemStore.java | 6 +-
.../hadoop/hbase/regionserver/TestHStore.java | 267 +++--
4 files changed, 303 insertions(+), 64 deletions(-)
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactingMemStore.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactingMemStore.java
index 973d5a4..5da0de9 100644
---
a/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactingMemStore.java
+++
b/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactingMemStore.java
@@ -208,7 +208,7 @@ public class CompactingMemStore extends AbstractMemStore {
stopCompaction();
// region level lock ensures pushing active to pipeline is done in
isolation
// no concurrent update operations trying to flush the active segment
- pushActiveToPipeline(getActive());
+ pushActiveToPipeline(getActive(), true);
resetTimeOfOldestEdit();
snapshotId = EnvironmentEdgeManager.currentTime();
// in both cases whatever is pushed to snapshot is cleared from the
pipeline
@@ -413,33 +413,61 @@ public class CompactingMemStore extends AbstractMemStore {
}
/**
- * Check whether anything need to be done based on the current active set
size.
- * The method is invoked upon every addition to the active set.
- * For CompactingMemStore, flush the active set to the read-only memory if
it's
- * size is above threshold
+ * Check whether anything need to be done based on the current active set
size. The method is
+ * invoked upon every addition to the active set. For CompactingMemStore,
flush the active set to
+ * the read-only memory if it's size is above threshold
* @param currActive intended segment to update
* @param cellToAdd cell to be added to the segment
* @param memstoreSizing object to accumulate changed size
- * @return true if the cell can be added to the
+ * @return true if the cell can be added to the currActive
*/
- private boolean checkAndAddToActiveSize(MutableSegment currActive, Cell
cellToAdd,
+ protected boolean checkAndAddToActiveSize(MutableSegment currActive, Cell
cellToAdd,
MemStoreSizing memstoreSizing) {
-if (shouldFlushInMemory(currActive, cellToAdd, memstoreSizing)) {
- if (currActive.setInMemoryFlushed()) {
-flushInMemory(currActive);
-if (setInMemoryCompactionFlag()) {
- // The thread is dispatched to do in-memory compaction in the
background
- InMemoryCompactionRunnable runnable = new
InMemoryCompactionRunnable();
- if (LOG.isTraceEnabled()) {
-LOG.trace("Dispatching the MemStore in-memory flush for store " +
store
-.getColumnFamilyName());
- }
- getPool().execute(runnable);
+long cellSize = MutableSegment.getCellLength(cellToAdd);
+boolean successAdd = false;
+while (true) {
+ long segmentDataSize = currActive.getDataSize();
+ if (!inWalReplay && segmentDataSize > inmemoryFlushSize) {
+// when replaying edits from WAL there is no need in in-memory flush
regardless the size
+// otherwise size below flush threshold try to update atomically
+break;
+ }
+ if (currActive.compareAndSetDataSize(segmentDataSize, segmentDataSize +
cellSize)) {
+if (memstoreSizing != null) {
+ memstoreSizing.incMemStoreSize(cellSize, 0, 0, 0);
}
+successAdd = true;
+break;
+ }
+}
+
+if (!inWalReplay && currActive.getDataSize() > inmemoryFlushSize) {
+ // size above flush threshold so we flush in memory
+ this.tryFlushInMemoryAndCompactingAsync(currActive);
+}
+return successAdd;
+ }
+
+ /**
+ * Try to flush the currActive in memory and submit the background
+ * {@link InMemoryCompactionRunnable} to
+ * {@link RegionServicesForStores#getInMemoryCompactionPool()}. Just one
thread can do the actual
+ * flushing in memory.
+ * @param currActive current Active Segment to be flush in memory.
+ */
+ private void tryFlushInMemoryAndCompactingAsync(MutableSegment
[hbase] branch branch-2.3 updated: HBASE-26210 HBase Write should be doomed to hang when cell size exceeds InmemoryFlushSize for CompactingMemStore (#3604)
This is an automated email from the ASF dual-hosted git repository.
zhangduo pushed a commit to branch branch-2.3
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/branch-2.3 by this push:
new ce4974f HBASE-26210 HBase Write should be doomed to hang when cell
size exceeds InmemoryFlushSize for CompactingMemStore (#3604)
ce4974f is described below
commit ce4974f313296e8515701af2c738779376f2d5bb
Author: chenglei
AuthorDate: Wed Sep 1 15:50:51 2021 +0800
HBASE-26210 HBase Write should be doomed to hang when cell size exceeds
InmemoryFlushSize for CompactingMemStore (#3604)
Signed-off-by: Duo Zhang
---
.../hbase/regionserver/CompactingMemStore.java | 93 +++
.../hbase/regionserver/CompactionPipeline.java | 2 +-
.../TestCompactingToCellFlatMapMemStore.java | 6 +-
.../hadoop/hbase/regionserver/TestHStore.java | 267 +++--
4 files changed, 303 insertions(+), 65 deletions(-)
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactingMemStore.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactingMemStore.java
index a8fee3e..e20c924 100644
---
a/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactingMemStore.java
+++
b/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactingMemStore.java
@@ -211,7 +211,7 @@ public class CompactingMemStore extends AbstractMemStore {
stopCompaction();
// region level lock ensures pushing active to pipeline is done in
isolation
// no concurrent update operations trying to flush the active segment
- pushActiveToPipeline(getActive());
+ pushActiveToPipeline(getActive(), true);
resetTimeOfOldestEdit();
snapshotId = EnvironmentEdgeManager.currentTime();
// in both cases whatever is pushed to snapshot is cleared from the
pipeline
@@ -419,33 +419,61 @@ public class CompactingMemStore extends AbstractMemStore {
}
/**
- * Check whether anything need to be done based on the current active set
size.
- * The method is invoked upon every addition to the active set.
- * For CompactingMemStore, flush the active set to the read-only memory if
it's
- * size is above threshold
+ * Check whether anything need to be done based on the current active set
size. The method is
+ * invoked upon every addition to the active set. For CompactingMemStore,
flush the active set to
+ * the read-only memory if it's size is above threshold
* @param currActive intended segment to update
* @param cellToAdd cell to be added to the segment
* @param memstoreSizing object to accumulate changed size
- * @return true if the cell can be added to the
+ * @return true if the cell can be added to the currActive
*/
- private boolean checkAndAddToActiveSize(MutableSegment currActive, Cell
cellToAdd,
+ protected boolean checkAndAddToActiveSize(MutableSegment currActive, Cell
cellToAdd,
MemStoreSizing memstoreSizing) {
-if (shouldFlushInMemory(currActive, cellToAdd, memstoreSizing)) {
- if (currActive.setInMemoryFlushed()) {
-flushInMemory(currActive);
-if (setInMemoryCompactionFlag()) {
- // The thread is dispatched to do in-memory compaction in the
background
- InMemoryCompactionRunnable runnable = new
InMemoryCompactionRunnable();
- if (LOG.isTraceEnabled()) {
-LOG.trace("Dispatching the MemStore in-memory flush for store " +
store
-.getColumnFamilyName());
- }
- getPool().execute(runnable);
+long cellSize = MutableSegment.getCellLength(cellToAdd);
+boolean successAdd = false;
+while (true) {
+ long segmentDataSize = currActive.getDataSize();
+ if (!inWalReplay && segmentDataSize > inmemoryFlushSize) {
+// when replaying edits from WAL there is no need in in-memory flush
regardless the size
+// otherwise size below flush threshold try to update atomically
+break;
+ }
+ if (currActive.compareAndSetDataSize(segmentDataSize, segmentDataSize +
cellSize)) {
+if (memstoreSizing != null) {
+ memstoreSizing.incMemStoreSize(cellSize, 0, 0, 0);
}
+successAdd = true;
+break;
+ }
+}
+
+if (!inWalReplay && currActive.getDataSize() > inmemoryFlushSize) {
+ // size above flush threshold so we flush in memory
+ this.tryFlushInMemoryAndCompactingAsync(currActive);
+}
+return successAdd;
+ }
+
+ /**
+ * Try to flush the currActive in memory and submit the background
+ * {@link InMemoryCompactionRunnable} to
+ * {@link RegionServicesForStores#getInMemoryCompactionPool()}. Just one
thread can do the actual
+ * flushing in memory.
+ * @param currActive current Active Segment to be flush in memory.
+ */
+ private void tryFlushInMemoryAndCompactingAsync(MutableSegment
[hbase] branch master updated (3c64f86 -> 36884c3)
This is an automated email from the ASF dual-hosted git repository. zhangduo pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hbase.git. from 3c64f86 HBASE-26244 Avoid trim the error stack trace when running UT with maven (#3654) add 36884c3 HBASE-26210 HBase Write should be doomed to hang when cell size exceeds InmemoryFlushSize for CompactingMemStore (#3604) No new revisions were added by this update. Summary of changes: .../hbase/regionserver/CompactingMemStore.java | 94 .../hbase/regionserver/CompactionPipeline.java | 2 +- .../TestCompactingToCellFlatMapMemStore.java | 6 +- .../hadoop/hbase/regionserver/TestHStore.java | 267 +++-- 4 files changed, 304 insertions(+), 65 deletions(-)
[hbase] branch branch-1.4 updated: HBASE-26244 Avoid trim the error stack trace when running UT with maven (#3654)
This is an automated email from the ASF dual-hosted git repository.
zhangduo pushed a commit to branch branch-1.4
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/branch-1.4 by this push:
new a6da5fa HBASE-26244 Avoid trim the error stack trace when running UT
with maven (#3654)
a6da5fa is described below
commit a6da5fa0a25d618ff08c1b1612cf60abee74bb31
Author: chenglei
AuthorDate: Wed Sep 1 15:17:05 2021 +0800
HBASE-26244 Avoid trim the error stack trace when running UT with maven
(#3654)
Signed-off-by: Geoffrey Jacoby
Signed-off-by: Duo Zhang
---
pom.xml | 1 +
1 file changed, 1 insertion(+)
diff --git a/pom.xml b/pom.xml
index 5bc05e8..3d90b69 100644
--- a/pom.xml
+++ b/pom.xml
@@ -523,6 +523,7 @@
false
+false
${surefire.skipFirstPart}
${surefire.firstPartForkCount}
false
[hbase] branch branch-1 updated: HBASE-26244 Avoid trim the error stack trace when running UT with maven (#3654)
This is an automated email from the ASF dual-hosted git repository.
zhangduo pushed a commit to branch branch-1
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/branch-1 by this push:
new 08ff55d HBASE-26244 Avoid trim the error stack trace when running UT
with maven (#3654)
08ff55d is described below
commit 08ff55df7105d5f5f3cbad48f4623e5e2a7e9721
Author: chenglei
AuthorDate: Wed Sep 1 15:17:05 2021 +0800
HBASE-26244 Avoid trim the error stack trace when running UT with maven
(#3654)
Signed-off-by: Geoffrey Jacoby
Signed-off-by: Duo Zhang
---
pom.xml | 1 +
1 file changed, 1 insertion(+)
diff --git a/pom.xml b/pom.xml
index fab0019..8a266e8 100644
--- a/pom.xml
+++ b/pom.xml
@@ -482,6 +482,7 @@
false
+false
${surefire.skipFirstPart}
${surefire.firstPartForkCount}
false
[hbase] branch branch-2.3 updated: HBASE-26244 Avoid trim the error stack trace when running UT with maven (#3654)
This is an automated email from the ASF dual-hosted git repository.
zhangduo pushed a commit to branch branch-2.3
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/branch-2.3 by this push:
new 8bc971c HBASE-26244 Avoid trim the error stack trace when running UT
with maven (#3654)
8bc971c is described below
commit 8bc971cb914d10363035acbc700434a479ef28b4
Author: chenglei
AuthorDate: Wed Sep 1 15:17:05 2021 +0800
HBASE-26244 Avoid trim the error stack trace when running UT with maven
(#3654)
Signed-off-by: Geoffrey Jacoby
Signed-off-by: Duo Zhang
---
pom.xml | 1 +
1 file changed, 1 insertion(+)
diff --git a/pom.xml b/pom.xml
index aa03fbd..d15489d 100755
--- a/pom.xml
+++ b/pom.xml
@@ -576,6 +576,7 @@
${surefire.firstPartGroups}
false
+false
${surefire.skipFirstPart}
${surefire.firstPartForkCount}
false
[hbase] branch branch-2.4 updated: HBASE-26244 Avoid trim the error stack trace when running UT with maven (#3654)
This is an automated email from the ASF dual-hosted git repository.
zhangduo pushed a commit to branch branch-2.4
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/branch-2.4 by this push:
new 8085e4c HBASE-26244 Avoid trim the error stack trace when running UT
with maven (#3654)
8085e4c is described below
commit 8085e4c5de030c8a18261545c44024d339513258
Author: chenglei
AuthorDate: Wed Sep 1 15:17:05 2021 +0800
HBASE-26244 Avoid trim the error stack trace when running UT with maven
(#3654)
Signed-off-by: Geoffrey Jacoby
Signed-off-by: Duo Zhang
---
pom.xml | 1 +
1 file changed, 1 insertion(+)
diff --git a/pom.xml b/pom.xml
index 02b89ed..479517b 100755
--- a/pom.xml
+++ b/pom.xml
@@ -584,6 +584,7 @@
${surefire.firstPartGroups}
false
+false
${surefire.skipFirstPart}
${surefire.firstPartForkCount}
false
[hbase] branch branch-2 updated: HBASE-26244 Avoid trim the error stack trace when running UT with maven (#3654)
This is an automated email from the ASF dual-hosted git repository.
zhangduo pushed a commit to branch branch-2
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/branch-2 by this push:
new 51233c3 HBASE-26244 Avoid trim the error stack trace when running UT
with maven (#3654)
51233c3 is described below
commit 51233c3c9bafb6155bca9e186bc156ad7a063621
Author: chenglei
AuthorDate: Wed Sep 1 15:17:05 2021 +0800
HBASE-26244 Avoid trim the error stack trace when running UT with maven
(#3654)
Signed-off-by: Geoffrey Jacoby
Signed-off-by: Duo Zhang
---
pom.xml | 1 +
1 file changed, 1 insertion(+)
diff --git a/pom.xml b/pom.xml
index 87e3b47..c6866e6 100755
--- a/pom.xml
+++ b/pom.xml
@@ -584,6 +584,7 @@
${surefire.firstPartGroups}
false
+false
${surefire.skipFirstPart}
${surefire.firstPartForkCount}
false
[hbase] branch master updated: HBASE-26244 Avoid trim the error stack trace when running UT with maven (#3654)
This is an automated email from the ASF dual-hosted git repository.
zhangduo pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/master by this push:
new 3c64f86 HBASE-26244 Avoid trim the error stack trace when running UT
with maven (#3654)
3c64f86 is described below
commit 3c64f8621fa5355b5001e6e59bb137ee750437bc
Author: chenglei
AuthorDate: Wed Sep 1 15:17:05 2021 +0800
HBASE-26244 Avoid trim the error stack trace when running UT with maven
(#3654)
Signed-off-by: Geoffrey Jacoby
Signed-off-by: Duo Zhang
---
pom.xml | 1 +
1 file changed, 1 insertion(+)
diff --git a/pom.xml b/pom.xml
index 3d24d9d..2c1e5b9 100755
--- a/pom.xml
+++ b/pom.xml
@@ -788,6 +788,7 @@
${surefire.firstPartGroups}
false
+false
${surefire.skipFirstPart}
${surefire.firstPartForkCount}
false
