This is an automated email from the ASF dual-hosted git repository.
wy471x pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/shenyu.git
The following commit(s) were added to refs/heads/master by this push:
new 04f7c2e8b7 test: cover the tracing of the gateway with an integrated
test (#7617) (#7784)
04f7c2e8b7 is described below
commit 04f7c2e8b7d282f14510157a9ab0bfaab538ecac
Author: wy471x <[email protected]>
AuthorDate: Sun Oct 18 11:37:49 2026 +0800
test: cover the tracing of the gateway with an integrated test (#7617)
(#7784)
* test: cover the tracing of the gateway with an integrated test (#7617)
A docker compose stack with an admin, a gateway, an http upstream and an
OTLP collector verifies the tracing of the gateway in a deployment, on top of
the unit tests of the plugins.
- The client starts a trace, the gateway continues it, and the upstream
call becomes a child of the gateway span.
- The trace id of the request log is the trace id of the span, and the
trace state and the baggage of the client reach the upstream.
- A request is forwarded while the collector is down, and the sampler is
covered with parent_based, always_on, always_off and ratio, which only decides
the requests without a parent context.
- The test case is added to the integration test matrix, and the collector
image is pulled with a retry in the CI.
Part of #7566.
* docs: document how the sampler of the tracing decides (#7617)
The ratio sampler only decides the requests which do not carry a parent
context: a request with a valid trace context keeps the decision of the client,
so a ratio lower than one does not resample the traffic which is already
traced. Say so in the javadoc of the configuration and in the bootstrap
configuration, so that an operator does not expect a ratio to lower the
sampling of the traced traffic.
Part of #7566.
* test: verify the continuity of a trace across the gateway (#7617)
Drive a request through the real tracing plugin and the real http client
plugin against a local upstream, so that the whole path is covered: the client
starts the trace, the gateway continues it, and the upstream call becomes a
child of the gateway span.
- Assert that the upstream receives exactly one traceparent whose parent is
the gateway span, and that its trace is the trace of the client.
- Assert that the gateway starts the trace when the client sends no
context, and that the trace state and the baggage of the client are forwarded.
- Chain the plugins lazily, the way the plugin chain of the gateway invokes
them, because the trace context is stored while the request is subscribed.
Part of #7566.
---
.github/scripts/resolve-test-case-matrix.sh | 11 +
.github/workflows/integrated-test.yml | 1 +
.../src/main/resources/application.yml | 6 +-
.../apache/shenyu/common/config/ShenyuConfig.java | 18 +-
shenyu-integrated-test/pom.xml | 1 +
.../shenyu-integrated-test-tracing/Dockerfile | 37 ++
.../config/otel-collector.yml | 34 ++
.../docker-compose.yml | 94 +++++
.../shenyu-integrated-test-tracing/pom.xml | 119 ++++++
.../script/healthcheck.sh | 36 ++
.../script/services.list | 19 +
.../test/tracing/TracingIntegratedBootstrap.java | 37 ++
.../src/main/resources/application-local.yml | 67 +++
.../src/main/resources/application.yml | 19 +
.../shenyu/integrated/test/tracing/Deployment.java | 209 +++++++++
.../tracing/TracingGatewayIntegrationTest.java | 468 +++++++++++++++++++++
shenyu-plugin/shenyu-plugin-tracing/pom.xml | 8 +
.../shenyu/plugin/tracing/TraceContinuityTest.java | 206 +++++++++
18 files changed, 1383 insertions(+), 7 deletions(-)
diff --git a/.github/scripts/resolve-test-case-matrix.sh
b/.github/scripts/resolve-test-case-matrix.sh
index c6efba0b5e..7b6453be0f 100644
--- a/.github/scripts/resolve-test-case-matrix.sh
+++ b/.github/scripts/resolve-test-case-matrix.sh
@@ -73,6 +73,7 @@ add_integration_all() {
add_unique integration_cases "shenyu-integrated-test-spring-cloud"
add_unique integration_cases "shenyu-integrated-test-websocket"
add_unique integration_cases "shenyu-integrated-test-rewrite"
+ add_unique integration_cases "shenyu-integrated-test-tracing"
add_unique integration_cases "shenyu-integrated-test-combination"
add_unique integration_cases "shenyu-integrated-test-sdk-apache-dubbo"
add_unique integration_cases "shenyu-integrated-test-sdk-http"
@@ -297,6 +298,9 @@ map_direct_case_path() {
shenyu-integrated-test/shenyu-integrated-test-rewrite/*)
add_unique integration_cases "shenyu-integrated-test-rewrite"
;;
+ shenyu-integrated-test/shenyu-integrated-test-tracing/*)
+ add_unique integration_cases "shenyu-integrated-test-tracing"
+ ;;
shenyu-integrated-test/shenyu-integrated-test-combination/*)
add_unique integration_cases "shenyu-integrated-test-combination"
;;
@@ -377,6 +381,13 @@ map_domain_path() {
add_unique integration_cases "shenyu-integrated-test-rewrite"
fi
;;
+ *tracing*)
+ if [[ "${mode}" == "e2e" ]]; then
+ add_unique e2e_cases "e2e-http-sync-compose"
+ else
+ add_unique integration_cases "shenyu-integrated-test-tracing"
+ fi
+ ;;
*https*)
if [[ "${mode}" == "e2e" ]]; then
add_unique e2e_cases "e2e-http-sync-compose"
diff --git a/.github/workflows/integrated-test.yml
b/.github/workflows/integrated-test.yml
index 0c7fea491e..ead01a2c63 100644
--- a/.github/workflows/integrated-test.yml
+++ b/.github/workflows/integrated-test.yml
@@ -166,6 +166,7 @@ jobs:
pull_with_retry centos:7
pull_with_retry eclipse-temurin:17-centos7
pull_with_retry elasticsearch:8.2.3
+ pull_with_retry otel/opentelemetry-collector-contrib:0.111.0
- name: Build with Maven
shell: bash
run: |
diff --git a/shenyu-bootstrap/src/main/resources/application.yml
b/shenyu-bootstrap/src/main/resources/application.yml
index 7953a96628..0bb10820dd 100644
--- a/shenyu-bootstrap/src/main/resources/application.yml
+++ b/shenyu-bootstrap/src/main/resources/application.yml
@@ -314,9 +314,11 @@ shenyu:
serviceName: shenyu-gateway
createSpan: true
# parent_based continues the decision of the client and samples a request
which has none,
- # always_on, always_off and ratio are the alternatives
+ # always_on and always_off decide about every request, even about the
request of a client which
+ # has decided already, and ratio continues the decision of the client as
well while it samples a
+ # request which carries no parent context with the given ratio
sampler: parent_based
- # only used by the ratio sampler
+ # only used by the ratio sampler, and only for the requests which carry no
parent context
# sampleRatio: 0.1
# resourceAttributes:
# deployment.environment: prod
diff --git
a/shenyu-common/src/main/java/org/apache/shenyu/common/config/ShenyuConfig.java
b/shenyu-common/src/main/java/org/apache/shenyu/common/config/ShenyuConfig.java
index b194dcf9bd..4391c980e2 100644
---
a/shenyu-common/src/main/java/org/apache/shenyu/common/config/ShenyuConfig.java
+++
b/shenyu-common/src/main/java/org/apache/shenyu/common/config/ShenyuConfig.java
@@ -2070,7 +2070,10 @@ public class ShenyuConfig {
/**
* Gets sampler.
*
- * @return the sampler, one of parent_based, always_on, always_off and
ratio
+ * @return the sampler, one of parent_based, always_on, always_off and
ratio, where
+ * parent_based continues the decision of the client,
always_on and always_off decide
+ * about every request, and ratio continues the decision of
the client while it
+ * samples a request which carries no parent context with the
given ratio
*/
public String getSampler() {
return sampler;
@@ -2079,8 +2082,10 @@ public class ShenyuConfig {
/**
* Sets sampler, one of parent_based, always_on, always_off and ratio.
The parent based
* sampler, which is the default, continues the decision of the client
and samples a request
- * which does not carry a decision, while the ratio sampler samples
the given ratio of the
- * requests which do not carry a decision.
+ * which does not carry a decision, always_on and always_off decide
about every request,
+ * including the request of a client which has decided already, and
the ratio sampler
+ * continues the decision of the client as well, while it samples a
request which carries no
+ * parent context with the given ratio.
*
* @param sampler the sampler
*/
@@ -2091,14 +2096,17 @@ public class ShenyuConfig {
/**
* Gets sample ratio.
*
- * @return the sample ratio, which is only used by the ratio sampler
+ * @return the sample ratio, which is only used by the ratio sampler,
and only for the
+ * requests which carry no parent context
*/
public double getSampleRatio() {
return sampleRatio;
}
/**
- * Sets sample ratio, which is only used by the ratio sampler.
+ * Sets sample ratio, which is only used by the ratio sampler, and
only for the requests
+ * which carry no parent context, because a request which carries a
valid trace context
+ * keeps the sampling decision of the client.
*
* @param sampleRatio the sample ratio
*/
diff --git a/shenyu-integrated-test/pom.xml b/shenyu-integrated-test/pom.xml
index 5e243fe33c..7e1b11dc7f 100644
--- a/shenyu-integrated-test/pom.xml
+++ b/shenyu-integrated-test/pom.xml
@@ -37,6 +37,7 @@
<module>shenyu-integrated-test-grpc</module>
<module>shenyu-integrated-test-websocket</module>
<module>shenyu-integrated-test-rewrite</module>
+ <module>shenyu-integrated-test-tracing</module>
<module>shenyu-integrated-test-combination</module>
<module>shenyu-integrated-test-sdk-apache-dubbo</module>
<module>shenyu-integrated-test-sdk-http</module>
diff --git a/shenyu-integrated-test/shenyu-integrated-test-tracing/Dockerfile
b/shenyu-integrated-test/shenyu-integrated-test-tracing/Dockerfile
new file mode 100644
index 0000000000..51476d646c
--- /dev/null
+++ b/shenyu-integrated-test/shenyu-integrated-test-tracing/Dockerfile
@@ -0,0 +1,37 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+FROM eclipse-temurin:17-centos7
+
+ENV APP_NAME shenyu-integrated-test-tracing
+LABEL org.opencontainers.image.title="${APP_NAME}" \
+ org.opencontainers.image.source="https://github.com/apache/shenyu"; \
+ org.opencontainers.image.licenses="Apache-2.0"
+ENV LOCAL_PATH /opt/${APP_NAME}
+
+RUN mkdir -p ${LOCAL_PATH}
+
+ADD target/${APP_NAME}.jar ${LOCAL_PATH}
+
+WORKDIR ${LOCAL_PATH}
+RUN groupadd --system shenyu && \
+ useradd --system --gid shenyu --create-home --home-dir /home/shenyu shenyu
&& \
+ chown -R shenyu:shenyu ${LOCAL_PATH}
+USER shenyu
+HEALTHCHECK --interval=30s --timeout=5s --start-period=30s --retries=3 CMD
["sh", "-c", "wget -q -O - http://127.0.0.1:9195/actuator/health | grep -q UP"]
+EXPOSE 9195
+
+CMD java -jar ${APP_NAME}.jar
diff --git
a/shenyu-integrated-test/shenyu-integrated-test-tracing/config/otel-collector.yml
b/shenyu-integrated-test/shenyu-integrated-test-tracing/config/otel-collector.yml
new file mode 100644
index 0000000000..45dc1f73c0
--- /dev/null
+++
b/shenyu-integrated-test/shenyu-integrated-test-tracing/config/otel-collector.yml
@@ -0,0 +1,34 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# The collector the gateway of this integrated test exports its spans to. It
receives OTLP over
+# http and prints every span it accepted, which is how the test verifies the
export without parsing
+# the protobuf of the transport.
+receivers:
+ otlp:
+ protocols:
+ http:
+ endpoint: 0.0.0.0:4318
+
+exporters:
+ debug:
+ verbosity: detailed
+
+service:
+ pipelines:
+ traces:
+ receivers: [ otlp ]
+ exporters: [ debug ]
diff --git
a/shenyu-integrated-test/shenyu-integrated-test-tracing/docker-compose.yml
b/shenyu-integrated-test/shenyu-integrated-test-tracing/docker-compose.yml
new file mode 100644
index 0000000000..359be36f92
--- /dev/null
+++ b/shenyu-integrated-test/shenyu-integrated-test-tracing/docker-compose.yml
@@ -0,0 +1,94 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+services:
+ shenyu-admin:
+ image: apache/shenyu-admin:latest
+ container_name: shenyu-admin
+ restart: always
+ networks:
+ - shenyu
+ ports:
+ - "9095:9095"
+ environment:
+ - SPRING_PROFILES_ACTIVE=h2
+ - shenyu.sync.websocket.token=shenyu-sync-token
+ - shenyu.database.init_script=sql-script/h2/schema.sql
+ - shenyu.jwt.secretKey=shenyu-e2e-jwt-secret-key-2024
+ healthcheck:
+ test: ["CMD-SHELL", "wget -q -O -
http://shenyu-admin:9095/actuator/health | grep UP || exit 1"]
+ timeout: 2s
+ retries: 30
+
+ shenyu-otel-collector:
+ image: otel/opentelemetry-collector-contrib:0.111.0
+ container_name: shenyu-otel-collector
+ networks:
+ - shenyu
+ volumes:
+ - type: bind
+ source: ./config/otel-collector.yml
+ target: /etc/otelcol-contrib/config.yaml
+
+ shenyu-examples-http:
+ deploy:
+ resources:
+ limits:
+ memory: 2048M
+ container_name: shenyu-examples-http
+ image: shenyu-examples-http:latest
+ restart: always
+ environment:
+ - shenyu.register.serverLists=http://shenyu-admin:9095
+ healthcheck:
+ test: [ "CMD", "wget",
"http://shenyu-examples-http:8189/test/path/123?name=tom"; ]
+ timeout: 2s
+ retries: 30
+ ports:
+ - "8189:8189"
+ depends_on:
+ shenyu-integrated-test-tracing:
+ condition: service_healthy
+ networks:
+ - shenyu
+
+ shenyu-integrated-test-tracing:
+ container_name: shenyu-integrated-test-tracing
+ image: apache/shenyu-integrated-test-tracing:latest
+ restart: always
+ deploy:
+ resources:
+ limits:
+ memory: 2048M
+ environment:
+ - shenyu.sync.websocket.urls=ws://shenyu-admin:9095/websocket
+ - shenyu.sync.websocket.token=shenyu-sync-token
+ # the collector of the compose file receives the spans of the gateway
+ -
shenyu.tracing.otlp.endpoint=http://shenyu-otel-collector:4318/v1/traces
+ depends_on:
+ shenyu-admin:
+ condition: service_healthy
+ ports:
+ - "9195:9195"
+ networks:
+ - shenyu
+ healthcheck:
+ test: [ "CMD", "wget",
"http://shenyu-integrated-test-tracing:9195/actuator/health"; ]
+ timeout: 2s
+ retries: 30
+
+networks:
+ shenyu:
+ name: shenyu
diff --git a/shenyu-integrated-test/shenyu-integrated-test-tracing/pom.xml
b/shenyu-integrated-test/shenyu-integrated-test-tracing/pom.xml
new file mode 100644
index 0000000000..70b22351ef
--- /dev/null
+++ b/shenyu-integrated-test/shenyu-integrated-test-tracing/pom.xml
@@ -0,0 +1,119 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ ~ Licensed to the Apache Software Foundation (ASF) under one or more
+ ~ contributor license agreements. See the NOTICE file distributed with
+ ~ this work for additional information regarding copyright ownership.
+ ~ The ASF licenses this file to You under the Apache License, Version 2.0
+ ~ (the "License"); you may not use this file except in compliance with
+ ~ the License. You may obtain a copy of the License at
+ ~
+ ~ http://www.apache.org/licenses/LICENSE-2.0
+ ~
+ ~ Unless required by applicable law or agreed to in writing, software
+ ~ distributed under the License is distributed on an "AS IS" BASIS,
+ ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ ~ See the License for the specific language governing permissions and
+ ~ limitations under the License.
+ -->
+<project xmlns="http://maven.apache.org/POM/4.0.0";
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd";>
+ <parent>
+ <artifactId>shenyu-integrated-test</artifactId>
+ <groupId>org.apache.shenyu</groupId>
+ <version>2.7.2-SNAPSHOT</version>
+ </parent>
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>shenyu-integrated-test-tracing</artifactId>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.shenyu</groupId>
+ <artifactId>shenyu-integrated-test-common</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+
+ <!-- the gateway routes the request of the test to the http example
over the divide plugin -->
+ <dependency>
+ <groupId>org.apache.shenyu</groupId>
+ <artifactId>shenyu-spring-boot-starter-plugin-divide</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.shenyu</groupId>
+
<artifactId>shenyu-spring-boot-starter-plugin-httpclient</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+
+ <!-- the plugin under test, the starter wires the OpenTelemetry SDK
and the OTLP exporter -->
+ <dependency>
+ <groupId>org.apache.shenyu</groupId>
+ <artifactId>shenyu-spring-boot-starter-plugin-tracing</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ </dependencies>
+
+ <profiles>
+ <profile>
+ <id>it</id>
+ <properties>
+
<docker.buildArg.APP_NAME>shenyu-integrated-test-tracing</docker.buildArg.APP_NAME>
+
<docker.image.tag.repo>apache/shenyu-integrated-test-tracing</docker.image.tag.repo>
+ <docker.image.tag.tagName>latest</docker.image.tag.tagName>
+ </properties>
+ <build>
+ <finalName>shenyu-integrated-test-tracing</finalName>
+ <plugins>
+ <plugin>
+ <groupId>org.springframework.boot</groupId>
+ <artifactId>spring-boot-maven-plugin</artifactId>
+ <version>${spring-boot.version}</version>
+ <executions>
+ <execution>
+ <phase>package</phase>
+ <goals>
+ <goal>repackage</goal>
+ </goals>
+ </execution>
+ </executions>
+ <configuration>
+
<mainClass>org.apache.shenyu.integrated.test.tracing.TracingIntegratedBootstrap</mainClass>
+ <executable>true</executable>
+ </configuration>
+ </plugin>
+ <plugin>
+ <groupId>io.fabric8</groupId>
+ <artifactId>docker-maven-plugin</artifactId>
+ <version>${docker-maven-plugin.version}</version>
+ <configuration>
+ <images>
+ <image>
+
<name>apache/shenyu-integrated-test-tracing</name>
+ <build>
+
<contextDir>${project.basedir}</contextDir>
+ </build>
+ </image>
+ </images>
+ </configuration>
+ <executions>
+ <execution>
+ <id>start</id>
+ <goals>
+ <goal>build</goal>
+ </goals>
+ </execution>
+ </executions>
+ </plugin>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-surefire-plugin</artifactId>
+ <configuration>
+ <skipTests>false</skipTests>
+ </configuration>
+ </plugin>
+ </plugins>
+ </build>
+ </profile>
+ </profiles>
+</project>
diff --git
a/shenyu-integrated-test/shenyu-integrated-test-tracing/script/healthcheck.sh
b/shenyu-integrated-test/shenyu-integrated-test-tracing/script/healthcheck.sh
new file mode 100755
index 0000000000..22a76034ab
--- /dev/null
+++
b/shenyu-integrated-test/shenyu-integrated-test-tracing/script/healthcheck.sh
@@ -0,0 +1,36 @@
+#!/bin/bash
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+PRGDIR=`dirname "$0"`
+for service in `grep -v -E "^$|^#" ${PRGDIR}/services.list`
+do
+ for loop in `seq 1 30`
+ do
+ status=`curl -o /dev/null -s -w %{http_code} $service`
+ echo -e "curl $service response $status"
+
+ if [ $status -eq 200 ]; then
+ break
+ fi
+
+ sleep 2
+ done
+done
+
+sleep 3
+echo -e "\n-------------------"
diff --git
a/shenyu-integrated-test/shenyu-integrated-test-tracing/script/services.list
b/shenyu-integrated-test/shenyu-integrated-test-tracing/script/services.list
new file mode 100644
index 0000000000..e48dc7e039
--- /dev/null
+++ b/shenyu-integrated-test/shenyu-integrated-test-tracing/script/services.list
@@ -0,0 +1,19 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+http://localhost:9095/actuator/health
+http://localhost:8189/test/path/123?name=tom
+http://localhost:9195/actuator/health
diff --git
a/shenyu-integrated-test/shenyu-integrated-test-tracing/src/main/java/org/apache/shenyu/integrated/test/tracing/TracingIntegratedBootstrap.java
b/shenyu-integrated-test/shenyu-integrated-test-tracing/src/main/java/org/apache/shenyu/integrated/test/tracing/TracingIntegratedBootstrap.java
new file mode 100644
index 0000000000..8189531341
--- /dev/null
+++
b/shenyu-integrated-test/shenyu-integrated-test-tracing/src/main/java/org/apache/shenyu/integrated/test/tracing/TracingIntegratedBootstrap.java
@@ -0,0 +1,37 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.shenyu.integrated.test.tracing;
+
+import org.springframework.boot.SpringApplication;
+import org.springframework.boot.autoconfigure.SpringBootApplication;
+
+/**
+ * The tracing integrated bootstrap, the gateway of the tracing integrated
test.
+ */
+@SpringBootApplication
+public class TracingIntegratedBootstrap {
+
+ /**
+ * main method of App.
+ *
+ * @param args args
+ */
+ public static void main(final String[] args) {
+ SpringApplication.run(TracingIntegratedBootstrap.class);
+ }
+}
diff --git
a/shenyu-integrated-test/shenyu-integrated-test-tracing/src/main/resources/application-local.yml
b/shenyu-integrated-test/shenyu-integrated-test-tracing/src/main/resources/application-local.yml
new file mode 100644
index 0000000000..c8bd5c5bb6
--- /dev/null
+++
b/shenyu-integrated-test/shenyu-integrated-test-tracing/src/main/resources/application-local.yml
@@ -0,0 +1,67 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+server:
+ port: 9195
+ address: 0.0.0.0
+
+spring:
+ main:
+ allow-bean-definition-overriding: true
+ application:
+ name: shenyu-bootstrap
+
+management:
+ health:
+ defaults:
+ enabled: false
+
+shenyu:
+ switchConfig:
+ local: true
+ cross:
+ enabled: true
+ sync:
+ websocket:
+ urls: ws://localhost:9095/websocket
+ token: shenyu-sync-token
+ allowOrigin: ws://localhost:9195
+ exclude:
+ enabled: true
+ paths:
+ - /favicon.ico
+ - /actuator/health
+ local:
+ enabled: true
+ sha512Key:
"BA3253876AED6BC22D4A6FF53D8406C6AD864195ED144AB5C87621B6C233B548BAEAE6956DF346EC8C17F5EA10F35EE3CBC514797ED7DDD3145464E2A0BAB413"
+ sharedPool:
+ enable: true
+ tracing:
+ enabled: true
+ service-name: shenyu-integrated-test-tracing
+ sampler: parent_based
+ otlp:
+ # the compose file points the gateway to the collector of the compose
file
+ endpoint: http://localhost:4318/v1/traces
+
+logging:
+ level:
+ root: info
+ org.springframework.boot: info
+ org.apache.ibatis: info
+ org.apache.shenyu: info
+ # the request scoped log line reports the trace id of the gateway span,
which the test verifies
+ org.apache.shenyu.web.handler: debug
diff --git
a/shenyu-integrated-test/shenyu-integrated-test-tracing/src/main/resources/application.yml
b/shenyu-integrated-test/shenyu-integrated-test-tracing/src/main/resources/application.yml
new file mode 100644
index 0000000000..c75ffb0595
--- /dev/null
+++
b/shenyu-integrated-test/shenyu-integrated-test-tracing/src/main/resources/application.yml
@@ -0,0 +1,19 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+spring:
+ profiles:
+ active: local
diff --git
a/shenyu-integrated-test/shenyu-integrated-test-tracing/src/test/java/org/apache/shenyu/integrated/test/tracing/Deployment.java
b/shenyu-integrated-test/shenyu-integrated-test-tracing/src/test/java/org/apache/shenyu/integrated/test/tracing/Deployment.java
new file mode 100644
index 0000000000..d3d28f7b12
--- /dev/null
+++
b/shenyu-integrated-test/shenyu-integrated-test-tracing/src/test/java/org/apache/shenyu/integrated/test/tracing/Deployment.java
@@ -0,0 +1,209 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.shenyu.integrated.test.tracing;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.ByteArrayOutputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.List;
+import java.util.Objects;
+import java.util.concurrent.TimeUnit;
+
+/**
+ * The docker deployment of the integrated test.
+ *
+ * <p>The tests of the sampler configure the gateway after the deployment is
started, so the gateway
+ * of the compose file is recreated with an override of the compose file
instead of shipping one
+ * gateway per sampler. The override only adds the tracing settings to the
environment of the
+ * gateway, everything else stays what the compose file declares.</p>
+ */
+final class Deployment {
+
+ static final String GATEWAY_CONTAINER = "shenyu-integrated-test-tracing";
+
+ static final String COLLECTOR_CONTAINER = "shenyu-otel-collector";
+
+ private static final Logger LOG =
LoggerFactory.getLogger(Deployment.class);
+
+ private static final String COMPOSE_FILE = "docker-compose.yml";
+
+ private static final Path OVERRIDE_FILE = Paths.get("target",
"tracing-sampler-override.yml");
+
+ private static final String SAMPLER_PROPERTY = "shenyu.tracing.sampler";
+
+ private static final String SAMPLE_RATIO_PROPERTY =
"shenyu.tracing.sample-ratio";
+
+ private static final long COMMAND_TIMEOUT_SECONDS = 180L;
+
+ private Deployment() {
+ }
+
+ /**
+ * Recreates the gateway with a sampler, the deployment of the compose
file is restored by
+ * {@link #restoreGateway()} afterwards.
+ *
+ * @param sampler the sampler of the gateway
+ * @throws IOException while the override of the compose file cannot be
written
+ * @throws InterruptedException while a command of the deployment is
interrupted
+ */
+ static void restartGatewayWithSampler(final String sampler) throws
IOException, InterruptedException {
+ restartGatewayWithSampler(sampler, null);
+ }
+
+ /**
+ * Recreates the gateway with a sampler and its sample ratio.
+ *
+ * @param sampler the sampler of the gateway
+ * @param sampleRatio the sample ratio, null for a sampler which takes no
ratio
+ * @throws IOException while the override of the compose file cannot be
written
+ * @throws InterruptedException while a command of the deployment is
interrupted
+ */
+ static void restartGatewayWithSampler(final String sampler, final Double
sampleRatio)
+ throws IOException, InterruptedException {
+ final StringBuilder override = new StringBuilder()
+ .append("services:").append(System.lineSeparator())
+ .append("
").append(GATEWAY_CONTAINER).append(':').append(System.lineSeparator())
+ .append(" environment:").append(System.lineSeparator())
+ .append(" -
").append(SAMPLER_PROPERTY).append('=').append(sampler).append(System.lineSeparator());
+ if (Objects.nonNull(sampleRatio)) {
+ override.append(" -
").append(SAMPLE_RATIO_PROPERTY).append('=').append(sampleRatio)
+ .append(System.lineSeparator());
+ }
+ Files.createDirectories(OVERRIDE_FILE.toAbsolutePath().getParent());
+ Files.writeString(OVERRIDE_FILE, override.toString(),
StandardCharsets.UTF_8);
+ composeUpGateway(OVERRIDE_FILE.toString());
+ }
+
+ /**
+ * Restores the gateway of the compose file, so that a deployment is left
the way the compose
+ * file declares it and the tests of the next run find the default sampler.
+ *
+ * @throws IOException while a command of the deployment cannot be executed
+ * @throws InterruptedException while a command of the deployment is
interrupted
+ */
+ static void restoreGateway() throws IOException, InterruptedException {
+ composeUpGateway(null);
+ }
+
+ /**
+ * Stops the collector, which is what an unreachable exporter looks like
to the gateway.
+ *
+ * @throws IOException while a command of the deployment cannot be executed
+ * @throws InterruptedException while a command of the deployment is
interrupted
+ */
+ static void stopCollector() throws IOException, InterruptedException {
+ run(Arrays.asList("docker", "stop", COLLECTOR_CONTAINER));
+ }
+
+ /**
+ * Starts the collector again.
+ *
+ * @throws IOException while a command of the deployment cannot be executed
+ * @throws InterruptedException while a command of the deployment is
interrupted
+ */
+ static void startCollector() throws IOException, InterruptedException {
+ run(Arrays.asList("docker", "start", COLLECTOR_CONTAINER));
+ }
+
+ /**
+ * Acquires the log of the collector, which reports every span it accepted.
+ *
+ * @return the log of the collector
+ * @throws IOException while a command of the deployment cannot be executed
+ * @throws InterruptedException while a command of the deployment is
interrupted
+ */
+ static String collectorLogs() throws IOException, InterruptedException {
+ return logs(COLLECTOR_CONTAINER);
+ }
+
+ /**
+ * Acquires the log of the gateway.
+ *
+ * @return the log of the gateway
+ * @throws IOException while a command of the deployment cannot be executed
+ * @throws InterruptedException while a command of the deployment is
interrupted
+ */
+ static String gatewayLogs() throws IOException, InterruptedException {
+ return logs(GATEWAY_CONTAINER);
+ }
+
+ private static String logs(final String container) throws IOException,
InterruptedException {
+ return run(Arrays.asList("docker", "logs", "--tail", "20000",
container));
+ }
+
+ private static void composeUpGateway(final String overrideFile) throws
IOException, InterruptedException {
+ final List<String> command = new ArrayList<>();
+ command.addAll(Arrays.asList("docker", "compose", "-f", COMPOSE_FILE));
+ if (Objects.nonNull(overrideFile)) {
+ command.addAll(Arrays.asList("-f", overrideFile));
+ }
+ command.addAll(Arrays.asList("up", "-d", "--force-recreate",
"--no-deps", GATEWAY_CONTAINER));
+ run(command);
+ }
+
+ private static String run(final List<String> command) throws IOException,
InterruptedException {
+ final Process process = new
ProcessBuilder(command).redirectErrorStream(true).start();
+ // the output is drained by a thread of its own: a log which is longer
than the buffer of the
+ // pipe would block the child until the test reads it, while reading
until the end of the
+ // stream would block the test until the child ends, so neither side
may depend on the other
+ final ByteArrayOutputStream output = new ByteArrayOutputStream();
+ final Thread drain = new Thread(() -> {
+ try (InputStream stream = process.getInputStream()) {
+ stream.transferTo(output);
+ } catch (IOException ex) {
+ LOG.warn("the output of {} cannot be read: {}", String.join("
", command), ex.getMessage());
+ }
+ }, "deployment-command-output");
+ drain.setDaemon(true);
+ drain.start();
+ final boolean finished;
+ try {
+ finished = process.waitFor(COMMAND_TIMEOUT_SECONDS,
TimeUnit.SECONDS);
+ if (!finished) {
+ process.destroyForcibly();
+ }
+ } finally {
+ // the drain of a finished command ends with the end of its
output, while a command which
+ // is killed may leave a process behind which holds the pipe, and
the collected output is
+ // then incomplete
+ drain.join(TimeUnit.SECONDS.toMillis(5L));
+ if (drain.isAlive()) {
+ LOG.warn("the output of {} is still being read, the reported
output may be truncated",
+ String.join(" ", command));
+ }
+ }
+ final String collected = output.toString(StandardCharsets.UTF_8);
+ if (!finished) {
+ throw new IllegalStateException(String.join(" ", command) + " did
not finish in "
+ + COMMAND_TIMEOUT_SECONDS + "s: " + collected);
+ }
+ if (0 != process.exitValue()) {
+ throw new IllegalStateException(String.join(" ", command) + "
failed: " + collected);
+ }
+ return collected;
+ }
+}
diff --git
a/shenyu-integrated-test/shenyu-integrated-test-tracing/src/test/java/org/apache/shenyu/integrated/test/tracing/TracingGatewayIntegrationTest.java
b/shenyu-integrated-test/shenyu-integrated-test-tracing/src/test/java/org/apache/shenyu/integrated/test/tracing/TracingGatewayIntegrationTest.java
new file mode 100644
index 0000000000..31f28cd427
--- /dev/null
+++
b/shenyu-integrated-test/shenyu-integrated-test-tracing/src/test/java/org/apache/shenyu/integrated/test/tracing/TracingGatewayIntegrationTest.java
@@ -0,0 +1,468 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.shenyu.integrated.test.tracing;
+
+import org.apache.shenyu.integratedtest.common.AbstractTest;
+import org.junit.jupiter.api.AfterAll;
+import org.junit.jupiter.api.BeforeAll;
+import org.junit.jupiter.api.MethodOrderer;
+import org.junit.jupiter.api.Order;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.TestMethodOrder;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.net.URI;
+import java.net.http.HttpClient;
+import java.net.http.HttpRequest;
+import java.net.http.HttpResponse;
+import java.time.Duration;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.concurrent.TimeUnit;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertFalse;
+import static org.junit.jupiter.api.Assertions.assertNotEquals;
+import static org.junit.jupiter.api.Assertions.assertNotNull;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+import static org.junit.jupiter.api.Assertions.fail;
+
+/**
+ * The Test Case For the tracing plugin of a docker deployment.
+ *
+ * <p>The client of the test, the gateway container, the http example
container and the OTLP
+ * collector container take part in every request, so the whole deployment is
covered: the trace
+ * context of the client is continued by the gateway, the call to the upstream
carries the span of
+ * the gateway as its parent, the span is accepted by a real collector, the
trace id of the request
+ * reaches the log of the gateway, and the sampler of the gateway decides
about the trace of a
+ * request which is forwarded either way.</p>
+ *
+ * <p>The tracing plugin exposes the trace id through the {@code
shenyu-agent-trace-id} exchange
+ * attribute, which the request scoped log line of the gateway reports and
which the logging plugins
+ * write into the access log. The test verifies the attribute through that log
line, because the
+ * console logging plugin of the deployment does not print the attribute
itself.</p>
+ *
+ * <p>The upstream is the http example, whose {@code /request/header} endpoint
echoes the headers it
+ * received, which is how the test reads the trace context the upstream really
got. The collector is
+ * the {@code debug} exporter of the collector image, which reports the ids of
every span it
+ * accepted, which is how the test reads what the gateway exported.</p>
+ */
+@TestMethodOrder(MethodOrderer.OrderAnnotation.class)
+public class TracingGatewayIntegrationTest extends AbstractTest {
+
+ private static final String GATEWAY_ENDPOINT = "http://localhost:9195";;
+
+ private static final String ROUTE = "/http/request/header";
+
+ private static final String HEADER_KEY = "header_key1";
+
+ private static final String TRACEPARENT = "traceparent";
+
+ private static final String TRACESTATE = "tracestate";
+
+ private static final String BAGGAGE = "baggage";
+
+ private static final String SAMPLED = "01";
+
+ private static final String NOT_SAMPLED = "00";
+
+ private static final String SERVICE_NAME =
"shenyu-integrated-test-tracing";
+
+ private static final String INSTRUMENTATION_SCOPE =
"org.apache.shenyu.plugin.tracing";
+
+ private static final String CHAIN_LOG_MARKER = "shenyu chain handle uri:";
+
+ private static final String SAMPLER_PARENT_BASED = "parent_based";
+
+ private static final String SAMPLER_ALWAYS_ON = "always_on";
+
+ private static final String SAMPLER_ALWAYS_OFF = "always_off";
+
+ private static final String SAMPLER_RATIO = "ratio";
+
+ private static final String CONTINUITY_TRACE_ID =
"4bf92f3577b34da6a3ce929d0e0e4736";
+
+ private static final String CONTINUITY_CLIENT_SPAN_ID = "00f067aa0ba902b7";
+
+ private static final String LOG_TRACE_ID =
"1af7651916cd43dd8448eb211c80319c";
+
+ private static final String LOG_CLIENT_SPAN_ID = "b7ad6b7169203331";
+
+ private static final String PROPAGATION_TRACE_ID =
"0af7651916cd43dd8448eb211c80319c";
+
+ private static final String PROPAGATION_CLIENT_SPAN_ID =
"b7ad6b7169203332";
+
+ private static final String UNSAMPLED_TRACE_ID =
"5af7651916cd43dd8448eb211c80319c";
+
+ private static final String UNSAMPLED_CLIENT_SPAN_ID = "b7ad6b7169203334";
+
+ private static final String ALWAYS_ON_TRACE_ID =
"6af7651916cd43dd8448eb211c80319c";
+
+ private static final String ALWAYS_ON_CLIENT_SPAN_ID = "b7ad6b7169203335";
+
+ private static final String ALWAYS_OFF_TRACE_ID =
"7af7651916cd43dd8448eb211c80319c";
+
+ private static final String ALWAYS_OFF_CLIENT_SPAN_ID = "b7ad6b7169203336";
+
+ private static final String UNREACHABLE_TRACE_ID =
"9af7651916cd43dd8448eb211c80319c";
+
+ private static final String UNREACHABLE_CLIENT_SPAN_ID =
"b7ad6b7169203338";
+
+ private static final Duration AWAIT_TIMEOUT = Duration.ofSeconds(60);
+
+ /**
+ * The time the batch processor of the gateway needs to export the spans
it has collected. The
+ * batch processor exports on a fixed schedule, every five seconds by
default, so a request which
+ * is not sampled is only proven to be dropped once twice that time has
passed.
+ */
+ private static final Duration EXPORT_SETTLE_TIME = Duration.ofSeconds(12);
+
+ /**
+ * The marker the debug exporter of the collector writes before the spans
of a resource, the text
+ * is the one the debug exporter of the collector 0.111.0 writes.
+ */
+ private static final String RESOURCE_REPORT_MARKER = "ResourceSpans #";
+
+ private static final Logger LOG =
LoggerFactory.getLogger(TracingGatewayIntegrationTest.class);
+
+ private static final HttpClient CLIENT =
HttpClient.newBuilder().connectTimeout(Duration.ofSeconds(10)).build();
+
+ private static boolean samplerOverridden;
+
+ /**
+ * Waits until the gateway routes the requests of the test.
+ *
+ * @throws IOException while a request cannot be sent
+ * @throws InterruptedException while the wait is interrupted
+ */
+ @BeforeAll
+ public static void awaitRouteReady() throws IOException,
InterruptedException {
+ final Map<String, String> headers = new HashMap<>(1, 1);
+ headers.put(HEADER_KEY, "tracing");
+ final long deadline = System.nanoTime() + AWAIT_TIMEOUT.toNanos();
+ while (System.nanoTime() < deadline) {
+ try {
+ final HttpResponse<String> response = get(headers);
+ if (200 == response.statusCode() &&
response.body().contains("response success:")) {
+ return;
+ }
+ } catch (IOException ex) {
+ // the gateway is recreated by the tests of the sampler, so it
may not listen yet
+ LOG.info("the gateway does not answer yet: {}",
ex.getMessage());
+ }
+ TimeUnit.MILLISECONDS.sleep(500L);
+ }
+ fail("the gateway did not route " + ROUTE + " to the http example in
time");
+ }
+
+ /**
+ * Restores the gateway of the compose file, so that the sampler of the
deployment is the
+ * declared one again.
+ *
+ * @throws IOException while a command of the deployment cannot be executed
+ * @throws InterruptedException while a command of the deployment is
interrupted
+ */
+ @AfterAll
+ public static void restoreDeployment() throws IOException,
InterruptedException {
+ if (samplerOverridden) {
+ Deployment.restoreGateway();
+ }
+ }
+
+ @Test
+ @Order(1)
+ public void testTheClientTheGatewayAndTheUpstreamShareOneTrace() throws
IOException, InterruptedException {
+ final HttpResponse<String> response = get(headers(CONTINUITY_TRACE_ID,
CONTINUITY_CLIENT_SPAN_ID, SAMPLED));
+ assertEquals(200, response.statusCode());
+ final String[] traceparent = upstreamTraceparent(response.body());
+ // the trace of the client is the trace of the upstream call, while
the gateway continued it
+ // with a span of its own instead of forwarding the span of the client
+ assertEquals(CONTINUITY_TRACE_ID, traceparent[1]);
+ assertNotEquals(CONTINUITY_CLIENT_SPAN_ID, traceparent[2]);
+ assertEquals(SAMPLED, traceparent[3]);
+ // the span the gateway exported belongs to that trace, its parent is
the span of the client,
+ // and its own span is the parent of the call to the upstream: the
whole path is one trace
+ final String exported = awaitExportedSpan(CONTINUITY_TRACE_ID,
CONTINUITY_CLIENT_SPAN_ID, traceparent[2]);
+ assertTrue(exported.contains("-> service.name: Str(" + SERVICE_NAME +
")"),
+ "the spans have to carry the configured service name");
+ assertTrue(exported.contains("InstrumentationScope " +
INSTRUMENTATION_SCOPE),
+ "the spans have to be reported for the instrumentation scope
of the tracing plugin");
+ }
+
+ @Test
+ @Order(2)
+ public void testTheRequestScopedLogReportsTheTraceIdOfTheTrace() throws
IOException, InterruptedException {
+ final HttpResponse<String> response = get(headers(LOG_TRACE_ID,
LOG_CLIENT_SPAN_ID, SAMPLED));
+ assertEquals(200, response.statusCode());
+ assertEquals(LOG_TRACE_ID, upstreamTraceparent(response.body())[1]);
+ // the id of the log is the id the tracing plugin put into the
exchange, which is the id of
+ // the trace the upstream continued, so a request is searchable by the
trace of its log
+ awaitGatewayLog(CHAIN_LOG_MARKER + ROUTE + ", traceId:" +
LOG_TRACE_ID);
+ }
+
+ @Test
+ @Order(3)
+ public void testTheTraceStateAndTheBaggageReachTheUpstream() throws
IOException, InterruptedException {
+ final Map<String, String> headers = headers(PROPAGATION_TRACE_ID,
PROPAGATION_CLIENT_SPAN_ID, SAMPLED);
+ headers.put(TRACESTATE, "vendor=value");
+ headers.put(BAGGAGE, "userId=alice");
+ final HttpResponse<String> response = get(headers);
+ assertEquals(200, response.statusCode());
+ final String[] traceparent = upstreamTraceparent(response.body());
+ assertEquals(PROPAGATION_TRACE_ID, traceparent[1]);
+ assertEquals("vendor=value", echoedHeader(response.body(),
TRACESTATE));
+ assertEquals("userId=alice", echoedHeader(response.body(), BAGGAGE));
+ // the trace state is part of the span as well, the upstream and the
collector see the same
+ final String exported = awaitExportedSpan(PROPAGATION_TRACE_ID,
PROPAGATION_CLIENT_SPAN_ID, traceparent[2]);
+
assertTrue(Pattern.compile("TraceState\\s*:\\s*vendor=value").matcher(exported).find(),
+ "the span has to carry the trace state of the client");
+ }
+
+ @Test
+ @Order(4)
+ public void testTheParentBasedSamplerDoesNotTraceAnUnsampledClient()
throws IOException, InterruptedException {
+ // the parent based sampler continues the decision of the client, so
the request of a client
+ // which did not sample is forwarded with its own decision and its
context is not exported
+ final HttpResponse<String> response = get(headers(UNSAMPLED_TRACE_ID,
UNSAMPLED_CLIENT_SPAN_ID, NOT_SAMPLED));
+ assertEquals(200, response.statusCode());
+ final String[] traceparent = upstreamTraceparent(response.body());
+ assertEquals(UNSAMPLED_TRACE_ID, traceparent[1]);
+ assertEquals(NOT_SAMPLED, traceparent[3]);
+ // the gateway still continues the trace with a span of its own,
dropping the export of the
+ // span must not degrade the propagation to a plain copy of the
context of the client
+ assertNotEquals(UNSAMPLED_CLIENT_SPAN_ID, traceparent[2]);
+ assertNotExported(UNSAMPLED_TRACE_ID);
+ }
+
+ @Test
+ @Order(5)
+ public void testTheRequestIsForwardedWhileTheCollectorIsUnreachable()
throws IOException, InterruptedException {
+ // the collector of the compose file is the exporter of the gateway,
once it is stopped the
+ // export of the spans fails, which must not affect the forwarding of
the request
+ Deployment.stopCollector();
+ try {
+ TimeUnit.SECONDS.sleep(2L);
+ final HttpResponse<String> response =
get(headers(UNREACHABLE_TRACE_ID, UNREACHABLE_CLIENT_SPAN_ID, SAMPLED));
+ assertEquals(200, response.statusCode());
+ assertTrue(response.body().contains("response success:"), "the
upstream has to answer the request");
+ final String[] traceparent = upstreamTraceparent(response.body());
+ assertEquals(UNREACHABLE_TRACE_ID, traceparent[1]);
+ assertEquals(SAMPLED, traceparent[3]);
+ } finally {
+ Deployment.startCollector();
+ }
+ }
+
+ @Test
+ @Order(6)
+ public void testTheAlwaysOnSamplerTracesAnUnsampledClient() throws
IOException, InterruptedException {
+ // the sampler of the gateway overrides the decision of the client, so
the request is traced
+ // and exported although the client did not sample it
+ samplerOverridden = true;
+ Deployment.restartGatewayWithSampler(SAMPLER_ALWAYS_ON);
+ awaitRouteReady();
+ awaitGatewayLog("sampler AlwaysOnSampler");
+ // the sampler of the gateway overrides the decision of the client, so
the request is traced
+ // and exported although the client did not sample it
+ final HttpResponse<String> response = get(headers(ALWAYS_ON_TRACE_ID,
ALWAYS_ON_CLIENT_SPAN_ID, NOT_SAMPLED));
+ assertEquals(200, response.statusCode());
+ final String[] traceparent = upstreamTraceparent(response.body());
+ assertEquals(ALWAYS_ON_TRACE_ID, traceparent[1]);
+ assertEquals(SAMPLED, traceparent[3]);
+ assertNotEquals(ALWAYS_ON_CLIENT_SPAN_ID, traceparent[2]);
+ awaitExportedSpan(ALWAYS_ON_TRACE_ID, ALWAYS_ON_CLIENT_SPAN_ID,
traceparent[2]);
+ }
+
+ @Test
+ @Order(7)
+ public void testTheAlwaysOffSamplerDoesNotTraceASampledClient() throws
IOException, InterruptedException {
+ // the sampler of the gateway overrides the decision of the client
again, the request is
+ // forwarded but nothing is traced or exported
+ samplerOverridden = true;
+ Deployment.restartGatewayWithSampler(SAMPLER_ALWAYS_OFF);
+ awaitRouteReady();
+ awaitGatewayLog("sampler AlwaysOffSampler");
+ // the sampler of the gateway overrides the decision of the client
again, the request is
+ // forwarded but nothing is traced or exported
+ final HttpResponse<String> response = get(headers(ALWAYS_OFF_TRACE_ID,
ALWAYS_OFF_CLIENT_SPAN_ID, SAMPLED));
+ assertEquals(200, response.statusCode());
+ final String[] traceparent = upstreamTraceparent(response.body());
+ assertEquals(ALWAYS_OFF_TRACE_ID, traceparent[1]);
+ assertEquals(NOT_SAMPLED, traceparent[3]);
+ // the gateway still starts a span of its own for the request,
dropping the export must not
+ // degrade the propagation to a plain copy of the context of the client
+ assertNotEquals(ALWAYS_OFF_CLIENT_SPAN_ID, traceparent[2]);
+ assertNotExported(ALWAYS_OFF_TRACE_ID);
+ }
+
+ @Test
+ @Order(8)
+ public void testTheRatioSamplerHonoursTheSampleRatio() throws IOException,
InterruptedException {
+ // the log of the starter reports the sampler which was really built
from the configuration,
+ // the description of the sdk is only matched up to its root sampler
+ samplerOverridden = true;
+ Deployment.restartGatewayWithSampler(SAMPLER_RATIO, 0.0D);
+ awaitRouteReady();
+ awaitGatewayLog("sampler ParentBased{root:TraceIdRatioBased{");
+ // the ratio decides about the requests which carry no decision of a
client, so a ratio of
+ // zero drops the trace of a request which comes without a trace
context, while the request
+ // is forwarded with a context the gateway sampled no
+ final Map<String, String> headers = new HashMap<>(1, 1);
+ headers.put(HEADER_KEY, "tracing");
+ final HttpResponse<String> response = get(headers);
+ assertEquals(200, response.statusCode());
+ final String[] traceparent = upstreamTraceparent(response.body());
+ assertEquals(NOT_SAMPLED, traceparent[3]);
+ assertNotExported(traceparent[1]);
+ }
+
+ private static Map<String, String> headers(final String traceId, final
String spanId, final String sampled) {
+ final Map<String, String> headers = new HashMap<>(4, 1);
+ headers.put(HEADER_KEY, "tracing");
+ headers.put(TRACEPARENT, "00-" + traceId + "-" + spanId + "-" +
sampled);
+ return headers;
+ }
+
+ private static HttpResponse<String> get(final Map<String, String> headers)
throws IOException, InterruptedException {
+ final HttpRequest.Builder builder =
HttpRequest.newBuilder(URI.create(GATEWAY_ENDPOINT + ROUTE)).GET();
+ headers.forEach(builder::header);
+ return CLIENT.send(builder.build(),
HttpResponse.BodyHandlers.ofString());
+ }
+
+ /**
+ * Acquires the trace context which the upstream received, from the
headers the example echoes.
+ *
+ * @param body the body of the response of the upstream
+ * @return the traceparent the upstream received, split into its four
fields
+ */
+ private static String[] upstreamTraceparent(final String body) {
+ final String traceparent = echoedHeader(body, TRACEPARENT);
+ assertNotNull(traceparent, "the upstream did not receive a
traceparent");
+ assertFalse(traceparent.contains(","),
+ "the upstream received more than one traceparent: " +
traceparent);
+ final String[] parts = traceparent.split("-");
+ assertEquals(4, parts.length, "the upstream received a malformed
traceparent " + traceparent);
+ assertEquals("00", parts[0]);
+ return parts;
+ }
+
+ /**
+ * Acquires the value of a header from the body of the response of the
upstream.
+ *
+ * <p>The example echoes the headers it received with the form of the
{@code HttpHeaders} of the
+ * framework, which may change between the releases, so the value is read
by its delimiter rather
+ * than by the exact format of the map. The value ends at the end of the
entry and not at a comma,
+ * so the values of a header which the upstream received more than once
are returned together,
+ * which lets the caller reject a duplicated header.</p>
+ *
+ * @param body the body of the response of the upstream
+ * @param name the name of the header
+ * @return the value of the header, null while the upstream did not
receive it
+ */
+ private static String echoedHeader(final String body, final String name) {
+ final Matcher matcher = Pattern.compile(Pattern.quote(name) +
"\\s*[:=]\\s*\\[?\"?", Pattern.CASE_INSENSITIVE)
+ .matcher(body);
+ if (!matcher.find()) {
+ return null;
+ }
+ int end = matcher.end();
+ while (end < body.length() && '"' != body.charAt(end) && ']' !=
body.charAt(end)) {
+ end++;
+ }
+ return body.substring(matcher.end(), end).trim();
+ }
+
+ /**
+ * Waits until the collector reports a span of the given trace whose
parent is the span of the
+ * client and whose own id is the one the upstream received.
+ *
+ * @param traceId the trace id of the request
+ * @param parentSpanId the span id of the client
+ * @param spanId the span id of the gateway, which the upstream received
+ * @return the report of the collector about the resource of the span and
its scope
+ * @throws IOException while the log cannot be read
+ * @throws InterruptedException while the wait is interrupted
+ */
+ private static String awaitExportedSpan(final String traceId, final String
parentSpanId, final String spanId)
+ throws IOException, InterruptedException {
+ final Pattern span = Pattern.compile("Trace ID\\s*:\\s*" + traceId
+ + "[\\s\\S]*?Parent ID\\s*:\\s*" + parentSpanId
+ + "[\\s\\S]*?^\\s+ID\\s*:\\s*" + spanId, Pattern.MULTILINE);
+ final long deadline = System.nanoTime() + AWAIT_TIMEOUT.toNanos();
+ while (System.nanoTime() < deadline) {
+ final String logs = Deployment.collectorLogs();
+ final Matcher matcher = span.matcher(logs);
+ if (matcher.find()) {
+ return resourceReport(logs, matcher.start());
+ }
+ TimeUnit.MILLISECONDS.sleep(500L);
+ }
+ fail("the collector did not accept the span " + spanId + " of the
trace " + traceId
+ + " whose parent is " + parentSpanId);
+ return "";
+ }
+
+ /**
+ * Narrows the log to the report of one resource, so that the assertions
about the resource and
+ * the scope belong to the span which was found instead of any other span
of the log.
+ *
+ * @param logs the log of the collector
+ * @param position the position of the span in the log
+ * @return the report of the resource which contains the span
+ */
+ private static String resourceReport(final String logs, final int
position) {
+ final int start = logs.lastIndexOf(RESOURCE_REPORT_MARKER, position);
+ // a report which cannot be delimited would turn the assertions of the
caller into a check of
+ // the whole log, so the marker is required instead
+ assertTrue(start >= 0, "the log of the collector does not report a
resource before the span");
+ final int end = logs.indexOf(RESOURCE_REPORT_MARKER, position);
+ return logs.substring(start, end < 0 ? logs.length() : end);
+ }
+
+ /**
+ * Verifies that the collector accepts no span of the given trace.
+ *
+ * @param traceId the trace id of the request
+ * @throws IOException while the log cannot be read
+ * @throws InterruptedException while the wait is interrupted
+ */
+ private static void assertNotExported(final String traceId) throws
IOException, InterruptedException {
+ TimeUnit.MILLISECONDS.sleep(EXPORT_SETTLE_TIME.toMillis());
+ final String logs = Deployment.collectorLogs();
+ assertFalse(Pattern.compile("Trace ID\\s*:\\s*" +
traceId).matcher(logs).find(),
+ "the sampler has to drop the trace " + traceId + " instead of
exporting it");
+ }
+
+ private static String awaitGatewayLog(final String marker) throws
IOException, InterruptedException {
+ final long deadline = System.nanoTime() + AWAIT_TIMEOUT.toNanos();
+ String logs = "";
+ while (System.nanoTime() < deadline) {
+ logs = Deployment.gatewayLogs();
+ if (logs.contains(marker)) {
+ return logs;
+ }
+ TimeUnit.MILLISECONDS.sleep(500L);
+ }
+ fail("the log of the gateway container does not contain " + marker);
+ return logs;
+ }
+}
diff --git a/shenyu-plugin/shenyu-plugin-tracing/pom.xml
b/shenyu-plugin/shenyu-plugin-tracing/pom.xml
index 6b5fd9bbb8..ba4bff00cc 100644
--- a/shenyu-plugin/shenyu-plugin-tracing/pom.xml
+++ b/shenyu-plugin/shenyu-plugin-tracing/pom.xml
@@ -56,6 +56,14 @@
<artifactId>opentelemetry-sdk-testing</artifactId>
<scope>test</scope>
</dependency>
+ <!-- the continuity of a trace is verified against the real http
client plugin, so that the
+ span of the gateway and the injected context are covered together
-->
+ <dependency>
+ <groupId>org.apache.shenyu</groupId>
+ <artifactId>shenyu-plugin-httpclient</artifactId>
+ <version>${project.version}</version>
+ <scope>test</scope>
+ </dependency>
</dependencies>
</project>
diff --git
a/shenyu-plugin/shenyu-plugin-tracing/src/test/java/org/apache/shenyu/plugin/tracing/TraceContinuityTest.java
b/shenyu-plugin/shenyu-plugin-tracing/src/test/java/org/apache/shenyu/plugin/tracing/TraceContinuityTest.java
new file mode 100644
index 0000000000..2ae2ec42dc
--- /dev/null
+++
b/shenyu-plugin/shenyu-plugin-tracing/src/test/java/org/apache/shenyu/plugin/tracing/TraceContinuityTest.java
@@ -0,0 +1,206 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.shenyu.plugin.tracing;
+
+import com.sun.net.httpserver.HttpServer;
+import io.opentelemetry.api.trace.Span;
+import io.opentelemetry.api.trace.SpanContext;
+import io.opentelemetry.sdk.testing.exporter.InMemorySpanExporter;
+import io.opentelemetry.sdk.trace.SdkTracerProvider;
+import io.opentelemetry.sdk.trace.data.SpanData;
+import io.opentelemetry.sdk.trace.export.SimpleSpanProcessor;
+import io.opentelemetry.sdk.trace.samplers.Sampler;
+import org.apache.shenyu.common.constant.Constants;
+import org.apache.shenyu.common.enums.RpcTypeEnum;
+import org.apache.shenyu.plugin.api.ShenyuPluginChain;
+import org.apache.shenyu.plugin.api.context.ShenyuContext;
+import org.apache.shenyu.plugin.httpclient.WebClientPlugin;
+import org.apache.shenyu.plugin.tracing.extract.TraceContextExtractor;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import org.springframework.http.HttpHeaders;
+import org.springframework.mock.http.server.reactive.MockServerHttpRequest;
+import org.springframework.mock.web.server.MockServerWebExchange;
+import org.springframework.web.reactive.function.client.WebClient;
+import org.springframework.web.server.ServerWebExchange;
+import reactor.core.publisher.Mono;
+import reactor.test.StepVerifier;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.net.InetSocketAddress;
+import java.net.URI;
+import java.util.List;
+import java.util.Objects;
+import java.util.concurrent.atomic.AtomicReference;
+
+/**
+ * The Test Case For the continuity of a trace across the gateway.
+ *
+ * <p>The real tracing plugin and the real http client plugin are wired
together against a local
+ * upstream, so the whole path of a request is covered: the span of the
gateway, the context which is
+ * injected into the outgoing call, and the parent relationship the upstream
receives.</p>
+ */
+public class TraceContinuityTest {
+
+ private static final String PATH = "/http/order";
+
+ private static final String CLIENT_TRACE_ID =
"4bf92f3577b34da6a3ce929d0e0e4736";
+
+ private static final String CLIENT_SPAN_ID = "00f067aa0ba902b7";
+
+ private static final String BAGGAGE = "baggage";
+
+ private final TraceContextExtractor extractor = new
TraceContextExtractor();
+
+ private final AtomicReference<List<String>> receivedTraceParents = new
AtomicReference<>();
+
+ private final AtomicReference<String> receivedTraceState = new
AtomicReference<>();
+
+ private final AtomicReference<String> receivedBaggage = new
AtomicReference<>();
+
+ private InMemorySpanExporter spanExporter;
+
+ private SdkTracerProvider tracerProvider;
+
+ private HttpServer upstream;
+
+ @BeforeEach
+ public void setUp() throws IOException {
+ this.spanExporter = InMemorySpanExporter.create();
+ this.tracerProvider = SdkTracerProvider.builder()
+ .addSpanProcessor(SimpleSpanProcessor.create(spanExporter))
+ .setSampler(Sampler.parentBased(Sampler.alwaysOn()))
+ .build();
+ this.upstream = HttpServer.create(new InetSocketAddress("127.0.0.1",
0), 0);
+ this.upstream.createContext(PATH, received -> {
+
receivedTraceParents.set(received.getRequestHeaders().get(TracingConstants.TRACEPARENT));
+
receivedTraceState.set(received.getRequestHeaders().getFirst(TracingConstants.TRACESTATE));
+
receivedBaggage.set(received.getRequestHeaders().getFirst(BAGGAGE));
+ try (InputStream body = received.getRequestBody()) {
+ body.readAllBytes();
+ }
+ received.sendResponseHeaders(200, -1);
+ received.close();
+ });
+ this.upstream.start();
+ }
+
+ @AfterEach
+ public void tearDown() {
+ upstream.stop(0);
+ tracerProvider.close();
+ }
+
+ @Test
+ public void testTheUpstreamContinuesTheTraceOfTheClient() {
+ ServerWebExchange exchange = createExchange("00-" + CLIENT_TRACE_ID +
"-" + CLIENT_SPAN_ID + "-01", null, null);
+ route(exchange);
+ // the whole path carries one trace: the client started it, the
gateway continued it, and the
+ // upstream call became a child of the gateway span
+ List<String> traceParents = receivedTraceParents.get();
+ Assertions.assertEquals(1, traceParents.size());
+ SpanContext upstreamParent = upstreamParent(traceParents.get(0));
+ Assertions.assertEquals(CLIENT_TRACE_ID, upstreamParent.getTraceId());
+ SpanData gatewaySpan = onlyFinishedSpan();
+ Assertions.assertEquals(gatewaySpan.getSpanId(),
upstreamParent.getSpanId());
+ Assertions.assertEquals(CLIENT_SPAN_ID, gatewaySpan.getParentSpanId());
+ Assertions.assertEquals(CLIENT_TRACE_ID, gatewaySpan.getTraceId());
+ }
+
+ @Test
+ public void testTheGatewayStartsTheTraceWithoutAnIncomingContext() {
+ ServerWebExchange exchange = createExchange(null, null, null);
+ route(exchange);
+ List<String> traceParents = receivedTraceParents.get();
+ Assertions.assertEquals(1, traceParents.size());
+ SpanData gatewaySpan = onlyFinishedSpan();
+ Assertions.assertEquals(gatewaySpan.getTraceId(),
upstreamParent(traceParents.get(0)).getTraceId());
+ Assertions.assertFalse(gatewaySpan.getParentSpanContext().isValid());
+ }
+
+ @Test
+ public void testTheTraceStateAndTheBaggageAreForwardedToTheUpstream() {
+ ServerWebExchange exchange = createExchange("00-" + CLIENT_TRACE_ID +
"-" + CLIENT_SPAN_ID + "-01",
+ "vendor=value", "userId=alice");
+ route(exchange);
+ // the propagation entries of the client are forwarded, while the
traceparent is rebuilt from
+ // the context of the gateway span
+ Assertions.assertEquals("vendor=value", receivedTraceState.get());
+ Assertions.assertEquals("userId=alice", receivedBaggage.get());
+ Assertions.assertEquals(1, receivedTraceParents.get().size());
+ }
+
+ /**
+ * Runs the request through the tracing plugin and the http client plugin,
ending at the upstream.
+ *
+ * @param exchange the current exchange
+ */
+ private void route(final ServerWebExchange exchange) {
+ final ShenyuPluginChain resource = finalExchange -> Mono.empty();
+ final WebClientPlugin webClientPlugin = new
WebClientPlugin(WebClient.builder().build(), Constants.BYTES_PER_MB);
+ // the plugin chain of the gateway invokes a plugin when the request
is subscribed, which is
+ // after the tracing plugin has created its span, so the chain of this
test is lazy as well
+ final ShenyuPluginChain client = finalExchange -> Mono.defer(
+ () -> webClientPlugin.execute(finalExchange, resource));
+ final TracingPlugin tracingPlugin = new TracingPlugin(
+ tracerProvider.get(TracingConstants.INSTRUMENTATION_NAME));
+ StepVerifier.create(tracingPlugin.execute(exchange,
client)).expectSubscription().verifyComplete();
+ }
+
+ private ServerWebExchange createExchange(final String traceparent, final
String traceState,
+ final String baggage) {
+ MockServerHttpRequest.BaseBuilder<?> request =
MockServerHttpRequest.get(PATH);
+ if (Objects.nonNull(traceparent)) {
+ request.header(TracingConstants.TRACEPARENT, traceparent);
+ }
+ if (Objects.nonNull(traceState)) {
+ request.header(TracingConstants.TRACESTATE, traceState);
+ }
+ if (Objects.nonNull(baggage)) {
+ request.header(BAGGAGE, baggage);
+ }
+ ServerWebExchange exchange =
MockServerWebExchange.from(request.build());
+ ShenyuContext shenyuContext = new ShenyuContext();
+ shenyuContext.setRpcType(RpcTypeEnum.HTTP.getName());
+ exchange.getAttributes().put(Constants.CONTEXT, shenyuContext);
+ exchange.getAttributes().put(Constants.HTTP_URI,
+ URI.create("http://127.0.0.1:"; +
upstream.getAddress().getPort() + PATH));
+ return exchange;
+ }
+
+ /**
+ * Parses the context the upstream received, the way the upstream service
would continue it.
+ *
+ * @param traceparent the received traceparent
+ * @return the parent context of the upstream call
+ */
+ private SpanContext upstreamParent(final String traceparent) {
+ HttpHeaders headers = new HttpHeaders();
+ headers.add(TracingConstants.TRACEPARENT, traceparent);
+ return Span.fromContext(extractor.extract(headers)).getSpanContext();
+ }
+
+ private SpanData onlyFinishedSpan() {
+ List<SpanData> spans = spanExporter.getFinishedSpanItems();
+ Assertions.assertEquals(1, spans.size());
+ return spans.get(0);
+ }
+}
