This is an automated email from the ASF dual-hosted git repository.
Gabriel39 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new cce55108667 [fix](fe) Bundle Azure io_uring transport dependencies
(#69420)
cce55108667 is described below
commit cce551086679dbdc2c2b244e16643dadab986cd5
Author: Gabriel <[email protected]>
AuthorDate: Thu Oct 22 21:04:37 2026 +0800
[fix](fe) Bundle Azure io_uring transport dependencies (#69420)
### What problem does this PR solve?
Two Azure regression suites fail because the FE Azure filesystem plugin
mixes its own Netty core classes with io_uring transport classes loaded
from the FE host. When Reactor Netty creates the HTTP client's native
event loop, the JVM rejects the incompatible class identities with an
`IoHandlerFactory` loader-constraint violation.
This is a Doris plugin dependency-packaging problem. Both failures share
the same cause; their test assertions and Azure credentials do not need
to change.
### Failing regression cases
Both suites are under
`regression-test/suites/external_table_p2/refactor_catalog_param/`:
| Suite | Source file | Observed failure |
| --- | --- | --- |
| `azure_non_catalog_all_test` | `azure_non_catalog_all_test.groovy` |
FE-side Azure file access fails while the Azure SDK creates its HTTP
event loop. This prevents the suite from completing its Azure
file-access checks. |
| `refactor_storage_backup_restore_azure` |
`backup_restore_azure.groovy` | Azure repository creation encounters the
same class-loading failure. The suite fails before exercising backup and
restore. |
The common failure path is:
```text
Azure SDK HTTP request
-> NettyAsyncHttpClient.sendSync
-> DefaultLoopResources.cacheNativeClientLoops / cacheNativeServerLoops
-> DefaultLoopIOUring.newEventLoopGroup
-> MultiThreadIoEventLoopGroup / IoUringIoHandler
-> LinkageError: loader constraint violation involving
io.netty.channel.IoHandlerFactory
```
The exception occurs during local HTTP transport initialization, before
the affected HTTP request can reach Azure. It is not an Azure
authentication rejection or a regression-test expected-result mismatch.
### Root cause and trigger conditions
The inspected failing artifact is Doris commit `6ca5ef36d8`, with Netty
`4.2.17.Final`, Reactor Netty `1.3.7`, and Azure Core HTTP Netty
`1.16.4`.
1. The Azure plugin bundles Reactor Netty and Netty core, but omits
`netty-transport-classes-io_uring` and the corresponding native
libraries.
2. The FE host does contain the io_uring artifacts. Doris loads
plugin-private classes child-first, then falls back to the parent when a
class is absent from the plugin.
3. Reactor's native transport detection can therefore discover the
host's io_uring implementation. When io_uring is available and selected,
it combines that implementation with the plugin's own Netty event-loop
classes.
4. Java identifies a type by both its name and its defining class
loader. The two copies of `IoHandlerFactory` are incompatible even
though the host and plugin use the same Netty version.
The original directory-loaded artifact produced this class ownership in
the local reproduction:
| Class | Defining loader |
| --- | --- |
| `reactor.netty.resources.DefaultLoopIOUring` | Plugin
`ChildFirstClassLoader` |
| `io.netty.channel.MultiThreadIoEventLoopGroup` | Plugin
`ChildFirstClassLoader` |
| `io.netty.channel.IoHandlerFactory` | Plugin `ChildFirstClassLoader` |
| `io.netty.channel.uring.IoUringIoHandler` | Host `AppClassLoader` |
Directly exercising `DefaultLoopIOUring.newEventLoopGroup` reproduced
the same category of `IoHandlerFactory` linkage failure without
accessing Azure. The exact error wording depends on which loader
resolves the interface first.
### Impact scope
- **Confirmed affected:** FE Azure file access and Azure repository
creation in the two suites above.
- **Potentially affected through the same client:** FE operations that
use `AzureObjStorage` and its Azure SDK HTTP client, including object
listing, metadata lookup, reads, writes, copies, and deletes. These
operations share the transport initialization path; they have not all
been independently reproduced.
- **Environment-dependent trigger:** the directory-loaded Azure plugin
has the incomplete dependency set, the host supplies io_uring, and
Reactor selects that native transport. Environments that use another
transport may not exhibit this failure, which explains why an ordinary
flat-classpath unit test can pass.
- **Boundary:** the evidence establishes an FE Azure plugin failure. It
does not establish the same defect in BE-native Azure clients or other
filesystem providers. This PR changes only Azure dependency packaging
and its test support, not global class-loading policy, SQL semantics, or
credentials.
- **Version boundary:** the affected artifact above is confirmed. The
first introducing commit and the oldest affected release have not been
established; this PR does not claim that every Azure deployment or
release is affected.
### Fix
Bundle these dependencies in the Azure plugin, using the existing Netty
BOM for their versions:
- `netty-transport-classes-io_uring`
- `netty-transport-native-io_uring:linux-x86_64`
- `netty-transport-native-io_uring:linux-aarch_64`
This keeps the io_uring implementation, its Netty interfaces, and native
resources inside the plugin's loader. Bundling both native classifiers
also avoids relying on the host for native resources. No
single-interface parent-first exception or process-wide Netty loading
change is introduced.
Add three isolated-loader tests using the Maven **runtime** dependency
set, excluding dependencies supplied only by the test/provided
classpath:
1. Verify io_uring and its Netty interfaces are defined by the plugin
loader and method signatures resolve to the same interface type.
2. Verify both supported Linux native resources are available in the
plugin and resource lookup does not mix host resources into the result.
3. Initialize host io_uring first, then create a plugin event loop,
execute a task, and await shutdown. Dispose plugin-local Reactor
schedulers as well so they do not retain the test loader.
Regenerate the test runtime-classpath file on partial Maven cache
restores, since tests may execute after cached generation steps and the
file contains checkout-specific dependency paths.
### Validation and remaining coverage
- Reproduced the original directory-plugin class-loader conflict before
the fix without an Azure connection.
- The new tests fail before the dependency fix because io_uring
classes/native resources are absent. After the fix, all **94 Azure
module unit tests pass**, including the three new tests.
- Inspected the actual assembled plugin zip and confirmed it contains
all three dependencies. Loaded that bundle through the real directory
plugin manager: the relevant Netty classes all come from the plugin
loader, and an event-loop task executes successfully.
- FE build/package, final FE Checkstyle, Maven cache-configuration
validation, and `git diff --check` pass. The FE-only local build
bypassed an unrelated preflight check for missing BE native archives;
that temporary build-script adjustment was restored and is not included
in this PR.
- **Local limitation:** the kernel disables io_uring ring creation.
Class-identity and native-resource checks run unconditionally, while
successful event-loop execution exercises automatic fallback on this
host. An io_uring-enabled execution and complete credential-backed runs
of the two original Azure suites have not been validated locally and
require CI coverage.
### Release note
Fix FE Azure file access and repository creation failures caused by
mixed host/plugin Netty classes when io_uring is selected.
### Check List (For Author)
- Test:
- [x] Unit test
- [x] Manual test
- Behavior changed:
- [x] Fix dependency packaging; no SQL or configuration changes.
- Does this need documentation?
- [x] No.
---
fe/.mvn/maven-build-cache-config.xml | 7 ++
fe/fe-filesystem/fe-filesystem-azure/pom.xml | 49 +++++++++
.../azure/AzureNettyClassLoaderTest.java | 111 +++++++++++++++++++++
3 files changed, 167 insertions(+)
diff --git a/fe/.mvn/maven-build-cache-config.xml
b/fe/.mvn/maven-build-cache-config.xml
index 3643d9a32b1..905cc232355 100644
--- a/fe/.mvn/maven-build-cache-config.xml
+++ b/fe/.mvn/maven-build-cache-config.xml
@@ -70,6 +70,13 @@ under the License.
<execId>validate</execId>
</execIds>
</execution>
+ <!-- Partial cache restores may still run plugin isolation
tests. Regenerate their
+ runtime classpath with this checkout's paths instead of
caching the local file. -->
+ <execution artifactId="maven-dependency-plugin">
+ <execIds>
+ <execId>plugin-runtime-classpath</execId>
+ </execIds>
+ </execution>
<!-- The wipe of target/lib that be-java-extensions/pom.xml
binds right before the
copy-dependencies forced below. The two run as a pair or
not at all: a copy
without the wipe deploys the union of every dependency
set the directory has
diff --git a/fe/fe-filesystem/fe-filesystem-azure/pom.xml
b/fe/fe-filesystem/fe-filesystem-azure/pom.xml
index cb76d8c2f09..159e0bb7328 100644
--- a/fe/fe-filesystem/fe-filesystem-azure/pom.xml
+++ b/fe/fe-filesystem/fe-filesystem-azure/pom.xml
@@ -100,11 +100,36 @@ under the License.
<artifactId>azure-identity</artifactId>
</dependency>
+ <!-- Reactor probes optional transports. Keep io_uring in the plugin
so its Netty types and
+ native resources cannot fall back to the host's different
ClassLoader. -->
+ <dependency>
+ <groupId>io.netty</groupId>
+ <artifactId>netty-transport-classes-io_uring</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>io.netty</groupId>
+ <artifactId>netty-transport-native-io_uring</artifactId>
+ <classifier>linux-x86_64</classifier>
+ <scope>runtime</scope>
+ </dependency>
+ <dependency>
+ <groupId>io.netty</groupId>
+ <artifactId>netty-transport-native-io_uring</artifactId>
+ <classifier>linux-aarch_64</classifier>
+ <scope>runtime</scope>
+ </dependency>
+
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api</artifactId>
</dependency>
+ <dependency>
+ <groupId>org.apache.doris</groupId>
+ <artifactId>fe-extension-loader</artifactId>
+ <version>${revision}</version>
+ <scope>test</scope>
+ </dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter</artifactId>
@@ -125,6 +150,30 @@ under the License.
<build>
<finalName>doris-fe-filesystem-azure</finalName>
<plugins>
+ <plugin>
+ <artifactId>maven-dependency-plugin</artifactId>
+ <executions>
+ <execution>
+ <id>plugin-runtime-classpath</id>
+ <phase>generate-test-resources</phase>
+ <goals>
+ <goal>build-classpath</goal>
+ </goals>
+ <configuration>
+ <includeScope>runtime</includeScope>
+
<outputFile>${project.build.directory}/plugin-runtime-classpath.txt</outputFile>
+ </configuration>
+ </execution>
+ </executions>
+ </plugin>
+ <plugin>
+ <artifactId>maven-surefire-plugin</artifactId>
+ <configuration>
+ <systemPropertyVariables>
+
<plugin.runtime.classpath>${project.build.directory}/plugin-runtime-classpath.txt</plugin.runtime.classpath>
+ </systemPropertyVariables>
+ </configuration>
+ </plugin>
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<configuration>
diff --git
a/fe/fe-filesystem/fe-filesystem-azure/src/test/java/org/apache/doris/filesystem/azure/AzureNettyClassLoaderTest.java
b/fe/fe-filesystem/fe-filesystem-azure/src/test/java/org/apache/doris/filesystem/azure/AzureNettyClassLoaderTest.java
new file mode 100644
index 00000000000..bc47093d35b
--- /dev/null
+++
b/fe/fe-filesystem/fe-filesystem-azure/src/test/java/org/apache/doris/filesystem/azure/AzureNettyClassLoaderTest.java
@@ -0,0 +1,111 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.filesystem.azure;
+
+import org.apache.doris.extension.loader.ChildFirstClassLoader;
+
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.Test;
+
+import java.io.File;
+import java.net.URL;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.time.Duration;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.TimeUnit;
+import java.util.regex.Pattern;
+
+class AzureNettyClassLoaderTest {
+ private static final String IO_HANDLER_FACTORY =
"io.netty.channel.IoHandlerFactory";
+ private static final String URING_HANDLER =
"io.netty.channel.uring.IoUringIoHandler";
+
+ private ChildFirstClassLoader pluginLoader() throws Exception {
+ // Surefire also exposes provided/test dependencies, which can hide a
missing plugin runtime dependency.
+ String classpath =
Files.readString(Path.of(System.getProperty("plugin.runtime.classpath"))).trim();
+ List<URL> urls = new ArrayList<>();
+ for (String entry :
classpath.split(Pattern.quote(File.pathSeparator))) {
+ urls.add(Path.of(entry).toUri().toURL());
+ }
+ return new ChildFirstClassLoader(urls.toArray(new URL[0]),
getClass().getClassLoader(),
+ ChildFirstClassLoader.DEFAULT_PARENT_FIRST_PACKAGES);
+ }
+
+ @Test
+ void ioUringTypesStayInThePluginLoader() throws Exception {
+ try (ChildFirstClassLoader loader = pluginLoader()) {
+ Class<?> factory = Class.forName(IO_HANDLER_FACTORY, false,
loader);
+ Class<?> handler = Class.forName(URING_HANDLER, false, loader);
+ Assertions.assertSame(loader, factory.getClassLoader());
+ Assertions.assertSame(loader, handler.getClassLoader());
+ Assertions.assertSame(loader,
Class.forName("io.netty.channel.MultiThreadIoEventLoopGroup", false, loader)
+ .getClassLoader());
+ // Resolving the signature must use the plugin's interface even
when the host also carries Netty.
+ Assertions.assertSame(factory,
handler.getMethod("newFactory").getReturnType());
+ }
+ }
+
+ @Test
+ void nativeLibrariesAreAvailableWithoutTheHost() throws Exception {
+ try (ChildFirstClassLoader loader = pluginLoader()) {
+ for (String architecture : List.of("x86_64", "aarch_64")) {
+ String name =
"META-INF/native/libnetty_transport_native_io_uring42_" + architecture + ".so";
+ URL resource = loader.findResource(name);
+ Assertions.assertNotNull(resource, "Missing plugin native
library: " + name);
+ Assertions.assertEquals(List.of(resource),
Collections.list(loader.getResources(name)));
+ }
+ }
+ }
+
+ @Test
+ void nativeSelectionWorksAfterHostInitialization() throws Exception {
+ // Initialize the host first to cover native libraries already loaded
by a different ClassLoader.
+
Class.forName("io.netty.channel.uring.IoUring").getMethod("isAvailable").invoke(null);
+ try (ChildFirstClassLoader loader = pluginLoader()) {
+ ClassLoader previous =
Thread.currentThread().getContextClassLoader();
+ Thread.currentThread().setContextClassLoader(loader);
+ try {
+ Class<?> loops =
Class.forName("reactor.netty.resources.LoopResources", true, loader);
+ Object resources = loops.getMethod("create", String.class,
int.class, boolean.class)
+ .invoke(null, "azure-plugin-test", 1, true);
+ try {
+ Object group = loops.getMethod("onClient",
boolean.class).invoke(resources, true);
+ Assertions.assertSame(loader,
group.getClass().getClassLoader());
+ Assertions.assertEquals("executed", ((ExecutorService)
group)
+ .submit(() -> "executed").get(5,
TimeUnit.SECONDS));
+ } finally {
+ try {
+ Object completion = loops.getMethod("disposeLater",
Duration.class, Duration.class)
+ .invoke(resources, Duration.ZERO,
Duration.ofSeconds(5));
+ Class.forName("reactor.core.publisher.Mono", true,
loader)
+ .getMethod("block",
Duration.class).invoke(completion, Duration.ofSeconds(10));
+ } finally {
+ // The disposal timeout initializes plugin-local
schedulers that otherwise retain the loader.
+ Class.forName("reactor.core.scheduler.Schedulers",
true, loader)
+ .getMethod("shutdownNow").invoke(null);
+ }
+ }
+ } finally {
+ Thread.currentThread().setContextClassLoader(previous);
+ }
+ }
+ }
+}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
