This is an automated email from the ASF dual-hosted git repository.
Gabriel39 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 601b259ce51 [fix](flight-sql) Report actual server and Arrow versions
(#70728)
601b259ce51 is described below
commit 601b259ce51d80e600504263ec581c23913091dc
Author: Mingyu Chen (Rayner) <[email protected]>
AuthorDate: Thu Nov 19 12:04:17 2026 +0800
[fix](flight-sql) Report actual server and Arrow versions (#70728)
### Background
Flight SQL clients use `GetSqlInfo` to discover server and library
metadata. Doris returns a FlightInfo ticket, and the client reads the
requested values through DoGet.
### What problem does this PR solve?
Issue Number: #67578
Related PR: #68770 (same fix on branch-4.1), #24314 (introduced the
hardcoded server version)
PR Type: Bugfix
Problem Summary:
Clients always receive server version `1.0` and Arrow version `18.2.0`
on master, even when the running FE and Arrow dependency have different
versions. Report the FE build version with its short commit hash and the
implementation version of the loaded Flight SQL library. Most of the
diff adds three Flight RPC tests.
Observed on the local build, for both all-info and explicit-code
requests:
| SqlInfo | Before | After |
|---|---|---|
| Server version | `1.0` | `doris-0.0.0-b706e757481` |
| Arrow version | `18.2.0` | `19.0.0` |
### Details
- **Frequency:** Always when requesting either version through
`GetSqlInfo` on the unfixed master.
- **Root cause:** `DorisFlightSqlProducer` registers literal versions in
its `SqlInfoBuilder`, so metadata cannot follow the running build or
loaded library.
- **Why existing tests missed it:** `DorisFlightSqlProducerTest` covers
session, prepare and error handling; `FlightSqlTableTypesTest` checks
table-type RPC metadata. Neither asserts SqlInfo versions. Regression
suites such as `test_session_options` and `test_get_tables_schema`
exercise other Flight APIs and do not request SqlInfo.
- **Fix:** Use the same FE version format as heartbeat metadata and read
Arrow's loaded package implementation version. If the manifest version
is missing or empty, return `unknown` and keep its info code registered
so explicit requests still succeed.
- **Other paths and branches:** Full and filtered requests share the
same builder; no other production version registration was found. The
server literal dates to #24314. Checked release heads: branch-3.1 and
branch-4.0 still use literals; branch-4.1 and branch-4.2 already use
runtime versions. This change targets master.
### Testing
- Locally ran `run-fe-ut.sh` for `FlightSqlServerInfoTest`,
`DorisFlightSqlProducerTest` and `FlightSqlTableTypesTest`: 21 passed,
none skipped, with Arrow allocator debug checks enabled. The three new
RPC tests all fail on the unfixed baseline because the version values
differ.
- Reran the three RPC tests with Flight SQL loaded from an exploded
directory without a manifest, then a JAR with an empty implementation
version: 3/3 passed in each case, including explicit Arrow-version
requests returning `unknown`.
- Rebuilt and restarted the local FE in a single-FE/single-BE cluster.
All-info, each version alone, and `[0, 1, 2]` GetFlightInfo/DoGet
requests match independent `SHOW FRONTENDS` and the manifest of the
Flight SQL JAR opened by the FE.
### Release note
Fix stale server and Arrow library versions returned by Flight SQL
GetSqlInfo.
### Check List (For Author)
- Behavior changed: No
- Does this need documentation: No
- Pre-submission status: Ready
### AI Disclosure
- Tool: Codex (GPT-6)
- Generated: code, tests, and this description
- Human review: not stated
---
.../doris/arrowflight/DorisFlightSqlProducer.java | 11 ++-
.../doris/arrowflight/FlightSqlServerInfoTest.java | 102 +++++++++++++++++++++
2 files changed, 111 insertions(+), 2 deletions(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/arrowflight/DorisFlightSqlProducer.java
b/fe/fe-core/src/main/java/org/apache/doris/arrowflight/DorisFlightSqlProducer.java
index ddf2d57549c..52d8b97bf6d 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/arrowflight/DorisFlightSqlProducer.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/arrowflight/DorisFlightSqlProducer.java
@@ -28,6 +28,7 @@ import
org.apache.doris.arrowflight.results.FlightSqlResultCacheEntry;
import org.apache.doris.arrowflight.sessions.FlightSessionsManager;
import org.apache.doris.common.IncrWindowNotReadyException;
import org.apache.doris.common.Status;
+import org.apache.doris.common.Version;
import org.apache.doris.common.util.DebugUtil;
import org.apache.doris.common.util.Util;
import org.apache.doris.mysql.MysqlCommand;
@@ -146,11 +147,17 @@ public class DorisFlightSqlProducer implements
FlightSqlProducer, AutoCloseable
this.location = location;
this.flightSessionsManager = flightSessionsManager;
sqlInfoBuilder = new SqlInfoBuilder();
-
sqlInfoBuilder.withFlightSqlServerName("DorisFE").withFlightSqlServerVersion("1.0")
-
.withFlightSqlServerArrowVersion("18.2.0").withFlightSqlServerReadOnly(false)
+ // Keep discovery metadata aligned with the running FE build and its
loaded Arrow dependency.
+ sqlInfoBuilder.withFlightSqlServerName("DorisFE")
+ .withFlightSqlServerVersion(Version.DORIS_BUILD_VERSION + "-"
+ Version.DORIS_BUILD_SHORT_HASH)
+ .withFlightSqlServerReadOnly(false)
.withSqlIdentifierQuoteChar("`").withSqlDdlCatalog(true).withSqlDdlSchema(false).withSqlDdlTable(false)
.withSqlIdentifierCase(SqlSupportedCaseSensitivity.SQL_CASE_SENSITIVITY_CASE_INSENSITIVE)
.withSqlQuotedIdentifierCase(SqlSupportedCaseSensitivity.SQL_CASE_SENSITIVITY_CASE_INSENSITIVE);
+ String arrowVersion =
SqlInfoBuilder.class.getPackage().getImplementationVersion();
+ // Keep the info code registered even without a manifest:
SqlInfoBuilder rejects unknown codes.
+ sqlInfoBuilder.withFlightSqlServerArrowVersion(
+ arrowVersion == null || arrowVersion.isEmpty() ? "unknown" :
arrowVersion);
}
private static ByteBuffer serializeMetadata(final Schema schema) {
diff --git
a/fe/fe-core/src/test/java/org/apache/doris/arrowflight/FlightSqlServerInfoTest.java
b/fe/fe-core/src/test/java/org/apache/doris/arrowflight/FlightSqlServerInfoTest.java
new file mode 100644
index 00000000000..ab81af26d62
--- /dev/null
+++
b/fe/fe-core/src/test/java/org/apache/doris/arrowflight/FlightSqlServerInfoTest.java
@@ -0,0 +1,102 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.arrowflight;
+
+import org.apache.doris.arrowflight.sessions.FlightSessionsManager;
+import org.apache.doris.common.Version;
+
+import org.apache.arrow.flight.FlightClient;
+import org.apache.arrow.flight.FlightInfo;
+import org.apache.arrow.flight.FlightServer;
+import org.apache.arrow.flight.FlightStream;
+import org.apache.arrow.flight.Location;
+import org.apache.arrow.flight.sql.FlightSqlClient;
+import org.apache.arrow.flight.sql.FlightSqlProducer.Schemas;
+import org.apache.arrow.flight.sql.SqlInfoBuilder;
+import org.apache.arrow.flight.sql.impl.FlightSql.SqlInfo;
+import org.apache.arrow.memory.RootAllocator;
+import org.apache.arrow.vector.VectorSchemaRoot;
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.Test;
+import org.mockito.Mockito;
+
+import java.util.HashMap;
+import java.util.Map;
+
+public class FlightSqlServerInfoTest {
+ @Test
+ public void reportsBuildVersionsInAllSqlInfo() throws Exception {
+ Map<Integer, Object> info = readSqlInfo();
+ Assertions.assertEquals("DorisFE",
info.get(SqlInfo.FLIGHT_SQL_SERVER_NAME_VALUE));
+ Assertions.assertEquals(Version.DORIS_BUILD_VERSION + "-" +
Version.DORIS_BUILD_SHORT_HASH,
+ info.get(SqlInfo.FLIGHT_SQL_SERVER_VERSION_VALUE));
+ assertArrowVersion(info);
+ Assertions.assertEquals(false,
info.get(SqlInfo.FLIGHT_SQL_SERVER_READ_ONLY_VALUE));
+ }
+
+ @Test
+ public void reportsOnlyRequestedServerVersion() throws Exception {
+ Map<Integer, Object> info =
readSqlInfo(SqlInfo.FLIGHT_SQL_SERVER_VERSION);
+ Assertions.assertEquals(1, info.size());
+ Assertions.assertEquals(Version.DORIS_BUILD_VERSION + "-" +
Version.DORIS_BUILD_SHORT_HASH,
+ info.get(SqlInfo.FLIGHT_SQL_SERVER_VERSION_VALUE));
+ }
+
+ @Test
+ public void reportsOnlyRequestedArrowVersion() throws Exception {
+ Map<Integer, Object> info =
readSqlInfo(SqlInfo.FLIGHT_SQL_SERVER_ARROW_VERSION);
+ Assertions.assertEquals(1, info.size());
+ assertArrowVersion(info);
+ }
+
+ private static void assertArrowVersion(Map<Integer, Object> info) {
+ String version =
SqlInfoBuilder.class.getPackage().getImplementationVersion();
+ // Exploded dependencies may omit the manifest; the code must still be
registered.
+ Assertions.assertEquals(version == null || version.isEmpty() ?
"unknown" : version,
+ info.get(SqlInfo.FLIGHT_SQL_SERVER_ARROW_VERSION_VALUE));
+ }
+
+ private static Map<Integer, Object> readSqlInfo(SqlInfo... requestedInfo)
throws Exception {
+ Location location = Location.forGrpcInsecure("127.0.0.1", 0);
+ FlightSessionsManager sessions =
Mockito.mock(FlightSessionsManager.class);
+ try (DorisFlightSqlProducer producer = new
DorisFlightSqlProducer(location, sessions);
+ RootAllocator allocator = new RootAllocator();
+ FlightServer server = FlightServer.builder(allocator,
location, producer).build().start();
+ FlightClient client = FlightClient.builder(allocator,
+ Location.forGrpcInsecure("127.0.0.1",
server.getPort())).build()) {
+ FlightSqlClient sql = new FlightSqlClient(client);
+ FlightInfo info = sql.getSqlInfo(requestedInfo);
+ Assertions.assertEquals(Schemas.GET_SQL_INFO_SCHEMA,
info.getSchema());
+ Assertions.assertEquals(1, info.getEndpoints().size());
+ Map<Integer, Object> values = new HashMap<>();
+ try (FlightStream stream =
client.getStream(info.getEndpoints().get(0).getTicket())) {
+ Assertions.assertEquals(info.getSchema(), stream.getSchema());
+ while (stream.next()) {
+ VectorSchemaRoot root = stream.getRoot();
+ for (int row = 0; row < root.getRowCount(); ++row) {
+ int code = ((Number)
root.getVector("info_name").getObject(row)).intValue();
+ Object value = root.getVector("value").getObject(row);
+ Assertions.assertNull(values.put(code, value
instanceof Boolean ? value : value.toString()));
+ }
+ }
+ }
+ Mockito.verifyNoInteractions(sessions);
+ return values;
+ }
+ }
+}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]