This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 0c01156be7f [feat](thirdparty) add arrow-adbc to the thirdparty build
(#66358)
0c01156be7f is described below
commit 0c01156be7f468ce73a4d903f3f374668e7a7548
Author: Mingyu Chen (Rayner) <[email protected]>
AuthorDate: Sun Aug 2 23:44:09 2026 +0800
[feat](thirdparty) add arrow-adbc to the thirdparty build (#66358)
### What problem does this PR solve?
Related Issue: #65615
Related PR: #66331
Problem Summary:
Split out of #66331, which adds an `adbc` catalog type that reads an
external source
through an [Arrow Database Connectivity](https://arrow.apache.org/adbc/)
driver. This PR
carries only that PR's `thirdparty/` half, so the dependency can be
reviewed and the
build-env image rebuilt before the code that links against it lands.
Nothing in the tree consumes these artifacts yet -- this PR adds one
package to the
thirdparty build and declares its license, and changes nothing else.
**What comes out of it**
| Artifact | How | Used by |
|---|---|---|
| `libadbc_driver_manager.a` | built from source | to be statically
linked into `doris_be` (#66331) |
| `libadbc_driver_jni.so` | built from source | to be loaded by the FE
ADBC connector (#66331) |
| `libadbc_driver_sqlite.so` | built from source | tests only, **not
shipped** |
| `libadbc_driver_flightsql.so` | prebuilt, from the official release
wheel | tests only, **not shipped** |
Doris ships no ADBC driver to users; a deployment supplies its own. The
two drivers above
exist so the ADBC code paths can be tested at all.
**Three things upstream does that do not carry over**
- *The SQLite driver needs a system SQLite3 development package*, which
Doris does not
ship and most build hosts lack. The source tree vendors the amalgamation
but never
references it from CMake, so it is compiled here into a scratch static
library, handed
to `FindSQLite3`, and dropped afterwards. It ends up statically inside
the driver,
leaving no sqlite artifacts in thirdparty.
- *The JNI bridge header is generated by shelling out to Maven*
(`java/driver/jni/CMakeLists.txt` runs `mvn -Pjni,javah compile`). Doing
that would make
this the first thirdparty package to require Maven, a Maven Central
connection and a
JDK 11+, while the build-env image runs this script with `JAVA_HOME` on
JDK 8. The
`javah` output is checked in as a patch instead and `jni_wrapper.cc` is
compiled against
it directly, needing nothing but `jni.h`. The patch header records how
to regenerate it
on a version bump.
The prebuilt JNI binary inside upstream's Maven jar is not used either:
it requires
`GLIBC_2.34` and `GLIBCXX_3.4.31`, which excludes CentOS 7/8, Rocky 8
and Ubuntu 20.04.
- *The Flight SQL driver is written in Go and no bare shared library is
published*, so it
is taken from the official release wheel -- a zip the existing download
step already
knows how to unpack -- rather than adding a Go toolchain to the
thirdparty build. It is
skipped on platforms upstream publishes no prebuilt binary for, the same
way hyperscan
is.
**On the version pin**
The source tree is tag `apache-arrow-adbc-24`, which is release C/Go
1.12.0 (the tag
carries neither number). The prebuilt Flight SQL driver is pinned to
that same release,
and that is not cosmetic: ADBC partition descriptors are driver-private
bytes, so every
process that hands one to another must have loaded the very same driver
build.
`dist/LICENSE-dist.txt` gets the corresponding Apache-2.0 entry. It is
the only file
outside `thirdparty/` here.
---
dist/LICENSE-dist.txt | 3 +
thirdparty/build-thirdparty.sh | 95 +++++
thirdparty/download-thirdparty.sh | 12 +
...che-arrow-adbc-24-pregenerated-jni-header.patch | 455 +++++++++++++++++++++
thirdparty/vars.sh | 40 ++
5 files changed, 605 insertions(+)
diff --git a/dist/LICENSE-dist.txt b/dist/LICENSE-dist.txt
index 1b522d2df14..b2fa999e560 100644
--- a/dist/LICENSE-dist.txt
+++ b/dist/LICENSE-dist.txt
@@ -1463,6 +1463,9 @@ The Apache Software License, Version 2.0
* brpc: 1.1.0
* rocksdb: 5.14.2
* arrow: 7.0.0
+ * arrow-adbc: 1.12.0
+ - source: https://github.com/apache/arrow-adbc
+ - only the driver manager (C++) is linked into the BE binary
* S2: 0.10.0
* croaringbitmap: 0.4.0
* Abseil LTS 20220623
diff --git a/thirdparty/build-thirdparty.sh b/thirdparty/build-thirdparty.sh
index 2a468540484..0bdbb10d0b2 100755
--- a/thirdparty/build-thirdparty.sh
+++ b/thirdparty/build-thirdparty.sh
@@ -1152,6 +1152,91 @@ build_arrow() {
strip_lib libarrow_acero.a
}
+# arrow-adbc
+# Produces three artifacts from one source tree:
+# libadbc_driver_manager.a -- statically linked into doris_be
+# libadbc_driver_jni.so -- loaded by the FE adbc connector
+# libadbc_driver_sqlite.so -- BE unit tests only, not shipped
+# and installs a fourth that is not built here:
+# libadbc_driver_flightsql.so -- prebuilt, adbc tests only, not shipped
+build_arrow_adbc() {
+ check_if_source_exist "${ARROW_ADBC_SOURCE}"
+
+ local adbc_src="${TP_SOURCE_DIR}/${ARROW_ADBC_SOURCE}"
+
+ # The SQLite driver needs a SQLite3 development package, which Doris does
not
+ # ship and most build hosts lack. arrow-adbc vendors the amalgamation, so
build
+ # it here into a scratch prefix and hand the paths to FindSQLite3. It ends
up
+ # statically inside libadbc_driver_sqlite.so, so nothing sqlite is
installed.
+ local sqlite_host="${adbc_src}/c/${BUILD_DIR}-sqlite-host"
+ rm -rf "${sqlite_host}"
+ mkdir -p "${sqlite_host}/include" "${sqlite_host}/lib"
+ "${CC}" -O2 -fPIC -DSQLITE_ENABLE_COLUMN_METADATA=1 \
+ -c "${adbc_src}/c/vendor/sqlite3/sqlite3.c" -o
"${sqlite_host}/sqlite3.o"
+ ar rcs "${sqlite_host}/lib/libsqlite3.a" "${sqlite_host}/sqlite3.o"
+ cp -f "${adbc_src}/c/vendor/sqlite3/sqlite3.h" "${sqlite_host}/include/"
+
+ # (1) driver manager + sqlite driver
+ cd "${adbc_src}/c"
+ rm -rf "${BUILD_DIR}"
+ mkdir -p "${BUILD_DIR}"
+ cd "${BUILD_DIR}"
+
+ "${CMAKE_CMD}" -G "${GENERATOR}" -DCMAKE_POLICY_VERSION_MINIMUM=3.5 \
+ -DCMAKE_BUILD_TYPE=Release \
+ -DCMAKE_INSTALL_PREFIX="${TP_INSTALL_DIR}" \
+ -DCMAKE_INSTALL_LIBDIR=lib64 \
+ -DCMAKE_POSITION_INDEPENDENT_CODE=ON \
+ -DADBC_DRIVER_MANAGER=ON \
+ -DADBC_DRIVER_SQLITE=ON \
+ -DADBC_BUILD_STATIC=ON \
+ -DADBC_BUILD_SHARED=ON \
+ -DADBC_BUILD_TESTS=OFF \
+ -DADBC_USE_CCACHE=OFF \
+ -DSQLite3_INCLUDE_DIR="${sqlite_host}/include" \
+ -DSQLite3_LIBRARY="${sqlite_host}/lib/libsqlite3.a" \
+ ..
+ "${BUILD_SYSTEM}" -j "${PARALLEL}"
+ "${BUILD_SYSTEM}" install
+
+ # (2) JNI bridge for the FE. Must come after (1): it links the driver
manager.
+ # Built directly rather than through upstream's java/CMakeLists.txt,
which
+ # shells out to Maven just to generate the JNI header; that header is
+ # checked in as a patch instead (see thirdparty/patches). Also not
taken
+ # from the Maven jar: the prebuilt binary there requires GLIBC_2.34 and
+ # GLIBCXX_3.4.31, which excludes CentOS 7/8, Rocky 8, Ubuntu 20.04 and
more.
+ # Only jni.h is needed, so any JDK will do.
+ if [[ ! -f "${JAVA_HOME}/include/jni.h" ]]; then
+ echo "arrow-adbc: JAVA_HOME must point at a JDK (no
${JAVA_HOME}/include/jni.h)"
+ exit 1
+ fi
+ local jni_md_dir='linux'
+ if [[ "${KERNEL}" == 'Darwin' ]]; then
+ jni_md_dir='darwin'
+ fi
+
+ "${CXX}" -std="c++${TP_CXX_STANDARD}" -O2 -fPIC -shared \
+ -I"${JAVA_HOME}/include" \
+ -I"${JAVA_HOME}/include/${jni_md_dir}" \
+ -I"${adbc_src}/java/driver/jni/doris_generated" \
+ -I"${TP_INCLUDE_DIR}" \
+ "${adbc_src}/java/driver/jni/src/main/cpp/jni_wrapper.cc" \
+ -o "${TP_INSTALL_DIR}/lib64/libadbc_driver_jni.so" \
+ "${TP_INSTALL_DIR}/lib64/libadbc_driver_manager.a"
+
+ # (3) Flight SQL driver. Not built: upstream implements it in Go, so it
comes
+ # prebuilt out of the release wheel that download-thirdparty.sh
unpacked
+ # (see vars.sh). Nothing links against it; the FE and BE dlopen it at
run
+ # time, and only the adbc tests ask for it.
+ if [[ -n "${ARROW_ADBC_FLIGHTSQL_SOURCE}" ]]; then
+ check_if_source_exist "${ARROW_ADBC_FLIGHTSQL_SOURCE}"
+ cp -f
"${TP_SOURCE_DIR}/${ARROW_ADBC_FLIGHTSQL_SOURCE}/libadbc_driver_flightsql.so" \
+ "${TP_INSTALL_DIR}/lib64/libadbc_driver_flightsql.so"
+ fi
+
+ rm -rf "${sqlite_host}"
+}
+
# abseil
build_abseil() {
check_if_source_exist "${ABSEIL_SOURCE}"
@@ -2228,6 +2313,7 @@ if [[ "${#packages[@]}" -eq 0 ]]; then
cares
grpc # after cares, protobuf
arrow
+ arrow_adbc
lance_c
s2
bitshuffle
@@ -2319,6 +2405,15 @@ cleanup_package_source() {
librdkafka) src_var="LIBRDKAFKA_SOURCE" ;;
flatbuffers) src_var="FLATBUFFERS_SOURCE" ;;
arrow) src_var="ARROW_SOURCE" ;;
+ arrow_adbc)
+ # arrow_adbc also unpacks the prebuilt flightsql driver, clean both
+ if [[ -n "${ARROW_ADBC_FLIGHTSQL_SOURCE}" && -d
"${TP_SOURCE_DIR}/${ARROW_ADBC_FLIGHTSQL_SOURCE}" ]]; then
+ echo "Cleaning up source: ${ARROW_ADBC_FLIGHTSQL_SOURCE}"
+ rm -rf "${TP_SOURCE_DIR}/${ARROW_ADBC_FLIGHTSQL_SOURCE}" \
+
"${TP_SOURCE_DIR}/${ARROW_ADBC_FLIGHTSQL_SOURCE}"-*.dist-info
+ fi
+ src_var="ARROW_ADBC_SOURCE"
+ ;;
brotli) src_var="BROTLI_SOURCE" ;;
cares) src_var="CARES_SOURCE" ;;
grpc) src_var="GRPC_SOURCE" ;;
diff --git a/thirdparty/download-thirdparty.sh
b/thirdparty/download-thirdparty.sh
index b4d3e5b42d7..a17e1f4412f 100755
--- a/thirdparty/download-thirdparty.sh
+++ b/thirdparty/download-thirdparty.sh
@@ -458,6 +458,18 @@ if [[ " ${TP_ARCHIVES[*]} " =~ " ARROW " ]]; then
echo "Finished patching ${ARROW_SOURCE}"
fi
+# arrow-adbc patch adds the pre-generated JNI header, so that building the JNI
+# bridge needs no Maven. See the patch header for how to regenerate it.
+if [[ " ${TP_ARCHIVES[*]} " =~ " ARROW_ADBC " ]]; then
+ cd "${TP_SOURCE_DIR}/${ARROW_ADBC_SOURCE}"
+ if [[ ! -f "${PATCHED_MARK}" ]]; then
+ patch -p1
<"${TP_PATCH_DIR}/apache-arrow-adbc-24-pregenerated-jni-header.patch"
+ touch "${PATCHED_MARK}"
+ fi
+ cd -
+ echo "Finished patching ${ARROW_ADBC_SOURCE}"
+fi
+
# patch librdkafka to avoid crash
if [[ " ${TP_ARCHIVES[*]} " =~ " LIBRDKAFKA " ]]; then
if [[ "${LIBRDKAFKA_SOURCE}" == "librdkafka-2.11.0" ]]; then
diff --git
a/thirdparty/patches/apache-arrow-adbc-24-pregenerated-jni-header.patch
b/thirdparty/patches/apache-arrow-adbc-24-pregenerated-jni-header.patch
new file mode 100644
index 00000000000..70766a1e281
--- /dev/null
+++ b/thirdparty/patches/apache-arrow-adbc-24-pregenerated-jni-header.patch
@@ -0,0 +1,455 @@
+Add the pre-generated JNI header for the adbc-driver-jni bridge.
+
+Upstream generates this header by running Maven over the Java sources
+(java/driver/jni/CMakeLists.txt calls `mvn -Pjni,javah compile`). Doing that
+here would make build-thirdparty.sh the first package to require Maven, a
+Maven Central connection and a JDK 11+ -- the official build-env image runs
+this script with JAVA_HOME pointing at JDK 8 (docker/compilation/Dockerfile).
+
+The header is pure javah output and is fully determined by the pinned version
+of
java/driver/jni/src/main/java/org/apache/arrow/adbc/driver/jni/impl/NativeAdbc.java,
+so it is checked in instead. build_arrow_adbc() compiles jni_wrapper.cc against
+it directly, needing nothing but jni.h.
+
+To regenerate when bumping arrow-adbc, run inside the extracted source tree:
+
+ cmake -S java -B /tmp/adbcjni -DCMAKE_PREFIX_PATH="${TP_INSTALL_DIR}"
+ cmake --build /tmp/adbcjni
+ cp
java/driver/jni/target/headers/org_apache_arrow_adbc_driver_jni_impl_NativeAdbc.h
\
+ java/driver/jni/doris_generated/
+
+then re-diff this file.
+
+diff -uNr
a/java/driver/jni/doris_generated/org_apache_arrow_adbc_driver_jni_impl_NativeAdbc.h
b/java/driver/jni/doris_generated/org_apache_arrow_adbc_driver_jni_impl_NativeAdbc.h
+---
a/java/driver/jni/doris_generated/org_apache_arrow_adbc_driver_jni_impl_NativeAdbc.h
1970-01-01 08:00:00.000000000 +0800
++++
b/java/driver/jni/doris_generated/org_apache_arrow_adbc_driver_jni_impl_NativeAdbc.h
2026-07-31 14:45:35.581346015 +0800
+@@ -0,0 +1,429 @@
++/* DO NOT EDIT THIS FILE - it is machine generated */
++#include <jni.h>
++/* Header for class org_apache_arrow_adbc_driver_jni_impl_NativeAdbc */
++
++#ifndef _Included_org_apache_arrow_adbc_driver_jni_impl_NativeAdbc
++#define _Included_org_apache_arrow_adbc_driver_jni_impl_NativeAdbc
++#ifdef __cplusplus
++extern "C" {
++#endif
++/*
++ * Class: org_apache_arrow_adbc_driver_jni_impl_NativeAdbc
++ * Method: openDatabase
++ * Signature:
(I[[B)Lorg/apache/arrow/adbc/driver/jni/impl/NativeDatabaseHandle;
++ */
++JNIEXPORT jobject JNICALL
Java_org_apache_arrow_adbc_driver_jni_impl_NativeAdbc_openDatabase
++ (JNIEnv *, jclass, jint, jobjectArray);
++
++/*
++ * Class: org_apache_arrow_adbc_driver_jni_impl_NativeAdbc
++ * Method: closeDatabase
++ * Signature: (J)V
++ */
++JNIEXPORT void JNICALL
Java_org_apache_arrow_adbc_driver_jni_impl_NativeAdbc_closeDatabase
++ (JNIEnv *, jclass, jlong);
++
++/*
++ * Class: org_apache_arrow_adbc_driver_jni_impl_NativeAdbc
++ * Method: openConnection
++ * Signature:
(J)Lorg/apache/arrow/adbc/driver/jni/impl/NativeConnectionHandle;
++ */
++JNIEXPORT jobject JNICALL
Java_org_apache_arrow_adbc_driver_jni_impl_NativeAdbc_openConnection
++ (JNIEnv *, jclass, jlong);
++
++/*
++ * Class: org_apache_arrow_adbc_driver_jni_impl_NativeAdbc
++ * Method: closeConnection
++ * Signature: (J)V
++ */
++JNIEXPORT void JNICALL
Java_org_apache_arrow_adbc_driver_jni_impl_NativeAdbc_closeConnection
++ (JNIEnv *, jclass, jlong);
++
++/*
++ * Class: org_apache_arrow_adbc_driver_jni_impl_NativeAdbc
++ * Method: openStatement
++ * Signature: (J)Lorg/apache/arrow/adbc/driver/jni/impl/NativeStatementHandle;
++ */
++JNIEXPORT jobject JNICALL
Java_org_apache_arrow_adbc_driver_jni_impl_NativeAdbc_openStatement
++ (JNIEnv *, jclass, jlong);
++
++/*
++ * Class: org_apache_arrow_adbc_driver_jni_impl_NativeAdbc
++ * Method: closeStatement
++ * Signature: (J)V
++ */
++JNIEXPORT void JNICALL
Java_org_apache_arrow_adbc_driver_jni_impl_NativeAdbc_closeStatement
++ (JNIEnv *, jclass, jlong);
++
++/*
++ * Class: org_apache_arrow_adbc_driver_jni_impl_NativeAdbc
++ * Method: statementBind
++ * Signature: (JJJ)V
++ */
++JNIEXPORT void JNICALL
Java_org_apache_arrow_adbc_driver_jni_impl_NativeAdbc_statementBind
++ (JNIEnv *, jclass, jlong, jlong, jlong);
++
++/*
++ * Class: org_apache_arrow_adbc_driver_jni_impl_NativeAdbc
++ * Method: statementBindStream
++ * Signature: (JJ)V
++ */
++JNIEXPORT void JNICALL
Java_org_apache_arrow_adbc_driver_jni_impl_NativeAdbc_statementBindStream
++ (JNIEnv *, jclass, jlong, jlong);
++
++/*
++ * Class: org_apache_arrow_adbc_driver_jni_impl_NativeAdbc
++ * Method: statementCancel
++ * Signature: (J)V
++ */
++JNIEXPORT void JNICALL
Java_org_apache_arrow_adbc_driver_jni_impl_NativeAdbc_statementCancel
++ (JNIEnv *, jclass, jlong);
++
++/*
++ * Class: org_apache_arrow_adbc_driver_jni_impl_NativeAdbc
++ * Method: statementExecuteUpdate
++ * Signature: (J)J
++ */
++JNIEXPORT jlong JNICALL
Java_org_apache_arrow_adbc_driver_jni_impl_NativeAdbc_statementExecuteUpdate
++ (JNIEnv *, jclass, jlong);
++
++/*
++ * Class: org_apache_arrow_adbc_driver_jni_impl_NativeAdbc
++ * Method: statementPrepare
++ * Signature: (J)V
++ */
++JNIEXPORT void JNICALL
Java_org_apache_arrow_adbc_driver_jni_impl_NativeAdbc_statementPrepare
++ (JNIEnv *, jclass, jlong);
++
++/*
++ * Class: org_apache_arrow_adbc_driver_jni_impl_NativeAdbc
++ * Method: statementExecutePartitions
++ * Signature: (J)Lorg/apache/arrow/adbc/driver/jni/impl/NativePartitionResult;
++ */
++JNIEXPORT jobject JNICALL
Java_org_apache_arrow_adbc_driver_jni_impl_NativeAdbc_statementExecutePartitions
++ (JNIEnv *, jclass, jlong);
++
++/*
++ * Class: org_apache_arrow_adbc_driver_jni_impl_NativeAdbc
++ * Method: statementExecuteQuery
++ * Signature: (J)Lorg/apache/arrow/adbc/driver/jni/impl/NativeQueryResult;
++ */
++JNIEXPORT jobject JNICALL
Java_org_apache_arrow_adbc_driver_jni_impl_NativeAdbc_statementExecuteQuery
++ (JNIEnv *, jclass, jlong);
++
++/*
++ * Class: org_apache_arrow_adbc_driver_jni_impl_NativeAdbc
++ * Method: statementExecuteSchema
++ * Signature: (J)Lorg/apache/arrow/adbc/driver/jni/impl/NativeSchemaResult;
++ */
++JNIEXPORT jobject JNICALL
Java_org_apache_arrow_adbc_driver_jni_impl_NativeAdbc_statementExecuteSchema
++ (JNIEnv *, jclass, jlong);
++
++/*
++ * Class: org_apache_arrow_adbc_driver_jni_impl_NativeAdbc
++ * Method: statementGetParameterSchema
++ * Signature: (J)Lorg/apache/arrow/adbc/driver/jni/impl/NativeSchemaResult;
++ */
++JNIEXPORT jobject JNICALL
Java_org_apache_arrow_adbc_driver_jni_impl_NativeAdbc_statementGetParameterSchema
++ (JNIEnv *, jclass, jlong);
++
++/*
++ * Class: org_apache_arrow_adbc_driver_jni_impl_NativeAdbc
++ * Method: statementSetSqlQuery
++ * Signature: (J[B)V
++ */
++JNIEXPORT void JNICALL
Java_org_apache_arrow_adbc_driver_jni_impl_NativeAdbc_statementSetSqlQuery
++ (JNIEnv *, jclass, jlong, jbyteArray);
++
++/*
++ * Class: org_apache_arrow_adbc_driver_jni_impl_NativeAdbc
++ * Method: statementSetSubstraitPlan
++ * Signature: (JLjava/nio/ByteBuffer;)V
++ */
++JNIEXPORT void JNICALL
Java_org_apache_arrow_adbc_driver_jni_impl_NativeAdbc_statementSetSubstraitPlan
++ (JNIEnv *, jclass, jlong, jobject);
++
++/*
++ * Class: org_apache_arrow_adbc_driver_jni_impl_NativeAdbc
++ * Method: statementGetOptionBytes
++ * Signature: (J[B)[B
++ */
++JNIEXPORT jbyteArray JNICALL
Java_org_apache_arrow_adbc_driver_jni_impl_NativeAdbc_statementGetOptionBytes
++ (JNIEnv *, jclass, jlong, jbyteArray);
++
++/*
++ * Class: org_apache_arrow_adbc_driver_jni_impl_NativeAdbc
++ * Method: statementGetOptionDouble
++ * Signature: (J[B)D
++ */
++JNIEXPORT jdouble JNICALL
Java_org_apache_arrow_adbc_driver_jni_impl_NativeAdbc_statementGetOptionDouble
++ (JNIEnv *, jclass, jlong, jbyteArray);
++
++/*
++ * Class: org_apache_arrow_adbc_driver_jni_impl_NativeAdbc
++ * Method: statementGetOptionLong
++ * Signature: (J[B)J
++ */
++JNIEXPORT jlong JNICALL
Java_org_apache_arrow_adbc_driver_jni_impl_NativeAdbc_statementGetOptionLong
++ (JNIEnv *, jclass, jlong, jbyteArray);
++
++/*
++ * Class: org_apache_arrow_adbc_driver_jni_impl_NativeAdbc
++ * Method: statementGetOptionString
++ * Signature: (J[B)[B
++ */
++JNIEXPORT jbyteArray JNICALL
Java_org_apache_arrow_adbc_driver_jni_impl_NativeAdbc_statementGetOptionString
++ (JNIEnv *, jclass, jlong, jbyteArray);
++
++/*
++ * Class: org_apache_arrow_adbc_driver_jni_impl_NativeAdbc
++ * Method: statementSetOptionBytes
++ * Signature: (J[B[B)V
++ */
++JNIEXPORT void JNICALL
Java_org_apache_arrow_adbc_driver_jni_impl_NativeAdbc_statementSetOptionBytes
++ (JNIEnv *, jclass, jlong, jbyteArray, jbyteArray);
++
++/*
++ * Class: org_apache_arrow_adbc_driver_jni_impl_NativeAdbc
++ * Method: statementSetOptionDouble
++ * Signature: (J[BD)V
++ */
++JNIEXPORT void JNICALL
Java_org_apache_arrow_adbc_driver_jni_impl_NativeAdbc_statementSetOptionDouble
++ (JNIEnv *, jclass, jlong, jbyteArray, jdouble);
++
++/*
++ * Class: org_apache_arrow_adbc_driver_jni_impl_NativeAdbc
++ * Method: statementSetOptionLong
++ * Signature: (J[BJ)V
++ */
++JNIEXPORT void JNICALL
Java_org_apache_arrow_adbc_driver_jni_impl_NativeAdbc_statementSetOptionLong
++ (JNIEnv *, jclass, jlong, jbyteArray, jlong);
++
++/*
++ * Class: org_apache_arrow_adbc_driver_jni_impl_NativeAdbc
++ * Method: statementSetOptionString
++ * Signature: (J[B[B)V
++ */
++JNIEXPORT void JNICALL
Java_org_apache_arrow_adbc_driver_jni_impl_NativeAdbc_statementSetOptionString
++ (JNIEnv *, jclass, jlong, jbyteArray, jbyteArray);
++
++/*
++ * Class: org_apache_arrow_adbc_driver_jni_impl_NativeAdbc
++ * Method: connectionCancel
++ * Signature: (J)V
++ */
++JNIEXPORT void JNICALL
Java_org_apache_arrow_adbc_driver_jni_impl_NativeAdbc_connectionCancel
++ (JNIEnv *, jclass, jlong);
++
++/*
++ * Class: org_apache_arrow_adbc_driver_jni_impl_NativeAdbc
++ * Method: connectionGetObjects
++ * Signature:
(JI[B[B[B[[B[B)Lorg/apache/arrow/adbc/driver/jni/impl/NativeQueryResult;
++ */
++JNIEXPORT jobject JNICALL
Java_org_apache_arrow_adbc_driver_jni_impl_NativeAdbc_connectionGetObjects
++ (JNIEnv *, jclass, jlong, jint, jbyteArray, jbyteArray, jbyteArray,
jobjectArray, jbyteArray);
++
++/*
++ * Class: org_apache_arrow_adbc_driver_jni_impl_NativeAdbc
++ * Method: connectionGetInfo
++ * Signature: (J[I)Lorg/apache/arrow/adbc/driver/jni/impl/NativeQueryResult;
++ */
++JNIEXPORT jobject JNICALL
Java_org_apache_arrow_adbc_driver_jni_impl_NativeAdbc_connectionGetInfo
++ (JNIEnv *, jclass, jlong, jintArray);
++
++/*
++ * Class: org_apache_arrow_adbc_driver_jni_impl_NativeAdbc
++ * Method: connectionGetTableSchema
++ * Signature:
(J[B[B[B)Lorg/apache/arrow/adbc/driver/jni/impl/NativeSchemaResult;
++ */
++JNIEXPORT jobject JNICALL
Java_org_apache_arrow_adbc_driver_jni_impl_NativeAdbc_connectionGetTableSchema
++ (JNIEnv *, jclass, jlong, jbyteArray, jbyteArray, jbyteArray);
++
++/*
++ * Class: org_apache_arrow_adbc_driver_jni_impl_NativeAdbc
++ * Method: connectionGetTableTypes
++ * Signature: (J)Lorg/apache/arrow/adbc/driver/jni/impl/NativeQueryResult;
++ */
++JNIEXPORT jobject JNICALL
Java_org_apache_arrow_adbc_driver_jni_impl_NativeAdbc_connectionGetTableTypes
++ (JNIEnv *, jclass, jlong);
++
++/*
++ * Class: org_apache_arrow_adbc_driver_jni_impl_NativeAdbc
++ * Method: connectionCommit
++ * Signature: (J)V
++ */
++JNIEXPORT void JNICALL
Java_org_apache_arrow_adbc_driver_jni_impl_NativeAdbc_connectionCommit
++ (JNIEnv *, jclass, jlong);
++
++/*
++ * Class: org_apache_arrow_adbc_driver_jni_impl_NativeAdbc
++ * Method: connectionRollback
++ * Signature: (J)V
++ */
++JNIEXPORT void JNICALL
Java_org_apache_arrow_adbc_driver_jni_impl_NativeAdbc_connectionRollback
++ (JNIEnv *, jclass, jlong);
++
++/*
++ * Class: org_apache_arrow_adbc_driver_jni_impl_NativeAdbc
++ * Method: connectionReadPartition
++ * Signature:
(JLjava/nio/ByteBuffer;)Lorg/apache/arrow/adbc/driver/jni/impl/NativeQueryResult;
++ */
++JNIEXPORT jobject JNICALL
Java_org_apache_arrow_adbc_driver_jni_impl_NativeAdbc_connectionReadPartition
++ (JNIEnv *, jclass, jlong, jobject);
++
++/*
++ * Class: org_apache_arrow_adbc_driver_jni_impl_NativeAdbc
++ * Method: connectionGetOptionBytes
++ * Signature: (J[B)[B
++ */
++JNIEXPORT jbyteArray JNICALL
Java_org_apache_arrow_adbc_driver_jni_impl_NativeAdbc_connectionGetOptionBytes
++ (JNIEnv *, jclass, jlong, jbyteArray);
++
++/*
++ * Class: org_apache_arrow_adbc_driver_jni_impl_NativeAdbc
++ * Method: connectionGetOptionDouble
++ * Signature: (J[B)D
++ */
++JNIEXPORT jdouble JNICALL
Java_org_apache_arrow_adbc_driver_jni_impl_NativeAdbc_connectionGetOptionDouble
++ (JNIEnv *, jclass, jlong, jbyteArray);
++
++/*
++ * Class: org_apache_arrow_adbc_driver_jni_impl_NativeAdbc
++ * Method: connectionGetOptionLong
++ * Signature: (J[B)J
++ */
++JNIEXPORT jlong JNICALL
Java_org_apache_arrow_adbc_driver_jni_impl_NativeAdbc_connectionGetOptionLong
++ (JNIEnv *, jclass, jlong, jbyteArray);
++
++/*
++ * Class: org_apache_arrow_adbc_driver_jni_impl_NativeAdbc
++ * Method: connectionGetOptionString
++ * Signature: (J[B)[B
++ */
++JNIEXPORT jbyteArray JNICALL
Java_org_apache_arrow_adbc_driver_jni_impl_NativeAdbc_connectionGetOptionString
++ (JNIEnv *, jclass, jlong, jbyteArray);
++
++/*
++ * Class: org_apache_arrow_adbc_driver_jni_impl_NativeAdbc
++ * Method: connectionSetOptionBytes
++ * Signature: (J[B[B)V
++ */
++JNIEXPORT void JNICALL
Java_org_apache_arrow_adbc_driver_jni_impl_NativeAdbc_connectionSetOptionBytes
++ (JNIEnv *, jclass, jlong, jbyteArray, jbyteArray);
++
++/*
++ * Class: org_apache_arrow_adbc_driver_jni_impl_NativeAdbc
++ * Method: connectionSetOptionDouble
++ * Signature: (J[BD)V
++ */
++JNIEXPORT void JNICALL
Java_org_apache_arrow_adbc_driver_jni_impl_NativeAdbc_connectionSetOptionDouble
++ (JNIEnv *, jclass, jlong, jbyteArray, jdouble);
++
++/*
++ * Class: org_apache_arrow_adbc_driver_jni_impl_NativeAdbc
++ * Method: connectionSetOptionLong
++ * Signature: (J[BJ)V
++ */
++JNIEXPORT void JNICALL
Java_org_apache_arrow_adbc_driver_jni_impl_NativeAdbc_connectionSetOptionLong
++ (JNIEnv *, jclass, jlong, jbyteArray, jlong);
++
++/*
++ * Class: org_apache_arrow_adbc_driver_jni_impl_NativeAdbc
++ * Method: connectionSetOptionString
++ * Signature: (J[B[B)V
++ */
++JNIEXPORT void JNICALL
Java_org_apache_arrow_adbc_driver_jni_impl_NativeAdbc_connectionSetOptionString
++ (JNIEnv *, jclass, jlong, jbyteArray, jbyteArray);
++
++/*
++ * Class: org_apache_arrow_adbc_driver_jni_impl_NativeAdbc
++ * Method: connectionGetStatisticNames
++ * Signature: (J)Lorg/apache/arrow/adbc/driver/jni/impl/NativeQueryResult;
++ */
++JNIEXPORT jobject JNICALL
Java_org_apache_arrow_adbc_driver_jni_impl_NativeAdbc_connectionGetStatisticNames
++ (JNIEnv *, jclass, jlong);
++
++/*
++ * Class: org_apache_arrow_adbc_driver_jni_impl_NativeAdbc
++ * Method: connectionGetStatistics
++ * Signature:
(J[B[B[BZ)Lorg/apache/arrow/adbc/driver/jni/impl/NativeQueryResult;
++ */
++JNIEXPORT jobject JNICALL
Java_org_apache_arrow_adbc_driver_jni_impl_NativeAdbc_connectionGetStatistics
++ (JNIEnv *, jclass, jlong, jbyteArray, jbyteArray, jbyteArray, jboolean);
++
++/*
++ * Class: org_apache_arrow_adbc_driver_jni_impl_NativeAdbc
++ * Method: databaseGetOptionBytes
++ * Signature: (J[B)[B
++ */
++JNIEXPORT jbyteArray JNICALL
Java_org_apache_arrow_adbc_driver_jni_impl_NativeAdbc_databaseGetOptionBytes
++ (JNIEnv *, jclass, jlong, jbyteArray);
++
++/*
++ * Class: org_apache_arrow_adbc_driver_jni_impl_NativeAdbc
++ * Method: databaseGetOptionDouble
++ * Signature: (J[B)D
++ */
++JNIEXPORT jdouble JNICALL
Java_org_apache_arrow_adbc_driver_jni_impl_NativeAdbc_databaseGetOptionDouble
++ (JNIEnv *, jclass, jlong, jbyteArray);
++
++/*
++ * Class: org_apache_arrow_adbc_driver_jni_impl_NativeAdbc
++ * Method: databaseGetOptionLong
++ * Signature: (J[B)J
++ */
++JNIEXPORT jlong JNICALL
Java_org_apache_arrow_adbc_driver_jni_impl_NativeAdbc_databaseGetOptionLong
++ (JNIEnv *, jclass, jlong, jbyteArray);
++
++/*
++ * Class: org_apache_arrow_adbc_driver_jni_impl_NativeAdbc
++ * Method: databaseGetOptionString
++ * Signature: (J[B)[B
++ */
++JNIEXPORT jbyteArray JNICALL
Java_org_apache_arrow_adbc_driver_jni_impl_NativeAdbc_databaseGetOptionString
++ (JNIEnv *, jclass, jlong, jbyteArray);
++
++/*
++ * Class: org_apache_arrow_adbc_driver_jni_impl_NativeAdbc
++ * Method: databaseSetOptionBytes
++ * Signature: (J[B[B)V
++ */
++JNIEXPORT void JNICALL
Java_org_apache_arrow_adbc_driver_jni_impl_NativeAdbc_databaseSetOptionBytes
++ (JNIEnv *, jclass, jlong, jbyteArray, jbyteArray);
++
++/*
++ * Class: org_apache_arrow_adbc_driver_jni_impl_NativeAdbc
++ * Method: databaseSetOptionDouble
++ * Signature: (J[BD)V
++ */
++JNIEXPORT void JNICALL
Java_org_apache_arrow_adbc_driver_jni_impl_NativeAdbc_databaseSetOptionDouble
++ (JNIEnv *, jclass, jlong, jbyteArray, jdouble);
++
++/*
++ * Class: org_apache_arrow_adbc_driver_jni_impl_NativeAdbc
++ * Method: databaseSetOptionLong
++ * Signature: (J[BJ)V
++ */
++JNIEXPORT void JNICALL
Java_org_apache_arrow_adbc_driver_jni_impl_NativeAdbc_databaseSetOptionLong
++ (JNIEnv *, jclass, jlong, jbyteArray, jlong);
++
++/*
++ * Class: org_apache_arrow_adbc_driver_jni_impl_NativeAdbc
++ * Method: databaseSetOptionString
++ * Signature: (J[B[B)V
++ */
++JNIEXPORT void JNICALL
Java_org_apache_arrow_adbc_driver_jni_impl_NativeAdbc_databaseSetOptionString
++ (JNIEnv *, jclass, jlong, jbyteArray, jbyteArray);
++
++/*
++ * Class: org_apache_arrow_adbc_driver_jni_impl_NativeAdbc
++ * Method: internalGetByteBuffer
++ * Signature: (Ljava/nio/ByteBuffer;)[B
++ */
++JNIEXPORT jbyteArray JNICALL
Java_org_apache_arrow_adbc_driver_jni_impl_NativeAdbc_internalGetByteBuffer
++ (JNIEnv *, jclass, jobject);
++
++#ifdef __cplusplus
++}
++#endif
++#endif
diff --git a/thirdparty/vars.sh b/thirdparty/vars.sh
index f8d18a3a2be..4b72026ebaa 100644
--- a/thirdparty/vars.sh
+++ b/thirdparty/vars.sh
@@ -268,6 +268,39 @@ ARROW_NAME="apache-arrow-17.0.0.tar.gz"
ARROW_SOURCE="arrow-apache-arrow-17.0.0"
ARROW_MD5SUM="ba18bf83e2164abd34b9ac4cb164f0f0"
+# arrow-adbc
+# One source tree, three artifacts: the driver manager (statically linked into
+# doris_be), the JNI bridge (loaded by the FE adbc connector) and the SQLite
+# driver (BE unit tests only, not shipped).
+# Release "N" ships C/Go 1.12.0 and Java 0.24.0; the tag carries neither
number.
+ARROW_ADBC_DOWNLOAD="https://github.com/apache/arrow-adbc/archive/refs/tags/apache-arrow-adbc-24.tar.gz";
+ARROW_ADBC_NAME="apache-arrow-adbc-24.tar.gz"
+ARROW_ADBC_SOURCE="arrow-adbc-apache-arrow-adbc-24"
+ARROW_ADBC_MD5SUM="2b2a18e95c33bdfd2bfa33a8b57c78d6"
+
+# arrow-adbc Flight SQL driver, prebuilt.
+# Upstream writes this driver in Go and publishes no bare shared library, so
it is
+# taken from the official release wheel (a zip) rather than adding a Go
toolchain to
+# the thirdparty build. It must come from the same release as the source tree
above
+# (C/Go 1.12.0): partition descriptors are driver-private bytes, so the FE and
every
+# BE have to load the very same file. ADBC tests only, not shipped -- same
treatment
+# as the SQLite driver. Linux only; the macOS wheels carry a .dylib no test
looks for.
+if [[ "$(uname -s)" == 'Linux' ]]; then
+ ARROW_ADBC_FLIGHTSQL_SOURCE="adbc_driver_flightsql"
+ if [[ "${MACHINE_TYPE}" == 'x86_64' ]]; then
+
ARROW_ADBC_FLIGHTSQL_DOWNLOAD="https://files.pythonhosted.org/packages/2b/5b/f8566aa6d05eb40225f5874f8f5f5e8b8b4a4ce866d817a5ff352f326302/adbc_driver_flightsql-1.12.0-py3-none-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl";
+ ARROW_ADBC_FLIGHTSQL_NAME="adbc_driver_flightsql-1.12.0-x86_64.zip"
+ ARROW_ADBC_FLIGHTSQL_MD5SUM="975a283dc72db0c646204e512b396e2a"
+ elif [[ "${MACHINE_TYPE}" == "aarch64" || "${MACHINE_TYPE}" == 'arm64' ]];
then
+
ARROW_ADBC_FLIGHTSQL_DOWNLOAD="https://files.pythonhosted.org/packages/57/45/6468fb425f3c0ae868c1ca9ab8e15cba76b59e941edd7e80e892e5476728/adbc_driver_flightsql-1.12.0-py3-none-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl";
+ ARROW_ADBC_FLIGHTSQL_NAME="adbc_driver_flightsql-1.12.0-aarch64.zip"
+ ARROW_ADBC_FLIGHTSQL_MD5SUM="4ab31967910f9d2af20663b06412e6aa"
+ else
+ echo "no prebuilt adbc flightsql driver for ${MACHINE_TYPE}, skipping
it"
+ ARROW_ADBC_FLIGHTSQL_SOURCE=""
+ fi
+fi
+
# Abseil
ABSEIL_DOWNLOAD="https://github.com/abseil/abseil-cpp/releases/download/20250512.1/abseil-cpp-20250512.1.tar.gz";
ABSEIL_NAME="abseil-cpp-20250512.1.tar.gz"
@@ -610,6 +643,7 @@ export TP_ARCHIVES=(
'LIBRDKAFKA'
'FLATBUFFERS'
'ARROW'
+ 'ARROW_ADBC'
'BROTLI'
'ZSTD'
'ABSEIL'
@@ -667,6 +701,12 @@ export TP_ARCHIVES=(
'LANCE_C'
)
+# Only defined on the platforms upstream ships a prebuilt driver for (see
above).
+if [[ -n "${ARROW_ADBC_FLIGHTSQL_SOURCE}" ]]; then
+ read -r -a TP_ARCHIVES <<<"${TP_ARCHIVES[*]} ARROW_ADBC_FLIGHTSQL"
+ export TP_ARCHIVES
+fi
+
if [[ "$(uname -s)" == 'Darwin' ]]; then
#binutils
BINUTILS_DOWNLOAD='https://ftp.nluug.nl/pub/gnu/binutils/binutils-2.39.tar.gz'
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
