morningman opened a new pull request, #68510:
URL: https://github.com/apache/doris/pull/68510
### What problem does this PR solve?
Issue Number: None
Related PR: #66399
Problem Summary:
**Context.** `fe-connector-adbc` is not a Maven prerequisite of fe-core, so
`-am` never reached it and its tests never ran in FE UT; #66399 therefore names
the module in the runner. Running it there for the first time turned 23 of its
tests red on the CI agent, all with the same error:
```
java.lang.UnsatisfiedLinkError: /tmp/jnilib-*.tmp: /lib64/libstdc++.so.6:
version `CXXABI_1.3.9' not found (required by /tmp/jnilib-*.tmp)
```
Two different native libraries are on that path, and only one of them was
ours. The ADBC JNI shim is built by `thirdparty` locally, which is why it loads
anywhere. Reading a driver's result as Arrow additionally needs arrow-c-data's
`arrow_cdata_jni`, which travels inside the arrow-c-data jar as an **upstream**
build: it is selected through the `arrow.cdata.library.path` property, nothing
in this repository sets that property, so the jar's binary is always the one
loaded -- and it needs CXXABI_1.3.9 (GCC 4.9+), which a host Doris still
supports does not have (CentOS 7 stops at CXXABI_1.3.7).
**1. The problem, and what it cost**
- Every ADBC test that materializes Arrow data (23 of the module's 204)
fails FE UT on such a host, with an error that reads like a test bug rather
than a platform limit.
- The rule was nowhere written down, and it is not visible from the test: a
case that iterates an `ArrowReader` looks exactly like one that does not, so
the next person adding a test to this module walks into the same wall.
- The suites delegate layer-level questions to these unit tests
(`test_adbc_metadata_ops`, `test_adbc_sqlite_catalog_scan`,
`test_adbc_catalog_scan` and `test_adbc_predicate_pushdown` each say so in a
comment), so deleting them without replacements would have moved that coverage
out of CI silently.
**2. What this PR does, and why it helps**
Commit 1 -- `[test](fe) Drop the ADBC tests that need arrow-c-data's JNI
shim, keep the rest in FE UT`:
- The three classes that read a driver's result are deleted:
`AdbcQueryBuilderNativeTest` (its text-level twin `AdbcQueryBuilderTest` stays,
and "the source accepts the generated SQL" is now asserted end to end),
`AdbcMetadataCacheNativeTest` (the cache's mechanics stay covered by the
pure-Java `AdbcMetadataCacheTest`, 18 cases), and `AdbcConnectorMetadataTest`.
- Of that last class, the two cases that need a driver but never materialize
a result -- a missing table must not be reported as a driver gap, and the table
descriptor must be typed for the scan path -- move to
`AdbcConnectorMetadataNativeTest` unchanged.
- `AdbcObjectsReaderTest` is rewritten against hand-built `getObjects`
responses (the file already carried the in-memory Arrow technique), so its four
parsing cases run in FE UT with no native library at all; two more cases are
added for the shapes a driver reports for an absent schema level, and for a
namespace reported twice.
- `AdbcNativeTestSupport` now states the boundary: tests that iterate an
`ArrowReader` or touch `org.apache.arrow.c` belong in
`regression-test/suites/external_table_p0/adbc`, not in this module's test
sources.
Commit 2 -- `[test](regression) Cover the ADBC cases the removed unit tests
pinned`: the end-to-end half of those behaviours, as assertions rather than
baselines, in the four suites listed below.
**What it buys:** FE UT runs 187 ADBC tests with no dependency on the
upstream C-data binary; the behaviours that need a real driver or a real
cluster are asserted where they can be; and the rule lives in the class that
owns the boundary.
**3. The classes, and how they call each other**
- `AdbcObjectsReader` -- parses the nested `getObjects` response. Now fed by
fixtures that reproduce the same shapes a driver emits (a schema entry named
the empty string, a null schema list, advisory filters that return other
namespaces' rows), and covered again in FE UT.
- `AdbcNativeTestSupport` -- the single funnel every driver-backed ADBC test
goes through; its javadoc is where the two-library distinction is now recorded.
- `AdbcConnectorMetadata` / `AdbcMetadataCache` -- the two surviving
driver-backed cases never reach a result (one throws before it, one never asks
the source), which is what makes them safe in FE UT.
- Regression suites -- `test_adbc_sqlite_catalog_scan` (empty-named
database, BLOB mapping, schema served until REFRESH), `test_adbc_metadata_ops`
(same REFRESH negative), `test_adbc_catalog_scan` (a second database in the
source; the sibling listing asserted as an exact set),
`test_adbc_predicate_pushdown` (the remote statement's two-level qualification).
- Not touched: no FE source change, and `run-fe-ut.sh` keeps the module
line, which is the point of the whole change.
```
run-fe-ut.sh ──▶ fe-connector-adbc (187 tests)
│
├─ driver-backed, reads no result ──▶
AdbcNativeTestSupport ──▶ thirdparty-built libadbc_driver_jni.so
│ (skips when absent;
never touches arrow-c-data)
└─ pure Java ──▶ AdbcObjectsReaderTest fixtures
(in-memory Arrow IPC)
▲
regression suites (real driver, real cluster) ──┘ same shapes, asserted end
to end
```
### Release note
None
### Check List (For Author)
- Test: Unit Test / Regression test
- `mvn -f fe/pom.xml -pl :fe-connector-adbc -am test` -> 187 tests, 0
failures (7 skipped locally: the SQLite driver is absent on macOS; they run in
CI).
- Local single-FE/single-BE cluster built from this branch: `bash
run-regression-test.sh --run -d external_table_p0/adbc -suiteParallel 1` ->
Test 20 suites, failed 0 suites, fatal 0 scripts, skipped 0 scripts
(`test_adbc_multi_backend` self-skips on one backend by design, as in CI).
- Behavior changed: No (test sources only; FE UT now runs the ADBC module's
tests)
- Does this need documentation: No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
