hutiefang76 opened a new pull request, #12543:
URL: https://github.com/apache/seatunnel/pull/12543
### Purpose of this pull request
DuckDB Source queries currently use the generic JDBC type-code converter:
DECIMAL(20,0) becomes BIGINT, zoned timestamps become TIMESTAMP, and
UUID/JSON/complex types fail discovery. Use the existing DuckDB mapper for
query metadata, following the other database catalogs.
Fixes #12542.
### Does this PR introduce any user-facing change?
Yes. Query discovery preserves native DECIMAL precision/scale and
TIMESTAMP_TZ, and reads UUID/JSON/INTERVAL/ARRAY/STRUCT/MAP as STRING through
the existing converter. Collection detection precedes scalar DECIMAL
normalization, so DECIMAL(p,s)[] remains STRING. Column aliases, native source
type, nullability and decimal scale are retained.
EN/ZH Source docs and incompatible-change notes explain the corrected output
types and explicit SQL casts for downstream compatibility. Complex STRING
values are driver text rather than nested SeaTunnel values; raw STRUCT text is
not a valid native STRUCT input in JDBC1.3.1. The documented `to_json`
projection was verified against a pre-created Lake target.
`table_path` discovery is unchanged. No new options, dependencies, row
serializers or standalone DuckLake connector.
### How was this patch tested?
- Extended existing DuckDBCatalogTest and DuckDBSourceAndSinkTest. Against
original production code: 13 tests, two failures and two unsupported-type
errors; corrected connection setup closes the write connection before readonly
Source discovery.
- Verified the DECIMAL-array regression independently (STRING expected,
DECIMAL(0,0) observed before collection-order correction).
- JDK8 and JDK17 each: 89 tests, zero failures/errors/skips, with clean
connector-jdbc `verify`:
```shell
mvn -q -o -pl seatunnel-connectors-v2/connector-jdbc
'-Dtest=DuckDB*Test,JdbcFieldTypeUtilsTest' clean verify
```
- Actual Source reads a value beyond signed BIGINT range, a microsecond
timestamp instant, UUID/JSON/ARRAY/STRUCT/MAP text, a decimal expression,
aliases and nulls. Existing table-path Source/Sink and ordinary catalog tests
still pass.
- DuckDB JDBC1.3.1 + real DuckLake extension, SQLite metadata/local data:
query Source → explicit JDBC insert → independent reconnect passes decimal,
zero-scale decimal, date, timestamp, zoned timestamp, UUID, JSON, BLOB, ARRAY
and MAP. STRUCT succeeds with the documented `to_json` projection; unnormalized
driver text fails native STRUCT insertion.
- Spotless and official MarkdownTest (4/4) pass. No existing DuckDB engine
IT was found; this uses the existing real-JDBC factory flow tests and an
additional Lake smoke probe.
These are representative local cases, not a claim of CDC, commit
deduplication, cross-host recovery or performance acceptance. Remote CI and
review are pending.
### Check list
- [x] EN/ZH Source documentation updated.
- [x] EN/ZH incompatible-change notes and migration guidance added.
- [x] Existing connector tests extended; no new dependencies or connector
registration needed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
