This is an automated email from the ASF dual-hosted git repository.
morningman pushed a change to branch branch-catalog-spi
in repository https://gitbox.apache.org/repos/asf/doris.git
discard f672e789674 [chore](handoff) record the 2026-07-29b rebase onto
8305fe71867 (upstream #66112)
discard 728f7bbb9c3 [fix](catalog) port #66112's CTAS atomicity + IF NOT
EXISTS cache refresh onto the plugin bridge
discard 8876656911f [fix](catalog) port #66112's iceberg SQL MERGE cardinality
enforcement onto the connector SPI
discard 5da0bc9110e [feature](plugin) replace the always-true connector
apiVersion() with a manifest-carried API version gate for all four plugin
families (#66211)
discard d23be037c44 [refactor](catalog) fe-core: delete the unreachable
metastore property cluster and the dead storage/AWS-credential surface (#66210)
discard 46e2e21a905 [fix](iceberg) persist a CREATE TABLE COMMENT clause
instead of dropping it
discard b09e84562e9 [fix](docker) make the thirdparty hive/iceberg stacks
start on macOS
discard 0af41ff5639 [test](iceberg) assert the nested-DDL refusal message this
branch actually emits
discard 719e40ec5f0 [fix](catalog) position-map a plugin system table against
its own pin, not latest
discard 635cea20617 [test](iceberg) assert gateway/dedicated parity instead of
an absolute nested-DDL capability
discard 4ce9d76d805 [fix](catalog) bind a plugin system table's schema at its
own pin, not at latest
discard dcfcc2a1d75 [fix](paimon) read an @options tag pin's schemaId from the
tag, not the snapshot file
discard 07f7073f4d3 [test](catalog) port #65870 partition-evolution coverage
into fe-connector-iceberg
discard ca348e6e353 [refactor](catalog) port #65984 paimon query-level
@options into fe-connector-paimon
discard c030af740ad [chore](ci) waive the unpatchable jackson-mapper-asl
advisories in dependency review
discard 2a7c2aff582 [chore](license) satisfy the ASF header check on this
branch
discard f2de7f46e6d [fix](ci) hand the license-check file list over as a file,
not an env var
discard bbae82c5aa2 [fix](build) keep CONTAINER_UID assignment literal so
pipeline sed still matches
discard 2e0678fb1b6 [chore](handoff) record the 2026-07-27c rebase onto
e7b7f1d1359 (upstream #66004 storage facade)
discard 91ba58dca8b [fix](catalog) reconcile the connector SPI with upstream
#66004's fe-filesystem storage facade
discard 3e965be94b4 [fix](build) Support Iceberg Docker stacks on Apple Silicon
discard ffc15f003e6 [refactor](catalog) connector public interface cleanup: no
common-module edit to add a connector, dead-surface deletion, 11 correctness
fixes (#66135)
discard ef5afa5e36e [fix](catalog) port #65955's paimon table-option
passthrough + jni.* rename to the connector
discard 0d88c8ac805 [chore](handoff) record the 2026-07-27 rebase onto
5b3ac63f8b4
discard 8c8b3329291 [fix](catalog) port #66008's paimon-cpp removal to the
paimon connector scan path
discard 5def7742c73 [fix](iceberg) dedupe partition columns so a
multi-transform spec cannot crash partition pruning
discard 300bfc91cf9 [opt](catalog) fe-connector: unify the connector cache
foundation, memoize per-statement hot paths, fix iceberg case-insensitive
column DDL (#66029)
discard 3185e4f5754 [test](nereids) rebase-adapt PullUpJoinFromUnionAllTest to
the connector-migrated Hudi scan path
discard a780e47e3ae [test](paimon) align stale schema-evolution matrix tests
to shipped behavior
discard 02dfaf0f5f2 [fix](paimon) read a branch's independent schema via its
own schema dir on the scan path
discard 4fbf40ea6df [fix](iceberg) scan same-table multi-version references
against each reference's pinned snapshot schema
discard 1905446eb26 [refactor](catalog) drop generic-S3 minio.* folding
superseded by upstream #65834's dedicated fe-filesystem-minio
discard 6fb0cb53424 [fix](be) fail the query instead of aborting BE on a
missing projected column
discard e60d367b6fb [fix](paimon) read latest schema in getColumnHandles so a
renamed column is not dropped
discard b11fcf51162 [fix](catalog) preserve nested STRUCT field comment on
omitted COMMENT in complex MODIFY (build 1004182)
discard 35f739744d7 [fix](catalog) catalog-SPI abstraction-mismatch review:
paimon time-travel crash, partition/schema fidelity, SPI convergence &
dead-code cleanup (#65970)
discard 2328e50951e [feat](catalog) port upstream #65329 iceberg nested column
schema change to the connector SPI (Tier 2)
discard 75701a80672 [fix](catalog) reconcile upstream #65329 iceberg nested
column schema change with connector SPI (Tier 1: compile + flat DDL routing)
discard d6d3223adaf [fix](catalog) port #65904 paimon special-char partition
names into fe-connector
discard 9a273760e89 [fix](catalog) bundle fastutil-core into BE
java-extensions that eager-load ConcurrentLong2LongHashMap
discard 1c46dfc179e [fix](catalog) migrate rebased-in
PhysicalStorageLayerAggregateTest to PluginDrivenExternalTable
discard 097ef392c1d [refactor](catalog) extract fe-trino-connector-common so
fe-common no longer depends on Trino
discard c11dac754b0 [refactor](catalog) fe-core: strip residual
iceberg/hive/hudi deps & dead code, delegate DDL validation to connectors,
remove hudi_meta TVF (#65893)
discard 9e113cf0d82 [fix](catalog) port #65784 authoritative iceberg name
mapping to the connector scan path
discard ab2139c87ce [fix](catalog) port #65782 collect_column_stats sink flag
+ write-metrics tests to the iceberg connector
discard 5df944d09a0 [perf](catalog) two-level cross-query cache for external
partition derived views (#65829)
discard bd6adec0bd2 [perf](catalog) fe-connector-iceberg hot-path caching +
fe-core per-statement metadata funnel, with session=user cache isolation
discard 3b3eb932c9f [fix](catalog) port #65676 iceberg deletion-vector
metadata validation to the connector scan path
discard b7c8aef9c7b [fix](catalog) port #65548 external COUNT(*)/COUNT(col)
semantics to the connector scan path
discard ba87851d68b [fix](catalog) iceberg system-table scan: restore #65262
positional JNI read + order-preserving projection
discard 6df9e918b00 [fix](catalog) restore jfs hive-write hadoop config +
iceberg system-table scan projection
discard 731c2932d80 [feat](catalog) fe-connector-iceberg: port #64966 REST 401
re-auth to the connector
discard 34ada8f0a52 [refactor](catalog) drop
ReauthenticatingRestSessionCatalog test orphaned by master rebase
discard c513d2acba0 [refactor](catalog) fe-core: remove 6 dead datasource
classes orphaned by the SPI migration (#65740)
discard 1afe69a6980 [refactor](catalog) fe-core: remove dead
ExternalMetadataOps layer, deprecated LakeSoul catalog, and odbc/statistics
stubs (#65736)
discard bba393b4a8e [refactor](catalog) fe-core: remove dead Trino-ported
DirectoryListercache cluster (#65734)
discard d2cc1171b47 [refactor](catalog) fe-connector: replace fat
hive-catalog-shade with a slim HMS metastore-client shade (#65721)
discard e6ff95e90bc [refactor](catalog) Catalog spi 11 hive (#65473)
discard fbf50a09c52 [doc](catalog) HANDOFF: $position_deletes port done; e2e
is the only remaining flip gate
discard e34214a344c [feat](catalog) P6.6: support the iceberg
$position_deletes system table on the connector SPI
discard 66ee7696dc2 [doc](catalog) HANDOFF: point next step at the
$position_deletes port design (T0 evidence in)
discard 30d288fb54d [doc](catalog) P6.6 $position_deletes connector port:
research notes + design (T0 evidence in, ready to implement)
discard 54015b28690 [doc](catalog) HANDOFF: rebase to upstream 30a44d61a8a +
register $position_deletes connector gap as flip blocker
discard 60a3cec07ab [fix](build) fix parallel -T shade race in
fe-connector-paimon-hive-shade
discard cab432a309b [fix](catalog) reconcile upstream #65365 with connector
SPI after rebase to 4f48ccfaa7d
discard 626952704b2 [doc](catalog) P6.6-C6 #63068 HANDOFF: record
cache-leakage data-flow test + Task 0 retained-base verify
discard 66cd0145f80 [feat](catalog) P6.6-C6 #63068 FE: re-migrate
ExternalDatabaseSessionContextTest (cross-user cache-leakage data-flow)
discard b5bb4ae472f [doc](catalog) P6.6-C6 #63068 OIDC session migration:
design + research notes + HANDOFF (Tasks 0-7 DONE)
discard 73194b56456 [feat](catalog) P6.6-C6 #63068 iceberg: per-user REST
session routing (metadata + scan/write/procedure)
discard a9d6dd6a850 [feat](catalog) P6.6-C6 #63068 FE: capability-gated
credential injection + per-user cache bypass
discard f4b901e723f [feat](catalog) P6.6-C6 #63068 SPI: neutral
delegated-credential DTO + user-session capability
discard a02f0d77173 [fix](catalog) tier3 #65094: preserve external-table
top-level column-name case on the read path
discard 3c867c9b751 [fix](catalog) port upstream #65332/#65094 datasource
fixes into SPI connectors
discard 70dedb98b4a [fix](catalog) P4 maxcompute: drop dangling legacy
MaxComputeExternalTableTest after rebase
discard a23b819d3f8 [fix](catalog) jdbc: keep driver classloaders alive per
URL to stop Metaspace leak
discard 72f7f3a6d9d [refactor](catalog) P6 iceberg: migrate to catalog SPI +
cutover + remove legacy fe-core subsystem (#64688)
discard 00182f5fe42 [refactor](catalog) P3b: consolidate kerberos
authentication into fe-kerberos (#64655)
discard bc24f170754 [refactor](catalog) P5 paimon: remove legacy subsystem
from fe-core + make fe-core paimon-SDK-free (T29) (#64653)
discard c549d74bd4c [refactor](catalog) P5 paimon: migrate to catalog SPI +
cutover (#64446) (#64446)
discard c89e2a66c68 update P5 handoff and fix compile issue
discard caf979497b7 [refactor](connector) P4 maxcompute: remove legacy
subsystem from fe-core + make fe-core odps-free (T07-T09) (#64300)
discard 92dfc9f97d3 [feat](connector) P3 hudi connector hardening + test
baseline + dispatch design (hybrid, T02-T08) (#64143)
discard 66a03ee9352 [feat](connector) P2 migrate trino-connector to catalog
SPI (T01-T13) (#64096)
discard 095482ab17b [P1-T03-T05] route plugin-driven scans first in nereids
translator (#63641)
discard 790c4767697 [feat](connector) P0 SPI baseline + DDL/Partition + import
gate (T03-T27) (#63582)
discard e70b8a895b3 [doc](connector) add project tracking system for catalog
SPI migration
add 794d514479e [feature](hive) Support static partition overwrite for
hive tables (#65991)
add 92948149269 [doc](connector) add project tracking system for catalog
SPI migration
add e5d064184f6 [feat](connector) P0 SPI baseline + DDL/Partition + import
gate (T03-T27) (#63582)
add c508264586d [P1-T03-T05] route plugin-driven scans first in nereids
translator (#63641)
add 343a5ff6789 [feat](connector) P2 migrate trino-connector to catalog
SPI (T01-T13) (#64096)
add 088863347e0 [feat](connector) P3 hudi connector hardening + test
baseline + dispatch design (hybrid, T02-T08) (#64143)
add c64f0167b38 [refactor](connector) P4 maxcompute: remove legacy
subsystem from fe-core + make fe-core odps-free (T07-T09) (#64300)
add aff44a0207c update P5 handoff and fix compile issue
add 6b5cca7508f [refactor](catalog) P5 paimon: migrate to catalog SPI +
cutover (#64446) (#64446)
add 6c4879425d4 [refactor](catalog) P5 paimon: remove legacy subsystem
from fe-core + make fe-core paimon-SDK-free (T29) (#64653)
add 4dcacc60e6f [refactor](catalog) P3b: consolidate kerberos
authentication into fe-kerberos (#64655)
add fe362e13b71 [refactor](catalog) P6 iceberg: migrate to catalog SPI +
cutover + remove legacy fe-core subsystem (#64688)
add 28b0699ae79 [fix](catalog) jdbc: keep driver classloaders alive per
URL to stop Metaspace leak
add 47bd01e1341 [fix](catalog) P4 maxcompute: drop dangling legacy
MaxComputeExternalTableTest after rebase
add cc29e938335 [fix](catalog) port upstream #65332/#65094 datasource
fixes into SPI connectors
add 5bf9daa97de [fix](catalog) tier3 #65094: preserve external-table
top-level column-name case on the read path
add 24d8e4e38f4 [feat](catalog) P6.6-C6 #63068 SPI: neutral
delegated-credential DTO + user-session capability
add 9265e17ddf6 [feat](catalog) P6.6-C6 #63068 FE: capability-gated
credential injection + per-user cache bypass
add f2ad073a258 [feat](catalog) P6.6-C6 #63068 iceberg: per-user REST
session routing (metadata + scan/write/procedure)
add 5675e998284 [doc](catalog) P6.6-C6 #63068 OIDC session migration:
design + research notes + HANDOFF (Tasks 0-7 DONE)
add cb5a5f49ec7 [feat](catalog) P6.6-C6 #63068 FE: re-migrate
ExternalDatabaseSessionContextTest (cross-user cache-leakage data-flow)
add f069445337e [doc](catalog) P6.6-C6 #63068 HANDOFF: record
cache-leakage data-flow test + Task 0 retained-base verify
add af55ff6931c [fix](catalog) reconcile upstream #65365 with connector
SPI after rebase to 4f48ccfaa7d
add b69dbe3e5de [fix](build) fix parallel -T shade race in
fe-connector-paimon-hive-shade
add 0b14ac75a1e [doc](catalog) HANDOFF: rebase to upstream 30a44d61a8a +
register $position_deletes connector gap as flip blocker
add 1cc8bd15f43 [doc](catalog) P6.6 $position_deletes connector port:
research notes + design (T0 evidence in, ready to implement)
add a114ad6f4f9 [doc](catalog) HANDOFF: point next step at the
$position_deletes port design (T0 evidence in)
add 7e73143b435 [feat](catalog) P6.6: support the iceberg
$position_deletes system table on the connector SPI
add adc3222a971 [doc](catalog) HANDOFF: $position_deletes port done; e2e
is the only remaining flip gate
add c5628881538 [refactor](catalog) Catalog spi 11 hive (#65473)
add e618e84c2a7 [refactor](catalog) fe-connector: replace fat
hive-catalog-shade with a slim HMS metastore-client shade (#65721)
add 16f5e88349d [refactor](catalog) fe-core: remove dead Trino-ported
DirectoryListercache cluster (#65734)
add b3234fbf425 [refactor](catalog) fe-core: remove dead
ExternalMetadataOps layer, deprecated LakeSoul catalog, and odbc/statistics
stubs (#65736)
add e134cf2a0b4 [refactor](catalog) fe-core: remove 6 dead datasource
classes orphaned by the SPI migration (#65740)
add 654c6d1a084 [refactor](catalog) drop
ReauthenticatingRestSessionCatalog test orphaned by master rebase
add 33541394766 [feat](catalog) fe-connector-iceberg: port #64966 REST 401
re-auth to the connector
add de7a7af05c3 [fix](catalog) restore jfs hive-write hadoop config +
iceberg system-table scan projection
add 62bd5794ac9 [fix](catalog) iceberg system-table scan: restore #65262
positional JNI read + order-preserving projection
add ddc8509564f [fix](catalog) port #65548 external COUNT(*)/COUNT(col)
semantics to the connector scan path
add 3212ad0f3c9 [fix](catalog) port #65676 iceberg deletion-vector
metadata validation to the connector scan path
add 493584edc15 [perf](catalog) fe-connector-iceberg hot-path caching +
fe-core per-statement metadata funnel, with session=user cache isolation
add 2f7622e2af3 [perf](catalog) two-level cross-query cache for external
partition derived views (#65829)
add 6dcf13b8dc7 [fix](catalog) port #65782 collect_column_stats sink flag
+ write-metrics tests to the iceberg connector
add f7ef1b7f19b [fix](catalog) port #65784 authoritative iceberg name
mapping to the connector scan path
add 66afc37ad30 [refactor](catalog) fe-core: strip residual
iceberg/hive/hudi deps & dead code, delegate DDL validation to connectors,
remove hudi_meta TVF (#65893)
add 6a323619a75 [refactor](catalog) extract fe-trino-connector-common so
fe-common no longer depends on Trino
add 448c8fe31e0 [fix](catalog) migrate rebased-in
PhysicalStorageLayerAggregateTest to PluginDrivenExternalTable
add af40e7f88d1 [fix](catalog) bundle fastutil-core into BE
java-extensions that eager-load ConcurrentLong2LongHashMap
add bf0604252c4 [fix](catalog) port #65904 paimon special-char partition
names into fe-connector
add 667acebf527 [fix](catalog) reconcile upstream #65329 iceberg nested
column schema change with connector SPI (Tier 1: compile + flat DDL routing)
add 4906bc4c7bf [feat](catalog) port upstream #65329 iceberg nested column
schema change to the connector SPI (Tier 2)
add 0a4a20c0f0d [fix](catalog) catalog-SPI abstraction-mismatch review:
paimon time-travel crash, partition/schema fidelity, SPI convergence &
dead-code cleanup (#65970)
add 5b1cac0f84e [fix](catalog) preserve nested STRUCT field comment on
omitted COMMENT in complex MODIFY (build 1004182)
add 3686d6cb963 [fix](paimon) read latest schema in getColumnHandles so a
renamed column is not dropped
add cbfe3096dff [fix](be) fail the query instead of aborting BE on a
missing projected column
add cc8d7acb679 [refactor](catalog) drop generic-S3 minio.* folding
superseded by upstream #65834's dedicated fe-filesystem-minio
add a7f9bca5017 [fix](iceberg) scan same-table multi-version references
against each reference's pinned snapshot schema
add 8cd1beba03c [fix](paimon) read a branch's independent schema via its
own schema dir on the scan path
add 74c7a14a90e [test](paimon) align stale schema-evolution matrix tests
to shipped behavior
add 3a2b623696e [test](nereids) rebase-adapt PullUpJoinFromUnionAllTest to
the connector-migrated Hudi scan path
add 129aee13ef5 [opt](catalog) fe-connector: unify the connector cache
foundation, memoize per-statement hot paths, fix iceberg case-insensitive
column DDL (#66029)
add fd44aadd919 [fix](iceberg) dedupe partition columns so a
multi-transform spec cannot crash partition pruning
add 7c2f8f283f6 [fix](catalog) port #66008's paimon-cpp removal to the
paimon connector scan path
add fbff92e6823 [chore](handoff) record the 2026-07-27 rebase onto
5b3ac63f8b4
add ba78f2ca3b1 [fix](catalog) port #65955's paimon table-option
passthrough + jni.* rename to the connector
add 8fbcb0e7a19 [refactor](catalog) connector public interface cleanup: no
common-module edit to add a connector, dead-surface deletion, 11 correctness
fixes (#66135)
add e2b06b06732 [fix](build) Support Iceberg Docker stacks on Apple Silicon
add daa4602d6ad [fix](catalog) reconcile the connector SPI with upstream
#66004's fe-filesystem storage facade
add ab44e1f2791 [chore](handoff) record the 2026-07-27c rebase onto
e7b7f1d1359 (upstream #66004 storage facade)
add df0cdca39af [fix](build) keep CONTAINER_UID assignment literal so
pipeline sed still matches
add d37388f5c16 [fix](ci) hand the license-check file list over as a file,
not an env var
add bfd87eac9b3 [chore](license) satisfy the ASF header check on this
branch
add fec089aa63d [chore](ci) waive the unpatchable jackson-mapper-asl
advisories in dependency review
add efe8751dafb [refactor](catalog) port #65984 paimon query-level
@options into fe-connector-paimon
add 5a9dc529527 [test](catalog) port #65870 partition-evolution coverage
into fe-connector-iceberg
add 59a64dc5d91 [fix](paimon) read an @options tag pin's schemaId from the
tag, not the snapshot file
add 89ec5508aa8 [fix](catalog) bind a plugin system table's schema at its
own pin, not at latest
add f5c08907024 [test](iceberg) assert gateway/dedicated parity instead of
an absolute nested-DDL capability
add 6350588d639 [fix](catalog) position-map a plugin system table against
its own pin, not latest
add 4879554ca6f [test](iceberg) assert the nested-DDL refusal message this
branch actually emits
add 94aefd5cd86 [fix](docker) make the thirdparty hive/iceberg stacks
start on macOS
add 120172cb09a [fix](iceberg) persist a CREATE TABLE COMMENT clause
instead of dropping it
add 486ce433609 [refactor](catalog) fe-core: delete the unreachable
metastore property cluster and the dead storage/AWS-credential surface (#66210)
add 88abe41a4e3 [feature](plugin) replace the always-true connector
apiVersion() with a manifest-carried API version gate for all four plugin
families (#66211)
add 8a8482b33a6 [fix](catalog) port #66112's iceberg SQL MERGE cardinality
enforcement onto the connector SPI
add 3d88dcb32db [fix](catalog) port #66112's CTAS atomicity + IF NOT
EXISTS cache refresh onto the plugin bridge
add 2670497718b [chore](handoff) record the 2026-07-29b rebase onto
8305fe71867 (upstream #66112)
add 32c659744aa [fix](hive) port #65991's static-partition write onto the
connector SPI
add 0da96f1ad3e [chore](handoff) record the 2026-07-30 rebase onto
794d514479e (upstream #65991)
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (f672e789674)
\
N -- N -- N refs/heads/branch-catalog-spi (0da96f1ad3e)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.../api/write/ConnectorWritePlanProvider.java | 11 +-
.../connector/hive/HiveConnectorMetadata.java | 44 ++-
.../connector/hive/HiveWritePlanProvider.java | 14 +
.../connector/hive/HiveConnectorContractTest.java | 8 +
.../HiveConnectorMetadataStaticPartitionTest.java | 136 ++++++++
.../maxcompute/MaxComputeConnectorMetadata.java | 48 +++
.../MaxComputeStaticPartitionValidationTest.java | 97 ++++++
.../plugin/PluginDrivenExternalTable.java | 3 +-
.../nereids/analyzer/UnboundTableSinkCreator.java | 5 +-
.../doris/nereids/rules/analysis/BindSink.java | 73 +++-
.../trees/plans/commands/insert/InsertUtils.java | 15 +-
.../BindConnectorSinkStaticPartitionTest.java | 133 ++++++-
plan-doc/HANDOFF.md | 75 +++-
.../write/test_hive_write_static_partition.groovy | 383 +++++++++++++++++++++
14 files changed, 1015 insertions(+), 30 deletions(-)
create mode 100644
fe/fe-connector/fe-connector-hive/src/test/java/org/apache/doris/connector/hive/HiveConnectorMetadataStaticPartitionTest.java
create mode 100644
fe/fe-connector/fe-connector-maxcompute/src/test/java/org/apache/doris/connector/maxcompute/MaxComputeStaticPartitionValidationTest.java
create mode 100644
regression-test/suites/external_table_p0/hive/write/test_hive_write_static_partition.groovy
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
