This is an automated email from the ASF dual-hosted git repository.
morningman pushed a change to branch branch-catalog-spi
in repository https://gitbox.apache.org/repos/asf/doris.git
discard f9c96e1e37a [fix](build) Support Iceberg Docker stacks on Apple Silicon
discard f041a83e4dd [refactor](catalog) connector public interface cleanup: no
common-module edit to add a connector, dead-surface deletion, 11 correctness
fixes (#66135)
omit e48a96ae488 [fix](catalog) port #65955's paimon table-option
passthrough + jni.* rename to the connector
omit 8aec7238ddf [chore](handoff) record the 2026-07-27 rebase onto
5b3ac63f8b4
omit d09b1b51503 [fix](catalog) port #66008's paimon-cpp removal to the
paimon connector scan path
omit 4ed4714d3a1 [fix](iceberg) dedupe partition columns so a
multi-transform spec cannot crash partition pruning
omit dbbab3652e6 [opt](catalog) fe-connector: unify the connector cache
foundation, memoize per-statement hot paths, fix iceberg case-insensitive
column DDL (#66029)
omit 957b670c9f9 [test](nereids) rebase-adapt PullUpJoinFromUnionAllTest to
the connector-migrated Hudi scan path
omit 1978fe6ec36 [test](paimon) align stale schema-evolution matrix tests
to shipped behavior
omit 7e547346a4f [fix](paimon) read a branch's independent schema via its
own schema dir on the scan path
omit cae06ef3942 [fix](iceberg) scan same-table multi-version references
against each reference's pinned snapshot schema
omit bb83f846f85 [refactor](catalog) drop generic-S3 minio.* folding
superseded by upstream #65834's dedicated fe-filesystem-minio
omit b94d9ea3e81 [fix](be) fail the query instead of aborting BE on a
missing projected column
omit 50e77e247e7 [fix](paimon) read latest schema in getColumnHandles so a
renamed column is not dropped
omit ec7380334e3 [fix](catalog) preserve nested STRUCT field comment on
omitted COMMENT in complex MODIFY (build 1004182)
omit bc82352fe9a [fix](catalog) catalog-SPI abstraction-mismatch review:
paimon time-travel crash, partition/schema fidelity, SPI convergence &
dead-code cleanup (#65970)
omit 755051b4cba [feat](catalog) port upstream #65329 iceberg nested column
schema change to the connector SPI (Tier 2)
omit 1edd78e318e [fix](catalog) reconcile upstream #65329 iceberg nested
column schema change with connector SPI (Tier 1: compile + flat DDL routing)
omit 16d93ea35d8 [fix](catalog) port #65904 paimon special-char partition
names into fe-connector
omit 22ca0f168ba [fix](catalog) bundle fastutil-core into BE
java-extensions that eager-load ConcurrentLong2LongHashMap
omit 95af37d0e5d [fix](catalog) migrate rebased-in
PhysicalStorageLayerAggregateTest to PluginDrivenExternalTable
omit bf53596fbad [refactor](catalog) extract fe-trino-connector-common so
fe-common no longer depends on Trino
omit f2a14d4a7f9 [refactor](catalog) fe-core: strip residual
iceberg/hive/hudi deps & dead code, delegate DDL validation to connectors,
remove hudi_meta TVF (#65893)
omit be13de2eb98 [fix](catalog) port #65784 authoritative iceberg name
mapping to the connector scan path
omit 71e7d44a446 [fix](catalog) port #65782 collect_column_stats sink flag
+ write-metrics tests to the iceberg connector
omit 3a033b5c8dc [perf](catalog) two-level cross-query cache for external
partition derived views (#65829)
omit 3b56b794c55 [perf](catalog) fe-connector-iceberg hot-path caching +
fe-core per-statement metadata funnel, with session=user cache isolation
omit a83813add05 [fix](catalog) port #65676 iceberg deletion-vector
metadata validation to the connector scan path
omit 8ece644991f [fix](catalog) port #65548 external COUNT(*)/COUNT(col)
semantics to the connector scan path
omit 1616ecc9d92 [fix](catalog) iceberg system-table scan: restore #65262
positional JNI read + order-preserving projection
omit 3229456925d [fix](catalog) restore jfs hive-write hadoop config +
iceberg system-table scan projection
omit e4466b50526 [feat](catalog) fe-connector-iceberg: port #64966 REST 401
re-auth to the connector
omit be89f06bab0 [refactor](catalog) drop
ReauthenticatingRestSessionCatalog test orphaned by master rebase
omit f4693453ce1 [refactor](catalog) fe-core: remove 6 dead datasource
classes orphaned by the SPI migration (#65740)
omit 9f20e42243c [refactor](catalog) fe-core: remove dead
ExternalMetadataOps layer, deprecated LakeSoul catalog, and odbc/statistics
stubs (#65736)
omit 0fc7007b35c [refactor](catalog) fe-core: remove dead Trino-ported
DirectoryListercache cluster (#65734)
omit 4e39a39f2bc [refactor](catalog) fe-connector: replace fat
hive-catalog-shade with a slim HMS metastore-client shade (#65721)
omit 246eb5de2c3 [refactor](catalog) Catalog spi 11 hive (#65473)
omit 33da408760c [doc](catalog) HANDOFF: $position_deletes port done; e2e
is the only remaining flip gate
omit c3deb797c51 [feat](catalog) P6.6: support the iceberg
$position_deletes system table on the connector SPI
omit 2169ac305d1 [doc](catalog) HANDOFF: point next step at the
$position_deletes port design (T0 evidence in)
omit 6cf66158cb6 [doc](catalog) P6.6 $position_deletes connector port:
research notes + design (T0 evidence in, ready to implement)
omit 7f45de86b2b [doc](catalog) HANDOFF: rebase to upstream 30a44d61a8a +
register $position_deletes connector gap as flip blocker
omit a51040dcc14 [fix](build) fix parallel -T shade race in
fe-connector-paimon-hive-shade
omit b54307ef471 [fix](catalog) reconcile upstream #65365 with connector
SPI after rebase to 4f48ccfaa7d
omit 7f42a457bb3 [doc](catalog) P6.6-C6 #63068 HANDOFF: record
cache-leakage data-flow test + Task 0 retained-base verify
omit 04020f0d516 [feat](catalog) P6.6-C6 #63068 FE: re-migrate
ExternalDatabaseSessionContextTest (cross-user cache-leakage data-flow)
omit 117342a4805 [doc](catalog) P6.6-C6 #63068 OIDC session migration:
design + research notes + HANDOFF (Tasks 0-7 DONE)
omit 25ba722b60c [feat](catalog) P6.6-C6 #63068 iceberg: per-user REST
session routing (metadata + scan/write/procedure)
omit e5e07ae7a80 [feat](catalog) P6.6-C6 #63068 FE: capability-gated
credential injection + per-user cache bypass
omit affc033a871 [feat](catalog) P6.6-C6 #63068 SPI: neutral
delegated-credential DTO + user-session capability
omit a6b27dfad8e [fix](catalog) tier3 #65094: preserve external-table
top-level column-name case on the read path
omit ea05668f6a8 [fix](catalog) port upstream #65332/#65094 datasource
fixes into SPI connectors
omit dcb4744a0ef [fix](catalog) P4 maxcompute: drop dangling legacy
MaxComputeExternalTableTest after rebase
omit f7caa067ead [fix](catalog) jdbc: keep driver classloaders alive per
URL to stop Metaspace leak
omit c276f8ed737 [refactor](catalog) P6 iceberg: migrate to catalog SPI +
cutover + remove legacy fe-core subsystem (#64688)
omit 5fc8cb65e9f [refactor](catalog) P3b: consolidate kerberos
authentication into fe-kerberos (#64655)
omit 23ab32e2e1d [refactor](catalog) P5 paimon: remove legacy subsystem
from fe-core + make fe-core paimon-SDK-free (T29) (#64653)
omit 2d65237c5f8 [refactor](catalog) P5 paimon: migrate to catalog SPI +
cutover (#64446) (#64446)
omit 423b488e6e3 update P5 handoff and fix compile issue
omit 46153e26be6 [refactor](connector) P4 maxcompute: remove legacy
subsystem from fe-core + make fe-core odps-free (T07-T09) (#64300)
omit d5890d009c8 [feat](connector) P3 hudi connector hardening + test
baseline + dispatch design (hybrid, T02-T08) (#64143)
omit 1bb6dea1ec8 [feat](connector) P2 migrate trino-connector to catalog
SPI (T01-T13) (#64096)
omit d904e005f08 [P1-T03-T05] route plugin-driven scans first in nereids
translator (#63641)
omit 08c05bb371e [feat](connector) P0 SPI baseline + DDL/Partition + import
gate (T03-T27) (#63582)
omit 73da94b6dc7 [doc](connector) add project tracking system for catalog
SPI migration
add f70306ce044 [Opt](bitmap) Optimize bitmap op count rewrites (#65483)
add 0dde27390ac [Fix](user-property) Preserve legacy user properties
during desrialization (#65781)
add f499c78c67c [refactor](fs) Migrate fe-core storage consumers onto the
fe-filesystem SPI facade (#66004)
add efa3748b9ed [chore](deps) Upgrade vulnerable FE dependencies (#65700)
add e7b7f1d1359 [opt](cloud) Add warm up job count metric on BE (#64734)
add cb5f9d70c41 [doc](connector) add project tracking system for catalog
SPI migration
add 9e5c596059f [feat](connector) P0 SPI baseline + DDL/Partition + import
gate (T03-T27) (#63582)
add 6b624aef7ff [P1-T03-T05] route plugin-driven scans first in nereids
translator (#63641)
add 8a0554b5011 [feat](connector) P2 migrate trino-connector to catalog
SPI (T01-T13) (#64096)
add 7232e1c6ddb [feat](connector) P3 hudi connector hardening + test
baseline + dispatch design (hybrid, T02-T08) (#64143)
add 457257ef2ae [refactor](connector) P4 maxcompute: remove legacy
subsystem from fe-core + make fe-core odps-free (T07-T09) (#64300)
add c043691f607 update P5 handoff and fix compile issue
add 142cc821883 [refactor](catalog) P5 paimon: migrate to catalog SPI +
cutover (#64446) (#64446)
add 2f2a47b787c [refactor](catalog) P5 paimon: remove legacy subsystem
from fe-core + make fe-core paimon-SDK-free (T29) (#64653)
add 93d5754117a [refactor](catalog) P3b: consolidate kerberos
authentication into fe-kerberos (#64655)
add 8700870bfa6 [refactor](catalog) P6 iceberg: migrate to catalog SPI +
cutover + remove legacy fe-core subsystem (#64688)
add e16c8341583 [fix](catalog) jdbc: keep driver classloaders alive per
URL to stop Metaspace leak
add 3624a117877 [fix](catalog) P4 maxcompute: drop dangling legacy
MaxComputeExternalTableTest after rebase
add f68004b6909 [fix](catalog) port upstream #65332/#65094 datasource
fixes into SPI connectors
add 78e2853e660 [fix](catalog) tier3 #65094: preserve external-table
top-level column-name case on the read path
add 32a1c3c030c [feat](catalog) P6.6-C6 #63068 SPI: neutral
delegated-credential DTO + user-session capability
add b79a3cecaf4 [feat](catalog) P6.6-C6 #63068 FE: capability-gated
credential injection + per-user cache bypass
add 89a7397ccce [feat](catalog) P6.6-C6 #63068 iceberg: per-user REST
session routing (metadata + scan/write/procedure)
add cf5c690b801 [doc](catalog) P6.6-C6 #63068 OIDC session migration:
design + research notes + HANDOFF (Tasks 0-7 DONE)
add ccc2f936682 [feat](catalog) P6.6-C6 #63068 FE: re-migrate
ExternalDatabaseSessionContextTest (cross-user cache-leakage data-flow)
add 8902d92169f [doc](catalog) P6.6-C6 #63068 HANDOFF: record
cache-leakage data-flow test + Task 0 retained-base verify
add 4968cf14eaa [fix](catalog) reconcile upstream #65365 with connector
SPI after rebase to 4f48ccfaa7d
add 63cbee44c1f [fix](build) fix parallel -T shade race in
fe-connector-paimon-hive-shade
add 21dd757efff [doc](catalog) HANDOFF: rebase to upstream 30a44d61a8a +
register $position_deletes connector gap as flip blocker
add 0ae42404f18 [doc](catalog) P6.6 $position_deletes connector port:
research notes + design (T0 evidence in, ready to implement)
add 83467da912a [doc](catalog) HANDOFF: point next step at the
$position_deletes port design (T0 evidence in)
add 0e9ccf93b2b [feat](catalog) P6.6: support the iceberg
$position_deletes system table on the connector SPI
add d90b695dab1 [doc](catalog) HANDOFF: $position_deletes port done; e2e
is the only remaining flip gate
add 2844bc17962 [refactor](catalog) Catalog spi 11 hive (#65473)
add 675a56cfc27 [refactor](catalog) fe-connector: replace fat
hive-catalog-shade with a slim HMS metastore-client shade (#65721)
add 4839f614fc1 [refactor](catalog) fe-core: remove dead Trino-ported
DirectoryListercache cluster (#65734)
add f889c080151 [refactor](catalog) fe-core: remove dead
ExternalMetadataOps layer, deprecated LakeSoul catalog, and odbc/statistics
stubs (#65736)
add be22a7bd293 [refactor](catalog) fe-core: remove 6 dead datasource
classes orphaned by the SPI migration (#65740)
add 63a82f6ff51 [refactor](catalog) drop
ReauthenticatingRestSessionCatalog test orphaned by master rebase
add a5ada44b263 [feat](catalog) fe-connector-iceberg: port #64966 REST 401
re-auth to the connector
add 087097b5b60 [fix](catalog) restore jfs hive-write hadoop config +
iceberg system-table scan projection
add 2582ace313f [fix](catalog) iceberg system-table scan: restore #65262
positional JNI read + order-preserving projection
add 2f092cafd89 [fix](catalog) port #65548 external COUNT(*)/COUNT(col)
semantics to the connector scan path
add d1df9809fe2 [fix](catalog) port #65676 iceberg deletion-vector
metadata validation to the connector scan path
add 942829ddd9d [perf](catalog) fe-connector-iceberg hot-path caching +
fe-core per-statement metadata funnel, with session=user cache isolation
add 594fa9d1f0c [perf](catalog) two-level cross-query cache for external
partition derived views (#65829)
add cadf02e5706 [fix](catalog) port #65782 collect_column_stats sink flag
+ write-metrics tests to the iceberg connector
add 9ad4d4935eb [fix](catalog) port #65784 authoritative iceberg name
mapping to the connector scan path
add 009c59ecafe [refactor](catalog) fe-core: strip residual
iceberg/hive/hudi deps & dead code, delegate DDL validation to connectors,
remove hudi_meta TVF (#65893)
add 8559a82e0f2 [refactor](catalog) extract fe-trino-connector-common so
fe-common no longer depends on Trino
add 51d694fc4fa [fix](catalog) migrate rebased-in
PhysicalStorageLayerAggregateTest to PluginDrivenExternalTable
add fffc74f1020 [fix](catalog) bundle fastutil-core into BE
java-extensions that eager-load ConcurrentLong2LongHashMap
add fe9fa1a3298 [fix](catalog) port #65904 paimon special-char partition
names into fe-connector
add bdfff9c2a57 [fix](catalog) reconcile upstream #65329 iceberg nested
column schema change with connector SPI (Tier 1: compile + flat DDL routing)
add f9cbd1b8ebb [feat](catalog) port upstream #65329 iceberg nested column
schema change to the connector SPI (Tier 2)
add be71c254653 [fix](catalog) catalog-SPI abstraction-mismatch review:
paimon time-travel crash, partition/schema fidelity, SPI convergence &
dead-code cleanup (#65970)
add c2155b300fe [fix](catalog) preserve nested STRUCT field comment on
omitted COMMENT in complex MODIFY (build 1004182)
add 8da6076f178 [fix](paimon) read latest schema in getColumnHandles so a
renamed column is not dropped
add ff3251d135d [fix](be) fail the query instead of aborting BE on a
missing projected column
add a78045b3ea3 [refactor](catalog) drop generic-S3 minio.* folding
superseded by upstream #65834's dedicated fe-filesystem-minio
add 18df253ae94 [fix](iceberg) scan same-table multi-version references
against each reference's pinned snapshot schema
add d88fbbda243 [fix](paimon) read a branch's independent schema via its
own schema dir on the scan path
add f1afb5fe7bc [test](paimon) align stale schema-evolution matrix tests
to shipped behavior
add 7f778ebfded [test](nereids) rebase-adapt PullUpJoinFromUnionAllTest to
the connector-migrated Hudi scan path
add e473ac77e0e [opt](catalog) fe-connector: unify the connector cache
foundation, memoize per-statement hot paths, fix iceberg case-insensitive
column DDL (#66029)
add 638109c0946 [fix](iceberg) dedupe partition columns so a
multi-transform spec cannot crash partition pruning
add 5f3fdbc63d9 [fix](catalog) port #66008's paimon-cpp removal to the
paimon connector scan path
add 079dbace4a5 [chore](handoff) record the 2026-07-27 rebase onto
5b3ac63f8b4
add 77caeba35a9 [fix](catalog) port #65955's paimon table-option
passthrough + jni.* rename to the connector
add fce5af4e041 [refactor](catalog) connector public interface cleanup: no
common-module edit to add a connector, dead-surface deletion, 11 correctness
fixes (#66135)
add e1a8963dadb [fix](build) Support Iceberg Docker stacks on Apple Silicon
add e29884df07f [fix](catalog) reconcile the connector SPI with upstream
#66004's fe-filesystem storage facade
add a0c10f0672b [chore](handoff) record the 2026-07-27c rebase onto
e7b7f1d1359 (upstream #66004 storage facade)
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (f9c96e1e37a)
\
N -- N -- N refs/heads/branch-catalog-spi (a0c10f0672b)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
be/src/cloud/cloud_warm_up_manager.cpp | 22 +-
be/src/core/value/bitmap_value.h | 61 +
be/src/exprs/function/function.h | 1 +
be/src/exprs/function/function_bitmap.cpp | 101 +-
be/src/exprs/function/function_bitmap_variadic.cpp | 166 +-
be/src/exprs/function/simple_function_factory.h | 5 +
be/src/exprs/vectorized_fn_call.cpp | 5 +-
be/src/format_v2/expr/cast.cpp | 5 +-
be/test/core/value/bitmap_value_test.cpp | 5 +-
be/test/exprs/function/function_bitmap_test.cpp | 81 +-
.../function/simple_function_factory_test.cpp | 35 +
fe/1.pom | 255 ++
fe/1.tree | 4842 ++++++++++++++++++++
.../paimon-scanner/${sys:logPath} | 6 +
fe/fe-core/pom.xml | 105 +
.../java/org/apache/doris/analysis/BrokerDesc.java | 19 +-
.../org/apache/doris/analysis/OutFileClause.java | 17 +-
.../org/apache/doris/analysis/StorageBackend.java | 2 +-
.../org/apache/doris/analysis/StorageDesc.java | 32 +-
.../org/apache/doris/backup/BackupHandler.java | 6 +-
.../java/org/apache/doris/backup/Repository.java | 37 +-
.../org/apache/doris/catalog/AzureResource.java | 55 +-
.../doris/catalog/BuiltinScalarFunctions.java | 10 +-
.../org/apache/doris/catalog/HdfsStorageVault.java | 4 +-
.../java/org/apache/doris/catalog/S3Resource.java | 97 +-
.../org/apache/doris/catalog/S3StorageVault.java | 22 +-
.../org/apache/doris/catalog/StorageVaultMgr.java | 11 +-
.../apache/doris/cloud/load/CleanCopyJobTask.java | 6 +-
.../doris/cloud/load/CopyLoadPendingTask.java | 10 +-
.../org/apache/doris/cloud/stage/StageUtil.java | 6 +-
.../doris/cloud/storage/ObjectInfoAdapter.java | 53 +-
.../doris/common/util/DatasourcePrintableMap.java | 60 +-
.../org/apache/doris/common/util/LocationPath.java | 243 +-
.../doris/connector/DefaultConnectorContext.java | 49 +-
.../apache/doris/datasource/CatalogProperty.java | 120 +-
.../apache/doris/datasource/ExternalCatalog.java | 10 +-
.../datasource/credentials/CredentialUtils.java | 11 +-
.../plugin/PluginDrivenExternalCatalog.java | 2 +-
.../property/metastore/MetastoreProperties.java | 31 +
.../storage/AbstractS3CompatibleProperties.java | 342 --
.../property/storage/AzureProperties.java | 324 --
.../property/storage/AzurePropertyUtils.java | 246 -
.../property/storage/BrokerProperties.java | 116 -
.../datasource/property/storage/COSProperties.java | 188 -
.../datasource/property/storage/GCSProperties.java | 204 -
.../property/storage/HdfsCompatibleProperties.java | 59 -
.../property/storage/HdfsProperties.java | 221 -
.../property/storage/HdfsPropertiesUtils.java | 270 --
.../property/storage/HttpProperties.java | 92 -
.../property/storage/LocalProperties.java | 89 -
.../property/storage/MinioProperties.java | 151 -
.../datasource/property/storage/OBSProperties.java | 219 -
.../property/storage/OSSHdfsProperties.java | 217 -
.../datasource/property/storage/OSSProperties.java | 395 --
.../property/storage/OzoneProperties.java | 153 -
.../datasource/property/storage/S3Properties.java | 757 ---
.../property/storage/S3PropertyUtils.java | 229 -
.../property/storage/StorageProperties.java | 369 --
.../storage/CloudObjectStoreAdapter.java | 148 +
.../doris/datasource/storage/S3ResourceCompat.java | 206 +
.../doris/datasource/storage/S3ThriftAdapter.java | 140 +
.../doris/datasource/storage/StorageAdapter.java | 878 ++++
.../doris/datasource/storage/StorageRegistry.java | 179 +
.../StorageTypeId.java} | 52 +-
.../doris/datasource/storage/StorageUriUtils.java | 360 ++
.../doris/datasource/storage/package-info.java | 42 +
.../java/org/apache/doris/fs/FileSystemCache.java | 8 +-
.../org/apache/doris/fs/FileSystemDescriptor.java | 12 +-
.../org/apache/doris/fs/FileSystemFactory.java | 84 +-
.../apache/doris/fs/FileSystemPluginManager.java | 232 +-
.../java/org/apache/doris/fs/SchemaTypeMapper.java | 168 -
.../apache/doris/fs/SpiSwitchingFileSystem.java | 37 +-
.../doris/fs/StoragePropertiesConverter.java | 99 -
.../apache/doris/httpv2/rest/CopyIntoAction.java | 6 +-
.../job/offset/s3/S3SourceOffsetProvider.java | 18 +-
.../apache/doris/load/loadv2/BrokerLoadJob.java | 4 +-
.../mysql/privilege/CommonUserProperties.java | 4 +-
.../nereids/load/NereidsStreamLoadPlanner.java | 1 +
.../rules/expression/ExpressionNormalization.java | 4 +-
.../rules/expression/ExpressionRuleType.java | 3 +-
.../rules/BitmapCountToBitmapOpCount.java | 67 +
.../expression/rules/FoldConstantRuleOnBE.java | 1 +
.../functions/scalar/BitmapAndCount.java | 8 +-
.../expressions/functions/scalar/BitmapAndNot.java | 73 -
.../functions/scalar/BitmapAndNotAlias.java | 73 -
.../functions/scalar/BitmapAndNotCount.java | 8 +-
.../functions/scalar/BitmapAndNotCountAlias.java | 78 -
.../functions/scalar/BitmapOrCount.java | 7 +-
.../functions/scalar/BitmapXorCount.java | 8 +-
.../expressions/visitor/ScalarFunctionVisitor.java | 15 -
.../plans/commands/CreateRepositoryCommand.java | 6 +-
.../trees/plans/commands/CreateStageCommand.java | 6 +-
.../plans/commands/CreateStorageVaultCommand.java | 6 +-
.../nereids/trees/plans/commands/LoadCommand.java | 13 +-
.../trees/plans/commands/info/CopyIntoInfo.java | 33 +-
.../plans/commands/info/CreateResourceInfo.java | 5 +-
.../commands/insert/InsertIntoTVFCommand.java | 10 +-
.../org/apache/doris/planner/TVFTableSink.java | 12 +-
.../org/apache/doris/policy/StoragePolicy.java | 10 +-
.../main/java/org/apache/doris/qe/Coordinator.java | 3 +
.../org/apache/doris/qe/CoordinatorContext.java | 1 +
.../org/apache/doris/qe/NereidsCoordinator.java | 1 +
.../java/org/apache/doris/qe/SessionVariable.java | 1 +
.../ExternalFileTableValuedFunction.java | 21 +-
.../tablefunction/FileTableValuedFunction.java | 43 +-
.../tablefunction/HdfsTableValuedFunction.java | 15 +-
.../tablefunction/HttpTableValuedFunction.java | 36 +-
.../tablefunction/LocalTableValuedFunction.java | 20 +-
.../ParquetMetadataTableValuedFunction.java | 40 +-
.../doris/tablefunction/S3TableValuedFunction.java | 17 +-
.../apache/doris/task/PushStoragePolicyTask.java | 4 +-
.../doris/analysis/StorageDescPersistTest.java | 17 +-
.../org/apache/doris/backup/BackupJobTest.java | 4 +-
.../org/apache/doris/backup/RepositoryTest.java | 11 +-
.../org/apache/doris/backup/RestoreJobTest.java | 4 +-
.../doris/catalog/S3ResourcePersistParityTest.java | 124 +
.../org/apache/doris/catalog/S3ResourceTest.java | 60 +-
.../doris/cloud/backup/CloudRestoreJobTest.java | 4 +-
.../doris/cloud/catalog/CloudEnvFactoryTest.java | 32 +
.../doris/cloud/catalog/HdfsStorageVaultTest.java | 13 +-
.../doris/cloud/load/CopyLoadPendingTaskTest.java | 6 +-
.../apache/doris/cloud/stage/StageUtilTest.java | 4 +-
.../doris/cloud/storage/ObjectInfoAdapterTest.java | 57 +-
.../apache/doris/common/util/LocationPathTest.java | 60 +-
...ultConnectorContextBackendStoragePropsTest.java | 13 +-
.../DefaultConnectorContextFileSystemTest.java | 11 +-
.../DefaultConnectorContextNormalizeUriTest.java | 13 +-
.../DefaultConnectorContextStoragePropsTest.java | 8 +
...atalogPropertyEffectiveRawStoragePropsTest.java | 10 +-
...CatalogPropertyPluginStorageDerivationTest.java | 8 +-
.../credentials/CredentialUtilsTest.java | 38 +-
.../property/storage/AzurePropertiesTest.java | 294 --
.../property/storage/AzurePropertyUtilsTest.java | 203 -
.../property/storage/BrokerPropertiesTest.java | 79 -
.../property/storage/COSPropertiesTest.java | 200 -
.../doris/datasource/property/storage/GCSIT.java | 103 -
.../property/storage/GCSPropertiesTest.java | 131 -
.../property/storage/HdfsPropertiesTest.java | 209 -
.../property/storage/HdfsPropertiesUtilsTest.java | 225 -
.../property/storage/MinioPropertiesTest.java | 112 -
.../property/storage/OBSPropertyTest.java | 192 -
.../property/storage/OSSHdfsPropertiesTest.java | 219 -
.../property/storage/OSSPropertiesTest.java | 297 --
.../property/storage/OzonePropertiesTest.java | 180 -
.../storage/S3ConnectorPropertiesUtilsTest.java | 123 -
.../property/storage/S3PropertiesTest.java | 557 ---
.../property/storage/StoragePropertiesTest.java | 512 ---
.../storage/CloudObjectStoreAdapterParityTest.java | 130 +
.../storage/S3ThriftAdapterParityTest.java | 141 +
.../storage/StorageRegistryParityTest.java | 89 +
.../datasource/storage/StorageUriUtilsTest.java | 70 +
.../doris/fs/AzureGuessRoutingParityTest.java | 141 +
.../org/apache/doris/fs/FileSystemCacheTest.java | 6 +-
.../doris/fs/FileSystemFactoryBindAllTest.java | 8 +
.../doris/fs/FileSystemPluginManagerTest.java | 29 +-
.../org/apache/doris/fs/SchemaTypeMapperTest.java | 71 -
.../apache/doris/fs/SpiBindRoutingParityTest.java | 270 ++
.../doris/fs/SpiSwitchingFileSystemTest.java | 77 +-
.../fs/StoragePropertiesConverterHdfsDirTest.java | 60 -
.../fs/UnlistedProviderExtensibilityTest.java | 365 ++
.../mysql/privilege/CommonUserPropertiesTest.java | 86 +
.../rules/analysis/FunctionRegistryTest.java | 21 +
.../rules/BitmapCountToBitmapOpCountTest.java | 115 +
.../doris/qe/ShortCircuitQueryContextTest.java | 33 +
fe/fe-filesystem/fe-filesystem-api/pom.xml | 11 +-
.../properties/FileSystemProperties.java | 36 +
.../properties/HadoopStorageProperties.java | 23 +
.../S3CompatibleFileSystemProperties.java | 37 +
.../azure/AzureFileSystemProperties.java | 12 +
.../filesystem/azure/AzureFileSystemProvider.java | 27 +
.../broker/BrokerFileSystemProperties.java | 89 +
.../broker/BrokerFileSystemProvider.java | 24 +-
.../filesystem/cos/CosFileSystemProperties.java | 18 +
.../filesystem/cos/CosFileSystemProvider.java | 25 +
.../cos/CosFileSystemPropertiesTest.java | 46 +
.../filesystem/gcs/GcsFileSystemProperties.java | 6 +
.../filesystem/gcs/GcsFileSystemProvider.java | 10 +
.../doris/filesystem/hdfs/DFSFileSystem.java | 42 +-
.../doris/filesystem/hdfs/HdfsFileIterator.java | 8 +-
.../doris/filesystem/hdfs/HdfsInputFile.java | 8 +-
.../doris/filesystem/hdfs/HdfsOutputFile.java | 6 +-
.../hdfs/KerberosHadoopAuthenticator.java | 231 +
.../filesystem/hdfs/SimpleHadoopAuthenticator.java | 63 +
.../hdfs/properties/HdfsCompatibleProperties.java | 204 +-
.../filesystem/hdfs/HdfsFileIteratorTest.java | 22 +-
.../hdfs/KerberosHadoopAuthenticatorEnvTest.java | 26 +-
.../hdfs/SimpleHadoopAuthenticatorTest.java | 70 +
.../HdfsCompatiblePropertiesAuthTest.java | 166 +
.../filesystem/hdfs/HdfsFileSystemProperties.java | 378 --
.../filesystem/hdfs/HdfsFileSystemProvider.java | 74 +-
.../filesystem/hdfs/properties/HdfsProperties.java | 83 +-
.../filesystem/hdfs/DFSAuthenticatorTest.java | 84 -
.../hdfs/HdfsFileSystemPropertiesTest.java | 472 --
.../hdfs/HdfsPropertiesSpiParityTest.java | 146 +
.../hdfs/KerberosHadoopAuthenticatorTest.java | 165 +
.../filesystem/http/HttpFileSystemProperties.java | 18 +
.../filesystem/http/HttpFileSystemProvider.java | 12 +
.../filesystem/jfs/JfsFileSystemProvider.java | 40 +-
.../filesystem/jfs/properties/JfsProperties.java | 38 +-
.../filesystem/jfs/JfsPropertiesSpiParityTest.java | 106 +
.../jfs/properties/JfsPropertiesTest.java | 80 +-
.../local/LocalFileSystemProperties.java | 78 +
.../filesystem/local/LocalFileSystemProvider.java | 19 +-
.../minio/MinioFileSystemProperties.java | 18 +
.../filesystem/minio/MinioFileSystemProvider.java | 89 +
.../minio/MinioFileSystemPropertiesTest.java | 30 +
.../minio/MinioFileSystemProviderTest.java | 45 +
.../filesystem/obs/ObsFileSystemProperties.java | 18 +
.../filesystem/obs/ObsFileSystemProvider.java | 25 +
.../obs/ObsFileSystemPropertiesTest.java | 47 +
.../filesystem/hdfs/OssHdfsFileSystemProvider.java | 30 +-
.../hdfs/properties/OssHdfsProperties.java | 39 +
.../hdfs/OssHdfsPropertiesSpiParityTest.java | 106 +
.../filesystem/oss/OssFileSystemProperties.java | 27 +
.../filesystem/oss/OssFileSystemProvider.java | 59 +
.../oss/OssFileSystemPropertiesTest.java | 46 +
.../ozone/OzoneFileSystemProperties.java | 18 +
.../filesystem/ozone/OzoneFileSystemProvider.java | 7 +
.../ozone/OzoneFileSystemPropertiesTest.java | 30 +
.../filesystem/s3/S3FileSystemProperties.java | 44 +-
.../filesystem/s3/S3FileSystemPropertiesTest.java | 115 +-
.../doris/filesystem/s3/S3ObjStorageTest.java | 19 +-
.../doris/filesystem/s3/S3FileSystemProvider.java | 40 +
.../filesystem/spi/AzureBlobEndpointSignals.java | 166 +
.../doris/filesystem/spi/FileSystemProvider.java | 27 +
.../apache/doris/filesystem/spi/LegacyS3Uri.java | 275 ++
.../spi/AzureBlobEndpointSignalsTest.java | 90 +
.../security/ExecutionAuthenticator.java | 83 +
.../doris/foundation/security/IOCallable.java} | 17 +-
fe/fe-kerberos/pom.xml | 9 +-
.../doris/kerberos/ExecutionAuthenticator.java | 9 +-
fe/pom.xml | 5 +-
gensrc/thrift/PaloInternalService.thrift | 1 +
plan-doc/HANDOFF.md | 87 +
.../bitmap_functions/test_bitmap_function.out | 18 +
.../bitmap_functions/test_bitmap_function.groovy | 86 +
236 files changed, 14080 insertions(+), 11068 deletions(-)
create mode 100644 fe/1.pom
create mode 100644 fe/1.tree
create mode 100644 fe/be-java-extensions/paimon-scanner/${sys:logPath}
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/property/storage/AbstractS3CompatibleProperties.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/property/storage/AzureProperties.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/property/storage/AzurePropertyUtils.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/property/storage/BrokerProperties.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/property/storage/COSProperties.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/property/storage/GCSProperties.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/property/storage/HdfsCompatibleProperties.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/property/storage/HdfsProperties.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/property/storage/HdfsPropertiesUtils.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/property/storage/HttpProperties.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/property/storage/LocalProperties.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/property/storage/MinioProperties.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/property/storage/OBSProperties.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/property/storage/OSSHdfsProperties.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/property/storage/OSSProperties.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/property/storage/OzoneProperties.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/property/storage/S3Properties.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/property/storage/S3PropertyUtils.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/property/storage/StorageProperties.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/storage/CloudObjectStoreAdapter.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/storage/S3ResourceCompat.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/storage/S3ThriftAdapter.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/storage/StorageAdapter.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/storage/StorageRegistry.java
rename
fe/fe-core/src/main/java/org/apache/doris/datasource/{property/storage/ObjectStorageProperties.java
=> storage/StorageTypeId.java} (51%)
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/storage/StorageUriUtils.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/storage/package-info.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/fs/SchemaTypeMapper.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/fs/StoragePropertiesConverter.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/rules/BitmapCountToBitmapOpCount.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/scalar/BitmapAndNot.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/scalar/BitmapAndNotAlias.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/scalar/BitmapAndNotCountAlias.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/catalog/S3ResourcePersistParityTest.java
delete mode 100644
fe/fe-core/src/test/java/org/apache/doris/datasource/property/storage/AzurePropertiesTest.java
delete mode 100644
fe/fe-core/src/test/java/org/apache/doris/datasource/property/storage/AzurePropertyUtilsTest.java
delete mode 100644
fe/fe-core/src/test/java/org/apache/doris/datasource/property/storage/BrokerPropertiesTest.java
delete mode 100644
fe/fe-core/src/test/java/org/apache/doris/datasource/property/storage/COSPropertiesTest.java
delete mode 100644
fe/fe-core/src/test/java/org/apache/doris/datasource/property/storage/GCSIT.java
delete mode 100644
fe/fe-core/src/test/java/org/apache/doris/datasource/property/storage/GCSPropertiesTest.java
delete mode 100644
fe/fe-core/src/test/java/org/apache/doris/datasource/property/storage/HdfsPropertiesTest.java
delete mode 100644
fe/fe-core/src/test/java/org/apache/doris/datasource/property/storage/HdfsPropertiesUtilsTest.java
delete mode 100644
fe/fe-core/src/test/java/org/apache/doris/datasource/property/storage/MinioPropertiesTest.java
delete mode 100644
fe/fe-core/src/test/java/org/apache/doris/datasource/property/storage/OBSPropertyTest.java
delete mode 100644
fe/fe-core/src/test/java/org/apache/doris/datasource/property/storage/OSSHdfsPropertiesTest.java
delete mode 100644
fe/fe-core/src/test/java/org/apache/doris/datasource/property/storage/OSSPropertiesTest.java
delete mode 100644
fe/fe-core/src/test/java/org/apache/doris/datasource/property/storage/OzonePropertiesTest.java
delete mode 100644
fe/fe-core/src/test/java/org/apache/doris/datasource/property/storage/S3ConnectorPropertiesUtilsTest.java
delete mode 100644
fe/fe-core/src/test/java/org/apache/doris/datasource/property/storage/S3PropertiesTest.java
delete mode 100644
fe/fe-core/src/test/java/org/apache/doris/datasource/property/storage/StoragePropertiesTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/datasource/storage/CloudObjectStoreAdapterParityTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/datasource/storage/S3ThriftAdapterParityTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/datasource/storage/StorageRegistryParityTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/datasource/storage/StorageUriUtilsTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/fs/AzureGuessRoutingParityTest.java
delete mode 100644
fe/fe-core/src/test/java/org/apache/doris/fs/SchemaTypeMapperTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/fs/SpiBindRoutingParityTest.java
delete mode 100644
fe/fe-core/src/test/java/org/apache/doris/fs/StoragePropertiesConverterHdfsDirTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/fs/UnlistedProviderExtensibilityTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/mysql/privilege/CommonUserPropertiesTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/nereids/rules/expression/rules/BitmapCountToBitmapOpCountTest.java
create mode 100644
fe/fe-filesystem/fe-filesystem-broker/src/main/java/org/apache/doris/filesystem/broker/BrokerFileSystemProperties.java
create mode 100644
fe/fe-filesystem/fe-filesystem-hdfs-base/src/main/java/org/apache/doris/filesystem/hdfs/KerberosHadoopAuthenticator.java
create mode 100644
fe/fe-filesystem/fe-filesystem-hdfs-base/src/main/java/org/apache/doris/filesystem/hdfs/SimpleHadoopAuthenticator.java
create mode 100644
fe/fe-filesystem/fe-filesystem-hdfs-base/src/test/java/org/apache/doris/filesystem/hdfs/SimpleHadoopAuthenticatorTest.java
create mode 100644
fe/fe-filesystem/fe-filesystem-hdfs-base/src/test/java/org/apache/doris/filesystem/hdfs/properties/HdfsCompatiblePropertiesAuthTest.java
delete mode 100644
fe/fe-filesystem/fe-filesystem-hdfs/src/main/java/org/apache/doris/filesystem/hdfs/HdfsFileSystemProperties.java
delete mode 100644
fe/fe-filesystem/fe-filesystem-hdfs/src/test/java/org/apache/doris/filesystem/hdfs/DFSAuthenticatorTest.java
delete mode 100644
fe/fe-filesystem/fe-filesystem-hdfs/src/test/java/org/apache/doris/filesystem/hdfs/HdfsFileSystemPropertiesTest.java
create mode 100644
fe/fe-filesystem/fe-filesystem-hdfs/src/test/java/org/apache/doris/filesystem/hdfs/HdfsPropertiesSpiParityTest.java
create mode 100644
fe/fe-filesystem/fe-filesystem-hdfs/src/test/java/org/apache/doris/filesystem/hdfs/KerberosHadoopAuthenticatorTest.java
create mode 100644
fe/fe-filesystem/fe-filesystem-jfs/src/test/java/org/apache/doris/filesystem/jfs/JfsPropertiesSpiParityTest.java
create mode 100644
fe/fe-filesystem/fe-filesystem-local/src/main/java/org/apache/doris/filesystem/local/LocalFileSystemProperties.java
create mode 100644
fe/fe-filesystem/fe-filesystem-oss-hdfs/src/test/java/org/apache/doris/filesystem/hdfs/OssHdfsPropertiesSpiParityTest.java
create mode 100644
fe/fe-filesystem/fe-filesystem-spi/src/main/java/org/apache/doris/filesystem/spi/AzureBlobEndpointSignals.java
create mode 100644
fe/fe-filesystem/fe-filesystem-spi/src/main/java/org/apache/doris/filesystem/spi/LegacyS3Uri.java
create mode 100644
fe/fe-filesystem/fe-filesystem-spi/src/test/java/org/apache/doris/filesystem/spi/AzureBlobEndpointSignalsTest.java
create mode 100644
fe/fe-foundation/src/main/java/org/apache/doris/foundation/security/ExecutionAuthenticator.java
rename
fe/{fe-core/src/main/java/org/apache/doris/datasource/property/storage/exception/AzureAuthType.java
=>
fe-foundation/src/main/java/org/apache/doris/foundation/security/IOCallable.java}
(64%)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
