This is an automated email from the ASF dual-hosted git repository.
morningman pushed a change to branch branch-catalog-spi
in repository https://gitbox.apache.org/repos/asf/doris.git
omit 9bc2038eb18 [fix](catalog) reconcile upstream #65365 with connector
SPI after rebase to 4f48ccfaa7d
omit 69a9c02f042 [doc](catalog) P6.6-C6 #63068 HANDOFF: record
cache-leakage data-flow test + Task 0 retained-base verify
omit 2a5cc99ec44 [feat](catalog) P6.6-C6 #63068 FE: re-migrate
ExternalDatabaseSessionContextTest (cross-user cache-leakage data-flow)
omit 0f77b8c63d9 [doc](catalog) P6.6-C6 #63068 OIDC session migration:
design + research notes + HANDOFF (Tasks 0-7 DONE)
omit 1e15fe8d61e [feat](catalog) P6.6-C6 #63068 iceberg: per-user REST
session routing (metadata + scan/write/procedure)
omit 28dbceecce1 [feat](catalog) P6.6-C6 #63068 FE: capability-gated
credential injection + per-user cache bypass
omit 49e86060e73 [feat](catalog) P6.6-C6 #63068 SPI: neutral
delegated-credential DTO + user-session capability
omit 4af6450fdc9 [fix](catalog) tier3 #65094: preserve external-table
top-level column-name case on the read path
omit 3797e2550c7 [fix](catalog) port upstream #65332/#65094 datasource
fixes into SPI connectors
omit adb08b5ce21 [fix](catalog) P4 maxcompute: drop dangling legacy
MaxComputeExternalTableTest after rebase
omit 1f346008e84 [fix](catalog) jdbc: keep driver classloaders alive per
URL to stop Metaspace leak
omit 03c5a315200 [refactor](catalog) P6 iceberg: migrate to catalog SPI +
cutover + remove legacy fe-core subsystem (#64688)
omit b417e7e4714 [refactor](catalog) P3b: consolidate kerberos
authentication into fe-kerberos (#64655)
omit 07d18d2a9e5 [refactor](catalog) P5 paimon: remove legacy subsystem
from fe-core + make fe-core paimon-SDK-free (T29) (#64653)
omit ea3798312a2 [refactor](catalog) P5 paimon: migrate to catalog SPI +
cutover (#64446) (#64446)
omit 6dc4bc10223 update P5 handoff and fix compile issue
omit a0983c330d9 [refactor](connector) P4 maxcompute: remove legacy
subsystem from fe-core + make fe-core odps-free (T07-T09) (#64300)
omit e38d36236e8 [feat](connector) P3 hudi connector hardening + test
baseline + dispatch design (hybrid, T02-T08) (#64143)
omit 222f7d53dcc [feat](connector) P2 migrate trino-connector to catalog
SPI (T01-T13) (#64096)
omit 35e09af1bb5 [P1-T03-T05] route plugin-driven scans first in nereids
translator (#63641)
omit 7c285e1a521 [feat](connector) P0 SPI baseline + DDL/Partition + import
gate (T03-T27) (#63582)
omit 7c44b34b0a4 [doc](connector) add project tracking system for catalog
SPI migration
add 7604fbf0773 [fix](be) Fix Iceberg row-level deletes across schema
evolution (#65502)
add edd50b6b1d4 [fix](fe) Relax nullable slot check in outer join to anti
join (#65110)
add 9b26dc6be40 [improvement](release) Improve release helper workflow
(#65488)
add 61c910f2cb5 [fix](fe) Fix stream scan normalization and snapshot/reset
pruning (#65468)
add 3ab1ff60952 [chore](build) Update ASF GitHub collaborators (#65563)
add 36bc996f449 [fix](be) Keep lazy map child columns consistent (#65355)
add f052d9da44e [fix](UDF) Fix create function rollback after Nereids
failure (#65100)
add 9345d787d23 [doc](connector) add project tracking system for catalog
SPI migration
add 21e0f021e6c [feat](connector) P0 SPI baseline + DDL/Partition + import
gate (T03-T27) (#63582)
add 3a2d2aa80b0 [P1-T03-T05] route plugin-driven scans first in nereids
translator (#63641)
add e0c1f8fbd9b [feat](connector) P2 migrate trino-connector to catalog
SPI (T01-T13) (#64096)
add 133708b4698 [feat](connector) P3 hudi connector hardening + test
baseline + dispatch design (hybrid, T02-T08) (#64143)
add 19b6d293834 [refactor](connector) P4 maxcompute: remove legacy
subsystem from fe-core + make fe-core odps-free (T07-T09) (#64300)
add 1b8b7d63840 update P5 handoff and fix compile issue
add 1baf73fe026 [refactor](catalog) P5 paimon: migrate to catalog SPI +
cutover (#64446) (#64446)
add 6cc203d1e0d [refactor](catalog) P5 paimon: remove legacy subsystem
from fe-core + make fe-core paimon-SDK-free (T29) (#64653)
add c6a1f94eeec [refactor](catalog) P3b: consolidate kerberos
authentication into fe-kerberos (#64655)
add ec3a0cd55f4 [refactor](catalog) P6 iceberg: migrate to catalog SPI +
cutover + remove legacy fe-core subsystem (#64688)
add 4363a8519f6 [fix](catalog) jdbc: keep driver classloaders alive per
URL to stop Metaspace leak
add 02e7bb44b49 [fix](catalog) P4 maxcompute: drop dangling legacy
MaxComputeExternalTableTest after rebase
add 97be32c43d6 [fix](catalog) port upstream #65332/#65094 datasource
fixes into SPI connectors
add 3670071e986 [fix](catalog) tier3 #65094: preserve external-table
top-level column-name case on the read path
add 1ea883b711d [feat](catalog) P6.6-C6 #63068 SPI: neutral
delegated-credential DTO + user-session capability
add 4ffdc391aec [feat](catalog) P6.6-C6 #63068 FE: capability-gated
credential injection + per-user cache bypass
add 2f9b1c9caa3 [feat](catalog) P6.6-C6 #63068 iceberg: per-user REST
session routing (metadata + scan/write/procedure)
add 7b26b49dd6b [doc](catalog) P6.6-C6 #63068 OIDC session migration:
design + research notes + HANDOFF (Tasks 0-7 DONE)
add 5c2263694c1 [feat](catalog) P6.6-C6 #63068 FE: re-migrate
ExternalDatabaseSessionContextTest (cross-user cache-leakage data-flow)
add 22af8613ef7 [doc](catalog) P6.6-C6 #63068 HANDOFF: record
cache-leakage data-flow test + Task 0 retained-base verify
add 83bc9259aa3 [fix](catalog) reconcile upstream #65365 with connector
SPI after rebase to 4f48ccfaa7d
add e0c392b9be0 [fix](build) fix parallel -T shade race in
fe-connector-paimon-hive-shade
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (9bc2038eb18)
\
N -- N -- N refs/heads/branch-catalog-spi (e0c392b9be0)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.asf.yaml | 4 +-
be/src/format/table/iceberg_reader.cpp | 203 +++-
be/src/format/table/iceberg_reader.h | 10 +-
be/src/format/table/iceberg_reader_mixin.h | 142 +++
be/src/format_v2/column_data.h | 6 +
be/src/format_v2/column_mapper.cpp | 88 +-
be/src/format_v2/column_mapper.h | 4 +
.../format_v2/expr/equality_delete_predicate.cpp | 47 +-
be/src/format_v2/expr/equality_delete_predicate.h | 5 +-
be/src/format_v2/file_reader.h | 3 +-
be/src/format_v2/materialized_reader_util.cpp | 26 +-
be/src/format_v2/table/iceberg_reader.cpp | 338 +++++--
be/src/format_v2/table/iceberg_reader.h | 30 +-
be/src/format_v2/table_reader.cpp | 54 ++
be/src/format_v2/table_reader.h | 5 +-
be/src/storage/segment/column_reader.cpp | 84 +-
.../format/table/iceberg/iceberg_reader_test.cpp | 641 ++++++++++++
.../expr/equality_delete_predicate_test.cpp | 21 +
be/test/format_v2/json/json_reader_test.cpp | 8 +-
be/test/format_v2/table/iceberg_reader_test.cpp | 1016 ++++++++++++++++++--
be/test/format_v2/table_reader_test.cpp | 57 ++
be/test/storage/segment/column_reader_test.cpp | 126 +++
.../fe-connector-paimon-hive-shade/pom.xml | 22 +
.../java/org/apache/doris/catalog/Database.java | 83 +-
.../main/java/org/apache/doris/catalog/Env.java | 7 +
.../org/apache/doris/catalog/FunctionRegistry.java | 8 +-
.../org/apache/doris/catalog/FunctionUtil.java | 18 +-
.../apache/doris/catalog/GlobalFunctionMgr.java | 4 +-
.../java/org/apache/doris/catalog/OlapTable.java | 4 +-
.../org/apache/doris/catalog/OlapTableWrapper.java | 7 +-
.../catalog/stream/OlapTableStreamWrapper.java | 8 +-
...tTableStreamUpdate.java => StreamReadMode.java} | 10 +-
.../org/apache/doris/datasource/ExternalUtil.java | 44 +-
.../doris/datasource/iceberg/IcebergUtils.java | 66 +-
.../datasource/iceberg/source/IcebergScanNode.java | 35 +-
.../org/apache/doris/journal/JournalEntity.java | 6 +
.../doris/nereids/jobs/executor/Rewriter.java | 13 +-
.../doris/nereids/rules/analysis/BindRelation.java | 7 +-
.../rules/exploration/mv/PartitionCompensator.java | 4 +-
.../rules/rewrite/ConvertOuterJoinToAntiJoin.java | 62 +-
.../nereids/rules/rewrite/PruneEmptyPartition.java | 2 +-
.../trees/expressions/functions/udf/AliasUdf.java | 15 +-
.../expressions/functions/udf/UdfBuilder.java | 4 +
.../plans/commands/CreateFunctionCommand.java | 12 +-
.../plans/commands/UpdateMvByPartitionCommand.java | 2 +-
.../plans/logical/LogicalCatalogRelation.java | 5 +
.../plans/logical/LogicalOlapTableStreamScan.java | 202 +++-
...leOperationLog.java => CreateFunctionInfo.java} | 36 +-
.../java/org/apache/doris/persist/EditLog.java | 9 +
.../org/apache/doris/persist/OperationType.java | 1 +
.../org/apache/doris/planner/OlapScanNode.java | 2 +-
.../apache/doris/catalog/CreateFunctionTest.java | 317 ++++++
.../apache/doris/datasource/ExternalUtilTest.java | 12 +-
.../doris/datasource/iceberg/IcebergUtilsTest.java | 43 +
.../iceberg/source/IcebergScanNodeTest.java | 46 +
.../exploration/mv/PartitionCompensatorTest.java | 5 +-
.../rewrite/ConvertOuterJoinToAntiJoinTest.java | 97 ++
.../doris/nereids/trees/expressions/UdfTest.java | 2 +-
.../trees/plans/ExplainTableStreamPlanTest.java | 62 ++
gensrc/thrift/ExternalTableSchema.thrift | 11 +-
.../struct_map_pruning_lazy_read.out} | 7 +-
.../test_min_delta_stream.out} | 4 +-
...eberg_equality_delete_with_schema_change.groovy | 16 +-
.../test_create_function_rollback.groovy | 52 +
.../struct_map_pruning_lazy_read.groovy | 55 ++
.../transform_outer_join_to_anti.groovy | 15 +
.../table_stream_p0/test_min_delta_stream.groovy | 60 +-
tools/release-tools/01-check-env.sh | 2 +-
tools/release-tools/04-release-complete.sh | 230 +++++
tools/release-tools/README.md | 336 +++++--
tools/release-tools/release.env | 13 +-
.../release-tools/tests/run.sh | 12 +-
.../tests/test-check-env-required-tools.sh | 21 +-
.../tests/test-readme-release-complete-guidance.sh | 19 +-
.../tests/test-release-complete-checksum.sh | 199 ++++
75 files changed, 4723 insertions(+), 529 deletions(-)
copy
fe/fe-core/src/main/java/org/apache/doris/catalog/stream/{AbstractTableStreamUpdate.java
=> StreamReadMode.java} (87%)
copy
fe/fe-core/src/main/java/org/apache/doris/persist/{DropSqlBlockRuleOperationLog.java
=> CreateFunctionInfo.java} (56%)
copy regression-test/data/{datatype_p0/agg_state/array/array.out =>
nereids_rules_p0/column_pruning/struct_map_pruning_lazy_read.out} (69%)
copy regression-test/data/{ann_index_p0/ivf_pq_full_buffer_train_recall.out =>
table_stream_p0/test_min_delta_stream.out} (58%)
create mode 100644
regression-test/suites/fault_injection_p0/test_create_function_rollback.groovy
create mode 100644
regression-test/suites/nereids_rules_p0/column_pruning/struct_map_pruning_lazy_read.groovy
create mode 100755 tools/release-tools/04-release-complete.sh
copy samples/datalake/hudi/scripts/start_doris.sh =>
tools/release-tools/tests/run.sh (81%)
copy docker/thirdparties/docker-compose/kerberos/health-checks/health.sh =>
tools/release-tools/tests/test-check-env-required-tools.sh (71%)
copy docker/thirdparties/docker-compose/kerberos/health-checks/health.sh =>
tools/release-tools/tests/test-readme-release-complete-guidance.sh (74%)
create mode 100755 tools/release-tools/tests/test-release-complete-checksum.sh
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
