This is an automated email from the ASF dual-hosted git repository.
924060929 pushed a change to branch fe_local_shuffle_rebase3
in repository https://gitbox.apache.org/repos/asf/doris.git
omit 8f1befd3016 [fix](local shuffle) doc: SortNode is before/below
AnalyticEvalNode, not above
omit 832ba5f0bf8 [fix](local shuffle) dedup OperatorBase::is_hash_shuffle
into is_shuffled_exchange
omit 6a48e0864df [fix](local shuffle) update stale HASH_SHUFFLE comments
after rename to GLOBAL_EXECUTION_HASH_SHUFFLE
omit 14ff5b48070 [doc](local shuffle) explain why RequireHash.preferType()
is GLOBAL (vs the LOCAL specialization in resolveExchangeType)
omit 19de375e12e [fix](local shuffle) address morrySnow review nits
omit fe9a3316be0 [fix](local shuffle) address morrySnow review (FE/BE)
omit 1d48e9ae984 [fix](local shuffle) drop unrelated macOS-build changes
omit 1d7cf6dbd60 [fix](local shuffle) doc: note the converse (id,name)->id
repartition needs cross-BE shuffle
omit 603c87096bf [fix](local shuffle) address morrySnow review: thrift enum
comments + optional partition_type
omit 73cc6458fd1 [fix](local shuffle) trim trailing blank lines at EOF (git
diff --check)
omit 8db568b5100 [fix](local shuffle) address /review: TVFScan leaf output
+ Bug 21 timeout
omit 9e03688df6a [fix](local shuffle) BE test: post-rename type fixes
(verified by local compile)
omit 3630071a131 [fix](local shuffle) remove serial-without-LE validation
omit 57a36a58112 [fix](local shuffle) serial validation throw to warn;
null-safe force_passthrough
omit a97d3347789 [opt](local shuffle) single topological sweep for pipeline
num_tasks
omit 7ce9d4a2c4b [fix](local shuffle) address review comments on PR #63366
omit 818dbab57ff [fix](build) macOS: python UDF boost/process via compat
shim; clang-format + license header
omit 79645fd6bc3 [fix](local shuffle) convert remaining ExchangeType refs
to TLocalPartitionType after rebase
omit eaa608631ee [fix](local shuffle) fall back to LOCAL hash for serial
source in PARTITIONED join/intersect
omit 47421053e50 [test](local shuffle) add DORIS-26103 regression: UNION
ALL + PartitionTopN + INTERSECT
omit db2d37a6097 [test](local shuffle) regression tests for
DORIS-26100/26101 global hash require
omit 31ab99ab14a [fix](local shuffle) Intersect/Except:
requireGlobalExecutionHash instead of autoRequireHash
omit 7ad0d52e3e6 [fix](local shuffle) PARTITIONED hash join:
requireGlobalExecutionHash instead of requireHash
omit b7de8783739 [test](local shuffle) update
LocalShuffleNodeCoverageTest.testRepeatNode for noRequire fix
omit 76b535df6c2 [test](local shuffle) cover RepeatNode noRequire +
return-child-distribution behaviors
omit 23d59442ca8 [fix](local shuffle) RepeatNode: stop forwarding hash
require below repeat
omit f7748e3324f [fix](local shuffle) Route to NereidsCoordinator based on
distributedPlans
omit ef43f96b5fd [doc](local shuffle) DANGER ZONE on sender_count to
prevent repeated mistakes
omit 151ff1d1778 [revert](local shuffle) Roll back the sender_count
std::max patch
omit cc7cf680d48 [fix](local shuffle) Mirror BE partition_exprs and
sender_count for FE-planned LE
omit d5a7b9f30ff [test](local shuffle) Switch order-sensitive checks to
order_qt_*
omit 727d3122f1e [fix](local shuffle) Skip LocalExchange under
RecursiveCteNode and mark it serial (DORIS-25865)
omit 8f83192f42a [fix](local shuffle) Restore LoadStatistic fileNum
tracking under FE local-shuffle planner
omit d360904ba7f [test](local shuffle) Drop stats-sensitive shape
assertions from test_multilevel_join_agg_local_shuffle
omit ea36b9b6f27 [test](local shuffle) Rewrite LocalExchangePlannerTest
with PlanShape DSL
omit 255cbd37e57 [refactor](local shuffle) Move local exchange planning
from BE to FE
add 9f6782e5d0f [fix](build) Fix build and test scripts failing due to
missing executable permission (#63330)
add f8c8dd7b619 [Fix](nereids) Fix month nullable datetimev2 literal
binding (#64459)
add 4c616855c0e [doc](fe) Clarify optimizer review output style (#64490)
add 818bc6d9376 [enhancement](workload policy) Add username-based backend
workload policy support (#60559)
add 1316d4bd633 [fix](bucket) Raise autobucket_min_buckets default from 1
to 3 (#63729)
add 28d590767af [fix](catalog) Handle incomplete dynamic partition
properties (#63831)
add 46129830bb9 [chore](dep)Upgrade dependencies (#64208)
add 989cca2875c [fix](mtmv) Add null-safety to getBaseViewsOneLevel for
backward compatibility (#64412)
add fad12a080a2 fix(regression): Make test_analyze_mv row_count assertion
stable after truncate (#64419)
add 9fb627b3469 [fix](test) Make test_analyze_long_string Case 5 stable
against sample rows randomness (#64408)
add 9ea1b774a3f [fix](lazy topn) Fix slot-not-found after
PullUpProjectExprUnderTopN with chained expressions (#64486)
add ec3346d03e3 [test](regression) Enable segment limit pushdown fuzzy
(#64479)
add 3534b85fdba [fix](case) fix insert_group_commit_into_max_filter_ratio
(#64402)
add 59383bf4d8c [codeowner] Update and fix codeowner (#64524)
add 5cf768f7b92 [feature](cloud) Add table-level event-driven warm up
(#63832)
add 0f1c6d2c387 [enhancement](CI) Force inject required AGENTS guides into
review prompt (#64536)
add 2a113a6eab0 [feat](fs)Add native filesystem SPI for object storage
(#63400)
add fef8ed342cb [improvement](cloud) Shard FSFileCacheStorage writer map
(#63789)
add 13ecb9d8cfd [test](regression) Remove no simdjson flexible update case
(#64513)
add a0a21f8482f [fix](regression) Wait for completed profiles in RF
pruning tests (#64392)
add 0050f18c709 [fix](regression): stabilize
push_down_filter_through_set_operation_with_unique_function plan shape (#64516)
add ca4af95e2c7 [refactor](storage) Drop PredicateColumnType (#64128)
add 085ac84bbd9 [test](nereids) stabilize flaky
prune_bucket_with_bucket_shuffle_join (#64530)
add a958942a3d9 [feature](ann-index) Add ann topn small candidate fallback
session var. (#64243)
add 513bf29622c [fix](fe) Add MergeProjectable after ColumnPruning in
DPHyp join reorder to merge consecutive projects (#64409)
add a57a2b36afe [fix](join) Restrict auto salt join rewrite (#64518)
add 28ef1b29daf [fix](fe) Prevent cast project pushdown through union
distinct (#64080)
add 117087fee5f [test](streaming-job) add cdc_client unit tests and
read-path Testcontainers ITCases (#64277)
add 8f0b8123087 [fix](binlog): fix GroupRowsetWriterTest coredump due to
incomplete binlog meta setting (#64545)
add 64e4ac94d6b [fix](cloud) Reset FDB test globals before exit (#64549)
add 28c17f83589 [fix](regression) fix regression test_hive_ctas_to_doris
failure (#64553)
add 800b6a100cf [Fix](test) Allow bitmap base64 serialization variants
(#64505)
add 7bef21c5be6 [fix](test) fix regression test eliminate_inner (#64457)
add e025050cdb0 [fix](streaming-job) streaming CDC reliability fixes
(scheduling latch, fetch-meta reason, PG db name, flaky case) (#64310)
add 7e8f1b21ea8 [fix](variant) Preserve json object when casting jsonb to
variant (#63792)
add 35089583cf8 [fix](variant) Handle truncated sparse path stats when
reading variant (#64205)
add f9e8d698327 [fix](cloud) Fix some docker case due to open
skip_writing_empty_rowset_metadata (#64565)
add 6e368807cf8 [test](mv) Add MV residual compensation coverage (#64051)
add 954e717b1bf [fix](audit) record per-query SET_VAR hint session
variables in audit log (#64569)
add 4fec15443ab [fix](topn) Skip TopN lazy materialization when
light_schema_change=false (#64441)
add 6879d610659 [fix](be) Normalize v1 date string cast result (#64575)
add a2eb4d6eed6 [fix](NestedColumnPruning) collect full-access paths for
array columns unreferenced in lambda body (#64436)
add 08cd09f4ff5 [fix](stream_load) Fix stream load IPv6 host parsing
(#64147)
add ab8f6572aea [chore](ci) declare least-privilege permissions on
clang-format and title-checker workflows (#63573)
add 05f10d95fa7 [refactor](local shuffle) Move local exchange planning
from BE to FE
add da1dea3167f [test](local shuffle) Rewrite LocalExchangePlannerTest
with PlanShape DSL
add b30dbcfdec9 [test](local shuffle) Drop stats-sensitive shape
assertions from test_multilevel_join_agg_local_shuffle
add 9f8c31ffb71 [fix](local shuffle) Restore LoadStatistic fileNum
tracking under FE local-shuffle planner
add 4db62ade053 [fix](local shuffle) Skip LocalExchange under
RecursiveCteNode and mark it serial (DORIS-25865)
add fa9d9658b9c [test](local shuffle) Switch order-sensitive checks to
order_qt_*
add 4869ca029dd [fix](local shuffle) Mirror BE partition_exprs and
sender_count for FE-planned LE
add a091555e73c [revert](local shuffle) Roll back the sender_count
std::max patch
add b67e6ace13a [doc](local shuffle) DANGER ZONE on sender_count to
prevent repeated mistakes
add cbe2cf7eacc [fix](local shuffle) Route to NereidsCoordinator based on
distributedPlans
add f81d9b5a26e [fix](local shuffle) RepeatNode: stop forwarding hash
require below repeat
add 9de0b210ffa [test](local shuffle) cover RepeatNode noRequire +
return-child-distribution behaviors
add 9705235d927 [test](local shuffle) update
LocalShuffleNodeCoverageTest.testRepeatNode for noRequire fix
add 45635941828 [fix](local shuffle) PARTITIONED hash join:
requireGlobalExecutionHash instead of requireHash
add 473d21b682d [fix](local shuffle) Intersect/Except:
requireGlobalExecutionHash instead of autoRequireHash
add b2348559fbb [test](local shuffle) regression tests for
DORIS-26100/26101 global hash require
add 75c5e0eeeb4 [test](local shuffle) add DORIS-26103 regression: UNION
ALL + PartitionTopN + INTERSECT
add 3d71fc7eb9e [fix](local shuffle) fall back to LOCAL hash for serial
source in PARTITIONED join/intersect
add 30f4bbc181e [fix](local shuffle) convert remaining ExchangeType refs
to TLocalPartitionType after rebase
add be0a171174d [fix](build) macOS: python UDF boost/process via compat
shim; clang-format + license header
add 25ede4a3b48 [fix](local shuffle) address review comments on PR #63366
add 5ee0bfb0b69 [opt](local shuffle) single topological sweep for pipeline
num_tasks
add e3f6c8260d8 [fix](local shuffle) serial validation throw to warn;
null-safe force_passthrough
add 545dcbbd82b [fix](local shuffle) remove serial-without-LE validation
add 9f106eb80d3 [fix](local shuffle) BE test: post-rename type fixes
(verified by local compile)
add aa406d8bfc4 [fix](local shuffle) address /review: TVFScan leaf output
+ Bug 21 timeout
add 483fe07b818 [fix](local shuffle) trim trailing blank lines at EOF (git
diff --check)
add cbb3a75d8bf [fix](local shuffle) address morrySnow review: thrift enum
comments + optional partition_type
add 985b0543b2a [fix](local shuffle) doc: note the converse (id,name)->id
repartition needs cross-BE shuffle
add 79eb33dd448 [fix](local shuffle) drop unrelated macOS-build changes
add a895f6ef8ae [fix](local shuffle) address morrySnow review (FE/BE)
add 35a1b2c4301 [fix](local shuffle) address morrySnow review nits
add 7648dbad674 [doc](local shuffle) explain why RequireHash.preferType()
is GLOBAL (vs the LOCAL specialization in resolveExchangeType)
add 29fac0c75af [fix](local shuffle) update stale HASH_SHUFFLE comments
after rename to GLOBAL_EXECUTION_HASH_SHUFFLE
add fbcd937e079 [fix](local shuffle) dedup OperatorBase::is_hash_shuffle
into is_shuffled_exchange
add c2c51332d7f [fix](local shuffle) doc: SortNode is before/below
AnalyticEvalNode, not above
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (8f1befd3016)
\
N -- N -- N refs/heads/fe_local_shuffle_rebase3 (c2c51332d7f)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.claude/skills/code-review/SKILL.md | 26 +
.github/CODEOWNERS | 7 +-
.github/scripts/prepare_review_agents.py | 90 ++
.github/workflows/build-extension.yml | 19 +
.github/workflows/clang-format.yml | 5 +
.github/workflows/code-review-runner.yml | 54 +
.github/workflows/title-checker.yml | 4 +
be/src/cloud/cloud_backend_service.cpp | 6 +-
be/src/cloud/cloud_internal_service.cpp | 139 ++-
be/src/cloud/cloud_meta_mgr.cpp | 2 +-
be/src/cloud/cloud_warm_up_manager.cpp | 223 +++-
be/src/cloud/cloud_warm_up_manager.h | 39 +-
be/src/cloud/cloud_warmup_metrics.cpp | 82 ++
be/src/cloud/cloud_warmup_metrics.h | 76 ++
be/src/core/column/column_decimal.h | 12 +
be/src/core/column/column_dictionary.h | 17 +-
be/src/core/column/column_execute_util.h | 48 +-
be/src/core/column/column_vector.h | 12 +
be/src/core/column/predicate_column.h | 467 ---------
be/src/exec/common/variant_util.cpp | 44 +-
be/src/exec/operator/olap_scan_operator.cpp | 8 +
be/src/exec/operator/olap_scan_operator.h | 4 +
be/src/exec/scan/olap_scanner.cpp | 8 +
be/src/exec/scan/vector_search_user_params.cpp | 24 +-
be/src/exec/scan/vector_search_user_params.h | 7 +
be/src/exprs/bloom_filter_func.h | 9 +-
be/src/exprs/bloom_filter_func_adaptor.h | 26 +-
be/src/exprs/bloom_filter_func_impl.h | 26 +-
be/src/exprs/create_predicate_function.h | 11 +-
.../cast/cast_to_date_or_datetime_impl.hpp | 2 +
be/src/exprs/function/like.h | 1 -
be/src/exprs/vectorized_fn_call.cpp | 19 +-
be/src/exprs/vectorized_fn_call.h | 5 +-
be/src/exprs/vexpr.cpp | 4 +-
be/src/exprs/vexpr.h | 5 +-
be/src/exprs/vexpr_context.cpp | 7 +-
be/src/exprs/vexpr_context.h | 5 +-
be/src/exprs/virtual_slot_ref.cpp | 9 +-
be/src/exprs/virtual_slot_ref.h | 5 +-
.../table/iceberg_delete_file_reader_helper.cpp | 6 +-
be/src/format/table/iceberg_reader.cpp | 3 +-
be/src/io/cache/fs_file_cache_storage.cpp | 22 +-
be/src/io/cache/fs_file_cache_storage.h | 28 +-
be/src/runtime/runtime_state.h | 8 +
.../workload_management/query_task_controller.cpp | 13 +
.../workload_management/query_task_controller.h | 2 +
.../runtime/workload_management/task_controller.h | 2 +
.../workload_management/workload_condition.cpp | 11 +
.../workload_management/workload_condition.h | 19 +-
.../workload_management/workload_sched_policy.cpp | 10 +-
be/src/service/http/action/warmup_stats_action.cpp | 121 +++
.../http/action/warmup_stats_action.h} | 21 +-
be/src/service/http_service.cpp | 5 +
be/src/storage/index/ann/ann_search_params.h | 12 +-
be/src/storage/index/ann/ann_topn_runtime.h | 2 +
be/src/storage/olap_common.h | 4 +
be/src/storage/predicate/bloom_filter_predicate.h | 9 +-
be/src/storage/predicate/comparison_predicate.h | 37 +-
be/src/storage/predicate/in_list_predicate.h | 36 +-
be/src/storage/predicate/like_column_predicate.cpp | 32 +-
be/src/storage/predicate/like_column_predicate.h | 7 +-
be/src/storage/predicate/predicate_creator.h | 2 +-
be/src/storage/schema.cpp | 102 +-
be/src/storage/schema.h | 2 +-
be/src/storage/segment/column_reader.cpp | 3 +-
.../storage/segment/row_binlog_segment_writer.cpp | 8 +-
be/src/storage/segment/segment_iterator.cpp | 37 +-
.../segment/variant/variant_column_reader.cpp | 17 +-
be/src/storage/tablet/tablet_meta.cpp | 8 +-
be/src/storage/tablet/tablet_reader.cpp | 3 +-
be/src/util/bvar_windowed_adder.h | 162 +++
.../cloud/cloud_warm_up_manager_filter_test.cpp | 278 +++++
be/test/cloud/cloud_warm_up_manager_test.cpp | 6 +-
be/test/core/column/column_dictionary_test.cpp | 23 +-
be/test/core/column/column_nullable_test.cpp | 5 +-
be/test/core/column/common_column_test.h | 1 +
be/test/core/column/predicate_column_test.cpp | 1082 --------------------
be/test/exec/common/schema_util_test.cpp | 95 +-
be/test/exprs/bloom_filter_func_test.cpp | 42 +-
be/test/exprs/function/cast/cast_to_date_test.cpp | 12 +
.../parquet/byte_array_dict_decoder_test.cpp | 6 +-
.../parquet/fix_length_dict_decoder_test.cpp | 6 +-
.../io/cache/block_file_cache_test_meta_store.cpp | 101 ++
.../query_task_controller_test.cpp | 64 ++
.../workload_condition_test.cpp} | 34 +-
be/test/runtime/workload_sched_policy_test.cpp | 157 +++
.../storage/index/ann/ann_index_edge_case_test.cpp | 57 ++
.../storage/index/ann/ann_range_search_test.cpp | 35 +-
.../predicate/block_column_predicate_test.cpp | 22 +-
be/test/util/bvar_windowed_adder_test.cpp | 140 +++
cloud/test/fdb_injection_test.cpp | 2 +
cloud/test/mem_txn_kv_test.cpp | 8 +-
cloud/test/txn_kv_test.cpp | 7 +-
cloud/test/txn_lazy_commit_test.cpp | 8 +-
docker/runtime/doris-compose/Dockerfile | 3 +-
.../main/java/org/apache/doris/common/Config.java | 21 +-
.../org/apache/doris/catalog/AzureResource.java | 2 +-
.../java/org/apache/doris/catalog/OlapTable.java | 6 +
.../java/org/apache/doris/catalog/S3Resource.java | 2 +-
.../org/apache/doris/catalog/TableProperty.java | 28 +
.../apache/doris/cloud/CacheHotspotManager.java | 492 ++++++++-
.../org/apache/doris/cloud/CloudWarmUpJob.java | 260 ++++-
.../org/apache/doris/cloud/JobWarmUpStats.java | 285 ++++++
.../org/apache/doris/cloud/OnTablesFilter.java | 169 +++
.../doris/cloud/TableWarmUpWindowedStats.java | 206 ++++
.../doris/cloud/catalog/CloudClusterChecker.java | 13 +-
.../cloud/catalog/CloudInstanceStatusChecker.java | 9 +-
.../doris/common/profile/ExecutionProfile.java | 66 +-
.../org/apache/doris/common/profile/Profile.java | 46 +
.../doris/common/profile/ProfileManager.java | 13 +-
.../doris/common/profile/SummaryProfile.java | 10 +
.../doris/common/util/DatasourcePrintableMap.java | 19 +
.../doris/common/util/DynamicPartitionUtil.java | 6 +-
.../apache/doris/common/util/PropertyAnalyzer.java | 8 +-
.../apache/doris/fs/FileSystemPluginManager.java | 7 +-
.../httpv2/controller/QueryProfileController.java | 3 +-
.../org/apache/doris/httpv2/rest/LoadAction.java | 31 +-
.../streaming/PostgresResourceValidator.java | 17 +
.../insert/streaming/StreamingInsertJob.java | 6 +-
.../job/offset/jdbc/JdbcSourceOffsetProvider.java | 7 -
.../java/org/apache/doris/metric/MetricRepo.java | 203 ++++
.../java/org/apache/doris/mtmv/MTMVRelation.java | 13 +-
.../doris/nereids/jobs/executor/Optimizer.java | 8 +-
.../nereids/load/NereidsStreamLoadPlanner.java | 8 +
.../doris/nereids/parser/LogicalPlanBuilder.java | 15 +-
.../parser/LogicalPlanBuilderForEncryption.java | 24 +
.../post/materialize/LazyMaterializeTopN.java | 11 +
.../rules/exploration/join/JoinReorderContext.java | 11 +
.../rewrite/AccessPathExpressionCollector.java | 40 +
.../rules/rewrite/PullUpProjectExprUnderTopN.java | 473 +++------
.../rules/rewrite/PushProjectThroughUnion.java | 33 +-
.../doris/nereids/rules/rewrite/SaltJoin.java | 1 +
.../doris/nereids/rules/rewrite/SkewJoin.java | 8 +-
.../trees/expressions/functions/scalar/Month.java | 2 +-
.../trees/plans/commands/ShowWarmUpCommand.java | 3 +
.../trees/plans/commands/WarmUpClusterCommand.java | 57 +-
.../commands/alter/AlterRepositoryCommand.java | 8 +-
.../org/apache/doris/nereids/types/ArrayType.java | 9 +
.../apache/doris/nereids/types/BooleanType.java | 8 +
.../org/apache/doris/nereids/types/DataType.java | 4 +
.../apache/doris/nereids/types/DateTimeType.java | 9 +
.../apache/doris/nereids/types/DateTimeV2Type.java | 13 +
.../org/apache/doris/nereids/types/DateType.java | 6 +
.../apache/doris/nereids/types/DecimalV2Type.java | 14 +
.../apache/doris/nereids/types/DecimalV3Type.java | 14 +
.../org/apache/doris/nereids/types/MapType.java | 10 +
.../org/apache/doris/nereids/types/StructType.java | 18 +
.../doris/nereids/types/TimeStampTzType.java | 10 +
.../org/apache/doris/nereids/types/TimeV2Type.java | 10 +
.../apache/doris/nereids/types/VariantType.java | 5 +
.../nereids/types/coercion/CharacterType.java | 5 +
.../doris/nereids/types/coercion/IntegralType.java | 15 +
.../apache/doris/nereids/util/ExpressionUtils.java | 14 +
.../java/org/apache/doris/qe/AuditLogHelper.java | 8 +-
.../main/java/org/apache/doris/qe/Coordinator.java | 8 +
.../java/org/apache/doris/qe/SessionVariable.java | 50 +-
.../java/org/apache/doris/qe/StmtExecutor.java | 19 +
.../qe/runtime/PipelineExecutionTaskBuilder.java | 9 +-
.../doris/qe/runtime/ThriftPlansBuilder.java | 8 +
.../WorkloadConditionUsername.java | 12 +-
.../WorkloadSchedPolicyMgr.java | 40 +-
.../org/apache/doris/backup/BackupJobTest.java | 29 +
.../doris/catalog/DynamicPartitionTableTest.java | 100 +-
.../apache/doris/catalog/TablePropertyTest.java | 108 ++
.../cloud/CacheHotspotManagerTableFilterTest.java | 1003 ++++++++++++++++++
.../doris/cloud/CloudWarmUpJobTableFilterTest.java | 461 +++++++++
.../org/apache/doris/cloud/OnTablesFilterTest.java | 141 +++
.../cloud/WarmUpClusterOnTablesParseTest.java | 447 ++++++++
.../org/apache/doris/cloud/WarmUpStatsTest.java | 497 +++++++++
.../catalog/CloudInstanceStatusCheckerTest.java | 260 +++++
.../doris/cloud/load/CopyLoadPendingTaskTest.java | 2 +-
.../apache/doris/common/PropertyAnalyzerTest.java | 22 +
.../doris/common/profile/ProfileManagerTest.java | 148 +++
.../common/util/DatasourcePrintableMapTest.java | 18 +
.../doris/fs/FileSystemPluginManagerTest.java | 57 ++
.../apache/doris/httpv2/rest/LoadActionTest.java | 71 ++
.../streaming/PostgresResourceValidatorTest.java | 45 +
.../java/org/apache/doris/metric/MetricsTest.java | 160 +++
.../apache/doris/nereids/mv/PredicatesTest.java | 234 ++++-
.../parser/RepositoryAuditEncryptionTest.java | 63 ++
.../postprocess/TopnLazyMaterializeTest.java | 70 ++
.../rewrite/PullUpProjectExprUnderTopNTest.java | 283 ++---
.../rules/rewrite/PushProjectThroughUnionTest.java | 57 ++
.../doris/nereids/rules/rewrite/SaltJoinTest.java | 14 +-
.../apache/doris/nereids/types/DataTypeTest.java | 66 ++
.../runtime/PipelineExecutionTaskBuilderTest.java | 96 ++
.../WorkloadSchedPolicyMgrTest.java | 168 ++-
.../org/apache/doris/filesystem/FileSystem.java | 26 +
.../apache/doris/filesystem}/UploadPartResult.java | 7 +-
.../capability/BatchDeleteCapability.java | 23 +-
.../doris/filesystem/capability/Capability.java | 27 +-
.../capability/MultipartUploadCapability.java} | 35 +-
.../capability/PresignedUrlCapability.java | 23 +-
.../filesystem/capability/UploadContent.java} | 32 +-
.../filesystem/capability/UploadSession.java} | 26 +-
.../properties/FileSystemProperties.java | 14 +-
.../S3CompatibleFileSystemProperties.java | 37 +-
...ystemProperties.java => StorageProperties.java} | 40 +-
.../doris/filesystem/FileSystemCapabilityTest.java | 127 +++
.../S3CompatibleFileSystemPropertiesTest.java | 155 +++
.../properties/StoragePropertiesInterfaceTest.java | 64 ++
fe/fe-filesystem/fe-filesystem-azure/pom.xml | 12 +
.../src/main/assembly/plugin-zip.xml | 9 +-
.../doris/filesystem/azure/AzureFileSystem.java | 4 +-
.../azure/AzureFileSystemProperties.java | 415 ++++++++
.../filesystem/azure/AzureFileSystemProvider.java | 54 +-
.../doris/filesystem/azure/AzureObjStorage.java | 137 +--
.../azure/AzureFileSystemPropertiesTest.java | 153 +++
.../azure/AzureFileSystemProviderTest.java | 7 +
.../filesystem/azure/AzureFileSystemTest.java | 4 +-
.../azure/AzureObjStorageExtensionTest.java | 7 +-
.../broker/BrokerFileSystemProvider.java | 3 +-
fe/fe-filesystem/fe-filesystem-cos/pom.xml | 16 +-
.../src/main/assembly/plugin-zip.xml | 12 +-
.../apache/doris/filesystem/cos/CosFileSystem.java | 24 +-
.../filesystem/cos/CosFileSystemProperties.java | 374 +++++++
.../filesystem/cos/CosFileSystemProvider.java | 72 +-
.../apache/doris/filesystem/cos/CosObjStorage.java | 501 +++++++--
.../doris/filesystem/cos/CosFileSystemEnvTest.java | 7 +-
.../cos/CosFileSystemPropertiesTest.java | 176 ++++
.../doris/filesystem/cos/CosObjStorageTest.java | 234 +++--
.../filesystem/hdfs/HdfsFileSystemProvider.java | 3 +-
.../filesystem/local/LocalFileSystemProvider.java | 3 +-
fe/fe-filesystem/fe-filesystem-obs/pom.xml | 16 +-
.../src/main/assembly/plugin-zip.xml | 12 +-
.../apache/doris/filesystem/obs/ObsFileSystem.java | 24 +-
.../filesystem/obs/ObsFileSystemProperties.java | 408 ++++++++
.../filesystem/obs/ObsFileSystemProvider.java | 72 +-
.../apache/doris/filesystem/obs/ObsObjStorage.java | 490 +++++++--
.../doris/filesystem/obs/ObsFileSystemEnvTest.java | 7 +-
.../obs/ObsFileSystemPropertiesTest.java | 179 ++++
.../doris/filesystem/obs/ObsObjStorageTest.java | 245 +++--
fe/fe-filesystem/fe-filesystem-oss/pom.xml | 16 +-
.../src/main/assembly/plugin-zip.xml | 12 +-
.../apache/doris/filesystem/oss/OssFileSystem.java | 24 +-
.../filesystem/oss/OssFileSystemProperties.java | 400 ++++++++
.../filesystem/oss/OssFileSystemProvider.java | 72 +-
.../apache/doris/filesystem/oss/OssObjStorage.java | 536 +++++++---
.../doris/filesystem/oss/OssFileSystemEnvTest.java | 7 +-
.../oss/OssFileSystemPropertiesTest.java | 183 ++++
.../doris/filesystem/oss/OssObjStorageTest.java | 269 +++--
fe/fe-filesystem/fe-filesystem-s3/pom.xml | 11 +-
.../src/main/assembly/plugin-zip.xml | 9 +-
.../apache/doris/filesystem/s3/S3FileSystem.java | 1080 +------------------
.../filesystem/s3/S3FileSystemProperties.java | 17 +-
.../doris/filesystem/s3/S3FileSystemProvider.java | 16 +-
.../apache/doris/filesystem/s3/S3ObjStorage.java | 95 +-
.../filesystem/s3/S3FileSystemPropertiesTest.java | 29 +
.../doris/filesystem/s3/S3FileSystemTest.java | 22 +-
.../doris/filesystem/s3/S3ObjStorageEnvTest.java | 2 +-
.../doris/filesystem/s3/S3ObjStorageMockTest.java | 2 +-
.../doris/filesystem/s3/S3ObjStorageTest.java | 78 +-
fe/fe-filesystem/fe-filesystem-spi/pom.xml | 8 +
.../doris/filesystem/spi/FileSystemProvider.java | 40 +-
.../apache/doris/filesystem/spi/ObjFileSystem.java | 5 +-
.../apache/doris/filesystem/spi/ObjStorage.java | 139 ++-
.../doris/filesystem/spi/ObjectListOptions.java | 91 ++
.../filesystem/spi/ObjectStorageOutputStream.java} | 47 +-
.../doris/filesystem/spi/ObjectStorageUri.java | 116 +++
.../filesystem/spi/S3CompatibleFileSystem.java} | 254 +++--
.../spi/FileSystemProviderTypedPropertiesTest.java | 151 +++
.../doris/filesystem/spi/ObjFileSystemTest.java | 9 +-
.../doris/filesystem/spi/ObjStorageApiTest.java} | 25 +-
.../spi/ObjectStorageOutputStreamTest.java} | 102 +-
.../property/ConnectorPropertiesUtils.java | 39 +
.../property/ConnectorPropertiesUtilsTest.java | 31 +
.../antlr4/org/apache/doris/nereids/DorisLexer.g4 | 1 +
.../antlr4/org/apache/doris/nereids/DorisParser.g4 | 12 +-
fe/hive-udf/pom.xml | 14 +
fe/pom.xml | 28 +-
fs_brokers/cdc_client/build.sh | 2 +-
fs_brokers/cdc_client/pom.xml | 67 ++
.../source/reader/mysql/MySqlSourceReader.java | 3 +
.../cdcclient/itcase/CdcClientReadHarness.java | 329 ++++++
.../cdcclient/itcase/CdcClientWriteHarness.java | 387 +++++++
.../doris/cdcclient/itcase/MockDorisServer.java | 122 +++
.../cdcclient/itcase/MySqlBasicReadITCase.java | 155 +++
.../doris/cdcclient/itcase/MySqlCharsetITCase.java | 169 +++
.../cdcclient/itcase/MySqlDateAdjusterITCase.java | 159 +++
.../cdcclient/itcase/MySqlEmptyTableITCase.java | 130 +++
.../itcase/MySqlSnapshotResumeITCase.java | 165 +++
.../cdcclient/itcase/MySqlStartupLatestITCase.java | 148 +++
.../itcase/MySqlStartupSpecificOffsetITCase.java | 165 +++
.../cdcclient/itcase/MySqlStreamDmlITCase.java | 166 +++
.../cdcclient/itcase/MySqlTimezoneITCase.java | 158 +++
.../cdcclient/itcase/MySqlVersionSmokeITCase.java | 154 +++
.../cdcclient/itcase/MySqlWriteDmlITCase.java | 153 +++
.../cdcclient/itcase/MySqlWriteRecordITCase.java | 145 +++
.../cdcclient/itcase/MySqlWriteResumeITCase.java | 189 ++++
.../cdcclient/itcase/MySqlWriteTypesITCase.java | 174 ++++
.../cdcclient/itcase/PostgresEmptyTableITCase.java | 112 ++
.../PostgresReplicaIdentityDefaultITCase.java | 131 +++
.../itcase/PostgresSnapshotResumeITCase.java | 143 +++
.../itcase/PostgresStartupLatestITCase.java | 131 +++
.../cdcclient/itcase/PostgresStreamReadITCase.java | 137 +++
.../itcase/PostgresVersionSmokeITCase.java | 133 +++
.../cdcclient/itcase/PostgresWriteDmlITCase.java | 135 +++
.../itcase/PostgresWriteResumeITCase.java | 130 +++
.../itcase/PostgresWriteSchemaChangeITCase.java | 129 +++
.../cdcclient/itcase/PostgresWriteTypesITCase.java | 133 +++
.../cdcclient/sink/BatchRecordBufferTest.java | 97 ++
.../doris/cdcclient/sink/HttpPutBuilderTest.java | 133 +++
.../doris/cdcclient/sink/LoadStatisticTest.java | 61 ++
.../deserialize/DebeziumConvertInternalTest.java | 255 +++++
.../DebeziumJsonDeserializeRecordTest.java | 179 ++++
.../source/deserialize/DeserializeResultTest.java | 73 ++
.../PostgresSchemaChangeDeserializeTest.java | 203 ++++
.../source/factory/SourceReaderFactoryTest.java | 51 +
.../doris/cdcclient/utils/ConfigUtilTest.java | 130 ++-
.../docker/server-allow-ancient-date-time/my.cnf} | 31 +-
.../resources/docker/server-with-binlog/my.cnf} | 27 +-
gensrc/proto/internal_service.proto | 2 +
gensrc/script/Makefile | 2 +-
gensrc/thrift/BackendService.thrift | 4 +-
gensrc/thrift/PaloInternalService.thrift | 8 +-
.../ddl/create_nestedtypes_with_schemachange.out | 12 +-
.../sql-functions/doc_bitmap_functions_test.out | 7 -
.../data/nereids_function_p0/scalar_function/M.out | 4 +-
.../column_pruning/lambda_null_pruning.out | 31 +
.../data/nereids_syntax_p0/set_operation.out | 9 +-
.../test_variant_compaction_with_sparse_limit.out | 6 +-
.../test_insert_json_object_to_variant.out | 7 +
.../org/apache/doris/regression/suite/Suite.groovy | 27 +
.../regression/util/WarmupMetricsUtils.groovy | 268 +++++
.../ann_index_p0/ann_const_expr_vector.groovy | 2 +-
.../suites/ann_index_p0/ann_index_basic.groovy | 2 +-
.../ann_index_build_min_segment_rows.groovy | 2 +-
.../ann_index_p0/ann_index_cache_additional.groovy | 2 +-
...ann_index_only_scan_compound_debug_point.groovy | 2 +-
.../ann_index_only_scan_debug_point.groovy | 2 +-
.../ann_index_only_scan_expr_debug_point.groovy | 2 +-
.../ann_index_p0/ann_index_result_cache.groovy | 2 +-
.../ann_index_p0/ann_index_topn_cache.groovy | 2 +-
.../ann_range_search_pushdown_regression.groovy | 2 +-
...ge_search_source_index_status_regression.groovy | 2 +-
.../ann_index_p0/ann_search_params_clamp.groovy | 2 +-
.../ann_topn_small_candidate_fallback.groovy | 231 +++++
.../suites/ann_index_p0/ann_with_fulltext.groovy | 6 +-
.../ann_index_p0/build_ann_index_test.groovy | 2 +-
.../ann_index_p0/cast_string_as_array.groovy | 12 +-
.../ann_index_p0/create_ann_index_test.groovy | 2 +-
.../create_tbl_with_ann_index_test.groovy | 2 +-
.../suites/ann_index_p0/delete_where.groovy | 4 +-
.../ann_index_p0/insert_with_invalid_array.groovy | 2 +-
.../suites/ann_index_p0/ivf_index_test.groovy | 2 +-
.../ann_index_p0/ivf_on_disk_index_test.groovy | 2 +-
.../ivf_pq_full_buffer_train_recall.groovy | 2 +-
.../suites/ann_index_p0/ivf_pq_recall.groovy | 2 +-
.../suites/ann_index_p0/memtbl_on_sink.groovy | 2 +-
.../ann_index_p0/quantizer_min_train_rows.groovy | 2 +-
.../suites/ann_index_p0/small_segment.groovy | 2 +-
.../test_audit_log_hint_session_context.groovy | 71 ++
.../suites/autobucket/test_autobucket.groovy | 4 +-
.../test_backup_restore_inverted_idx.groovy | 6 +-
...est_balance_use_compute_group_properties.groovy | 1 +
.../cloud_p0/balance/test_balance_warm_up.groovy | 1 +
.../test_balance_warm_up_sync_global_config.groovy | 1 +
.../test_balance_warm_up_task_abnormal.groovy | 1 +
.../test_balance_warm_up_use_peer_cache.groovy | 1 +
.../balance/test_peer_read_async_warmup.groovy | 1 +
...on_tables_abnormal_cancel_empty_recovery.groovy | 212 ++++
...ent_on_tables_abnormal_stats_and_failure.groovy | 261 +++++
...warm_up_event_on_tables_canonicalization.groovy | 117 +++
.../test_warm_up_event_on_tables_dynamic.groovy | 217 ++++
...m_up_event_on_tables_error_and_lifecycle.groovy | 387 +++++++
.../test_warm_up_event_on_tables_include.groovy | 167 +++
..._warm_up_event_on_tables_include_exclude.groovy | 153 +++
...t_warm_up_event_on_tables_mow_compaction.groovy | 221 ++++
.../test_warm_up_event_on_tables_multi_dst.groovy | 213 ++++
...st_warm_up_event_on_tables_multi_include.groovy | 142 +++
...t_warm_up_event_on_tables_overlap_and_mv.groovy | 332 ++++++
..._warm_up_event_on_tables_show_and_cancel.groovy | 384 +++++++
.../test_warm_up_event_on_tables_sync_stats.groovy | 300 ++++++
...up_event_on_tables_system_cluster_change.groovy | 168 +++
...nt_on_tables_system_compaction_sync_wait.groovy | 214 ++++
...m_up_event_on_tables_system_e2e_multi_be.groovy | 269 +++++
...rm_up_event_on_tables_system_packed_file.groovy | 155 +++
...vent_on_tables_system_restart_and_resize.groovy | 223 ++++
...m_up_event_on_tables_system_schema_index.groovy | 182 ++++
.../cloud_p0/cache/test_topn_broadcast.groovy | 2 +-
.../correctness/test_pushdown_common_expr.groovy | 8 +-
.../create_nestedtypes_with_schemachange.groovy | 2 +-
.../query/test_nested_type_with_resize.groovy | 8 +-
.../basic-elements/data-types/array-md.groovy | 18 +-
.../sql-functions/doc_bitmap_functions_test.groovy | 11 +-
.../hive/write/test_hive_ctas_to_doris.groovy | 2 +
.../test_all_index_hit_fault_injection.groovy | 2 +-
...dex_compaction_exception_fault_injection.groovy | 12 +-
...test_index_compound_file_fault_injection.groovy | 2 +-
...test_index_empty_segment_fault_injection.groovy | 2 +-
.../test_index_file_cache_fault_injection.groovy | 2 +-
.../test_index_inlist_fault_injection.groovy | 8 +-
.../test_index_io_context.groovy | 10 +-
.../test_index_mow_fault_injection.groovy | 4 +-
...test_match_without_index_fault_injection.groovy | 8 +-
.../test_need_read_data_fault_injection.groovy | 2 +-
...t_variant_count_on_index_fault_injection.groovy | 2 +-
...inverted_index_exception_fault_injection.groovy | 24 +-
...nsert_group_commit_into_max_filter_ratio.groovy | 7 +-
.../analyzer/test_basic_analyzer.groovy | 2 +-
...test_builtin_analyzer_in_custom_analyzer.groovy | 44 +-
.../analyzer/test_char_group_tokenizer.groovy | 2 +-
.../analyzer/test_custom_analyzer.groovy | 74 +-
.../analyzer/test_custom_analyzer1.groovy | 4 +-
.../analyzer/test_custom_analyzer2.groovy | 4 +-
.../analyzer/test_custom_normalizer.groovy | 2 +-
.../analyzer/test_icu_analyzer.groovy | 2 +-
.../analyzer/test_ik_analyzer.groovy | 2 +-
.../test_add_drop_index_with_data_arr.groovy | 3 +-
...test_add_drop_index_with_delete_data_arr.groovy | 5 +-
.../array_contains/test_add_index_for_arr.groovy | 2 +-
.../test_array_contains_estimate.groovy | 2 +-
.../array_contains/test_char_replace_arr.groovy | 5 +-
.../array_contains/test_compound_arr.groovy | 9 +-
.../test_count_on_index_httplogs_arr.groovy | 11 +-
.../test_equal_on_fulltext_arr.groovy | 5 +-
.../array_contains/test_index_bkd_null_arr.groovy | 5 +-
.../array_contains/test_index_change_2_arr.groovy | 3 +-
.../array_contains/test_index_change_4_arr.groovy | 7 +-
.../array_contains/test_index_change_6_arr.groovy | 7 +-
.../test_index_compaction_dup_keys_arr.groovy | 7 +-
.../test_index_compaction_null_arr.groovy | 11 +-
.../test_index_compaction_unique_keys_arr.groovy | 3 +-
...compaction_with_multi_index_segments_arr.groovy | 3 +-
.../array_contains/test_index_delete_arr.groovy | 5 +-
.../test_index_empty_string_arr.groovy | 9 +-
.../test_index_equal_select_arr.groovy | 9 +-
.../test_index_match_phrase_edge_arr.groovy | 3 +-
.../test_index_match_regexp_arr.groovy | 7 +-
.../array_contains/test_index_null_arr.groovy | 5 +-
.../array_contains/test_invalid_param.groovy | 3 +-
.../test_inverted_index_null_arr.groovy | 3 +-
.../array_contains/test_lowercase_arr.groovy | 3 +-
.../test_parser_with_none_stopwords.groovy | 3 +-
.../char_filter/test_char_replace.groovy | 4 +-
.../test_build_index_multi_analyzer_order.groovy | 2 +-
.../test_index_compaction_dup_keys.groovy | 2 +-
.../test_index_compaction_null.groovy | 12 +-
.../test_index_compaction_unique_keys.groovy | 2 +-
...dex_compaction_with_multi_index_segments.groovy | 2 +-
...est_cumulative_compaction_with_format_v2.groovy | 4 +-
.../test_drop_column_with_format_v2.groovy | 4 +-
.../test_drop_index_with_format_v2.groovy | 8 +-
.../test_mor_table_with_format_v2.groovy | 4 +-
.../test_mow_table_with_format_v2.groovy | 2 +-
.../test_rename_column_with_format_v2.groovy | 4 +-
.../storage_format/test_storage_format_v1.groovy | 8 +-
.../storage_format/test_storage_format_v2.groovy | 12 +-
.../test_arrayInvertedIdx_profile.groovy | 5 +-
.../test_array_contains_with_inverted_index.groovy | 43 +-
.../inverted_index_p0/test_array_index.groovy | 2 +-
.../sql/insert_with_query.sql | 2 +-
.../test_array_with_inverted_index_all_type.groovy | 25 +-
.../inverted_index_p0/test_bm25_score.groovy | 20 +-
.../test_bm25_score_range_filter.groovy | 2 +-
.../test_bm25_score_variant.groovy | 2 +-
.../inverted_index_p0/test_chinese_analyzer.groovy | 4 +-
.../test_complex_or_null_semantics.groovy | 2 +-
.../suites/inverted_index_p0/test_compound.groovy | 10 +-
.../inverted_index_p0/test_compound_1.groovy | 12 +-
.../inverted_index_p0/test_compound_inlist.groovy | 20 +-
.../inverted_index_p0/test_count_on_index.groovy | 10 +-
.../inverted_index_p0/test_count_on_index_2.groovy | 8 +-
.../test_cross_field_or_with_null.groovy | 6 +-
.../suites/inverted_index_p0/test_delete.groovy | 2 +-
.../test_empty_string_match.groovy | 2 +-
.../test_equal_on_fulltext.groovy | 4 +-
.../inverted_index_p0/test_expr_in_null.groovy | 4 +-
.../inverted_index_p0/test_float_double.groovy | 6 +-
.../test_ignore_above_in_index.groovy | 4 +-
.../test_index_chinese_column.groovy | 2 +-
.../test_index_complex_match.groovy | 8 +-
.../inverted_index_p0/test_index_delete.groovy | 4 +-
.../test_index_empty_string.groovy | 12 +-
.../test_index_equal_select.groovy | 4 +-
.../test_index_key_match_select.groovy | 2 +-
.../test_index_lowercase_fault_injection.groovy | 2 +-
.../test_index_match_phrase.groovy | 4 +-
.../test_index_match_phrase_edge.groovy | 8 +-
.../test_index_match_phrase_ordered.groovy | 6 +-
.../test_index_match_phrase_prefix.groovy | 6 +-
.../test_index_match_phrase_prefix_1.groovy | 2 +-
.../test_index_match_phrase_slop.groovy | 6 +-
.../test_index_match_regexp.groovy | 8 +-
.../test_index_match_select.groovy | 6 +-
.../test_index_multi_match.groovy | 10 +-
.../test_index_no_need_read_data.groovy | 2 +-
.../inverted_index_p0/test_index_null.groovy | 2 +-
.../test_index_range_between_select.groovy | 16 +-
...test_index_range_bigger_and_equal_select.groovy | 2 +-
.../test_index_range_bigger_select.groovy | 2 +-
.../test_index_range_in_select.groovy | 4 +-
.../test_index_range_not_in_select.groovy | 2 +-
...est_index_range_smaller_and_equal_select.groovy | 8 +-
.../test_index_range_smaller_select.groovy | 2 +-
.../inverted_index_p0/test_index_rqg_bug.groovy | 36 +-
.../inverted_index_p0/test_index_rqg_bug2.groovy | 2 +-
.../inverted_index_p0/test_index_rqg_bug3.groovy | 2 +-
.../inverted_index_p0/test_index_rqg_bug4.groovy | 2 +-
.../test_index_skip_read_data.groovy | 4 +-
.../inverted_index_p0/test_inverted_index.groovy | 4 +-
.../test_inverted_index_collection_stats.groovy | 17 +-
.../test_inverted_index_io_timer.groovy | 29 +-
.../test_inverted_index_keyword.groovy | 4 +-
.../test_inverted_index_mor.groovy | 2 +-
.../test_inverted_index_null.groovy | 2 +-
.../test_inverted_index_null_ram_dir.groovy | 4 +-
.../test_inverted_index_v2_file_size.groovy | 2 +-
.../inverted_index_p0/test_inverted_is_null.groovy | 2 +-
.../test_ip_cidr_search_with_inverted_index.groovy | 6 +-
.../inverted_index_p0/test_is_null_expr.groovy | 8 +-
.../suites/inverted_index_p0/test_lowercase.groovy | 2 +-
.../test_match_or_null_semantics.groovy | 6 +-
.../test_match_query_without_index.groovy | 2 +-
.../inverted_index_p0/test_need_read_data.groovy | 6 +-
.../inverted_index_p0/test_no_index_match.groovy | 6 +-
.../inverted_index_p0/test_null_index.groovy | 4 +-
.../inverted_index_p0/test_omit_norms.groovy | 12 +-
.../inverted_index_p0/test_or_not_match.groovy | 4 +-
.../inverted_index_p0/test_pinyin_phrase.groovy | 8 +-
.../test_pk_no_need_read_data.groovy | 2 +-
.../test_search_or_null_semantics.groovy | 26 +-
.../test_single_column_multi_index.groovy | 19 +-
.../test_single_column_multi_index1.groovy | 11 +-
.../suites/inverted_index_p0/test_stopwords.groovy | 2 +-
.../inverted_index_p0/topn_clear_block.groovy | 12 +-
.../test_mow_with_null_sequence.groovy | 6 +-
.../unique_with_mow/test_pk_uk_case.groovy | 28 +-
.../test_primary_key_simple_case.groovy | 4 +-
.../test_unique_mow_sequence.groovy | 2 +-
.../test_dup_table_inverted_index.groovy | 6 +-
...st_streaming_postgres_job_special_offset.groovy | 31 +
.../sql/basic_q01.sql | 26 +-
.../sql/basic_q02.sql | 26 +-
.../sql/fulltext_q01.sql | 38 +-
.../sql/fulltext_q02.sql | 38 +-
.../sql/large_records_q01.sql | 20 +-
.../sql/large_records_q02.sql | 20 +-
.../sql/large_records_q03.sql | 18 +-
.../sql/large_records_q04.sql | 18 +-
.../sql/large_records_q05.sql | 12 +-
.../sql/large_records_q06.sql | 12 +-
.../sql/large_records_q07.sql | 12 +-
.../sql/large_records_q08.sql | 12 +-
.../nereids_function_p0/scalar_function/M.groovy | 1 +
.../column_pruning/lambda_null_pruning.groovy | 150 +++
.../nereids_rules_p0/mv/variant/variant_mv.groovy | 2 +-
.../nereids_rules_p0/pkfk/eliminate_inner.groovy | 1 +
...rough_set_operation_with_unique_function.groovy | 1 +
.../prune_bucket_with_bucket_shuffle_join.groovy | 17 +
.../suites/nereids_syntax_p0/set_operation.groovy | 21 +
...gg_after_nested_loop_join_local_exchange.groovy | 3 +-
.../runtime_filter/rf_partition_pruning.groovy | 14 +-
.../rf_partition_pruning_type_matrix.groovy | 12 +-
.../test_validate_restore_inverted_idx.groovy | 6 +-
..._materialized_view_common_expr_push_down.groovy | 2 +-
.../search/test_disable_root_variant_match.groovy | 6 +-
.../search/test_search_boundary_cases.groovy | 40 +-
.../suites/search/test_search_cache.groovy | 20 +-
.../test_search_default_field_operator.groovy | 40 +-
.../suites/search/test_search_dsl_operators.groovy | 30 +-
.../suites/search/test_search_dsl_syntax.groovy | 136 +--
.../suites/search/test_search_escape.groovy | 30 +-
.../suites/search/test_search_exact_basic.groovy | 10 +-
.../search/test_search_exact_lowercase.groovy | 22 +-
.../suites/search/test_search_exact_match.groovy | 46 +-
.../search/test_search_exact_multi_index.groovy | 18 +-
.../search/test_search_field_group_query.groovy | 4 +-
.../suites/search/test_search_function.groovy | 100 +-
.../search/test_search_inverted_index.groovy | 54 +-
.../test_search_inverted_index_profile.groovy | 17 +-
.../test_search_inverted_is_null_pushdown.groovy | 2 +-
.../suites/search/test_search_lucene_mode.groovy | 38 +-
.../suites/search/test_search_mow_support.groovy | 4 +-
.../test_search_multi_analyzer_lucene.groovy | 34 +-
.../suites/search/test_search_multi_field.groovy | 62 +-
.../search/test_search_not_null_bitmap.groovy | 30 +-
.../search/test_search_null_regression.groovy | 44 +-
.../search/test_search_null_semantics.groovy | 86 +-
.../search/test_search_regexp_lowercase.groovy | 28 +-
.../suites/search/test_search_slash_in_term.groovy | 16 +-
.../search/test_search_usage_restrictions.groovy | 36 +-
.../test_search_variant_dual_index_reader.groovy | 14 +-
.../test_search_variant_subcolumn_analyzer.groovy | 18 +-
.../search/test_search_variant_wildcard.groovy | 26 +-
..._search_variant_wildcard_custom_analyzer.groovy | 28 +-
.../search/test_search_vs_match_consistency.groovy | 100 +-
.../statistics/test_analyze_long_string.groovy | 12 +-
.../suites/statistics/test_analyze_mv.groovy | 3 +-
.../suites/statistics/test_hot_value.groovy | 109 +-
.../flexible/test_f_no_simdjson.groovy | 166 ---
.../suites/variant_doc_mode_p2/load.groovy | 10 +-
.../variant_github_events_new_p2/load.groovy | 28 +-
.../load.groovy | 8 +-
.../suites/variant_p0/doc_mode/load.groovy | 40 +-
.../test_prdefine_insert_into_select.groovy | 12 +-
.../predefine/test_predefine_pattern.groovy | 10 +-
.../predefine/test_predefine_schema_change.groovy | 6 +-
.../predefine/test_predefine_type_index.groovy | 18 +-
.../test_array_contains_with_inverted_index.groovy | 43 +-
.../test_variant_external_meta_integration.groovy | 86 +-
regression-test/suites/variant_p0/load.groovy | 40 +-
.../predefine/predefined_typed_to_sparse.groovy | 22 +-
.../predefined_typed_to_sparse_1shard.groovy | 14 +-
.../predefine/test_custom_analyzer.groovy | 10 +-
.../predefine/test_multi_index_file.groovy | 4 +-
.../predefine/test_multi_index_nonCurrent.groovy | 22 +-
.../test_prdefine_insert_into_select.groovy | 12 +-
.../predefine/test_predefine_pattern.groovy | 12 +-
.../predefine/test_predefine_schema_change.groovy | 4 +-
.../predefine/test_predefine_type_index.groovy | 18 +-
.../test_predefine_type_multi_index.groovy | 8 +-
.../test_schema_template_auto_cast.groovy | 2 +-
.../test_types_with_indexes_profile.groovy | 14 +-
...est_variant_compaction_with_sparse_limit.groovy | 24 +-
.../predefine/test_variant_is_null_expr.groovy | 8 +-
.../suites/variant_p0/query_subcolumns.groovy | 6 +-
.../test_add_multi_var_mulit_indexes.groovy | 11 +-
.../test_insert_json_object_to_variant.groovy | 98 ++
.../suites/variant_p0/test_sub_path_pruning.groovy | 4 +-
.../variant_p0/test_variant_is_null_expr.groovy | 10 +-
.../test_variant_search_subcolumn.groovy | 20 +-
.../suites/variant_p0/with_index/load.groovy | 6 +-
.../test_array_contains_with_inverted_index.groovy | 43 +-
.../test_array_inverted_index_profile.groovy | 9 +-
.../suites/variant_p0/with_index/var_index.groovy | 26 +-
.../suites/variant_p1/doc_snapshot/load.groovy | 12 +-
.../suites/variant_p1/predefine/load.groovy | 12 +-
.../test_workload_sched_policy.groovy | 42 +-
run-fe-ut.sh | 2 +-
thirdparty/build-thirdparty.sh | 2 +-
631 files changed, 28059 insertions(+), 6945 deletions(-)
create mode 100644 .github/scripts/prepare_review_agents.py
create mode 100644 be/src/cloud/cloud_warmup_metrics.cpp
create mode 100644 be/src/cloud/cloud_warmup_metrics.h
delete mode 100644 be/src/core/column/predicate_column.h
create mode 100644 be/src/service/http/action/warmup_stats_action.cpp
copy be/src/{exec/scan/vector_search_user_params.h =>
service/http/action/warmup_stats_action.h} (67%)
create mode 100644 be/src/util/bvar_windowed_adder.h
create mode 100644 be/test/cloud/cloud_warm_up_manager_filter_test.cpp
delete mode 100644 be/test/core/column/predicate_column_test.cpp
create mode 100644
be/test/runtime/workload_management/query_task_controller_test.cpp
copy be/{src/exec/scan/vector_search_user_params.h =>
test/runtime/workload_management/workload_condition_test.cpp} (51%)
create mode 100644 be/test/util/bvar_windowed_adder_test.cpp
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/cloud/JobWarmUpStats.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/cloud/OnTablesFilter.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/cloud/TableWarmUpWindowedStats.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/catalog/TablePropertyTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/cloud/CacheHotspotManagerTableFilterTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/cloud/CloudWarmUpJobTableFilterTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/cloud/OnTablesFilterTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/cloud/WarmUpClusterOnTablesParseTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/cloud/WarmUpStatsTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/cloud/catalog/CloudInstanceStatusCheckerTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/fs/FileSystemPluginManagerTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/job/extensions/insert/streaming/PostgresResourceValidatorTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/nereids/parser/RepositoryAuditEncryptionTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/qe/runtime/PipelineExecutionTaskBuilderTest.java
copy
fe/fe-filesystem/{fe-filesystem-spi/src/main/java/org/apache/doris/filesystem/spi
=>
fe-filesystem-api/src/main/java/org/apache/doris/filesystem}/UploadPartResult.java
(81%)
copy be/src/exec/scan/vector_search_user_params.h =>
fe/fe-filesystem/fe-filesystem-api/src/main/java/org/apache/doris/filesystem/capability/BatchDeleteCapability.java
(68%)
copy be/src/exec/scan/vector_search_user_params.h =>
fe/fe-filesystem/fe-filesystem-api/src/main/java/org/apache/doris/filesystem/capability/Capability.java
(64%)
copy fe/{fe-core/src/main/java/org/apache/doris/nereids/types/BooleanType.java
=>
fe-filesystem/fe-filesystem-api/src/main/java/org/apache/doris/filesystem/capability/MultipartUploadCapability.java}
(53%)
copy be/src/exec/scan/vector_search_user_params.h =>
fe/fe-filesystem/fe-filesystem-api/src/main/java/org/apache/doris/filesystem/capability/PresignedUrlCapability.java
(66%)
copy fe/{fe-core/src/main/java/org/apache/doris/nereids/types/BooleanType.java
=>
fe-filesystem/fe-filesystem-api/src/main/java/org/apache/doris/filesystem/capability/UploadContent.java}
(56%)
copy
fe/fe-filesystem/{fe-filesystem-spi/src/main/java/org/apache/doris/filesystem/spi/UploadPartResult.java
=>
fe-filesystem-api/src/main/java/org/apache/doris/filesystem/capability/UploadSession.java}
(60%)
copy
fe/fe-filesystem/fe-filesystem-api/src/main/java/org/apache/doris/filesystem/properties/{FileSystemProperties.java
=> StorageProperties.java} (56%)
create mode 100644
fe/fe-filesystem/fe-filesystem-api/src/test/java/org/apache/doris/filesystem/FileSystemCapabilityTest.java
create mode 100644
fe/fe-filesystem/fe-filesystem-api/src/test/java/org/apache/doris/filesystem/properties/S3CompatibleFileSystemPropertiesTest.java
create mode 100644
fe/fe-filesystem/fe-filesystem-api/src/test/java/org/apache/doris/filesystem/properties/StoragePropertiesInterfaceTest.java
create mode 100644
fe/fe-filesystem/fe-filesystem-azure/src/main/java/org/apache/doris/filesystem/azure/AzureFileSystemProperties.java
create mode 100644
fe/fe-filesystem/fe-filesystem-azure/src/test/java/org/apache/doris/filesystem/azure/AzureFileSystemPropertiesTest.java
copy be/src/exec/scan/vector_search_user_params.h =>
fe/fe-filesystem/fe-filesystem-cos/src/main/java/org/apache/doris/filesystem/cos/CosFileSystem.java
(67%)
create mode 100644
fe/fe-filesystem/fe-filesystem-cos/src/main/java/org/apache/doris/filesystem/cos/CosFileSystemProperties.java
create mode 100644
fe/fe-filesystem/fe-filesystem-cos/src/test/java/org/apache/doris/filesystem/cos/CosFileSystemPropertiesTest.java
copy be/src/exec/scan/vector_search_user_params.h =>
fe/fe-filesystem/fe-filesystem-obs/src/main/java/org/apache/doris/filesystem/obs/ObsFileSystem.java
(67%)
create mode 100644
fe/fe-filesystem/fe-filesystem-obs/src/main/java/org/apache/doris/filesystem/obs/ObsFileSystemProperties.java
create mode 100644
fe/fe-filesystem/fe-filesystem-obs/src/test/java/org/apache/doris/filesystem/obs/ObsFileSystemPropertiesTest.java
copy be/src/exec/scan/vector_search_user_params.h =>
fe/fe-filesystem/fe-filesystem-oss/src/main/java/org/apache/doris/filesystem/oss/OssFileSystem.java
(67%)
create mode 100644
fe/fe-filesystem/fe-filesystem-oss/src/main/java/org/apache/doris/filesystem/oss/OssFileSystemProperties.java
create mode 100644
fe/fe-filesystem/fe-filesystem-oss/src/test/java/org/apache/doris/filesystem/oss/OssFileSystemPropertiesTest.java
create mode 100644
fe/fe-filesystem/fe-filesystem-spi/src/main/java/org/apache/doris/filesystem/spi/ObjectListOptions.java
rename
fe/fe-filesystem/{fe-filesystem-s3/src/main/java/org/apache/doris/filesystem/s3/S3OutputStream.java
=>
fe-filesystem-spi/src/main/java/org/apache/doris/filesystem/spi/ObjectStorageOutputStream.java}
(62%)
create mode 100644
fe/fe-filesystem/fe-filesystem-spi/src/main/java/org/apache/doris/filesystem/spi/ObjectStorageUri.java
copy
fe/fe-filesystem/{fe-filesystem-s3/src/main/java/org/apache/doris/filesystem/s3/S3FileSystem.java
=>
fe-filesystem-spi/src/main/java/org/apache/doris/filesystem/spi/S3CompatibleFileSystem.java}
(83%)
create mode 100644
fe/fe-filesystem/fe-filesystem-spi/src/test/java/org/apache/doris/filesystem/spi/FileSystemProviderTypedPropertiesTest.java
rename
fe/fe-filesystem/fe-filesystem-spi/src/{main/java/org/apache/doris/filesystem/spi/UploadPartResult.java
=> test/java/org/apache/doris/filesystem/spi/ObjStorageApiTest.java} (67%)
rename
fe/fe-filesystem/{fe-filesystem-s3/src/test/java/org/apache/doris/filesystem/s3/S3OutputStreamTest.java
=>
fe-filesystem-spi/src/test/java/org/apache/doris/filesystem/spi/ObjectStorageOutputStreamTest.java}
(75%)
create mode 100644
fs_brokers/cdc_client/src/test/java/org/apache/doris/cdcclient/itcase/CdcClientReadHarness.java
create mode 100644
fs_brokers/cdc_client/src/test/java/org/apache/doris/cdcclient/itcase/CdcClientWriteHarness.java
create mode 100644
fs_brokers/cdc_client/src/test/java/org/apache/doris/cdcclient/itcase/MockDorisServer.java
create mode 100644
fs_brokers/cdc_client/src/test/java/org/apache/doris/cdcclient/itcase/MySqlBasicReadITCase.java
create mode 100644
fs_brokers/cdc_client/src/test/java/org/apache/doris/cdcclient/itcase/MySqlCharsetITCase.java
create mode 100644
fs_brokers/cdc_client/src/test/java/org/apache/doris/cdcclient/itcase/MySqlDateAdjusterITCase.java
create mode 100644
fs_brokers/cdc_client/src/test/java/org/apache/doris/cdcclient/itcase/MySqlEmptyTableITCase.java
create mode 100644
fs_brokers/cdc_client/src/test/java/org/apache/doris/cdcclient/itcase/MySqlSnapshotResumeITCase.java
create mode 100644
fs_brokers/cdc_client/src/test/java/org/apache/doris/cdcclient/itcase/MySqlStartupLatestITCase.java
create mode 100644
fs_brokers/cdc_client/src/test/java/org/apache/doris/cdcclient/itcase/MySqlStartupSpecificOffsetITCase.java
create mode 100644
fs_brokers/cdc_client/src/test/java/org/apache/doris/cdcclient/itcase/MySqlStreamDmlITCase.java
create mode 100644
fs_brokers/cdc_client/src/test/java/org/apache/doris/cdcclient/itcase/MySqlTimezoneITCase.java
create mode 100644
fs_brokers/cdc_client/src/test/java/org/apache/doris/cdcclient/itcase/MySqlVersionSmokeITCase.java
create mode 100644
fs_brokers/cdc_client/src/test/java/org/apache/doris/cdcclient/itcase/MySqlWriteDmlITCase.java
create mode 100644
fs_brokers/cdc_client/src/test/java/org/apache/doris/cdcclient/itcase/MySqlWriteRecordITCase.java
create mode 100644
fs_brokers/cdc_client/src/test/java/org/apache/doris/cdcclient/itcase/MySqlWriteResumeITCase.java
create mode 100644
fs_brokers/cdc_client/src/test/java/org/apache/doris/cdcclient/itcase/MySqlWriteTypesITCase.java
create mode 100644
fs_brokers/cdc_client/src/test/java/org/apache/doris/cdcclient/itcase/PostgresEmptyTableITCase.java
create mode 100644
fs_brokers/cdc_client/src/test/java/org/apache/doris/cdcclient/itcase/PostgresReplicaIdentityDefaultITCase.java
create mode 100644
fs_brokers/cdc_client/src/test/java/org/apache/doris/cdcclient/itcase/PostgresSnapshotResumeITCase.java
create mode 100644
fs_brokers/cdc_client/src/test/java/org/apache/doris/cdcclient/itcase/PostgresStartupLatestITCase.java
create mode 100644
fs_brokers/cdc_client/src/test/java/org/apache/doris/cdcclient/itcase/PostgresStreamReadITCase.java
create mode 100644
fs_brokers/cdc_client/src/test/java/org/apache/doris/cdcclient/itcase/PostgresVersionSmokeITCase.java
create mode 100644
fs_brokers/cdc_client/src/test/java/org/apache/doris/cdcclient/itcase/PostgresWriteDmlITCase.java
create mode 100644
fs_brokers/cdc_client/src/test/java/org/apache/doris/cdcclient/itcase/PostgresWriteResumeITCase.java
create mode 100644
fs_brokers/cdc_client/src/test/java/org/apache/doris/cdcclient/itcase/PostgresWriteSchemaChangeITCase.java
create mode 100644
fs_brokers/cdc_client/src/test/java/org/apache/doris/cdcclient/itcase/PostgresWriteTypesITCase.java
create mode 100644
fs_brokers/cdc_client/src/test/java/org/apache/doris/cdcclient/sink/BatchRecordBufferTest.java
create mode 100644
fs_brokers/cdc_client/src/test/java/org/apache/doris/cdcclient/sink/HttpPutBuilderTest.java
create mode 100644
fs_brokers/cdc_client/src/test/java/org/apache/doris/cdcclient/sink/LoadStatisticTest.java
create mode 100644
fs_brokers/cdc_client/src/test/java/org/apache/doris/cdcclient/source/deserialize/DebeziumConvertInternalTest.java
create mode 100644
fs_brokers/cdc_client/src/test/java/org/apache/doris/cdcclient/source/deserialize/DebeziumJsonDeserializeRecordTest.java
create mode 100644
fs_brokers/cdc_client/src/test/java/org/apache/doris/cdcclient/source/deserialize/DeserializeResultTest.java
create mode 100644
fs_brokers/cdc_client/src/test/java/org/apache/doris/cdcclient/source/deserialize/PostgresSchemaChangeDeserializeTest.java
create mode 100644
fs_brokers/cdc_client/src/test/java/org/apache/doris/cdcclient/source/factory/SourceReaderFactoryTest.java
copy fs_brokers/cdc_client/{build.sh =>
src/test/resources/docker/server-allow-ancient-date-time/my.cnf} (62%)
mode change 100755 => 100644
copy fs_brokers/cdc_client/{build.sh =>
src/test/resources/docker/server-with-binlog/my.cnf} (62%)
mode change 100755 => 100644
create mode 100644
regression-test/data/nereids_rules_p0/column_pruning/lambda_null_pruning.out
create mode 100644
regression-test/data/variant_p0/test_insert_json_object_to_variant.out
create mode 100644
regression-test/framework/src/main/groovy/org/apache/doris/regression/util/WarmupMetricsUtils.groovy
create mode 100644
regression-test/suites/ann_index_p0/ann_topn_small_candidate_fallback.groovy
create mode 100644
regression-test/suites/audit/test_audit_log_hint_session_context.groovy
create mode 100644
regression-test/suites/cloud_p0/cache/multi_cluster/warm_up/on_tables/test_warm_up_event_on_tables_abnormal_cancel_empty_recovery.groovy
create mode 100644
regression-test/suites/cloud_p0/cache/multi_cluster/warm_up/on_tables/test_warm_up_event_on_tables_abnormal_stats_and_failure.groovy
create mode 100644
regression-test/suites/cloud_p0/cache/multi_cluster/warm_up/on_tables/test_warm_up_event_on_tables_canonicalization.groovy
create mode 100644
regression-test/suites/cloud_p0/cache/multi_cluster/warm_up/on_tables/test_warm_up_event_on_tables_dynamic.groovy
create mode 100644
regression-test/suites/cloud_p0/cache/multi_cluster/warm_up/on_tables/test_warm_up_event_on_tables_error_and_lifecycle.groovy
create mode 100644
regression-test/suites/cloud_p0/cache/multi_cluster/warm_up/on_tables/test_warm_up_event_on_tables_include.groovy
create mode 100644
regression-test/suites/cloud_p0/cache/multi_cluster/warm_up/on_tables/test_warm_up_event_on_tables_include_exclude.groovy
create mode 100644
regression-test/suites/cloud_p0/cache/multi_cluster/warm_up/on_tables/test_warm_up_event_on_tables_mow_compaction.groovy
create mode 100644
regression-test/suites/cloud_p0/cache/multi_cluster/warm_up/on_tables/test_warm_up_event_on_tables_multi_dst.groovy
create mode 100644
regression-test/suites/cloud_p0/cache/multi_cluster/warm_up/on_tables/test_warm_up_event_on_tables_multi_include.groovy
create mode 100644
regression-test/suites/cloud_p0/cache/multi_cluster/warm_up/on_tables/test_warm_up_event_on_tables_overlap_and_mv.groovy
create mode 100644
regression-test/suites/cloud_p0/cache/multi_cluster/warm_up/on_tables/test_warm_up_event_on_tables_show_and_cancel.groovy
create mode 100644
regression-test/suites/cloud_p0/cache/multi_cluster/warm_up/on_tables/test_warm_up_event_on_tables_sync_stats.groovy
create mode 100644
regression-test/suites/cloud_p0/cache/multi_cluster/warm_up/on_tables/test_warm_up_event_on_tables_system_cluster_change.groovy
create mode 100644
regression-test/suites/cloud_p0/cache/multi_cluster/warm_up/on_tables/test_warm_up_event_on_tables_system_compaction_sync_wait.groovy
create mode 100644
regression-test/suites/cloud_p0/cache/multi_cluster/warm_up/on_tables/test_warm_up_event_on_tables_system_e2e_multi_be.groovy
create mode 100644
regression-test/suites/cloud_p0/cache/multi_cluster/warm_up/on_tables/test_warm_up_event_on_tables_system_packed_file.groovy
create mode 100644
regression-test/suites/cloud_p0/cache/multi_cluster/warm_up/on_tables/test_warm_up_event_on_tables_system_restart_and_resize.groovy
create mode 100644
regression-test/suites/cloud_p0/cache/multi_cluster/warm_up/on_tables/test_warm_up_event_on_tables_system_schema_index.groovy
create mode 100644
regression-test/suites/nereids_rules_p0/column_pruning/lambda_null_pruning.groovy
delete mode 100644
regression-test/suites/unique_with_mow_p0/flexible/test_f_no_simdjson.groovy
create mode 100644
regression-test/suites/variant_p0/test_insert_json_object_to_variant.groovy
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
