This is an automated email from the ASF dual-hosted git repository.
dataroaring pushed a change to branch branch-3.0
in repository https://gitbox.apache.org/repos/asf/doris.git
from 92982d226c7 bump to 3.0.0-rc07
new e497a4dcbf5 [Fix](explain) fix tablet showing problem (#35830)
new 726225d3ecd [improve](cloud) make tablet sync_rowsets async to reduce
calculating delete bitmap time (#35838)
new 6f1d4e1bc9d [improve](cloud) The cloudWarmUpJob save/load is supported
for image (#35979)
new 6a1cde9c8b4 [enhancement](regression-test) modify a value type tests
(#35894)
new 9bd4d6fb01d [enhancement](regression-test) modify a value type tests
(#36011)
new 30110cca04e [fix](gc) fix move existing tablet to trash (#36027)
new 6dacaffac7f [fix](editlog) Make OP_INIT_CATALOG compatible with the
old cloud code (#35989)
new acc8f323591 [opt](parquet-writer) Specify the row group size when
writing data to Parquet files. (#35081)
new 6ca8083458c [doc] Update README (#35993)
new 85625af0e5d [minor](log) Add some debug logs (#36041)
new 38e90d0a6af [fix](statistics)Fix analyze twice after drop stats bug.
(#36046)
new 83cbd442641 [bugfix](iceberg)Fix the datafile path error issue (#35957)
new ba76f39911e [improve](routine-load) optimize routine load job auto
resume policy (#35266)
new 7909773822d [fix](script) Set DYLD_LIBRARY_PATH for macOS in be
startup script (#36067)
new 1ffbd887940 [log](cloud) Print txn id when prepare/commit rowset
(#36068)
new ed7b033f76d [fix](cloud) cloud mode don't start collect partition info
thread (#36012)
new 3930b44e378 [fix](clone) fix check replica failed due to replica had
drop (#35994)
new 4a1d9d62327 [fix](cloud) port CloudLoadManager#removeCopyJobs() to
master (#35915)
new 7894db48834 [fix] (inverted index v2) fix inverted index format is
lost during a schema change (#36059)
new 54094b6c985 [build](ci) modify the required checks for branch-2.1
(#36074)
new 0ecb24ebac2 [fix](ubsan) reinterpret_cast fix length types to int8 is
not safe (#35912)
new e9b19f3114f [fix](thirdparty) patch libevent to enable keepalive on
tcp (#36026)
new 9f999e2cecc (cloud-merge) Fix stale schema version in delete predicate
(#36083)
new 887900e7934 [fix](shuffle) fix tablets num calculation in shuffle
condition (#36050)
new 6670a30a3df [fix](cloud) Fix compatibility issues with old cloud auth
code (#36022)
new 8fe89b2912b [Chore](exec-version) fix some missing be_exec_version
request (#34653)
new 9f22d4f6cc8 [fix](compaction) fix http error when downloading varaint
inverted index file (#35668)
new f7ac5d6ef53 [enhancement](regression-test) Add clickhouse case for SQL
Dialect Convertor (#35231)
new fdf85d56d78 [Bug](function) fix week function get wrong result (#36000)
new 72e1092ff5d [minor](dependencies)binding a third-party Hadoop
component to version 1.2.0 (#35093)
new de7343ab9b5 [fix](nereids)decimal and datetime literal comparison
should compare datatype too (#36055)
new 688cd957659 [Fix]Fix show role stmt missing grouo info (#36032)
new 8239ef5ff3e [fix](eq_for_null) fix incorrect logic in function
eq_for_null (#36004)
new 154fcb86166 [feature](cloud) support get load state by HTTP API in
cloud mode (#35904)
new 5f5db0430e0 [enhance](cloud) Implement FullRangeGetIterator to
simplify iterating over a kv range (#33388)
new d0ab33dd389 [feature](Cloud) Add azure's support to BE (#35670)
new ae2b25439e3 [fix](scan) catch exceptions thrown in scanner (#36101)
new 3919c0241a9 [opt](inverted index) performance optimization for
need_read_data in compound (#35346)
new 084b53b36b5 [Refactor](Variant) refactor flush logic to support
partial update (#34925)
new a333f9d9793 [fix](routine-load) fix routine load schedule test failed
(#36113)
new 77666d1f067 [minor](dependencies)upgrade aircompressor to 0.27 (#36106)
new 3fb8204a884 [fix](function) fix date_format and from_unixtime core
when meet long format string (#35883)
new 440c763ea04 [build](ci) modify the required checks for branch-2.1
(#36074) (#36129)
new 71518f5749a [fix](Nereids) fix four phase aggregation compute wrong
result (#36128)
new af945d2ed56 [fix](default value) change throw AnalysisException order
(#35760)
new 116acbb0aa2 [build](ci) modify the required checks for branch-2.1
(#36074) (#36129) (#36151)
new 9396aa5dbb9 [Bug](runtime-filter) release dependency when rf rpc
failed or meet error status (#36126)
new 1a1a9bc5890 [fix](Nereids) Add Primary Key by Primary Relation Instead
of Foreign Relation (#36008)
new 1d6ee361dd9 [fix](profile) Fix duplicated profile (#36120)
new 13760aa8c30 [fix](ci) external pipeline use
regression-test/pipeline/external/conf/be.conf (#36132)
new 2c54f579e57 [opt](mtmv) Set query rewrite by materialized view default
enable (#35897)
new fb101e3b786 [fix](autobucket) fix autobucket config masterOnly=true
(#36116)
new 5281031b58d [feat](mtmv) Support grouping_sets rewrite when query
rewrite by materialized view (#36056)
new 153fcb6ba8f [opt](nereids) using mv's derived stats (#35721)
new 803053315fc [Fix](inverted index) fix memory leak in inverted index
writer for array values (#36144)
new 27496b52d4b [bugfix](iceberg)Fixed missing type of iceberg table for
timetravel (#36047)
new 4b8ca342dee [fix](memory) Refactor memory allocated failure processing
(#36090)
new 1a7a1ef2787 [ub](pod array) fix applying non-zero offset 16 to null
pointer (#36086)
new 5ee268b938e [feature](statistics)Fallback to sample analyze when one
partition contains too many rows. (#36115)
new 261ce455f79 [Bug](executor) fix get empty result when
max_query_retry_time=0 and reduce unused wa… (#36176)
new 13be5e7d038 [fix](mtmv) mtmv add case and fix bug (#36019)
new d4974b9909f [feature](pipeline) local merge sort ready when all queue
has data (#35992)
new 89f5ccf434a [fix](invert index) ensure that the pred result sign of
the inlist is in order(#36085)
new 9979d841b73 [fix](regression test) Single compaction is not supported
in cloud mode. (#36118)
new fa888ca0e4c [enhance](Vault) Refresh default storage vault info when
doing instance check in FE (#36104)
new 22ab5c4d3f8 [fix](alter) Remove rebuildAlterJob (#36196)
new 753b41e1dad [enhancement](regression-test) drop a value type tests
(#36112)
new 3edb55242fa [enhancement](regression-test) drop a key type tests
(#36178)
new 933d3898823 [fix](load) fix NPE in LoadManager#jobRemovedTrigger()
(#36173)
new abdbc6870ba [Fix](outfile) Add a configuration for exporting data in
Parquet format using `select into outfile` (#36142)
new f38a0797b20 [minor] better column name error description (#36154)
new 12fe7d2c7cc [fix](planner)remove constant expr in window function's
partition and order exprs (#36184)
new 62672e9f34b [fix](eq_for_null) Fix eq_for_null when both column is NOT
only_null (#36164)
new 87e2f940673 [improve](cloud txn insert) Add more ut; Remove duplicated
table_id to reduce fdb value length (#36135)
new 71d36fa8efc [fix](nereids) fix topn-filter expr bug (#35832)
new 166ef6d7946 [Feature](Point Query) fully support in nereids (#35823)
new 48b2dd92b9f [fix](spill) fix bug: spill disk data usage is not
decreased in abnormal situation (#36194)
new 58ec058ce86 [Fix](nereids) fix merge aggregate rule, rules should not
have mutable members (#36145)
new 7626392b3cd [fix](Azure) Use correct http range assignation to Azure
Block Blob Client (#36211)
new 4116da11dad [fix](clone) Fix some typo error (#36204)
new a1b0d9b9eee [fix](cloud) Fix missing table ids in TxnRunningPB (#36171)
new d8aa7b18e23 [fix](cloud) Fix CloudTabletRebalancer npe (#36162)
new 50bcfaa4911 [bug](metrics) fix be metric doris_be_process_thread_num
is zero (#35511)
new ee4c35a3e0e [Fix](Variant) fix checksum task read with unmatched
column type (#36201)
new 749f241547c [Chore](function) make check_input_type adapt with array
distance new signature (#36206)
new 58fd63e7a98 [test](mtmv)add increment create mtmv case (#35707)
new 86b1c31ba7d [fix] Process not destroy in DiskUtils df function (#34387)
new 0db641be265 [feat](Nereids)compute project cost according to
expression tree (#35864)
new ca6ce68a529 [Bug](function) fix yearweek function get wrong result
(#36159)
new c50cc270f01 [UT](test) add BE UT test case (#35851)
new db31de10cd1 [feature](statistics)Support analyze and drop single
partition stats. (#36031)
new 526f7e69314 [fix](regression) Fix test_disable_move_memtable (#36198)
new 9312046b6cb [fix](function) error scale set in unix_timestamp (#36110)
new 816de47e89b [opt](docker)git ignore docker generated files (#36242)
new 4f0dfa69e2b [Bug](drop) fix drop function has error when it's not
exists (#35878)
new 61857fec160 [Enhancement] (Nereids) let insert statement support CTE
(#36150)
new 352bd9639bc [fix](group commit) Group commit core for agg_state
(#36220)
new c8b985c0999 [Fix](sql function) memory overflow to the left of string
address when do_money_format has small negative value (#36226)
The 98 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.asf.yaml | 16 +-
.gitignore | 8 +-
README.md | 11 +-
be/cmake/thirdparty.cmake | 5 +
be/src/agent/be_exec_version_manager.h | 12 +-
be/src/agent/task_worker_pool.cpp | 28 +-
be/src/apache-orc | 2 +-
be/src/cloud/cloud_delete_task.cpp | 7 +
.../cloud/cloud_engine_calc_delete_bitmap_task.cpp | 134 +--
.../cloud/cloud_engine_calc_delete_bitmap_task.h | 9 +-
be/src/cloud/cloud_meta_mgr.cpp | 21 +-
be/src/cloud/cloud_rowset_writer.cpp | 13 +-
be/src/clucene | 2 +-
be/src/common/config.cpp | 3 +
be/src/common/config.h | 3 +
be/src/common/exception.h | 17 +
be/src/exprs/runtime_filter.cpp | 30 +-
be/src/io/fs/azure_obj_storage_client.cpp | 244 +++++
be/src/io/fs/azure_obj_storage_client.h | 63 ++
be/src/io/fs/obj_storage_client.h | 3 +-
be/src/io/fs/s3_file_system.cpp | 8 +-
be/src/olap/base_tablet.cpp | 22 +-
be/src/olap/data_dir.cpp | 3 +-
be/src/olap/delete_handler.cpp | 15 +-
be/src/olap/delete_handler.h | 2 +-
be/src/olap/rowset/beta_rowset_writer.cpp | 13 +-
be/src/olap/rowset/rowset.cpp | 7 +
be/src/olap/rowset/rowset.h | 2 +
be/src/olap/rowset/rowset_meta.cpp | 14 +
be/src/olap/rowset/rowset_writer_context.h | 3 +-
be/src/olap/rowset/segment_creator.cpp | 92 +-
be/src/olap/rowset/segment_creator.h | 8 +-
.../rowset/segment_v2/hierarchical_data_reader.cpp | 5 +-
.../rowset/segment_v2/hierarchical_data_reader.h | 25 +-
.../rowset/segment_v2/inverted_index_writer.cpp | 7 +-
be/src/olap/rowset/segment_v2/segment.cpp | 29 +-
be/src/olap/rowset/segment_v2/segment_iterator.cpp | 33 +-
be/src/olap/rowset/segment_v2/segment_iterator.h | 5 +-
be/src/olap/rowset/segment_v2/segment_writer.cpp | 285 ++++--
be/src/olap/rowset/segment_v2/segment_writer.h | 12 +-
.../rowset/segment_v2/vertical_segment_writer.cpp | 151 ++-
.../rowset/segment_v2/vertical_segment_writer.h | 11 +-
be/src/olap/rowset_builder.cpp | 14 +-
be/src/olap/schema_change.cpp | 2 +-
be/src/olap/single_replica_compaction.cpp | 22 +-
be/src/olap/storage_engine.cpp | 2 +-
be/src/olap/tablet.cpp | 6 +-
be/src/olap/tablet_manager.cpp | 8 +-
be/src/olap/tablet_reader.cpp | 10 +-
be/src/olap/tablet_schema.cpp | 26 +-
be/src/olap/tablet_schema.h | 4 +-
be/src/olap/task/engine_clone_task.cpp | 23 +-
be/src/olap/types.h | 16 +-
be/src/pipeline/exec/aggregation_sink_operator.cpp | 2 +-
.../distinct_streaming_aggregation_operator.cpp | 3 +-
be/src/pipeline/exec/exchange_sink_buffer.cpp | 13 +-
be/src/pipeline/exec/exchange_sink_buffer.h | 4 +-
be/src/pipeline/exec/exchange_sink_operator.cpp | 4 +-
.../exec/group_commit_block_sink_operator.cpp | 2 +
.../exec/group_commit_block_sink_operator.h | 6 +-
be/src/pipeline/exec/hashjoin_probe_operator.cpp | 58 +-
.../exec/join/process_hash_table_probe_impl.h | 4 +-
.../exec/nested_loop_join_probe_operator.cpp | 15 +-
.../exec/partitioned_aggregation_sink_operator.cpp | 16 +
.../partitioned_aggregation_source_operator.cpp | 28 +-
.../exec/partitioned_hash_join_probe_operator.cpp | 61 +-
.../exec/partitioned_hash_join_sink_operator.cpp | 83 +-
be/src/pipeline/exec/result_file_sink_operator.cpp | 17 +-
be/src/pipeline/exec/sort_source_operator.cpp | 3 +-
be/src/pipeline/exec/spill_sort_sink_operator.cpp | 21 +-
.../pipeline/exec/spill_sort_source_operator.cpp | 32 +-
.../exec/streaming_aggregation_operator.cpp | 2 +-
.../local_exchange_sink_operator.cpp | 9 +
.../local_exchange/local_exchange_sink_operator.h | 8 +-
.../local_exchange_source_operator.cpp | 9 +
.../local_exchange_source_operator.h | 2 +
be/src/pipeline/local_exchange/local_exchanger.cpp | 25 +
be/src/pipeline/local_exchange/local_exchanger.h | 19 +-
be/src/pipeline/pipeline_fragment_context.cpp | 16 +-
be/src/pipeline/pipeline_task.cpp | 4 +-
be/src/pipeline/task_scheduler.cpp | 41 +-
be/src/runtime/fragment_mgr.cpp | 26 +-
be/src/runtime/fragment_mgr.h | 3 +-
be/src/runtime/group_commit_mgr.cpp | 6 +-
be/src/runtime/load_channel.cpp | 6 +-
be/src/runtime/load_stream.cpp | 6 +-
be/src/runtime/runtime_predicate.cpp | 8 +-
be/src/runtime/runtime_predicate.h | 10 +-
be/src/runtime/runtime_state.cpp | 1 +
be/src/runtime/runtime_state.h | 6 +-
be/src/service/internal_service.cpp | 25 +-
be/src/util/doris_metrics.cpp | 2 +-
be/src/util/ref_count_closure.h | 32 +-
be/src/util/s3_util.cpp | 148 ++-
be/src/util/s3_util.h | 26 +-
be/src/vec/columns/column_object.cpp | 19 +-
be/src/vec/common/pod_array.h | 3 +-
be/src/vec/common/schema_util.cpp | 112 +-
be/src/vec/common/schema_util.h | 13 +-
be/src/vec/core/block.cpp | 3 +-
be/src/vec/data_types/data_type.h | 4 +
be/src/vec/data_types/data_type_string.cpp | 5 -
.../serde/data_type_datetimev2_serde.cpp | 16 +-
.../data_types/serde/data_type_object_serde.cpp | 19 +-
.../vec/data_types/serde/data_type_object_serde.h | 9 +-
.../format/parquet/fix_length_dict_decoder.hpp | 17 +-
.../exec/format/parquet/fix_length_plain_decoder.h | 13 +-
be/src/vec/exec/format/table/iceberg_reader.cpp | 15 +-
be/src/vec/exec/format/table/iceberg_reader.h | 3 +-
be/src/vec/exec/scan/new_olap_scanner.cpp | 2 +-
be/src/vec/exec/scan/scanner_scheduler.cpp | 163 +--
be/src/vec/exprs/vexpr.cpp | 7 +-
.../vec/functions/array/function_array_distance.h | 2 +-
be/src/vec/functions/comparison_equal_for_null.cpp | 139 ++-
be/src/vec/functions/date_time_transforms.h | 10 +-
be/src/vec/functions/function.cpp | 2 +-
be/src/vec/functions/function_string.h | 5 +-
be/src/vec/functions/function_timestamp.cpp | 2 +-
be/src/vec/functions/function_variant_element.cpp | 43 +-
be/src/vec/olap/olap_data_convertor.cpp | 34 +-
be/src/vec/olap/olap_data_convertor.h | 15 +-
be/src/vec/runtime/vdatetime_value.cpp | 35 +-
be/src/vec/runtime/vdatetime_value.h | 17 +-
be/src/vec/runtime/vparquet_transformer.cpp | 12 +-
be/src/vec/runtime/vparquet_transformer.h | 1 +
be/src/vec/sink/vdata_stream_sender.cpp | 2 +-
be/src/vec/sink/vdata_stream_sender.h | 22 +-
be/src/vec/spill/spill_stream.cpp | 45 +-
be/src/vec/spill/spill_stream.h | 4 +-
be/src/vec/spill/spill_stream_manager.cpp | 91 +-
be/src/vec/spill/spill_stream_manager.h | 15 +
be/test/io/fs/azure_test.cpp | 96 ++
be/test/io/fs/s3_file_writer_test.cpp | 3 +-
be/test/vec/core/accurate_comparison_test.cpp | 73 ++
be/test/vec/core/field_test.cpp | 46 +
be/test/vec/core/get_common_type_test.cpp | 152 +++
be/test/vec/core/number_traits_test.cpp | 102 ++
be/test/vec/function/function_eq_for_null_test.cpp | 647 ++++++++++++
be/test/vec/function/function_math_test.cpp | 3 +-
bin/start_be.sh | 9 +
cloud/src/meta-service/mem_txn_kv.cpp | 49 +
cloud/src/meta-service/mem_txn_kv.h | 26 +-
cloud/src/meta-service/meta_service.cpp | 8 +-
cloud/src/meta-service/meta_service_txn.cpp | 44 +-
cloud/src/meta-service/txn_kv.cpp | 131 +++
cloud/src/meta-service/txn_kv.h | 87 ++
cloud/test/CMakeLists.txt | 1 +
cloud/test/meta_service_test.cpp | 337 ++++++-
cloud/test/txn_kv_test.cpp | 263 +++++
.../docker-compose/iceberg/iceberg.yaml.tpl | 6 +
.../docker-compose/iceberg/spark-init.sql | 26 +
.../main/java/org/apache/doris/common/Config.java | 6 +-
.../java/org/apache/doris/common/io/DiskUtils.java | 20 +-
.../org/apache/doris/common/io/DiskUtilsTest.java | 38 +-
.../antlr4/org/apache/doris/nereids/DorisParser.g4 | 2 +-
.../org/apache/doris/alter/AlterJobV2Factory.java | 15 -
.../doris/alter/MaterializedViewHandler.java | 2 -
.../apache/doris/alter/SchemaChangeHandler.java | 2 -
.../org/apache/doris/alter/SchemaChangeJobV2.java | 2 +-
.../org/apache/doris/analysis/AddRollupClause.java | 2 +-
.../org/apache/doris/analysis/AnalyticExpr.java | 18 +-
.../org/apache/doris/analysis/AnalyzeTblStmt.java | 10 +-
.../java/org/apache/doris/analysis/CastExpr.java | 2 +-
.../java/org/apache/doris/analysis/ColumnDef.java | 44 +-
.../apache/doris/analysis/DropRollupClause.java | 2 +-
.../org/apache/doris/analysis/DropStatsStmt.java | 38 +-
.../java/org/apache/doris/analysis/GrantStmt.java | 17 +-
.../org/apache/doris/analysis/ResourcePattern.java | 3 -
.../apache/doris/analysis/RollupRenameClause.java | 2 +-
.../apache/doris/analysis/ShowColumnStatsStmt.java | 5 +-
.../org/apache/doris/analysis/ShowRolesStmt.java | 1 +
.../java/org/apache/doris/analysis/UpdateStmt.java | 11 +-
.../java/org/apache/doris/backup/RestoreJob.java | 2 +-
.../java/org/apache/doris/catalog/Database.java | 14 +-
.../main/java/org/apache/doris/catalog/Env.java | 9 +-
.../apache/doris/catalog/TabletInvertedIndex.java | 1 +
.../org/apache/doris/clone/BeLoadRebalancer.java | 10 +-
.../org/apache/doris/clone/DiskRebalancer.java | 14 +-
.../cloud/catalog/CloudInstanceStatusChecker.java | 5 +
.../doris/cloud/catalog/CloudTabletRebalancer.java | 25 +-
.../apache/doris/cloud/load/CloudLoadManager.java | 101 ++
.../transaction/CloudGlobalTransactionMgr.java | 37 +-
.../doris/common/profile/ExecutionProfile.java | 22 +-
.../apache/doris/common/util/ProfileManager.java | 2 +-
.../datasource/iceberg/source/IcebergScanNode.java | 4 +-
.../datasource/iceberg/source/IcebergSplit.java | 5 +-
.../main/java/org/apache/doris/load/DeleteJob.java | 10 +-
.../java/org/apache/doris/load/DppScheduler.java | 7 +-
.../org/apache/doris/load/loadv2/LoadManager.java | 12 +-
.../org/apache/doris/load/loadv2/SparkLoadJob.java | 9 +-
.../doris/load/routineload/RoutineLoadJob.java | 3 +-
.../doris/load/routineload/RoutineLoadManager.java | 3 +-
.../doris/load/routineload/ScheduleRule.java | 40 +-
.../org/apache/doris/master/ReportHandler.java | 2 +
.../apache/doris/mysql/privilege/Privilege.java | 13 +
.../doris/mysql/privilege/ResourcePrivEntry.java | 5 +-
.../apache/doris/mysql/privilege/RoleManager.java | 2 +-
.../org/apache/doris/nereids/StatementContext.java | 27 +-
.../nereids/cost/ExpressionCostEvaluator.java | 4 +-
.../glue/translator/PhysicalPlanTranslator.java | 3 +
.../doris/nereids/jobs/executor/Rewriter.java | 3 +
.../doris/nereids/parser/LogicalPlanBuilder.java | 13 +-
.../doris/nereids/processor/post/TopNScanOpt.java | 26 +-
.../processor/post/TopnFilterPushDownVisitor.java | 5 +-
.../org/apache/doris/nereids/rules/RuleType.java | 2 +
.../LogicalResultSinkToShortCircuitPointQuery.java | 108 ++
.../nereids/rules/analysis/NormalizeRepeat.java | 45 +-
.../mv/AbstractMaterializedViewAggregateRule.java | 275 +++--
.../mv/AbstractMaterializedViewRule.java | 28 +-
.../mv/InitMaterializationContextHook.java | 53 +-
.../exploration/mv/MaterializedViewUtils.java | 5 +-
.../nereids/rules/exploration/mv/StructInfo.java | 10 +-
.../rules/implementation/AggregateStrategies.java | 46 +-
.../ExtractAndNormalizeWindowExpression.java | 3 +-
.../nereids/rules/rewrite/ForeignKeyContext.java | 9 +-
.../nereids/rules/rewrite/MergeAggregate.java | 16 +-
.../doris/nereids/stats/StatsCalculator.java | 15 +
.../nereids/trees/expressions/Expression.java | 5 +
.../executable/DateTimeExtractAndTransform.java | 30 +-
.../expressions/literal/DateTimeV2Literal.java | 16 +
.../trees/expressions/literal/DecimalLiteral.java | 15 +
.../expressions/literal/DecimalV3Literal.java | 15 +
.../nereids/trees/expressions/literal/Literal.java | 2 +-
.../doris/nereids/trees/plans/PlaceholderId.java | 7 +-
.../trees/plans/commands/CreateTableCommand.java | 3 +-
.../plans/commands/DeleteFromUsingCommand.java | 4 +-
.../trees/plans/commands/ExecuteCommand.java | 26 +-
.../nereids/trees/plans/commands/LoadCommand.java | 2 +-
.../trees/plans/commands/UpdateCommand.java | 6 +-
.../plans/commands/UpdateMvByPartitionCommand.java | 2 +-
.../trees/plans/commands/info/CreateMTMVInfo.java | 2 +
.../trees/plans/commands/info/CreateTableInfo.java | 12 +-
.../commands/insert/GroupCommitInsertExecutor.java | 6 +-
.../commands/insert/InsertIntoTableCommand.java | 7 +-
.../insert/InsertOverwriteTableCommand.java | 11 +-
.../plans/physical/PhysicalOlapTableSink.java | 9 +-
.../java/org/apache/doris/persist/EditLog.java | 3 +-
.../org/apache/doris/persist/OperationType.java | 1 +
.../org/apache/doris/persist/meta/MetaReader.java | 9 -
.../org/apache/doris/planner/OlapScanNode.java | 11 +-
.../src/main/java/org/apache/doris/plsql/Exec.java | 7 +-
.../java/org/apache/doris/qe/ConnectProcessor.java | 4 +
.../org/apache/doris/qe/PointQueryExecutor.java | 353 +++++++
.../apache/doris/qe/PreparedStatementContext.java | 3 +
.../java/org/apache/doris/qe/SessionVariable.java | 23 +-
.../apache/doris/qe/ShortCircuitQueryContext.java | 88 ++
.../java/org/apache/doris/qe/StmtExecutor.java | 19 +-
.../apache/doris/service/FrontendServiceImpl.java | 12 +-
.../org/apache/doris/statistics/AnalysisJob.java | 6 +
.../apache/doris/statistics/AnalysisManager.java | 62 +-
.../apache/doris/statistics/BaseAnalysisTask.java | 69 +-
.../org/apache/doris/statistics/HistogramTask.java | 4 +
.../doris/statistics/InvalidateStatsTarget.java | 6 +-
.../apache/doris/statistics/OlapAnalysisTask.java | 1 +
.../doris/statistics/StatisticConstants.java | 1 +
.../doris/statistics/StatisticsAutoCollector.java | 17 +-
.../apache/doris/statistics/StatisticsCache.java | 20 +-
.../doris/statistics/StatisticsRepository.java | 73 +-
.../apache/doris/statistics/TableStatsMeta.java | 2 -
.../doris/statistics/util/StatisticsUtil.java | 10 +
.../main/java/org/apache/doris/task/PushTask.java | 10 +-
.../apache/doris/transaction/TransactionEntry.java | 14 +-
.../load/routineload/RoutineLoadManagerTest.java | 11 +-
.../org/apache/doris/mysql/privilege/AuthTest.java | 42 +
.../nereids/cost/ExpressionCostEvaluatorTest.java | 87 --
.../joinorder/hypergraph/CompareOuterJoinTest.java | 1 +
.../rules/expression/ExpressionRewriteTest.java | 8 +-
.../expression/rules/SimplifyCastRuleTest.java | 12 +-
.../rules/rewrite/EliminateJoinByFkTest.java | 11 +
.../rules/rewrite/SimplifyAggGroupByTest.java | 19 +-
.../functions/DateTimeExtractAndTransformTest.java | 59 ++
.../expressions/literal/DateTimeLiteralTest.java | 14 +-
fe/pom.xml | 39 +-
gensrc/proto/cloud.proto | 16 +-
gensrc/thrift/AgentService.thrift | 12 +
gensrc/thrift/PlanNodes.thrift | 1 +
regression-test/conf/regression-conf.groovy | 5 +-
...gle_compaction_with_variant_inverted_index.out} | 4 +-
.../data/datatype_p0/date/test_from_unixtime.out | 6 +-
regression-test/data/delete_p0/test_delete.out | 3 +
.../data/dialect_compatible/sql/clickhouse.out | 472 +++++++++
.../iceberg/test_iceberg_read_with_posdelete.out} | 7 +-
.../test_need_read_data_fault_injection.out | 22 +
.../test_auto_bucket_mtmv.out} | 6 +-
.../test_bitmap_mtmv.out} | 6 +-
.../test_bloom_filter_mtmv.out} | 6 +-
.../test_colocate_with_mtmv.out} | 6 +-
.../test_compression_mtmv.out} | 6 +-
.../test_create_mv_mtmv.out} | 16 +-
.../test_create_rollup_mtmv.out} | 16 +-
.../data/mtmv_p0/test_create_view_mtmv.out | 15 +
.../test_inverted_mtmv.out} | 6 +-
.../test_light_schema_change_mtmv.out} | 6 +-
.../test_n_gram_mtmv.out} | 6 +-
.../test_refresh_partition_num_mtmv.out} | 6 +-
.../test_schema_version_mtmv.out} | 6 +-
.../test_set_partition_version_mtmv.out} | 6 +-
.../test_tablet_type_mtmv.out} | 6 +-
.../test_version_info_mtmv.out} | 6 +-
.../nereids_p0/insert_into_table/insert_cte.out | 32 +
.../datetime_functions/test_date_function.out | 3 +
.../mv/grouping_sets/grouping_sets.out | 225 +++++
.../nereids_rules_p0/mv/partition_mv_rewrite.out | 64 --
.../data/nereids_syntax_p0/agg_4_phase.out | 12 +-
.../point_query_p0/test_point_query_partition.out | 15 +
.../datetime_functions/test_date_function.out | 3 +
.../test_unique_model_schema_value_add.out | 151 +++
.../test_unique_model_schema_value_change.out | 150 ---
.../test_unique_model_schema_value_change1.out | 81 ++
.../test_unique_model_schema_value_change_del.out | 9 +
regression-test/data/variant_p0/delete_update.out | 7 +
.../partial_update_parallel1.csv | 0
.../partial_update_parallel2.csv | 0
.../data/variant_p0/partial_update_parallel3.csv | 5 +
.../partial_update_parallel4.csv | 0
.../data/variant_p0/variant_with_rowstore.out | 9 +
regression-test/pipeline/external/conf/be.conf | 73 +-
.../pipeline/p0/conf/regression-conf.groovy | 3 +-
.../test_compaction_uniq_keys_row_store.groovy | 7 +-
...e_compaction_with_variant_inverted_index.groovy | 254 +++++
.../test_table_level_compaction_policy.groovy | 13 +-
.../suites/correctness_p0/test_current_date.groovy | 21 +-
.../datatype_p0/date/test_from_unixtime.groovy | 1 +
.../suites/delete_p0/test_delete.groovy | 5 +
.../suites/dialect_compatible/sql/clickhouse.sql | 465 +++++++++
.../test_iceberg_read_with_posdelete.groovy | 46 +
.../test_disable_move_memtable.groovy | 10 +-
.../test_need_read_data_fault_injection.groovy | 94 ++
.../test_schema_change_storage_format.groovy | 149 +++
.../routine_load/test_routine_load_schedule.groovy | 16 +-
.../stream_load/test_get_stream_load_state.groovy | 78 ++
.../test_stream_load_2pc_with_schema_change.groovy | 102 ++
.../test_alter_distribution_type_mtmv.groovy | 60 ++
.../suites/mtmv_p0/test_auto_bucket_mtmv.groovy | 64 ++
.../suites/mtmv_p0/test_bitmap_mtmv.groovy | 79 ++
.../suites/mtmv_p0/test_block_rule_mtmv.groovy | 70 ++
.../suites/mtmv_p0/test_bloom_filter_mtmv.groovy | 86 ++
.../suites/mtmv_p0/test_colocate_with_mtmv.groovy | 77 ++
.../suites/mtmv_p0/test_compression_mtmv.groovy | 68 ++
.../suites/mtmv_p0/test_create_mv_mtmv.groovy | 90 ++
.../suites/mtmv_p0/test_create_rollup_mtmv.groovy | 91 ++
.../suites/mtmv_p0/test_create_view_mtmv.groovy | 72 ++
.../suites/mtmv_p0/test_inverted_mtmv.groovy | 80 ++
.../mtmv_p0/test_light_schema_change_mtmv.groovy | 64 ++
.../suites/mtmv_p0/test_limit_op_mtmv.groovy | 66 +-
.../suites/mtmv_p0/test_n_gram_mtmv.groovy | 79 ++
.../mtmv_p0/test_refresh_partition_num_mtmv.groovy | 70 ++
.../suites/mtmv_p0/test_schema_version_mtmv.groovy | 64 ++
.../mtmv_p0/test_sequence_column_mtmv.groovy | 56 +
.../mtmv_p0/test_set_partition_version_mtmv.groovy | 73 ++
.../suites/mtmv_p0/test_tablet_type_mtmv.groovy | 64 ++
.../suites/mtmv_p0/test_version_info_mtmv.groovy | 64 ++
.../nereids_p0/insert_into_table/insert_cte.groovy | 70 ++
.../datetime_functions/test_date_function.groovy | 1 +
.../agg_with_roll_up/aggregate_with_roll_up.groovy | 3 -
.../aggregate_without_roll_up.groovy | 4 -
.../mv/availability/grace_period.groovy | 3 -
.../availability/materialized_view_switch.groovy | 3 -
.../mv/dimension/dimension_1.groovy | 3 -
.../mv/dimension/dimension_2_3.groovy | 3 -
.../mv/dimension/dimension_2_4.groovy | 3 -
.../mv/dimension/dimension_2_5.groovy | 3 -
.../mv/dimension/dimension_2_6.groovy | 3 -
.../mv/dimension/dimension_2_full_join.groovy | 3 -
.../mv/dimension/dimension_2_inner_join.groovy | 3 -
.../mv/dimension/dimension_2_left_anti_join.groovy | 3 -
.../mv/dimension/dimension_2_left_join.groovy | 3 -
.../mv/dimension/dimension_2_left_semi_join.groovy | 3 -
.../dimension/dimension_2_right_anti_join.groovy | 3 -
.../mv/dimension/dimension_2_right_join.groovy | 3 -
.../dimension/dimension_2_right_semi_join.groovy | 3 -
.../mv/dimension/dimension_self_conn.groovy | 3 -
.../dimension_2_join_agg.groovy | 3 -
.../filter_equal_or_notequal.groovy | 3 -
.../mv/grouping_sets/grouping_sets.groovy | 616 +++++++++++
.../cross_join_list_str_increment_create.groovy | 67 +-
.../cross_join_range_date_increment_create.groovy | 67 +-
...cross_join_range_number_increment_create.groovy | 67 +-
.../full_join_list_str_increment_create.groovy | 50 +-
.../full_join_range_date_increment_create.groovy | 48 +-
.../full_join_range_number_increment_create.groovy | 48 +-
.../inner_join_list_str_increment_create.groovy | 74 +-
.../inner_join_range_date_increment_create.groovy | 73 +-
...inner_join_range_number_increment_create.groovy | 73 +-
...left_anti_join_list_str_increment_create.groovy | 43 +-
...ft_anti_join_range_date_increment_create.groovy | 43 +-
..._anti_join_range_number_increment_create.groovy | 43 +-
.../left_join_list_str_increment_create.groovy | 43 +-
.../left_join_range_date_increment_create.groovy | 43 +-
.../left_join_range_number_increment_create.groovy | 43 +-
...left_semi_join_list_str_increment_create.groovy | 43 +-
...ft_semi_join_range_date_increment_create.groovy | 43 +-
..._semi_join_range_number_increment_create.groovy | 43 +-
...ight_anti_join_list_str_increment_create.groovy | 43 +-
...ht_anti_join_range_date_increment_create.groovy | 43 +-
..._anti_join_range_number_increment_create.groovy | 43 +-
.../right_join_list_str_increment_create.groovy | 71 +-
.../right_join_range_date_increment_create.groovy | 71 +-
...right_join_range_number_increment_create.groovy | 71 +-
...ight_semi_join_list_str_increment_create.groovy | 43 +-
...ht_semi_join_range_date_increment_create.groovy | 43 +-
..._semi_join_range_number_increment_create.groovy | 43 +-
.../self_conn_list_str_increment_create.groovy | 48 +-
.../self_conn_range_date_increment_create.groovy | 48 +-
.../self_conn_range_number_increment_create.groovy | 48 +-
.../mv/join/dphyp_inner/inner_join_dphyp.groovy | 3 -
.../mv/join/dphyp_outer/outer_join_dphyp.groovy | 3 -
.../mv/join/inner/inner_join.groovy | 3 -
.../mv/join/left_outer/outer_join.groovy | 3 -
.../mv/negative/negative_test.groovy | 3 -
.../mv/nested/nested_materialized_view.groovy | 3 -
.../mv/partition_mv_rewrite.groovy | 88 --
.../nereids_rules_p0/mv/scan/scan_table.groovy | 3 -
.../nereids_rules_p0/mv/ssb/mv_ssb_test.groovy | 3 -
.../nereids_rules_p0/mv/tpch/mv_tpch_test.groovy | 3 -
.../partition_curd_union_rewrite.groovy | 3 -
.../mv/union_rewrite/usercase_union_rewrite.groovy | 4 -
.../suites/nereids_syntax_p0/agg_4_phase.groovy | 27 +-
.../suites/nereids_syntax_p0/explain.groovy | 6 +
.../suites/point_query_p0/test_point_query.groovy | 33 +-
.../test_point_query_partition.groovy | 41 +-
.../datetime_functions/test_date_function.groovy | 4 +-
.../test_delete_schema_change_2.groovy | 82 ++
.../test_unique_model_schema_key_drop.groovy | 242 +++++
.../test_unique_model_schema_value_add.groovy | 279 +++++
.../test_unique_model_schema_value_change.groovy | 261 +----
.../test_unique_model_schema_value_change1.groovy | 1066 ++++++++++++++++++++
.../test_unique_model_schema_value_change3.groovy | 547 ++++++++++
...st_unique_model_schema_value_change_drop.groovy | 1019 +++++++++++++++++++
.../suites/statistics/test_partition_stats.groovy | 154 ++-
.../variant_github_events_p0_new/load.groovy | 30 +
.../suites/variant_p0/delete_update.groovy | 122 ++-
.../variant_p0/test_compaction_extract_root.groovy | 12 +-
.../suites/variant_p0/variant_with_rowstore.groovy | 47 +-
thirdparty/download-thirdparty.sh | 2 +
thirdparty/patches/libevent-1532.patch | 12 -
...ch => libevent-keepalive-accepted-socket.patch} | 4 +-
437 files changed, 15678 insertions(+), 2926 deletions(-)
create mode 100644 be/src/io/fs/azure_obj_storage_client.cpp
create mode 100644 be/src/io/fs/azure_obj_storage_client.h
create mode 100644 be/test/io/fs/azure_test.cpp
create mode 100644 be/test/vec/core/accurate_comparison_test.cpp
create mode 100644 be/test/vec/core/field_test.cpp
create mode 100644 be/test/vec/core/get_common_type_test.cpp
create mode 100644 be/test/vec/core/number_traits_test.cpp
create mode 100644 be/test/vec/function/function_eq_for_null_test.cpp
create mode 100644 docker/thirdparties/docker-compose/iceberg/spark-init.sql
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/analysis/LogicalResultSinkToShortCircuitPointQuery.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/qe/PointQueryExecutor.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/qe/ShortCircuitQueryContext.java
delete mode 100644
fe/fe-core/src/test/java/org/apache/doris/nereids/cost/ExpressionCostEvaluatorTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/nereids/trees/expressions/functions/DateTimeExtractAndTransformTest.java
copy regression-test/data/{nereids_syntax_p0/agg_4_phase.out =>
compaction/test_single_compaction_with_variant_inverted_index.out} (82%)
create mode 100644 regression-test/data/dialect_compatible/sql/clickhouse.out
copy regression-test/data/{nereids_syntax_p0/agg_4_phase.out =>
external_table_p0/iceberg/test_iceberg_read_with_posdelete.out} (75%)
create mode 100644
regression-test/data/fault_injection_p0/test_need_read_data_fault_injection.out
copy regression-test/data/{nereids_syntax_p0/agg_4_phase.out =>
mtmv_p0/test_auto_bucket_mtmv.out} (76%)
copy regression-test/data/{nereids_syntax_p0/agg_4_phase.out =>
mtmv_p0/test_bitmap_mtmv.out} (76%)
copy regression-test/data/{nereids_syntax_p0/agg_4_phase.out =>
mtmv_p0/test_bloom_filter_mtmv.out} (76%)
copy regression-test/data/{nereids_syntax_p0/agg_4_phase.out =>
mtmv_p0/test_colocate_with_mtmv.out} (76%)
copy regression-test/data/{nereids_syntax_p0/agg_4_phase.out =>
mtmv_p0/test_compression_mtmv.out} (76%)
copy regression-test/data/{nereids_syntax_p0/agg_4_phase.out =>
mtmv_p0/test_create_mv_mtmv.out} (56%)
copy regression-test/data/{nereids_syntax_p0/agg_4_phase.out =>
mtmv_p0/test_create_rollup_mtmv.out} (56%)
create mode 100644 regression-test/data/mtmv_p0/test_create_view_mtmv.out
copy regression-test/data/{nereids_syntax_p0/agg_4_phase.out =>
mtmv_p0/test_inverted_mtmv.out} (76%)
copy regression-test/data/{nereids_syntax_p0/agg_4_phase.out =>
mtmv_p0/test_light_schema_change_mtmv.out} (76%)
copy regression-test/data/{nereids_syntax_p0/agg_4_phase.out =>
mtmv_p0/test_n_gram_mtmv.out} (76%)
copy regression-test/data/{nereids_syntax_p0/agg_4_phase.out =>
mtmv_p0/test_refresh_partition_num_mtmv.out} (76%)
copy regression-test/data/{nereids_syntax_p0/agg_4_phase.out =>
mtmv_p0/test_schema_version_mtmv.out} (76%)
copy regression-test/data/{nereids_syntax_p0/agg_4_phase.out =>
mtmv_p0/test_set_partition_version_mtmv.out} (76%)
copy regression-test/data/{nereids_syntax_p0/agg_4_phase.out =>
mtmv_p0/test_tablet_type_mtmv.out} (76%)
copy regression-test/data/{nereids_syntax_p0/agg_4_phase.out =>
mtmv_p0/test_version_info_mtmv.out} (76%)
create mode 100644
regression-test/data/nereids_p0/insert_into_table/insert_cte.out
create mode 100644
regression-test/data/nereids_rules_p0/mv/grouping_sets/grouping_sets.out
create mode 100644
regression-test/data/schema_change_p0/test_unique_model_schema_value_add.out
create mode 100644
regression-test/data/schema_change_p0/test_unique_model_schema_value_change1.out
create mode 100644
regression-test/data/schema_change_p0/test_unique_model_schema_value_change_del.out
copy regression-test/data/{unique_with_mow_p0/partial_update =>
variant_p0}/partial_update_parallel1.csv (100%)
copy regression-test/data/{unique_with_mow_p0/partial_update =>
variant_p0}/partial_update_parallel2.csv (100%)
create mode 100644 regression-test/data/variant_p0/partial_update_parallel3.csv
copy regression-test/data/{unique_with_mow_p0/partial_update =>
variant_p0}/partial_update_parallel4.csv (100%)
create mode 100644
regression-test/suites/compaction/test_single_compaction_with_variant_inverted_index.groovy
create mode 100644 regression-test/suites/dialect_compatible/sql/clickhouse.sql
create mode 100644
regression-test/suites/external_table_p0/iceberg/test_iceberg_read_with_posdelete.groovy
create mode 100644
regression-test/suites/fault_injection_p0/test_need_read_data_fault_injection.groovy
create mode 100644
regression-test/suites/inverted_index_p0/storage_format/test_schema_change_storage_format.groovy
create mode 100644
regression-test/suites/load_p0/stream_load/test_get_stream_load_state.groovy
create mode 100644
regression-test/suites/load_p0/stream_load/test_stream_load_2pc_with_schema_change.groovy
create mode 100644
regression-test/suites/mtmv_p0/test_alter_distribution_type_mtmv.groovy
create mode 100644 regression-test/suites/mtmv_p0/test_auto_bucket_mtmv.groovy
create mode 100644 regression-test/suites/mtmv_p0/test_bitmap_mtmv.groovy
create mode 100644 regression-test/suites/mtmv_p0/test_block_rule_mtmv.groovy
create mode 100644 regression-test/suites/mtmv_p0/test_bloom_filter_mtmv.groovy
create mode 100644
regression-test/suites/mtmv_p0/test_colocate_with_mtmv.groovy
create mode 100644 regression-test/suites/mtmv_p0/test_compression_mtmv.groovy
create mode 100644 regression-test/suites/mtmv_p0/test_create_mv_mtmv.groovy
create mode 100644
regression-test/suites/mtmv_p0/test_create_rollup_mtmv.groovy
create mode 100644 regression-test/suites/mtmv_p0/test_create_view_mtmv.groovy
create mode 100644 regression-test/suites/mtmv_p0/test_inverted_mtmv.groovy
create mode 100644
regression-test/suites/mtmv_p0/test_light_schema_change_mtmv.groovy
create mode 100644 regression-test/suites/mtmv_p0/test_n_gram_mtmv.groovy
create mode 100644
regression-test/suites/mtmv_p0/test_refresh_partition_num_mtmv.groovy
create mode 100644
regression-test/suites/mtmv_p0/test_schema_version_mtmv.groovy
create mode 100644
regression-test/suites/mtmv_p0/test_sequence_column_mtmv.groovy
create mode 100644
regression-test/suites/mtmv_p0/test_set_partition_version_mtmv.groovy
create mode 100644 regression-test/suites/mtmv_p0/test_tablet_type_mtmv.groovy
create mode 100644 regression-test/suites/mtmv_p0/test_version_info_mtmv.groovy
create mode 100644
regression-test/suites/nereids_p0/insert_into_table/insert_cte.groovy
create mode 100644
regression-test/suites/nereids_rules_p0/mv/grouping_sets/grouping_sets.groovy
create mode 100644
regression-test/suites/schema_change_p0/test_delete_schema_change_2.groovy
create mode 100644
regression-test/suites/schema_change_p0/test_unique_model_schema_key_drop.groovy
create mode 100644
regression-test/suites/schema_change_p0/test_unique_model_schema_value_add.groovy
create mode 100644
regression-test/suites/schema_change_p0/test_unique_model_schema_value_change1.groovy
create mode 100644
regression-test/suites/schema_change_p0/test_unique_model_schema_value_change3.groovy
create mode 100644
regression-test/suites/schema_change_p0/test_unique_model_schema_value_change_drop.groovy
rename thirdparty/patches/{libevent-enable_keepalive_accepted_socket.patch =>
libevent-keepalive-accepted-socket.patch} (90%)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
