This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a change to branch fix_export_bug
in repository https://gitbox.apache.org/repos/asf/doris.git
discard 99830a93dcb [enhancement](export)export job for olap table will only
hit the base index, so skip other tablets of rollup index when do splitTablets
computing
add 02e71fb5964 [fix](spill) fix wrong disk usage of spill (#35423)
add 208f5bcfe3a [Fix](JobSchedual) Modify the default value of
`async_task_consumer_thread_num` (#35456)
add 9082e5349f2 [Bug](runtime-filter) do not process rf on
HashJoinBuildSinkLocalState::close when query ca… (#35487)
add a2fa3d8b300 [Fix](group commit) Fix group commit block queue mem
estimate faule (#35314)
add 37f7c8c7435 [github](pr) simplify the PR template (#35550)
add dc1a708fd57 [fix](Nereids) fix Illegal aggregate node: group by and
output is empty (#35497)
add 1a42a2acb68 [opt](fe) Optimize fe show table statistics (#35457)
add f41d35b8e68 [fix](ci) tmp exclude block case stress_test_insert_into
(#35553)
add 118ca08b123 [fix](restore) Fix restore table name when
lower_case_table_names enabled (#35508)
add 81a78e0defa [fix](statistics)Record update rows shouldn't fail load
publish. (#35549)
add de2d71aa501 [docker](hive) sync for hive initializing (#35479)
add d86cd1b7b2c [bugfix](iceberg)support null values as partition (#35503)
add 0e33ab4eda7 [Fix](executor)Fix backend_active_tasks only scan one be
(#35492)
add d6279fa0938 [improvement](page builder) avoid allocating big memory in
ctor (#35493)
add 40d8a9831c5 [fix](pipeline) Disable non-pipeline engine for stream
load (#35555)
add 99dbc0f3db4 [fix](compaction) fix inaccurate compaction cnt (#35539)
add 5d40bd62864 [fix](publish) fix publish failed when balance (#35462)
add 073df2cdece [fix](httpapi) restore compaction/run_status api can show
be's overall compaction status and refactor code (#35278)
add 0939ab12714 [regression-test](framework) add log for scpFiles (#35570)
add 74309a22d6c [fix](Nereids) aggregate combinator should be
case-insensitive (#35540)
add f63d5a4b2fe [fix](auth) Fix no auth,but can select count(*) (#35465)
add 961ccf11012 [Fix](hive-writer) Fix s3 file commiter not working.
(#35502)
add da71c0cb940 [fix](pipeline) Prevent concurrent accessing to
dependencies (#35560)
add 1afe0e94e69 [fix](snappy) avoid potential buffer overflow (#35537)
add 23aa43049ff [fix](Nereids) prune not required window expressions on
window operator (#35504)
add e4c6272dc42 [chore](fs) add a config to contorl if sync on close
(#35391)
add e2d7eca8875 [fix](auto-partition) fix auto partition load lost data in
multi sender (#35287)
add d7d52350859 [Feature](iceberg-writer) Implements iceberg sink basic
functionality for inserting into table. (#34929)
add fae2b4b4fe1 [pipeline](API) Add a new API to find pipeline tasks by a
specific query ID (#35563)
add 0b5dd110a5e [fix](mtmv) Fix partition mv rewrite result wrong (#35236)
add baf7ea3daf4 [fix](mtmv) Fix getting related partition table wrongly
when multi base partition table exists (#34781)
add c1676f03cae [enhance](mtmv) Mv refresh on commit (#34548)
add 85e7add9730 [fix](inverted index) add in list to fast execute logic
after hit index (#35344)
add a58f6aae1fd [fix](Nereids) adjust nullable for set operation may cause
IndexOutOfBound (#35588)
add d35888a93e7 [feature](statistics)Support partition stats cache (#35517)
add adacbc35d9a [enhancement](memory) Add `ThreadMemTrackerMgr` BE UT
(#35518)
add 85ba7992922 [fix](ub) fix uninitialized accesses in BE (#35370)
add e000330e616 [fix](short circurt) fix return default value issue
(#34186)
add 20b8e4a32aa [Fix](ci) release a testcase and remove some outdate log
settings (#35580)
add f6e624adaaf [fix](Nereids) remove getTableInMinidumpCache temporary
(#35571)
add df7fa82f3b2 [fix](test) fix
test_create_table_without_distribution.groovy bug (#35486)
add f3de70c8f37 [refactor](functions) clean some ip functions code and
make IS_IP_ADDRESS_IN_RANGE DEPENDS_ON_ARGUMENT (#35239)
add 2b0ead71e33 [pipeline](tracing) Fix pipeline tracing tools (#35595)
add 2affc16ed31 [fix](backup) save finished state for local repo backup
(#35491)
add f89f07fc827 [fix](group commit) Group commit http stream should not
begin txn (#35494)
add 2c7bfd89f1f [regression-test](framework) fix scp bugs in regression
framework (#35613)
add fdeaf689a00 [regression-test](case) remove unused files (#35614)
add 29107309c2f [fix](inverted index) Remove float/double support in index
writer (#35615)
add ec6abb6b432 [fix](set) incorrect result of set operator (#35607)
add 3c11f0972ac [fix](resource-tag) missing resource tag after forwarding
to master (#35618)
add fad30bf5364 [cases](array_contains)add cases for array_contains
supporting inverted index and fix stopwords as query string (#35299)
add 948460f9f9b [opt](audit) add timeout for audit log load and modify the
label format (#35535)
add 1c562d86e0d [fix](regression)Remove useless case which may cause
preHeat npe. (#35582)
add fe8a010bdf7 [fix](SenderQueue) error memlimit in add/sub
_blocks_memory_usage (#35428)
add bd82026b1b4 [fix](fe) fix start fe failed when recover fe in different
host (#35612)
add 5ab26462afe [feature-wip](Cloud) Introduce azure core C++ sdk (#35208)
add ce8b9bbb6f3 [enhance](mtmv)add truncate table case (#35599)
add 3ad78f99789 [fix](meta) fix catalog replay error (#35532)
add 45d99200719 [feature-wip](Cloud) Extract obj storage client interface
to suite different kinds of object storage (#35307)
add bb8fe3d0237 [fix](s3-load) refactor s3 parallelism case and remove it
to p2 (#35629)
add 94db52c8205 [FE](session-variable) Add a debug variable of constant
fold (#35584)
add 2b86d0ec941 [fix](stats) Add synchronize for some analysis maps to
avoid ConcurrentModificationException (#35591)
add e58fcf0f7a3 [Feature](column) do not change column nullable on
ColumnDef (#35574)
add a4691fc44da [opt](nereids) compare str literal as date literal to
compute selectivity (#35610)
add eeb706a3020 [chore](multi catalog) Improve network interface detection
in Hive container script (#35089)
add 456aad2d638 [fix](agg) Update agg counter in execution (#35624)
add ffc80cf9312 [fix](mtmv) Fix get mv statistics plan wrong and optimize
code usage (#35623)
add 5c747be6346 [feature] (recycle bin) Support delete catalog recycle bin
(#31893)
add 861ae815641 [fix](mtmv) ignore MTMV when backup and restore (#35586)
add 26ebe0e16b0 [bug](Compile) Add default value for optional field to
pass compile's missing-field-initializers check (#35648)
add 47e52e3fc4a [fix](auth)fix simple auth check and default username
(#35620)
add cc854a32f1d [fix](libjdk) Revert support loading libjvm at runtime
(#35557)
add 841f42564e1 [Bug](backend-options) fix use after free on
BackendOptions::get_local_backend() (#35634)
add 754b55f5b8b [fix](catalog) close connection on refresh (#35426)
add c2b667f0247 [Fix](Variant) fix variant with empty json key (#35546)
add d2a2e22e06d [fix](nereids) do not generate runtime filter on
schema-scan (#35655)
add ff2b1fe1e8a [improvement](page builder) catch exception when page
builder reset to avoid coredump (#35597)
add 75ddc679c75 [Test](external-table-writer) Use `use_meta_cache=true` to
create catalog when executing hive/iceberg writer regression tests. (#35640)
add b3c00561118 [fix](clone) fix calculate merge missing version (#35627)
add a6e99fb6e7d [refactor](cancel) disable cancel instance for pipeline
(#35632)
add a62fe7c26ac [feat](thrift) add NereidsId and cardinality to TPlanNode
(#35482)
add 2c958fd7b72 [opt](mv) calc Projection cost according to the
expressions in project list (#35652)
add 7b5bc731efc [Improvement](executor)Support kill query in be (#35602)
add a8df1d7e145 [Test](regression-test): Normalize S3 http url (#35666)
add cc1fef18005 [regression-test](load) add something like $.tag.[a.b]
key's json case (#35134)
add 6272315c402 [improve](txn-insert) Add txn insert p2 case (#35439)
add 42e173b987a [fix](binlog) Add is temp for UpsertRecord (#35636)
add b5034c20d36 [Performance](opt) opt the scan performance and remove
unless code (#35604)
add f5647edd869 [chore](third-party) Add a workflow: Build Third Party
Libraries (macOS-arm64) (#35705)
add 3c18026c762 [feature](statistics)Support clear stale partition stats.
(#35661)
add 35b17ee0e12 [chore](ASF) Make the workflow Build Third Party Libraries
(macOS-arm64) required (#35706)
add 75629133932 [fix](log) print log of error status (#35687)
add dc729311370 [Feature](Prepared Statment) Implement in nereids planner
(#35318)
add a939b59dd4c [opt](mtmv) Improve the mv rewrite performance by
optimize code usage (#35674)
add 7f8901578bd [fix](regression-test) Fix s3 load profile test failed due
to css schema change (#35633)
add 99388a08b29 [fix](segment cache) estimate momory consumed by segment
(#35647)
add 23c925591a5 [Fix](trino-connector) add regression tests for
trino-connector accessing Kafka. (#35282)
add a11a590368d [fix](pipeline) Fix query hang up if limited rows is
reached (#35513)
add 50446688891 [thirdparty](compatibility) restore dragonbox for compile
compatibility (#35605)
add f53aee4c8f3 [Fix](tvf) Pass through user-defined properties (#35515)
add e008bcfda22 [fix](decimal256) fix coredump when enable decimal256 but
fallback to old planner (#35731)
add d1c1c49d53c [regression-test](case) forbid unstable case (#35738)
add dd9beff5b72 [fix](orc)fix orc reader missing column and filter missing
column. (#35583)
add 3beb92b3b00 [bugfix](iceberg)fix datetime conversion error and data
path error (#35708)
add adfcbe86ceb (Fix)[hive-writer] Fixed the issue when partition values
contain spaces when writing to s3. (#35645)
add 8c797025992 [Fix](inverted index) fix fast execute condition for vexpr
(#35673)
add bf5736a8106 [Feature](function) support array_split and
array_reverse_split functions (#35619)
add 5368bb19b07 [feature](be jvm monitor)append enable_jvm_monitor in
be.conf to control jvm monitor. (#35608)
add 2ba385385a1 [Bug](load) fix s3 load not display the progress info
(#35719)
add 78431dbb80d [fix](Nereids): Convert VarcharLiteral to
StringLikeLiteral in Function Signatures (#35536)
add dc524a5f564 [refactor](cleancode) remove unused code from be (#35756)
add 74becf00d7e [Test](regression-test): do not use path style to access
aliyun OSS (#35725)
add 8f79f175f86 [chore](cloud) Add show hotspot tablet API on BE (#35527)
add 9a82c9f7288 [fix](partial update) duplicate key occur when BE restart
after conflict concurrent partial update (#35739)
add afbc93f44dd [fix](drop table) Improve error prompts when deleting
materialized views (#35437)
add 2879d8f7d10 [Performance](opt) opt the memcpy and string compare
performance (#35713)
add 67f1ae8a12f [Enhancement] support to erase all tables and partitions
with same DbId while drop a database which is not in catalog recycle bin
(#35750)
add 502721149f5 [enhancement](oom) add exception in olap data convertor
when memory is not enough to prevent oom (#35761)
add 8cae0848d09 [regression](show stmt) Add test checkpoint for the Show
Procedure status and Show create procedure command (#35395)
add ae50d7a614a [Scanner](revert) revert the scanner change by
apache#35604 (#31508)
add 76696da5610 [fix](audit log) Fix audit plugin run in cloud mode
(#35710)
add c5ba29a77d8 [fix](s3) move s3 providers to fe-common to be accessible
for jni reader (#35690)
add c9651614b0e [fix](rpc) fix transfer large data and enable
transfer_large_data_by_brpc by default (#35770)
add 4b6ba8adfb2 [fix](ub) fix some ub error (#35769)
add 8a64e8f050a [improve](session) print more error msg when set a wrong
session variable name (#35775)
add 7b8c520489e [fix](compile) avoid use jdk17 api Optional#isEmpty since
target is 1.8 (#35736)
add 6e1a2ce8e70 [enhancement](memory) Add PODArray UT and pick some patch
(#35639)
add d4730ad20c7 [opt](fe) Remove redundant fe config
`cloud_meta_service_rpc_failed_retry_times` (#35688)
add 2c2d2adc0af [fix](pipeline) Fix reporting status if prepare failed
(#35777)
add 1113f3d44d7 [fix](mtmv) Fix that the storage medium specified for the
mtmv is SSD, but the partition storage medium for the mtmv is still HDD (#35644)
add 539cad87b0e [fix](bit shift) fix undefined behavior of bit shift
functions. (#35723)
add f2a2c7082b8 [opt](exchange) Add buffer size limit for exchange sink
operator (#35642)
add 79ad00a0e15 [Chore](bitmap) set enable_set_in_bitmap_value default
value to true (#35730)
add 9147c046ce2 [fix](s3) fix invalid s3 properties checking logic (#35762)
add f744276c77d [fix](mtmv) Mapping materialization statistics's
expressionToColumnStats to mv scan plan based (#35749)
add b2f04fa9a7b [Feat](nereids) add transform rule MergePercentileToArray
(#34313)
add 3f7e7a0f0a4 [fix](fe) Add check editlog size mechanism for backupJob
(#35653)
add 645147a5cdd [enhancement](oom) return error when bloom filter allocate
memory failed (#35790)
add 21e23482ac4 [enhancement](debugtimeout) should print debug stack when
timeout (#35798)
add 896415aa68e [Chore](materialized-view) enable create mv on mow table
(#35797)
add f4d029dba03 [fix](Nereids) unix_timestamp compute signature and fold
const is wrong (#35727)
add 71c5b852cd9 [opt](nereids)calculate expression cost (#35701)
add 9e7345888c8 [opt](load) use notify to replace polling for FlushToken
(#35796)
add 45661f941e8 [feature](Cloud) Introduce obj storage client interface to
recycler (#35447)
add 70a120698f6 [enhancement](regression-test) modify a key type tests
(#35792)
add 255d80cf981 [Feature](exec) Support group by limit opt in BE code
(#29641)
add c0eb7e975f4 [fix](dynamic partition) fix dynamic partition thread met
uncatch exception (#35778)
add 3b79632e29b [fix](regression) Fix txn_insert execute timeout (#35801)
add 80cc735ac90 [Feature](IP) support ipv4/ipv6 with inverted index and
conjuncts for query (#35734)
add b4e7d5b6b78 [fix](be) Fix invalid memory order argument to atomic
operation (#35815)
add f7ca8fe424d [Fix](show data) fix show data total size wrong (#35818)
add afc00788ff4 [fix](third-party) enable keepalive on socket created by
libevent (#35805)
add 0ff395590ee [improvement](partial update)add more logs for partial
update (#35802)
add 585662b37af [improvement](http) Support to acquire md5 digest of the
file to download (#35807)
add cb7e8bf1aa6 [fix](simd-json-reader) fix SIMD json reader lose data and
support stream parser (#35781)
add c6622ea16aa [Improvement](coldheat) add statment to show objects which
use storage policy (#35741)
add 949e29e040f [feature](rowset) To support rowset on remote FS using
paths of different format (#35473)
add 41e495e723b [opt](log) refine the FE logger (#35679)
add df144a36aa7 [feature](workloadgroup)Add workload condition query used
memory (#35662)
add 5cfb985ad5a [Enchancement](scan) enable parallel scan when preagg is
on (#35810)
add 8eb6caf517e [Enchancement](bitmap) optimize bitmap deserialize and
remove some unused code (#35789)
add a205d82bf63 [Enhancement](inverted index) low level log for fulltext
query info (#35820)
add b33a1613abf [fix](multi-catalog)fix hive partition insert regression
case (#35821)
add 3a9addbca18 [Improvement](local shuffle) Use grouping expr for hash
shuffling (#35716)
add 25f01828a22 [opt](nereids) keep equal predicate as join conjunct even
if it can be fold to null literal (#35811)
add 4e85439c4c6 [fix](mtmv) mtmv disable dynamic partition (#35686)
add b3e0b89dc4c [refactor](pipeline) Delete non-pipeline exec nodes
(#35803)
add 044a24586d3 [Bug](udtf) fix udtf core dump use as normal function
(#35235)
add e8420ac72da [opt](nereids) transform grouping sets to trivial
aggregation as far as possible (#35813)
add 68d57bb4e90 [Fix](Tablet) Fix the issue of redundant loading of stale
rowset meta (#35768)
add f2d8dd438ad [fix](stream_load) fix stream load may failed caused by
column name with keyword (#35822)
add 7730e883f04 [fix](export) fix that export table retry with same label
should succeed when first export job failed (#35676)
add 7b24e3c7b8c [refactor](pipeline) Delete non-pipeline data sink (#35858)
add 4ba219f0470 [feature](statistics)Remove not exist partition stats
while analyzing. (#35814)
add 2b4b60abcfa [improve](fe) Support to config max msg/frame size of the
thrift server (#35845)
add 1a8a44b3529 [regression](backup-restore) Add a `huge tables`
backup-restore test case (#35862)
add e71e4bfdb45 [Bugfix] Fix nullptr cache object because init file cache
concurrently (#35722)
add b439484f7c7 [improve](cloud) Add profile for sync_rowsets while
quering (#35869)
add a02be87c09d [fix](broker) fix no error url when broker data quality
error (#35643)
add a10a959a52f [improve](cache) optimize the cache log (#35603)
add 5668770f03c [Fix](MS) Fix get tablet stats value api (#35880)
add 2b49e13e17c [fix](fe) Set is electable for checking compatibility mode
(#35853)
add 5cb5df30c84 [fix](clone) Fix clone and alter tablet use same tablet
path (#34889)
add edd501b5b1a [streamload](redirect) Support redirect-policy for
streamload which is used by audit plugin (#35840)
add 6a005b60728 [fix](regression test) test_alter_colocate_group due to
force_olap_table_replication_num (#35835)
add 327cbf8e967 [fix](move-memtable) fix move memtable core when use multi
table load (#35458)
add 2a57976238c [enhancement](delete-pred) enable delete sub predicate v2
for compaction (#35859)
add 174eea7246e (chore)[profile] remove non-pipeline logical from profile
(#35877)
add 3fb46c77114 [chore](FE) Fix issues with the activation of profile
protoc_rosetta (#35598)
add 70f39475565 [Chore](log) do not print stacktrace on
ABORTED/NOT_IMPLEMENTED_ERROR (#35867)
add 918141bc7e4 [enhancement](fe-meta) Support skip specified journal to
avoid FE can't be start due to a damaged journal (#35783)
add b60f5b822f2 [fix](case)fix regress case for test_ip_in_inverted_index
(#35881)
add a81364bf026 [fix](build) fix include for macOS compile (#35865)
add e93be1ae80f [fix](RecycleBin) fix race condition issue when do execute
ShowCatalogRecycleBinStmt (#35759)
add b1e4f5c3054 [improvement](JdbcScan) Change the mysql function that
does not support pushdown in JdbcScan to Config (#35196)
add d17c85601ba [enhancement](export)export job for olap table will only
hit the base index, so skip other tablets of rollup index when do splitTablets
computing
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (99830a93dcb)
\
N -- N -- N refs/heads/fix_export_bug (d17c85601ba)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.asf.yaml | 1 +
.github/PULL_REQUEST_TEMPLATE.md | 4 -
.github/workflows/build-thirdparty.yml | 60 +-
be/CMakeLists.txt | 22 +-
be/src/agent/be_exec_version_manager.h | 1 +
be/src/agent/task_worker_pool.cpp | 20 +-
be/src/cloud/cloud_base_compaction.cpp | 2 +-
be/src/cloud/cloud_compaction_action.cpp | 46 +-
be/src/cloud/cloud_cumulative_compaction.cpp | 3 +-
be/src/cloud/cloud_delete_task.cpp | 5 +-
be/src/cloud/cloud_full_compaction.cpp | 2 +-
be/src/cloud/cloud_meta_mgr.cpp | 3 +-
be/src/cloud/cloud_rowset_builder.cpp | 7 +-
be/src/cloud/cloud_rowset_writer.cpp | 15 +-
be/src/cloud/cloud_schema_change_job.cpp | 13 +-
be/src/cloud/cloud_storage_engine.cpp | 14 +-
be/src/cloud/cloud_storage_engine.h | 17 +-
be/src/cloud/cloud_tablet.cpp | 66 +-
be/src/cloud/cloud_tablet.h | 7 +
be/src/cloud/cloud_tablets_channel.cpp | 6 +-
be/src/cloud/cloud_warm_up_manager.cpp | 14 +-
be/src/common/config.cpp | 24 +-
be/src/common/config.h | 17 +-
be/src/common/status.h | 24 +-
be/src/common/utils.h | 3 +-
be/src/exec/data_sink.cpp | 367 -----
be/src/exec/data_sink.h | 131 --
be/src/exec/exec_node.cpp | 637 --------
be/src/exec/exec_node.h | 368 -----
be/src/exec/olap_common.h | 15 +-
be/src/exec/scan_node.cpp | 57 -
be/src/exec/scan_node.h | 109 --
be/src/exec/tablet_info.cpp | 17 +-
be/src/exprs/hybrid_set.h | 2 +-
be/src/gutil/strings/fastmem.h | 159 --
be/src/gutil/strings/stringpiece.h | 10 +-
be/src/http/action/compaction_action.cpp | 35 +-
be/src/http/action/http_stream.cpp | 1 -
be/src/http/action/pipeline_task_action.cpp | 30 +
be/src/http/action/pipeline_task_action.h | 9 +
be/src/http/action/show_hotspot_action.cpp | 170 +++
be/src/http/action/show_hotspot_action.h | 38 +
be/src/http/http_client.cpp | 46 +-
be/src/http/http_client.h | 5 +-
be/src/http/utils.cpp | 72 +-
be/src/http/utils.h | 3 +-
be/src/index-tools/index_tool.cpp | 34 +-
be/src/io/cache/block_file_cache.cpp | 13 +-
be/src/io/cache/block_file_cache_downloader.cpp | 14 +-
be/src/io/cache/block_file_cache_factory.cpp | 10 +-
be/src/io/cache/block_file_cache_factory.h | 1 +
be/src/io/file_factory.cpp | 25 +-
be/src/io/fs/err_utils.cpp | 7 +
be/src/io/fs/local_file_writer.cpp | 2 +-
be/src/io/fs/multi_table_pipe.cpp | 219 ++-
be/src/io/fs/multi_table_pipe.h | 11 +-
be/src/io/fs/obj_storage_client.h | 117 ++
be/src/io/fs/s3_file_bufferpool.cpp | 7 +-
be/src/io/fs/s3_file_bufferpool.h | 2 +
be/src/io/fs/s3_file_reader.cpp | 23 +-
be/src/io/fs/s3_file_reader.h | 6 +-
be/src/io/fs/s3_file_system.cpp | 332 ++---
be/src/io/fs/s3_file_system.h | 18 +-
be/src/io/fs/s3_file_writer.cpp | 244 +--
be/src/io/fs/s3_file_writer.h | 24 +-
be/src/io/fs/s3_obj_storage_client.cpp | 390 +++++
be/src/io/fs/s3_obj_storage_client.h | 66 +
be/src/olap/base_tablet.cpp | 82 +-
be/src/olap/base_tablet.h | 14 +-
be/src/olap/bitmap_filter_predicate.h | 2 -
be/src/olap/cold_data_compaction.cpp | 6 +-
be/src/olap/column_predicate.h | 4 +-
be/src/olap/compaction.cpp | 135 +-
be/src/olap/data_dir.cpp | 38 +-
be/src/olap/data_dir.h | 4 +-
be/src/olap/delete_handler.cpp | 85 +-
be/src/olap/delete_handler.h | 14 +-
be/src/olap/delta_writer.cpp | 20 +-
be/src/olap/hll.cpp | 19 +-
be/src/olap/hll.h | 73 +-
be/src/olap/memtable_flush_executor.cpp | 21 +-
be/src/olap/memtable_flush_executor.h | 4 +
be/src/olap/memtable_memory_limiter.h | 2 +-
be/src/olap/merger.cpp | 4 +-
be/src/olap/olap_common.h | 4 +-
be/src/olap/olap_define.h | 18 +-
be/src/olap/olap_server.cpp | 34 +-
be/src/olap/parallel_scanner_builder.cpp | 16 +-
be/src/olap/parallel_scanner_builder.h | 10 +-
be/src/olap/primary_key_index.cpp | 4 +-
be/src/olap/rowset/beta_rowset.cpp | 335 ++---
be/src/olap/rowset/beta_rowset.h | 29 +-
be/src/olap/rowset/beta_rowset_writer.cpp | 97 +-
be/src/olap/rowset/beta_rowset_writer.h | 6 +-
be/src/olap/rowset/beta_rowset_writer_v2.h | 4 +
be/src/olap/rowset/rowset.cpp | 29 +-
be/src/olap/rowset/rowset.h | 28 +-
be/src/olap/rowset/rowset_factory.cpp | 2 +-
be/src/olap/rowset/rowset_meta.cpp | 45 +-
be/src/olap/rowset/rowset_meta.h | 13 +-
be/src/olap/rowset/rowset_writer.h | 3 +
be/src/olap/rowset/rowset_writer_context.h | 37 +-
be/src/olap/rowset/segcompaction.cpp | 25 +-
be/src/olap/rowset/segment_creator.cpp | 12 +-
be/src/olap/rowset/segment_creator.h | 11 +
be/src/olap/rowset/segment_v2/binary_dict_page.cpp | 48 +-
be/src/olap/rowset/segment_v2/binary_dict_page.h | 11 +-
be/src/olap/rowset/segment_v2/binary_plain_page.h | 34 +-
be/src/olap/rowset/segment_v2/binary_prefix_page.h | 12 +-
be/src/olap/rowset/segment_v2/bitshuffle_page.h | 38 +-
be/src/olap/rowset/segment_v2/bloom_filter.cpp | 8 +-
.../segment_v2/bloom_filter_index_writer.cpp | 13 +-
.../rowset/segment_v2/bloom_filter_index_writer.h | 8 +-
be/src/olap/rowset/segment_v2/column_reader.cpp | 11 +-
be/src/olap/rowset/segment_v2/column_reader.h | 2 +-
be/src/olap/rowset/segment_v2/column_writer.cpp | 4 +-
be/src/olap/rowset/segment_v2/encoding_info.cpp | 32 +-
.../rowset/segment_v2/frame_of_reference_page.h | 15 +-
.../rowset/segment_v2/indexed_column_reader.cpp | 8 +
.../olap/rowset/segment_v2/indexed_column_reader.h | 2 +
.../rowset/segment_v2/indexed_column_writer.cpp | 2 +-
.../segment_v2/inverted_index_compaction.cpp | 12 +-
.../rowset/segment_v2/inverted_index_compaction.h | 10 +-
.../olap/rowset/segment_v2/inverted_index_desc.cpp | 59 +-
.../olap/rowset/segment_v2/inverted_index_desc.h | 39 +-
.../segment_v2/inverted_index_file_reader.cpp | 66 +-
.../rowset/segment_v2/inverted_index_file_reader.h | 19 +-
.../segment_v2/inverted_index_file_writer.cpp | 50 +-
.../rowset/segment_v2/inverted_index_file_writer.h | 21 +-
.../rowset/segment_v2/inverted_index_reader.cpp | 11 +-
.../rowset/segment_v2/inverted_index_writer.cpp | 26 +-
.../olap/rowset/segment_v2/ordinal_page_index.cpp | 14 +
be/src/olap/rowset/segment_v2/ordinal_page_index.h | 2 +
be/src/olap/rowset/segment_v2/page_builder.h | 18 +-
be/src/olap/rowset/segment_v2/plain_page.h | 26 +-
be/src/olap/rowset/segment_v2/rle_page.h | 30 +-
be/src/olap/rowset/segment_v2/segment.cpp | 29 +-
be/src/olap/rowset/segment_v2/segment.h | 7 +
be/src/olap/rowset/segment_v2/segment_iterator.cpp | 8 +-
be/src/olap/rowset/segment_v2/segment_writer.cpp | 28 +-
be/src/olap/rowset/segment_v2/segment_writer.h | 14 +-
.../rowset/segment_v2/vertical_segment_writer.cpp | 34 +-
.../rowset/segment_v2/vertical_segment_writer.h | 14 +-
be/src/olap/rowset/segment_v2/zone_map_index.cpp | 11 +
be/src/olap/rowset/segment_v2/zone_map_index.h | 2 +
be/src/olap/rowset/vertical_beta_rowset_writer.cpp | 13 +-
be/src/olap/rowset_builder.cpp | 11 +
be/src/olap/schema_change.cpp | 7 +-
be/src/olap/short_key_index.cpp | 16 +
be/src/olap/short_key_index.h | 1 +
be/src/olap/single_replica_compaction.cpp | 4 +-
be/src/olap/snapshot_manager.cpp | 6 +-
be/src/olap/storage_engine.cpp | 4 +-
be/src/olap/storage_policy.cpp | 95 +-
be/src/olap/storage_policy.h | 29 +-
be/src/olap/tablet.cpp | 118 +-
be/src/olap/tablet.h | 15 +-
be/src/olap/tablet_manager.cpp | 121 +-
be/src/olap/tablet_manager.h | 16 +-
be/src/olap/tablet_meta.cpp | 17 +-
be/src/olap/tablet_meta.h | 5 +
be/src/olap/tablet_reader.cpp | 6 -
be/src/olap/tablet_schema.cpp | 10 +
be/src/olap/tablet_schema.h | 4 +-
be/src/olap/task/engine_clone_task.cpp | 28 +-
be/src/olap/task/engine_storage_migration_task.cpp | 8 +
be/src/olap/task/index_builder.cpp | 97 +-
be/src/olap/utils.h | 4 +-
be/src/olap/wal/wal_manager.cpp | 1 -
be/src/olap/wal/wal_table.cpp | 1 -
be/src/pipeline/dependency.cpp | 84 +-
be/src/pipeline/dependency.h | 151 +-
be/src/pipeline/exec/aggregation_sink_operator.cpp | 306 +++-
be/src/pipeline/exec/aggregation_sink_operator.h | 19 +-
.../pipeline/exec/aggregation_source_operator.cpp | 31 +-
be/src/pipeline/exec/aggregation_source_operator.h | 2 +-
be/src/pipeline/exec/analytic_sink_operator.cpp | 22 +-
be/src/pipeline/exec/analytic_sink_operator.h | 14 +-
be/src/pipeline/exec/analytic_source_operator.cpp | 28 +-
be/src/pipeline/exec/analytic_source_operator.h | 21 +-
be/src/pipeline/exec/datagen_operator.h | 2 +-
.../distinct_streaming_aggregation_operator.cpp | 9 +-
.../exec/distinct_streaming_aggregation_operator.h | 5 +-
be/src/pipeline/exec/exchange_sink_buffer.cpp | 88 +-
be/src/pipeline/exec/exchange_sink_buffer.h | 45 +-
be/src/pipeline/exec/exchange_sink_operator.cpp | 40 +-
be/src/pipeline/exec/exchange_sink_operator.h | 7 +-
be/src/pipeline/exec/exchange_source_operator.cpp | 8 +-
be/src/pipeline/exec/file_scan_operator.cpp | 3 +-
.../exec/group_commit_block_sink_operator.cpp | 3 +-
.../exec/group_commit_block_sink_operator.h | 7 +-
be/src/pipeline/exec/hashjoin_build_sink.cpp | 3 +-
be/src/pipeline/exec/hashjoin_probe_operator.cpp | 5 +-
be/src/pipeline/exec/hashjoin_probe_operator.h | 29 +-
be/src/pipeline/exec/hive_table_sink_operator.h | 2 +-
.../pipeline/exec/iceberg_table_sink_operator.cpp | 46 +
be/src/pipeline/exec/iceberg_table_sink_operator.h | 94 ++
be/src/pipeline/exec/jdbc_scan_operator.cpp | 3 +-
be/src/pipeline/exec/jdbc_table_sink_operator.cpp | 4 -
be/src/pipeline/exec/jdbc_table_sink_operator.h | 9 +-
be/src/pipeline/exec/join_build_sink_operator.h | 2 +-
.../pipeline/exec/multi_cast_data_stream_sink.cpp | 30 +
be/src/pipeline/exec/multi_cast_data_stream_sink.h | 27 +-
.../exec/multi_cast_data_stream_source.cpp | 4 +-
be/src/pipeline/exec/multi_cast_data_streamer.h | 8 -
.../exec/nested_loop_join_build_operator.cpp | 33 +-
be/src/pipeline/exec/olap_scan_operator.cpp | 91 +-
be/src/pipeline/exec/olap_scan_operator.h | 20 +-
be/src/pipeline/exec/olap_table_sink_operator.h | 2 +-
be/src/pipeline/exec/olap_table_sink_v2_operator.h | 2 +-
be/src/pipeline/exec/operator.cpp | 25 +-
be/src/pipeline/exec/operator.h | 15 +-
.../pipeline/exec/partition_sort_sink_operator.cpp | 94 +-
.../pipeline/exec/partition_sort_sink_operator.h | 188 ++-
.../exec/partitioned_aggregation_sink_operator.h | 4 +
be/src/pipeline/exec/result_file_sink_operator.cpp | 37 +-
be/src/pipeline/exec/result_file_sink_operator.h | 15 +-
be/src/pipeline/exec/result_sink_operator.cpp | 4 +-
be/src/pipeline/exec/result_sink_operator.h | 86 +-
be/src/pipeline/exec/scan_operator.cpp | 173 ++-
be/src/pipeline/exec/scan_operator.h | 76 +-
be/src/pipeline/exec/schema_scan_operator.h | 2 +-
be/src/pipeline/exec/set_sink_operator.cpp | 4 +-
be/src/pipeline/exec/set_source_operator.cpp | 4 +-
be/src/pipeline/exec/sort_sink_operator.cpp | 5 +-
be/src/pipeline/exec/sort_sink_operator.h | 4 +-
be/src/pipeline/exec/spill_sort_sink_operator.h | 3 +
be/src/pipeline/exec/table_function_operator.h | 2 +-
.../local_exchange_sink_operator.cpp | 53 +
.../local_exchange/local_exchange_sink_operator.h | 54 +-
.../local_exchange_source_operator.cpp | 8 +
be/src/pipeline/local_exchange/local_exchanger.cpp | 26 +
be/src/pipeline/local_exchange/local_exchanger.h | 4 +
be/src/pipeline/pipeline.cpp | 9 +
be/src/pipeline/pipeline_fragment_context.cpp | 40 +-
be/src/pipeline/pipeline_fragment_context.h | 8 +
be/src/pipeline/pipeline_task.cpp | 60 +-
be/src/pipeline/pipeline_task.h | 17 +-
be/src/pipeline/pipeline_tracing.cpp | 48 +-
be/src/pipeline/pipeline_tracing.h | 16 +-
be/src/runtime/buffer_control_block.cpp | 245 ++-
be/src/runtime/buffer_control_block.h | 51 +-
be/src/runtime/decimalv2_value.cpp | 5 +-
be/src/runtime/descriptors.cpp | 5 +
be/src/runtime/descriptors.h | 8 +-
be/src/runtime/exec_env.h | 4 +-
be/src/runtime/external_scan_context_mgr.cpp | 9 +-
be/src/runtime/external_scan_context_mgr.h | 3 +-
be/src/runtime/fragment_mgr.cpp | 376 +----
be/src/runtime/fragment_mgr.h | 19 +-
be/src/runtime/group_commit_mgr.cpp | 66 +-
be/src/runtime/group_commit_mgr.h | 9 +-
be/src/runtime/load_channel.cpp | 25 +-
be/src/runtime/load_channel.h | 9 +-
be/src/runtime/load_channel_mgr.cpp | 8 -
be/src/runtime/load_stream.cpp | 2 +-
be/src/runtime/load_stream.h | 4 +
be/src/runtime/memory/thread_mem_tracker_mgr.h | 8 +-
be/src/runtime/plan_fragment_executor.cpp | 664 ---------
be/src/runtime/plan_fragment_executor.h | 280 ----
be/src/runtime/query_context.cpp | 96 +-
be/src/runtime/query_context.h | 27 +-
be/src/runtime/result_buffer_mgr.cpp | 8 +-
be/src/runtime/result_buffer_mgr.h | 3 +-
be/src/runtime/result_writer.h | 2 -
be/src/runtime/runtime_query_statistics_mgr.cpp | 157 +-
be/src/runtime/runtime_query_statistics_mgr.h | 42 +-
be/src/runtime/runtime_state.h | 4 +
be/src/runtime/tablets_channel.cpp | 43 +-
be/src/runtime/tablets_channel.h | 9 +-
be/src/runtime/thread_context.h | 6 -
.../workload_management/workload_action.cpp | 11 +-
.../workload_management/workload_condition.cpp | 13 +
.../workload_management/workload_condition.h | 17 +-
.../workload_management/workload_query_info.h | 2 +
.../workload_management/workload_sched_policy.cpp | 2 +
be/src/service/backend_options.cpp | 16 +-
be/src/service/backend_options.h | 5 +-
be/src/service/backend_service.cpp | 77 +-
be/src/service/http_service.cpp | 8 +
be/src/service/internal_service.cpp | 34 +-
be/src/util/CMakeLists.txt | 2 +
be/src/util/bitmap.h | 4 +-
be/src/util/bitmap_value.h | 413 +-----
be/src/util/block_compression.cpp | 4 +-
be/src/util/faststring.h | 4 +-
be/src/util/frame_of_reference_coding.h | 15 +-
be/src/util/jni-util.cpp | 4 +-
be/src/util/jvm_metrics.cpp | 112 +-
be/src/util/jvm_metrics.h | 16 +-
be/src/util/memcpy_inlined.h | 147 ++
be/src/util/proto_util.h | 11 +-
be/src/util/s3_util.cpp | 12 +-
be/src/util/timezone_utils.cpp | 6 +-
.../aggregate_function_approx_count_distinct.cpp | 1 -
be/src/vec/columns/column.h | 4 +-
be/src/vec/columns/column_array.cpp | 12 +-
be/src/vec/columns/column_array.h | 18 +-
be/src/vec/columns/column_complex.h | 30 +-
be/src/vec/columns/column_decimal.h | 1 +
be/src/vec/columns/column_nullable.cpp | 1 +
be/src/vec/columns/column_nullable.h | 3 +-
be/src/vec/columns/column_object.cpp | 2 +-
be/src/vec/columns/column_string.cpp | 16 +-
be/src/vec/columns/column_string.h | 1 +
be/src/vec/columns/column_vector.cpp | 12 +-
be/src/vec/columns/column_vector.h | 1 +
be/src/vec/common/hash_table/hash_map_context.h | 11 +-
be/src/vec/common/hash_table/hash_table.h | 3 +-
.../vec/common/hash_table/hash_table_set_build.h | 1 -
.../vec/common/hash_table/hash_table_set_probe.h | 1 -
be/src/vec/common/pod_array.h | 158 +-
be/src/vec/common/pod_array_fwd.h | 8 +-
be/src/vec/common/string_ref.h | 8 +-
be/src/vec/core/block.cpp | 57 +-
be/src/vec/core/block.h | 7 +-
be/src/vec/core/field.h | 15 +-
be/src/vec/data_types/data_type_hll.h | 12 +-
.../serde/data_type_datetimev2_serde.cpp | 2 +-
.../data_types/serde/data_type_nullable_serde.cpp | 10 +-
be/src/vec/exec/format/json/new_json_reader.cpp | 187 +--
be/src/vec/exec/format/json/new_json_reader.h | 6 +-
be/src/vec/exec/format/orc/vorc_reader.cpp | 18 +-
.../format/parquet/byte_array_dict_decoder.cpp | 10 +-
.../format/parquet/fix_length_dict_decoder.hpp | 4 +-
.../exec/format/table/iceberg/partition_spec.cpp | 71 +
.../vec/exec/format/table/iceberg/partition_spec.h | 97 ++
.../format/table/iceberg/partition_spec_parser.cpp | 67 +
.../format/table/iceberg/partition_spec_parser.h | 56 +
be/src/vec/exec/format/table/iceberg/schema.cpp | 53 +
be/src/vec/exec/format/table/iceberg/schema.h | 55 +
.../exec/format/table/iceberg/schema_parser.cpp | 163 ++
.../vec/exec/format/table/iceberg/schema_parser.h | 68 +
be/src/vec/exec/format/table/iceberg/struct_like.h | 37 +
be/src/vec/exec/format/table/iceberg/types.cpp | 194 +++
be/src/vec/exec/format/table/iceberg/types.h | 403 +++++
.../table/iceberg/unbound_partition_spec.cpp | 75 +
.../format/table/iceberg/unbound_partition_spec.h | 75 +
be/src/vec/exec/jni_connector.cpp | 38 +-
be/src/vec/exec/join/process_hash_table_probe.h | 9 +-
.../vec/exec/join/process_hash_table_probe_impl.h | 158 +-
be/src/vec/exec/join/vhash_join_node.cpp | 1041 -------------
be/src/vec/exec/join/vhash_join_node.h | 216 +--
be/src/vec/exec/join/vjoin_node_base.cpp | 350 -----
be/src/vec/exec/join/vjoin_node_base.h | 168 ---
be/src/vec/exec/join/vnested_loop_join_node.cpp | 723 ---------
be/src/vec/exec/join/vnested_loop_join_node.h | 268 ----
be/src/vec/exec/runtime_filter_consumer.cpp | 6 +-
be/src/vec/exec/runtime_filter_consumer.h | 1 -
be/src/vec/exec/scan/group_commit_scan_node.cpp | 73 -
be/src/vec/exec/scan/group_commit_scan_node.h | 49 -
be/src/vec/exec/scan/new_es_scan_node.cpp | 177 ---
be/src/vec/exec/scan/new_es_scan_node.h | 91 --
be/src/vec/exec/scan/new_es_scanner.cpp | 34 +-
be/src/vec/exec/scan/new_es_scanner.h | 5 -
be/src/vec/exec/scan/new_file_scan_node.cpp | 134 --
be/src/vec/exec/scan/new_file_scan_node.h | 73 -
be/src/vec/exec/scan/new_jdbc_scan_node.cpp | 75 -
be/src/vec/exec/scan/new_jdbc_scan_node.h | 56 -
be/src/vec/exec/scan/new_jdbc_scanner.cpp | 31 +-
be/src/vec/exec/scan/new_jdbc_scanner.h | 3 -
be/src/vec/exec/scan/new_odbc_scan_node.cpp | 74 -
be/src/vec/exec/scan/new_odbc_scan_node.h | 60 -
be/src/vec/exec/scan/new_odbc_scanner.cpp | 241 ---
be/src/vec/exec/scan/new_odbc_scanner.h | 85 --
be/src/vec/exec/scan/new_olap_scan_node.cpp | 681 ---------
be/src/vec/exec/scan/new_olap_scan_node.h | 222 ---
be/src/vec/exec/scan/new_olap_scanner.cpp | 96 +-
be/src/vec/exec/scan/new_olap_scanner.h | 10 +-
be/src/vec/exec/scan/pip_scanner_context.h | 88 --
be/src/vec/exec/scan/scanner_context.cpp | 119 +-
be/src/vec/exec/scan/scanner_context.h | 34 +-
be/src/vec/exec/scan/scanner_scheduler.cpp | 16 +-
be/src/vec/exec/scan/vfile_scanner.cpp | 78 +-
be/src/vec/exec/scan/vfile_scanner.h | 20 +-
be/src/vec/exec/scan/vmeta_scan_node.cpp | 84 --
be/src/vec/exec/scan/vmeta_scan_node.h | 65 -
be/src/vec/exec/scan/vmeta_scanner.cpp | 10 -
be/src/vec/exec/scan/vmeta_scanner.h | 4 -
be/src/vec/exec/scan/vscan_node.cpp | 1303 ----------------
be/src/vec/exec/scan/vscan_node.h | 368 -----
be/src/vec/exec/scan/vscanner.cpp | 53 +-
be/src/vec/exec/scan/vscanner.h | 6 -
be/src/vec/exec/vaggregation_node.cpp | 1556 --------------------
be/src/vec/exec/vaggregation_node.h | 379 -----
be/src/vec/exec/vanalytic_eval_node.cpp | 825 -----------
be/src/vec/exec/vanalytic_eval_node.h | 194 ---
be/src/vec/exec/vassert_num_rows_node.cpp | 155 --
be/src/vec/exec/vassert_num_rows_node.h | 51 -
be/src/vec/exec/vdata_gen_scan_node.cpp | 137 --
be/src/vec/exec/vdata_gen_scan_node.h | 71 -
be/src/vec/exec/vempty_set_node.cpp | 38 -
be/src/vec/exec/vempty_set_node.h | 40 -
be/src/vec/exec/vexchange_node.cpp | 145 --
be/src/vec/exec/vexchange_node.h | 76 -
be/src/vec/exec/vjdbc_connector.h | 15 +-
be/src/vec/exec/vmysql_scan_node.cpp | 234 ---
be/src/vec/exec/vmysql_scan_node.h | 83 --
be/src/vec/exec/vpartition_sort_node.cpp | 441 ------
be/src/vec/exec/vpartition_sort_node.h | 286 ----
be/src/vec/exec/vrepeat_node.cpp | 301 ----
be/src/vec/exec/vrepeat_node.h | 90 --
be/src/vec/exec/vschema_scan_node.cpp | 308 ----
be/src/vec/exec/vschema_scan_node.h | 81 -
be/src/vec/exec/vselect_node.cpp | 90 --
be/src/vec/exec/vselect_node.h | 46 -
be/src/vec/exec/vset_operation_node.cpp | 604 --------
be/src/vec/exec/vset_operation_node.h | 143 --
be/src/vec/exec/vsort_node.cpp | 205 ---
be/src/vec/exec/vsort_node.h | 106 --
be/src/vec/exec/vtable_function_node.cpp | 303 ----
be/src/vec/exec/vtable_function_node.h | 174 ---
be/src/vec/exec/vunion_node.cpp | 343 -----
be/src/vec/exec/vunion_node.h | 135 --
.../vec/exprs/table_function/vexplode_bitmap.cpp | 2 +-
be/src/vec/exprs/vectorized_fn_call.cpp | 37 +-
be/src/vec/exprs/vectorized_fn_call.h | 3 -
be/src/vec/exprs/vexpr.cpp | 39 +
be/src/vec/exprs/vexpr.h | 8 +
be/src/vec/exprs/vin_predicate.cpp | 13 +
be/src/vec/exprs/vin_predicate.h | 2 +
be/src/vec/exprs/vliteral.h | 2 +
.../vec/functions/array/function_array_element.h | 4 +-
be/src/vec/functions/array/function_array_index.h | 23 +-
.../vec/functions/array/function_array_popback.cpp | 6 +-
.../functions/array/function_array_register.cpp | 2 +
.../vec/functions/array/function_array_split.cpp | 157 ++
be/src/vec/functions/function.h | 3 +-
be/src/vec/functions/function_bit_shift.cpp | 16 +-
be/src/vec/functions/function_coalesce.cpp | 1 -
be/src/vec/functions/function_fake.h | 26 +-
be/src/vec/functions/function_ip.cpp | 2 +
be/src/vec/functions/function_ip.h | 223 +--
be/src/vec/json/parse2column.cpp | 6 +-
be/src/vec/json/path_in_data.cpp | 14 +-
be/src/vec/jsonb/serialize.cpp | 1 -
be/src/vec/olap/olap_data_convertor.cpp | 19 +-
be/src/vec/olap/olap_data_convertor.h | 4 +-
be/src/vec/runtime/vdata_stream_recvr.cpp | 17 +-
be/src/vec/runtime/vdata_stream_recvr.h | 5 +
be/src/vec/runtime/vorc_transformer.cpp | 110 +-
be/src/vec/runtime/vorc_transformer.h | 16 +-
be/src/vec/runtime/vparquet_transformer.cpp | 24 +-
be/src/vec/runtime/vparquet_transformer.h | 5 +-
be/src/vec/sink/async_writer_sink.h | 137 --
be/src/vec/sink/group_commit_block_sink.cpp | 332 -----
be/src/vec/sink/group_commit_block_sink.h | 86 --
be/src/vec/sink/load_stream_map_pool.cpp | 11 +-
be/src/vec/sink/load_stream_map_pool.h | 5 +-
be/src/vec/sink/load_stream_stub.cpp | 13 +-
be/src/vec/sink/load_stream_stub.h | 11 +-
be/src/vec/sink/multi_cast_data_stream_sink.h | 59 -
be/src/vec/sink/varrow_flight_result_writer.cpp | 4 -
be/src/vec/sink/varrow_flight_result_writer.h | 2 -
be/src/vec/sink/vdata_stream_sender.cpp | 518 -------
be/src/vec/sink/vdata_stream_sender.h | 202 +--
be/src/vec/sink/vhive_table_sink.cpp | 48 -
be/src/vec/sink/vhive_table_sink.h | 52 -
be/src/vec/sink/vmemory_scratch_sink.cpp | 117 --
be/src/vec/sink/vmemory_scratch_sink.h | 77 -
be/src/vec/sink/vmysql_result_writer.cpp | 6 -
be/src/vec/sink/vmysql_result_writer.h | 2 -
be/src/vec/sink/volap_table_sink.cpp | 136 --
be/src/vec/sink/volap_table_sink.h | 96 --
be/src/vec/sink/volap_table_sink_v2.cpp | 56 -
be/src/vec/sink/volap_table_sink_v2.h | 66 -
be/src/vec/sink/vresult_file_sink.cpp | 149 --
be/src/vec/sink/vresult_file_sink.h | 83 --
be/src/vec/sink/vresult_sink.cpp | 187 ---
be/src/vec/sink/vresult_sink.h | 163 --
be/src/vec/sink/writer/async_result_writer.h | 7 -
be/src/vec/sink/writer/iceberg/partition_data.h | 51 +
.../sink/writer/iceberg/partition_transformers.cpp | 149 ++
.../sink/writer/iceberg/partition_transformers.h | 81 +
.../writer/iceberg/viceberg_partition_writer.cpp | 210 +++
.../writer/iceberg/viceberg_partition_writer.h | 116 ++
.../sink/writer/iceberg/viceberg_table_writer.cpp | 493 +++++++
.../sink/writer/iceberg/viceberg_table_writer.h | 151 ++
be/src/vec/sink/writer/vfile_result_writer.cpp | 4 +-
be/src/vec/sink/writer/vfile_result_writer.h | 8 +-
be/src/vec/sink/writer/vhive_partition_writer.cpp | 6 +-
be/src/vec/sink/writer/vhive_utils.h | 5 +-
be/src/vec/sink/writer/vtablet_writer.cpp | 126 +-
be/src/vec/sink/writer/vtablet_writer.h | 68 +-
be/src/vec/sink/writer/vtablet_writer_v2.cpp | 60 +-
be/src/vec/sink/writer/vtablet_writer_v2.h | 2 +
be/src/vec/spill/spill_stream.cpp | 25 +-
be/src/vec/spill/spill_stream.h | 8 +-
be/src/vec/spill/spill_stream_manager.cpp | 59 +-
be/src/vec/spill/spill_stream_manager.h | 4 +-
be/src/vec/spill/spill_writer.h | 4 +-
be/test/http/http_client_test.cpp | 96 ++
be/test/io/fs/s3_file_writer_test.cpp | 1272 +++++++++++-----
be/test/olap/delete_bitmap_calculator_test.cpp | 2 +-
be/test/olap/delete_handler_test.cpp | 2 +-
be/test/olap/ordered_data_compaction_test.cpp | 4 +-
be/test/olap/path_gc_test.cpp | 12 +-
be/test/olap/rowid_conversion_test.cpp | 4 +-
be/test/olap/rowset/beta_rowset_test.cpp | 27 +-
.../rowset/segment_v2/binary_dict_page_test.cpp | 4 +
.../rowset/segment_v2/binary_plain_page_test.cpp | 2 +
.../rowset/segment_v2/binary_prefix_page_test.cpp | 6 +
.../rowset/segment_v2/bitshuffle_page_test.cpp | 4 +
.../bloom_filter_index_reader_writer_test.cpp | 2 +-
.../segment_v2/frame_of_reference_page_test.cpp | 8 +
.../segment_v2/inverted_index_array_test.cpp | 24 +-
be/test/olap/rowset/segment_v2/plain_page_test.cpp | 4 +
be/test/olap/rowset/segment_v2/rle_page_test.cpp | 6 +
be/test/olap/single_compaction_test.cpp | 4 +-
be/test/olap/tablet_cooldown_test.cpp | 13 +-
be/test/olap/tablet_mgr_test.cpp | 4 +-
be/test/olap/tablet_test.cpp | 16 +-
be/test/runtime/load_stream_test.cpp | 4 +-
be/test/runtime/mem_limit_test.cpp | 57 -
be/test/runtime/memory/mem_tracker_test.cpp | 57 +
.../runtime/memory/thread_mem_tracker_mgr_test.cpp | 455 ++++++
be/test/testutil/mock_rowset.h | 10 +-
be/test/testutil/run_all_tests.cpp | 1 +
be/test/util/bitmap_value_test.cpp | 178 +--
be/test/util/mysql_row_buffer_test.cpp | 3 -
be/test/vec/common/pod_array_test.cpp | 626 ++++++++
be/test/vec/core/column_complex_test.cpp | 9 +-
.../table/iceberg/partition_spec_parser_test.cpp | 87 ++
.../format/table/iceberg/schema_parser_test.cpp | 264 ++++
.../vec/exec/format/table/iceberg/schema_test.cpp | 70 +
.../vec/exec/format/table/iceberg/type_test.cpp | 176 +++
be/test/vec/exec/vtablet_sink_test.cpp | 858 -----------
be/test/vec/exec/vwal_scanner_test.cpp | 27 +-
be/test/vec/function/function_ip_test.cpp | 10 +-
be/test/vec/olap/vertical_compaction_test.cpp | 4 +-
be/test/vec/runtime/vdata_stream_test.cpp | 221 ---
bin/start_be.sh | 115 +-
bin/start_fe.sh | 26 +-
build.sh | 2 +-
cloud/src/meta-service/http_encode_key.cpp | 59 +-
.../src/meta-service/meta_service_tablet_stats.cpp | 98 +-
cloud/src/meta-service/meta_service_tablet_stats.h | 5 +
cloud/src/recycler/obj_store_accessor.h | 57 +
cloud/src/recycler/s3_accessor.cpp | 389 +----
cloud/src/recycler/s3_accessor.h | 6 +-
cloud/src/recycler/s3_obj_client.cpp | 371 +++++
cloud/src/recycler/s3_obj_client.h | 57 +
cloud/test/http_encode_key_test.cpp | 36 +-
cloud/test/s3_accessor_test.cpp | 18 +
.../docker-compose/hive/hive-2x.yaml.tpl | 13 +
.../docker-compose/hive/hive-3x.yaml.tpl | 13 +
.../hive/scripts/create_preinstalled_table.hql | 21 +-
docker/thirdparties/run-thirdparties-docker.sh | 15 +-
fe/be-java-extensions/avro-scanner/pom.xml | 31 -
fe/fe-common/pom.xml | 20 +
.../org/apache/doris/catalog/MysqlColType.java | 15 +
.../org/apache/doris/catalog/PrimitiveType.java | 4 +
.../doris/cloud/security/UrlSecurityChecker.java | 3 +-
.../main/java/org/apache/doris/common/Config.java | 39 +-
.../java/org/apache/doris/common/LogUtils.java | 97 ++
.../doris/common/credentials/CloudCredential.java | 60 +
.../credentials/CloudCredentialWithEndpoint.java | 61 +
.../DataLakeAWSCredentialsProvider.java | 59 +
.../common/security/authentication/HadoopUGI.java | 1 +
.../authentication/SimpleAuthenticationConfig.java | 3 +-
fe/fe-core/pom.xml | 11 +-
.../antlr4/org/apache/doris/nereids/DorisLexer.g4 | 1 +
.../antlr4/org/apache/doris/nereids/DorisParser.g4 | 5 +-
fe/fe-core/src/main/cup/sql_parser.cup | 58 +-
.../src/main/java/org/apache/doris/DorisFE.java | 22 +-
.../doris/alter/MaterializedViewHandler.java | 3 -
.../java/org/apache/doris/analysis/ColumnDef.java | 84 +-
.../apache/doris/analysis/ColumnNullableType.java | 44 +
.../org/apache/doris/analysis/DateLiteral.java | 23 +-
.../java/org/apache/doris/analysis/IndexDef.java | 2 +-
.../doris/analysis/LambdaFunctionCallExpr.java | 31 +-
.../analysis/ModifyTablePropertiesClause.java | 17 +
.../apache/doris/analysis/ShowColumnStatsStmt.java | 29 +
.../org/apache/doris/analysis/ShowDataStmt.java | 13 +-
.../doris/analysis/ShowStoragePolicyUsingStmt.java | 78 +
.../apache/doris/analysis/ShowTableStatsStmt.java | 2 +-
.../org/apache/doris/backup/BackupHandler.java | 4 +-
.../java/org/apache/doris/backup/BackupJob.java | 44 +-
.../java/org/apache/doris/backup/RestoreJob.java | 6 +-
.../java/org/apache/doris/binlog/UpsertRecord.java | 4 +
.../doris/catalog/BuiltinScalarFunctions.java | 4 +
.../apache/doris/catalog/CatalogRecycleBin.java | 119 +-
.../apache/doris/catalog/CloudTabletStatMgr.java | 71 +-
.../apache/doris/catalog/ColocateGroupSchema.java | 5 +-
.../java/org/apache/doris/catalog/Database.java | 45 +-
.../main/java/org/apache/doris/catalog/Env.java | 62 +-
.../org/apache/doris/catalog/InternalSchema.java | 161 +-
.../doris/catalog/InternalSchemaInitializer.java | 2 +-
.../apache/doris/catalog/ListPartitionItem.java | 2 +-
.../java/org/apache/doris/catalog/OlapTable.java | 123 +-
.../org/apache/doris/catalog/PartitionKey.java | 7 +
.../apache/doris/catalog/RangePartitionItem.java | 6 +-
.../java/org/apache/doris/catalog/ResourceMgr.java | 5 +
.../java/org/apache/doris/catalog/S3Resource.java | 17 +-
.../apache/doris/catalog/TabletInvertedIndex.java | 3 +
.../org/apache/doris/catalog/TabletStatMgr.java | 37 +-
.../org/apache/doris/catalog/TempPartitions.java | 4 +
.../doris/clone/DynamicPartitionScheduler.java | 79 +-
.../cloud/datasource/CloudInternalCatalog.java | 28 +-
.../org/apache/doris/cloud/rpc/VersionHelper.java | 4 +-
.../transaction/CloudGlobalTransactionMgr.java | 29 +-
.../java/org/apache/doris/common/ErrorCode.java | 7 +-
.../java/org/apache/doris/common/Log4jConfig.java | 108 +-
.../java/org/apache/doris/common/ThriftServer.java | 61 +-
.../org/apache/doris/common/proc/DbsProcDir.java | 2 +-
.../doris/common/profile/ExecutionProfile.java | 250 +---
.../org/apache/doris/common/profile/Profile.java | 20 +-
.../org/apache/doris/common/util/DebugUtil.java | 24 +
.../org/apache/doris/common/util/DocGenerator.java | 9 +-
.../doris/common/util/DynamicPartitionUtil.java | 23 +-
.../org/apache/doris/common/util/JdkUtils.java | 3 +-
.../org/apache/doris/common/util/PlatformName.java | 33 -
.../apache/doris/common/util/ProfileManager.java | 7 +-
.../java/org/apache/doris/common/util/S3URI.java | 39 +-
.../java/org/apache/doris/common/util/S3Util.java | 2 +-
.../java/org/apache/doris/common/util/Util.java | 28 -
.../apache/doris/datasource/CatalogProperty.java | 2 +-
.../apache/doris/datasource/InternalCatalog.java | 110 +-
.../datasource/credentials/CloudCredential.java | 60 -
.../credentials/CloudCredentialWithEndpoint.java | 61 -
.../DataLakeAWSCredentialsProvider.java | 59 -
.../doris/datasource/hive/HMSCachedClient.java | 5 +
.../doris/datasource/hive/HMSExternalCatalog.java | 8 +
.../datasource/hive/HiveMetaStoreClientHelper.java | 71 -
.../doris/datasource/hive/HiveMetadataOps.java | 9 +-
.../hive/PostgreSQLJdbcHMSCachedClient.java | 5 +
.../datasource/hive/ThriftHMSCachedClient.java | 18 +-
.../datasource/iceberg/IcebergMetadataOps.java | 4 +
.../datasource/iceberg/IcebergTransaction.java | 14 +-
.../doris/datasource/iceberg/IcebergUtils.java | 3 +-
.../doris/datasource/iceberg/dlf/DLFCatalog.java | 2 +-
.../jdbc/source/JdbcFunctionPushDownRule.java | 4 +
.../datasource/lakesoul/LakeSoulExternalTable.java | 1 -
.../maxcompute/MaxComputeExternalCatalog.java | 2 +-
.../datasource/operations/ExternalMetadataOps.java | 5 +
.../datasource/property/PropertyConverter.java | 4 +-
.../property/constants/BaseProperties.java | 2 +-
.../property/constants/CosProperties.java | 2 +-
.../property/constants/DLFProperties.java | 2 +-
.../property/constants/GCSProperties.java | 2 +-
.../property/constants/GlueProperties.java | 2 +-
.../property/constants/MCProperties.java | 2 +-
.../property/constants/MinioProperties.java | 2 +-
.../property/constants/ObsProperties.java | 2 +-
.../property/constants/OssProperties.java | 2 +-
.../property/constants/PaimonProperties.java | 2 +-
.../property/constants/S3Properties.java | 6 +-
.../org/apache/doris/event/DataChangeEvent.java | 24 +
.../org/apache/doris/event/DropPartitionEvent.java | 24 +
.../main/java/org/apache/doris/event/Event.java | 60 +
.../org/apache/doris/event/EventException.java | 34 +
.../java/org/apache/doris/event/EventListener.java | 23 +
.../org/apache/doris/event/EventProcessor.java | 57 +
.../java/org/apache/doris/event/EventType.java | 24 +
.../apache/doris/event/ReplacePartitionEvent.java | 24 +
.../java/org/apache/doris/event/TableEvent.java | 52 +
.../java/org/apache/doris/fs/obj/S3ObjStorage.java | 2 +-
.../org/apache/doris/fs/remote/S3FileSystem.java | 5 +-
.../src/main/java/org/apache/doris/ha/BDBHA.java | 3 +
.../apache/doris/ha/BDBStateChangeListener.java | 4 +-
.../org/apache/doris/httpv2/rest/LoadAction.java | 43 +-
.../apache/doris/job/extensions/mtmv/MTMVJob.java | 29 +-
.../apache/doris/job/extensions/mtmv/MTMVTask.java | 1 +
.../java/org/apache/doris/journal/Journal.java | 2 +
.../apache/doris/journal/bdbje/BDBEnvironment.java | 3 +-
.../apache/doris/journal/bdbje/BDBJEJournal.java | 37 +-
.../doris/journal/bdbje/BDBJournalCursor.java | 6 +-
.../org/apache/doris/journal/bdbje/BDBTool.java | 27 +-
.../apache/doris/journal/local/LocalJournal.java | 7 +-
.../doris/journal/local/LocalJournalCursor.java | 4 +-
.../main/java/org/apache/doris/load/ExportMgr.java | 6 +-
.../org/apache/doris/load/StreamLoadHandler.java | 7 +
.../apache/doris/load/loadv2/BrokerLoadJob.java | 3 +-
.../load/loadv2/BrokerLoadingTaskAttachment.java | 9 +-
.../apache/doris/load/loadv2/LoadLoadingTask.java | 6 +-
.../doris/metric/PrometheusMetricVisitor.java | 13 +-
.../java/org/apache/doris/mtmv/BaseTableInfo.java | 20 +-
.../main/java/org/apache/doris/mtmv/MTMVCache.java | 20 +-
.../java/org/apache/doris/mtmv/MTMVJobManager.java | 19 +-
.../org/apache/doris/mtmv/MTMVRefreshEnum.java | 1 +
.../java/org/apache/doris/mtmv/MTMVService.java | 31 +-
.../org/apache/doris/nereids/NereidsPlanner.java | 7 +
.../org/apache/doris/nereids/StatementContext.java | 26 +
.../nereids/analyzer/UnboundTableSinkCreator.java | 3 +
.../org/apache/doris/nereids/cost/CostModelV1.java | 9 +-
.../nereids/cost/ExpressionCostEvaluator.java | 86 ++
.../glue/translator/PhysicalPlanTranslator.java | 14 +-
.../glue/translator/PlanTranslatorContext.java | 10 +-
.../doris/nereids/jobs/executor/Rewriter.java | 4 +-
.../nereids/jobs/rewrite/CustomRewriteJob.java | 5 -
.../doris/nereids/parser/LogicalPlanBuilder.java | 48 +-
.../processor/post/RuntimeFilterGenerator.java | 4 +
.../post/RuntimeFilterPushDownVisitor.java | 4 +
.../org/apache/doris/nereids/rules/RuleType.java | 1 +
.../doris/nereids/rules/analysis/BindRelation.java | 9 +-
.../nereids/rules/analysis/ExpressionAnalyzer.java | 8 +
.../nereids/rules/analysis/NormalizeRepeat.java | 6 +
.../nereids/rules/analysis/UserAuthentication.java | 18 +-
.../mv/AbstractMaterializedViewAggregateRule.java | 11 +-
.../mv/AbstractMaterializedViewJoinRule.java | 4 +-
.../mv/AbstractMaterializedViewRule.java | 324 ++--
.../mv/AsyncMaterializationContext.java | 17 +-
.../mv/InitMaterializationContextHook.java | 10 +-
.../exploration/mv/MaterializationContext.java | 115 +-
.../exploration/mv/MaterializedViewScanRule.java | 4 +-
.../exploration/mv/MaterializedViewUtils.java | 144 +-
.../nereids/rules/exploration/mv/Predicates.java | 52 -
.../nereids/rules/exploration/mv/StructInfo.java | 126 +-
.../rules/expression/ExpressionRewrite.java | 5 +
.../expression/rules/FoldConstantRuleOnBE.java | 2 +
.../expression/rules/FoldConstantRuleOnFE.java | 2 +
.../nereids/rules/rewrite/AdjustNullable.java | 4 +-
.../doris/nereids/rules/rewrite/ColumnPruning.java | 25 +
.../rules/rewrite/MergePercentileToArray.java | 218 +++
.../rules/rewrite/SimplifyWindowExpression.java | 2 +-
.../doris/nereids/stats/FilterEstimation.java | 127 +-
.../nereids/trees/expressions/Placeholder.java | 78 +
.../functions/AggCombinerFunctionBuilder.java | 14 +-
.../executable/DateTimeExtractAndTransform.java | 54 +-
.../functions/executable/ExecutableFunctions.java | 5 +-
.../functions/scalar/ArrayReverseSort.java | 2 +-
.../functions/scalar/ArrayReverseSplit.java | 81 +
.../expressions/functions/scalar/ArraySplit.java | 81 +
.../functions/scalar/Ipv6NumToString.java | 3 +-
.../functions/scalar/IsIpAddressInRange.java | 4 +-
.../functions/scalar/UnixTimestamp.java | 5 +-
.../trees/expressions/functions/table/Hdfs.java | 3 +-
.../trees/expressions/functions/table/Local.java | 3 +-
.../trees/expressions/functions/table/S3.java | 3 +-
.../nereids/trees/expressions/literal/Literal.java | 169 +++
.../expressions/literal/StringLikeLiteral.java | 16 +-
.../expressions/visitor/ExpressionVisitor.java | 5 +
.../expressions/visitor/ScalarFunctionVisitor.java | 10 +
.../doris/nereids/trees/plans/PlaceholderId.java | 58 +
.../apache/doris/nereids/trees/plans/PlanType.java | 6 +-
.../commands/DropCatalogRecycleBinCommand.java | 81 +
.../trees/plans/commands/ExecuteCommand.java | 83 ++
.../trees/plans/commands/PrepareCommand.java | 120 ++
.../plans/commands/UpdateMvByPartitionCommand.java | 128 +-
.../plans/commands/info/ColumnDefinition.java | 29 +-
.../trees/plans/commands/info/CreateMTMVInfo.java | 4 +
.../trees/plans/commands/info/IndexDefinition.java | 2 +-
.../plans/commands/insert/OlapInsertExecutor.java | 24 +-
.../nereids/trees/plans/logical/LogicalWindow.java | 2 +-
.../trees/plans/visitor/CommandVisitor.java | 5 +
.../org/apache/doris/nereids/types/DataType.java | 4 +
.../java/org/apache/doris/persist/EditLog.java | 24 +-
.../planner/BackendPartitionedSchemaScanNode.java | 9 +-
.../org/apache/doris/planner/OlapTableSink.java | 111 +-
.../java/org/apache/doris/planner/PlanNode.java | 1 +
.../doris/plugin/audit/AuditStreamLoader.java | 23 +-
.../java/org/apache/doris/policy/PolicyMgr.java | 106 ++
.../java/org/apache/doris/qe/ConnectContext.java | 13 +
.../java/org/apache/doris/qe/ConnectProcessor.java | 10 +-
.../main/java/org/apache/doris/qe/Coordinator.java | 15 +-
.../java/org/apache/doris/qe/GlobalVariable.java | 4 +
.../org/apache/doris/qe/MysqlConnectProcessor.java | 145 +-
.../org/apache/doris/qe/PrepareStmtContext.java | 12 +-
.../apache/doris/qe/PreparedStatementContext.java | 47 +
.../java/org/apache/doris/qe/SessionVariable.java | 17 +-
.../java/org/apache/doris/qe/ShowExecutor.java | 55 +-
.../java/org/apache/doris/qe/StmtExecutor.java | 110 +-
.../main/java/org/apache/doris/qe/VariableMgr.java | 16 +-
.../workloadschedpolicy/WorkloadActionMeta.java | 12 +-
.../workloadschedpolicy/WorkloadCondition.java | 2 +
.../WorkloadConditionBeScanBytes.java | 11 +-
.../WorkloadConditionBeScanRows.java | 11 +-
.../workloadschedpolicy/WorkloadConditionMeta.java | 14 +-
.../WorkloadConditionQueryBeMemory.java | 56 +
.../WorkloadConditionQueryTime.java | 11 +-
.../workloadschedpolicy/WorkloadMetricType.java | 2 +-
.../workloadschedpolicy/WorkloadSchedPolicy.java | 27 +-
.../WorkloadSchedPolicyMgr.java | 46 +-
.../apache/doris/service/FrontendServiceImpl.java | 20 +-
.../apache/doris/statistics/AnalysisManager.java | 90 +-
.../apache/doris/statistics/BaseAnalysisTask.java | 15 +-
.../apache/doris/statistics/ColumnStatistic.java | 14 +-
.../statistics/ColumnStatisticsCacheLoader.java | 7 +-
.../doris/statistics/ExternalAnalysisTask.java | 5 +
.../apache/doris/statistics/HMSAnalysisTask.java | 21 +
.../org/apache/doris/statistics/HistogramTask.java | 5 +
.../apache/doris/statistics/OlapAnalysisTask.java | 42 +
.../doris/statistics/PartitionColumnStatistic.java | 168 +++
.../PartitionColumnStatisticBuilder.java | 158 ++
.../PartitionColumnStatisticCacheKey.java | 82 ++
.../PartitionColumnStatisticCacheLoader.java | 76 +
.../org/apache/doris/statistics/Statistics.java | 13 +
.../doris/statistics/StatisticsAutoCollector.java | 2 +-
.../apache/doris/statistics/StatisticsCache.java | 49 +-
.../apache/doris/statistics/StatisticsCleaner.java | 80 +-
.../doris/statistics/StatisticsRepository.java | 97 +-
.../apache/doris/statistics/TableStatsMeta.java | 6 +-
.../doris/statistics/util/StatisticsUtil.java | 13 +-
.../tablefunction/HdfsTableValuedFunction.java | 7 +-
.../doris/tablefunction/S3TableValuedFunction.java | 22 +-
.../doris/transaction/DatabaseTransactionMgr.java | 94 +-
.../doris/transaction/PartitionCommitInfo.java | 13 +-
.../apache/doris/transaction/TransactionEntry.java | 4 +-
.../java/org/apache/doris/alter/AlterTest.java | 10 +-
.../org/apache/doris/analysis/ColumnDefTest.java | 4 +-
.../apache/doris/analysis/CreateTableStmtTest.java | 32 +-
.../org/apache/doris/analysis/DateLiteralTest.java | 15 +
.../org/apache/doris/catalog/AdminStmtTest.java | 2 +-
.../apache/doris/catalog/ColocateTableTest.java | 4 +-
.../doris/catalog/DropMaterializedViewTest.java | 158 ++
.../doris/catalog/DynamicPartitionTableTest.java | 20 +
.../doris/cluster/DecommissionBackendTest.java | 8 +-
.../apache/doris/common/util/DebugUtilTest.java | 45 +
.../org/apache/doris/common/util/S3URITest.java | 13 +
.../org/apache/doris/common/util/UnitTestUtil.java | 1 +
.../org/apache/doris/common/util/UtilTest.java | 46 -
.../doris/datasource/TestHMSCachedClient.java | 4 +
.../property/PropertyPassThroughTest.java | 68 +
.../nereids/cost/ExpressionCostEvaluatorTest.java | 87 ++
.../doris/nereids/minidump/MinidumpUtTest.java | 2 +
.../doris/nereids/mv/IdStatisticsMapTest.java | 16 +-
.../rules/analysis/NormalizeRepeatTest.java | 39 +-
.../exploration/mv/MaterializedViewUtilsTest.java | 103 ++
.../nereids/rules/expression/FoldConstantTest.java | 102 +-
.../rules/rewrite/MergePercentileToArrayTest.java | 55 +
.../doris/nereids/stats/FilterEstimationTest.java | 31 +-
.../doris/nereids/trees/plans/MarkJoinTest.java | 12 +-
.../doris/nereids/trees/plans/PlanVisitorTest.java | 12 +-
.../org/apache/doris/planner/QueryPlanTest.java | 10 +-
.../doris/statistics/util/StatisticsUtilTest.java | 14 +-
fe/pom.xml | 5 +
gensrc/proto/internal_service.proto | 2 +
gensrc/script/doris_builtins_functions.py | 50 +-
gensrc/thrift/BackendService.thrift | 1 +
gensrc/thrift/Descriptors.thrift | 1 +
gensrc/thrift/PlanNodes.thrift | 1 +
.../qe/query_regression/result/issue_5121.result | 2 +-
.../qe/query_regression/result/issue_8850.result | 2 +-
pytest/qe/query_regression/sql/issue_5121.sql | 2 +-
pytest/qe/query_regression/sql/issue_8850.sql | 2 +-
.../test_sys_bitmap/test_sys_bitmap_function.py | 2 +-
regression-test/common/table/bitmap_basic_agg.sql | 2 +-
regression-test/common/table/hll_basic_agg.sql | 2 +-
regression-test/conf/regression-conf.groovy | 11 +-
.../test_compaction_uniq_keys_row_store.out | 8 +-
.../data/correctness_p0/test_bit_shift_lagency.out | 8 +-
.../data/correctness_p0/test_bit_shift_nereids.out | 8 +-
.../datatype_p0/ip/test_ip_in_inverted_index.out | 866 +++++++++++
.../test_create_table_without_distribution.out | 6 -
.../test_export_table_with_label_retry.out | 205 +++
.../hive/test_hive_orc_add_column.out | 185 +++
.../iceberg/test_iceberg_filter.out | 54 +
.../iceberg/write/test_iceberg_write_insert.out | 233 +++
.../write/test_iceberg_write_partitions.out | 169 +++
.../kafka/test_trino_kafka_base.out | 43 +
regression-test/data/insert_p0/txn_insert.out | 25 +
.../array_contains/documents-1000.json | 1000 +++++++++++++
.../array_contains/test_char_replace_arr.out | 100 ++
.../array_contains/test_compound_arr.out} | 0
.../test_count_on_index_httplogs_arr.out | 46 +
.../array_contains/test_equal_on_fulltext_arr.out | 6 +
.../test_index_bkd_null_arr.out} | 0
.../array_contains/test_index_change_2_arr.out | 37 +
.../array_contains/test_index_change_4_arr.out | 31 +
.../array_contains/test_index_change_6_arr.out | 31 +
.../test_index_compaction_dup_keys_arr.out | 105 ++
.../test_index_compaction_null_arr.out | 1531 +++++++++++++++++++
.../test_index_compaction_unique_keys_arr.out} | 0
...ex_compaction_with_multi_index_segments_arr.out | 219 +++
.../array_contains/test_index_delete_arr.out | 16 +
.../test_index_empty_string_arr.out} | 0
.../array_contains/test_index_equal_select_arr.out | 245 +++
.../test_index_match_phrase_edge_arr.out | 21 +
.../array_contains/test_index_match_regexp_arr.out | 19 +
.../array_contains/test_index_null_arr.out | 11 +
.../array_contains/test_invalid_param.out | 115 ++
.../test_inverted_index_null_arr.out | 130 ++
.../array_contains/test_lowercase_arr.out | 136 ++
.../test_parser_with_none_stopwords.out | 115 ++
.../inverted_index_p0/test_compound_inlist.out | 60 +
.../data/inverted_index_p0/test_index_rqg_bug2.out | 19 +
.../test_s3_load_with_load_parallelism.out | 4 -
.../load_p0/stream_load/iterate_read_json.json | 3 +
.../data/load_p0/stream_load/test_json_load.out | 8 +
.../data/load_p0/stream_load/test_keyword.csv | 10 +
.../load_p0/stream_load/test_special_key_json.json | 1 +
.../stream_load/test_stream_load_keyword.out | 13 +
.../broker_load/test_s3_load_parallelism.out | 7 -
.../test_s3_load_with_load_parallelism.out | 10 +
regression-test/data/mtmv_p0/test_commit_mtmv.out | 40 +
.../data/mtmv_p0/test_truncate_table_mtmv.out | 12 +
.../data/mv_p0/test_mv_mow/test_mv_mow.out | 5 +
.../insert_into_table/partial_update_seq_col.out | Bin 1412 -> 1416 bytes
.../data/nereids_p0/join/test_mark_join.out | 3 +
.../merge_percentile_to_array.out | 43 +
.../nereids_rules_p0/mv/partition_mv_rewrite.out | 112 +-
.../sql/two_instance_correctness.out} | 0
.../data/point_query_p0/test_rowstore.out | 6 +
.../data/prepared_stmt_p0/prepared_stmt.out | 53 +-
.../data/query_p0/operator/test_set_operator.out | 12 +
.../array_functions/test_array_split.out | 60 +
.../datetime_functions/test_date_function.out | 18 +-
.../test_is_ip_address_in_range_function.out | 13 +-
.../test_partial_update_insert_seq_col.out | Bin 1412 -> 1416 bytes
.../partial_update/test_partial_update_seq_col.out | Bin 1411 -> 1415 bytes
.../test_partial_update_seq_col_delete.out | Bin 1526 -> 1530 bytes
regression-test/data/variant_p0/column_name.out | 17 +-
.../org/apache/doris/regression/suite/Suite.groovy | 97 +-
.../pipeline/cloud_p0/conf/be_custom.conf | 3 -
.../pipeline/cloud_p0/conf/fe_custom.conf | 2 -
.../cloud_p0/conf/regression-conf-custom.groovy | 3 +
.../pipeline/cloud_p1/conf/be_custom.conf | 3 -
.../pipeline/cloud_p1/conf/fe_custom.conf | 3 -
.../cloud_p1/conf/regression-conf-custom.groovy | 1 +
regression-test/pipeline/external/conf/be.conf | 3 +
.../pipeline/external/conf/regression-conf.groovy | 3 +
regression-test/pipeline/p0/conf/be.conf | 5 +-
regression-test/pipeline/p0/conf/fe.conf | 3 -
.../pipeline/p0/conf/regression-conf.groovy | 3 +
regression-test/pipeline/p1/conf/be.conf | 3 +
regression-test/pipeline/p1/conf/fe.conf | 4 +-
.../plugins/plugin_curl_requester.groovy | 4 +
.../plugins/plugin_must_contains.groovy | 30 +
.../account_p0/test_nereids_authentication.groovy | 10 +
.../alter_p2/test_alter_colocate_group.groovy | 42 +-
.../backup_restore/test_backup_restore_mtmv.groovy | 99 ++
.../test_backup_restore_with_many_tables.groovy | 95 ++
.../test_drop_catalog_recycle_bin.groovy | 135 ++
.../clone_p0/test_clone_missing_version.groovy | 88 ++
.../test_drop_clone_tablet_path_race.groovy | 85 ++
.../test_show_storage_policy_using.groovy | 164 +++
.../suites/compaction/test_base_compaction.groovy | 5 +
.../test_lateral_view_in_subquery.groovy | 2 +-
.../correctness_p0/test_bit_shift_nereids.groovy | 2 +-
.../correctness_p0/test_table_function.groovy | 2 +-
.../test_union_subquery_groupby.groovy | 2 +-
.../ip/test_ip_in_inverted_index.groovy | 124 ++
.../test_create_table_without_distribution.groovy | 8 +-
.../export_p0/export/test_show_export.groovy | 4 +-
.../outfile/csv/test_outfile_csv_array_type.groovy | 28 +-
.../csv/test_outfile_csv_complex_type.groovy | 14 +-
.../outfile/csv/test_outfile_csv_map_type.groovy | 36 +-
.../csv/test_outfile_csv_one_nested_type.groovy | 26 +-
.../outfile/csv/test_outfile_empty_data.groovy | 4 +-
.../orc/test_outfile_orc_one_nested_type.groovy | 26 +-
.../outfile/orc/test_outfile_orc_timestamp.groovy | 2 +-
.../test_outfile_expr_generate_col_name.groovy | 4 +-
.../parquet/test_outfile_parquet_array_type.groovy | 28 +-
.../test_outfile_parquet_complex_type.groovy | 14 +-
.../parquet/test_outfile_parquet_map_type.groovy | 36 +-
.../test_outfile_parquet_one_nested_type.groovy | 24 +-
.../suites/export_p0/test_export_data_types.groovy | 10 +-
.../suites/export_p0/test_export_orc.groovy | 2 +-
.../suites/export_p0/test_export_parquet.groovy | 2 +-
.../test_export_table_with_label_retry.groovy | 249 ++++
.../suites/export_p0/test_export_view.groovy | 2 +-
.../export_p0/test_outfile_file_suffix.groovy | 4 +-
.../export_p0/test_outfile_orc_array_type.groovy | 28 +-
.../export_p0/test_outfile_orc_complex_type.groovy | 14 +-
.../export_p0/test_outfile_orc_map_type.groovy | 36 +-
.../suites/export_p0/test_with_bom.groovy | 8 +-
.../export_p2/test_export_with_parallelism.groovy | 4 +-
.../suites/export_p2/test_export_with_s3.groovy | 2 +-
.../hive/ddl/test_hive_ctas.groovy | 3 +-
.../hive/ddl/test_hive_ddl.groovy | 3 +-
.../hive/ddl/test_hive_show_create_table.groovy | 3 +-
.../hive/ddl/test_hive_write_type.groovy | 27 +-
.../hive/test_hive_orc_add_column.groovy | 95 ++
.../write/test_hive_write_different_path.groovy | 6 +-
.../iceberg/test_iceberg_filter.groovy | 102 ++
.../iceberg/write/test_iceberg_write_insert.groovy | 848 +++++++++++
.../write/test_iceberg_write_partitions.groovy | 228 +++
.../jdbc/test_doris_jdbc_catalog.groovy | 120 +-
.../jdbc/test_mysql_jdbc_catalog.groovy | 5 +
.../kafka/data/trino_kafka_basic_data.csv | 20 +
...trino_kafka_base_db.trino_kafka_basic_data.json | 100 ++
.../kafka/test_trino_kafka_base.groovy | 93 ++
.../test_insert_from_tvf_with_common_user.groovy | 2 +-
.../external_table_p0/tvf/test_s3_tvf.groovy | 6 +-
.../tvf/test_s3_tvf_compression.groovy | 20 +-
.../tvf/test_s3_tvf_with_resource.groovy | 11 +-
.../tvf/test_path_partition_keys.groovy | 8 +-
.../test_full_compaciton_run_status.groovy | 5 +
.../suites/function_p0/test_agg_foreach.groovy | 4 +-
.../insert_p0/test_group_commit_variant.groovy | 43 +
regression-test/suites/insert_p0/txn_insert.groovy | 62 +-
regression-test/suites/insert_p2/txn_insert.groovy | 1 +
.../txn_insert_concurrent_insert_duplicate.groovy | 170 +++
.../txn_insert_concurrent_insert_ud.groovy | 183 +++
.../txn_insert_concurrent_insert_unique.groovy | 170 +++
.../txn_insert_concurrent_insert_update.groovy | 195 +++
.../test_add_drop_index_with_data_arr.groovy | 338 +++++
...test_add_drop_index_with_delete_data_arr.groovy | 273 ++++
.../array_contains/test_char_replace_arr.groovy | 103 ++
.../array_contains/test_compound_arr.groovy | 69 +
.../test_count_on_index_httplogs_arr.groovy | 204 +++
.../test_equal_on_fulltext_arr.groovy | 52 +
.../array_contains/test_index_bkd_null_arr.groovy | 52 +
.../array_contains/test_index_change_2_arr.groovy | 144 ++
.../array_contains/test_index_change_4_arr.groovy | 145 ++
.../array_contains/test_index_change_6_arr.groovy | 149 ++
.../test_index_compaction_dup_keys_arr.groovy | 250 ++++
.../test_index_compaction_null_arr.groovy | 329 +++++
.../test_index_compaction_unique_keys_arr.groovy | 255 ++++
...compaction_with_multi_index_segments_arr.groovy | 410 ++++++
.../array_contains/test_index_delete_arr.groovy | 80 +
.../test_index_empty_string_arr.groovy | 59 +
.../test_index_equal_select_arr.groovy | 247 ++++
.../test_index_match_phrase_edge_arr.groovy | 76 +
.../test_index_match_regexp_arr.groovy | 100 ++
.../array_contains/test_index_null_arr.groovy | 56 +
.../array_contains/test_invalid_param.groovy | 118 ++
.../test_inverted_index_null_arr.groovy | 100 ++
.../array_contains/test_lowercase_arr.groovy | 249 ++++
.../test_parser_with_none_stopwords.groovy | 118 ++
.../inverted_index_p0/test_compound_inlist.groovy | 76 +-
.../inverted_index_p0/test_index_rqg_bug2.groovy | 106 ++
.../suites/inverted_index_p0/test_show_data.groovy | 53 +-
.../javaudf_p0/test_javaudtf_all_types.groovy | 19 +-
.../suites/javaudf_p0/test_javaudtf_int.groovy | 5 +-
.../load_p0/broker_load/ddl/agg_tbl_basic.sql | 12 +-
.../load_p0/broker_load/test_etl_failed.groovy | 2 +-
.../test_s3_load_with_load_parallelism.groovy | 181 ---
.../routine_load/ddl/agg_tbl_basic_create.sql | 12 +-
.../stream_load/ddl/agg_tbl_basic_create.sql | 12 +-
.../load_p0/stream_load/test_json_load.groovy | 40 +-
.../stream_load/test_stream_load_keyword.groovy | 75 +
.../suites/load_p0/tvf/ddl/agg_tbl_basic_tvf.sql | 12 +-
.../suites/load_p0/tvf/test_tvf_error_url.groovy | 8 +-
.../broker_load/test_s3_load_parallelism.groovy | 99 --
.../test_s3_load_with_load_parallelism.groovy | 198 +++
.../suites/load_p2/test_large_data_by_rpc.groovy | 64 +
.../suites/mtmv_p0/test_build_mtmv.groovy | 6 +
.../suites/mtmv_p0/test_commit_mtmv.groovy | 130 ++
.../suites/mtmv_p0/test_limit_op_mtmv.groovy | 18 +
.../suites/mtmv_p0/test_storage_medium_mtmv.groovy | 80 +
.../suites/mtmv_p0/test_truncate_table_mtmv.groovy | 87 ++
.../suites/mv_p0/test_mv_mow/test_mv_mow.groovy | 47 +
.../mv_p0/test_mv_useless/mow_invalid.groovy | 40 -
.../where_invalid/where_invalid.groovy | 4 +-
.../suites/nereids_p0/aggregate/aggregate.groovy | 15 +
.../nereids_p0/join/test_bitmap_filter.groovy | 4 +-
.../suites/nereids_p0/join/test_mark_join.groovy | 2 +
.../nereids_p0/lateral_view/test_issue_8850.sql | 2 +-
.../test_aggregate_all_functions.groovy | 4 +-
.../bitmap_functions/test_bitmap_function.groovy | 10 +-
.../adjust_nullable/set_operation.groovy | 46 +
.../column_pruning/window_column_pruning.groovy | 60 +
.../grouping_sets/grouping_normalize_test.groovy | 15 +
.../merge_percentile_to_array.groovy | 60 +
.../agg_with_roll_up/aggregate_with_roll_up.groovy | 1 -
.../mv/availability/grace_period.groovy | 1 -
.../mv/dimension/dimension_1.groovy | 1 -
.../mv/dimension/dimension_2_3.groovy | 1 -
.../mv/dimension/dimension_2_4.groovy | 1 -
.../mv/dimension/dimension_2_5.groovy | 1 -
.../mv/dimension/dimension_2_6.groovy | 1 -
.../mv/dimension/dimension_2_full_join.groovy | 1 -
.../mv/dimension/dimension_2_inner_join.groovy | 1 -
.../mv/dimension/dimension_2_left_anti_join.groovy | 1 -
.../mv/dimension/dimension_2_left_join.groovy | 1 -
.../mv/dimension/dimension_2_left_semi_join.groovy | 1 -
.../dimension/dimension_2_right_anti_join.groovy | 1 -
.../mv/dimension/dimension_2_right_join.groovy | 1 -
.../dimension/dimension_2_right_semi_join.groovy | 1 -
.../mv/dimension/dimension_self_conn.groovy | 1 -
.../dimension_2_join_agg.groovy | 1 -
.../filter_equal_or_notequal.groovy | 1 -
.../mv_contain_external_table.groovy | 1 -
.../cross_join_list_str_increment_create.groovy | 1 -
.../cross_join_range_date_increment_create.groovy | 1 -
...cross_join_range_number_increment_create.groovy | 1 -
.../full_join_list_str_increment_create.groovy | 1 -
.../full_join_range_date_increment_create.groovy | 1 -
.../full_join_range_number_increment_create.groovy | 1 -
.../inner_join_list_str_increment_create.groovy | 1 -
.../inner_join_range_date_increment_create.groovy | 1 -
...inner_join_range_number_increment_create.groovy | 1 -
...left_anti_join_list_str_increment_create.groovy | 1 -
...ft_anti_join_range_date_increment_create.groovy | 1 -
..._anti_join_range_number_increment_create.groovy | 1 -
.../left_join_list_str_increment_create.groovy | 1 -
.../left_join_range_date_increment_create.groovy | 1 -
.../left_join_range_number_increment_create.groovy | 1 -
...left_semi_join_list_str_increment_create.groovy | 1 -
...ft_semi_join_range_date_increment_create.groovy | 1 -
..._semi_join_range_number_increment_create.groovy | 1 -
...ight_anti_join_list_str_increment_create.groovy | 1 -
...ht_anti_join_range_date_increment_create.groovy | 1 -
..._anti_join_range_number_increment_create.groovy | 1 -
.../right_join_list_str_increment_create.groovy | 1 -
.../right_join_range_date_increment_create.groovy | 1 -
...right_join_range_number_increment_create.groovy | 1 -
...ight_semi_join_list_str_increment_create.groovy | 1 -
...ht_semi_join_range_date_increment_create.groovy | 1 -
..._semi_join_range_number_increment_create.groovy | 1 -
.../self_conn_list_str_increment_create.groovy | 1 -
.../self_conn_range_date_increment_create.groovy | 1 -
.../self_conn_range_number_increment_create.groovy | 1 -
.../mv/join/dphyp_inner/inner_join_dphyp.groovy | 1 -
.../mv/join/dphyp_outer/outer_join_dphyp.groovy | 1 -
.../mv/join/inner/inner_join.groovy | 1 -
.../mv/join/left_outer/outer_join.groovy | 1 -
.../mv/negative/negative_test.groovy | 1 -
.../mv/nested/nested_materialized_view.groovy | 1 -
.../mv/nested_mtmv/nested_mtmv.groovy | 1 -
.../mv/partition_mv_rewrite.groovy | 306 +++-
.../nereids_rules_p0/mv/scan/scan_table.groovy | 1 -
.../partition_curd_union_rewrite.groovy | 1 -
.../mv/union_rewrite/usercase_union_rewrite.groovy | 1 -
.../suites/nereids_syntax_p0/lateral_view.groovy | 2 +-
.../suites/nereids_syntax_p0/runtime_filter.groovy | 16 +
.../tpch/push_filter_window_eqset.groovy | 2 +-
.../test_auto_range_partition.groovy | 3 +-
.../test_dynamic_partition_failed.groovy | 74 +
.../auto_partition/sql/multi_thread_load.groovy | 2 +-
.../sql/stress_test_insert_into.groovy | 2 +-
.../sql/two_instance_correctness.groovy | 45 +
.../plsql_p0/test_plsql_show_procedure.groovy | 16 +
.../suites/point_query_p0/test_point_query.groovy | 5 +-
.../suites/point_query_p0/test_rowstore.groovy | 9 +
.../suites/prepared_stmt_p0/prepared_stmt.groovy | 249 ++--
.../suites/query_p0/join/test_bitmap_filter.groovy | 4 +-
.../join/test_bitmap_filter_nereids.groovy | 4 +-
.../query_p0/lateral_view/lateral_view.groovy | 2 +-
.../query_p0/lateral_view/test_issue_8850.sql | 2 +-
.../query_p0/operator/test_set_operator.groovy | 8 +
.../test_aggregate_all_functions.groovy | 4 +-
.../array_functions/test_array_split.groovy | 81 +
.../bitmap_functions/test_bitmap_function.groovy | 20 +-
.../bitmap_functions/test_issue_27409.groovy | 2 +-
.../test_count_distinct_with_case_function.groovy | 4 +-
.../conditional_functions/test_nullif.groovy | 2 +-
.../datetime_functions/test_date_function.groovy | 6 +-
.../test_is_ip_address_in_range_function.groovy | 19 +-
.../suites/query_p0/test_data_type_marks.groovy | 4 +-
.../return_binaray/test_return_binaray_hll.groovy | 2 +-
.../test_return_binary_bitmap.groovy | 2 +-
.../query_profile/s3_load_profile_test.groovy | 10 +-
.../test_unique_schema_key_change_modify.groovy | 1044 +++++++++++++
.../test_unique_schema_key_change_modify1.groovy | 547 -------
.../suites/statistics/test_agg_complex_type.groovy | 2 +-
regression-test/suites/statistics/test_ddl.groovy | 16 -
.../suites/statistics/test_partition_stats.groovy | 257 ++++
.../statistics/test_show_partition_stats.groovy | 92 --
.../suites/variant_p0/column_name.groovy | 23 +-
.../test_workload_sched_policy.groovy | 33 +-
.../conf/mixed_query_test_conf.groovy | 46 -
.../workload_manager_p1/mixed_query_test.groovy | 197 ---
run-be-ut.sh | 29 +
samples/read_bitmap/java/ReadBitmap.java | 2 +-
thirdparty/CHANGELOG.md | 4 +
thirdparty/build-thirdparty.sh | 36 +
thirdparty/patches/libevent-1532.patch | 200 +++
...libevent-enable_keepalive_accepted_socket.patch | 17 +
thirdparty/vars.sh | 14 +
tools/find_libjvm.sh | 61 +
tools/pipeline-tracing/origin-to-show.py | 47 +-
1153 files changed, 38229 insertions(+), 31018 deletions(-)
delete mode 100644 be/src/exec/data_sink.cpp
delete mode 100644 be/src/exec/data_sink.h
delete mode 100644 be/src/exec/exec_node.cpp
delete mode 100644 be/src/exec/exec_node.h
delete mode 100644 be/src/exec/scan_node.cpp
delete mode 100644 be/src/exec/scan_node.h
delete mode 100644 be/src/gutil/strings/fastmem.h
create mode 100644 be/src/http/action/show_hotspot_action.cpp
create mode 100644 be/src/http/action/show_hotspot_action.h
create mode 100644 be/src/io/fs/obj_storage_client.h
create mode 100644 be/src/io/fs/s3_obj_storage_client.cpp
create mode 100644 be/src/io/fs/s3_obj_storage_client.h
create mode 100644 be/src/pipeline/exec/iceberg_table_sink_operator.cpp
create mode 100644 be/src/pipeline/exec/iceberg_table_sink_operator.h
delete mode 100644 be/src/runtime/plan_fragment_executor.cpp
delete mode 100644 be/src/runtime/plan_fragment_executor.h
create mode 100644 be/src/util/memcpy_inlined.h
create mode 100644 be/src/vec/exec/format/table/iceberg/partition_spec.cpp
create mode 100644 be/src/vec/exec/format/table/iceberg/partition_spec.h
create mode 100644
be/src/vec/exec/format/table/iceberg/partition_spec_parser.cpp
create mode 100644 be/src/vec/exec/format/table/iceberg/partition_spec_parser.h
create mode 100644 be/src/vec/exec/format/table/iceberg/schema.cpp
create mode 100644 be/src/vec/exec/format/table/iceberg/schema.h
create mode 100644 be/src/vec/exec/format/table/iceberg/schema_parser.cpp
create mode 100644 be/src/vec/exec/format/table/iceberg/schema_parser.h
create mode 100644 be/src/vec/exec/format/table/iceberg/struct_like.h
create mode 100644 be/src/vec/exec/format/table/iceberg/types.cpp
create mode 100644 be/src/vec/exec/format/table/iceberg/types.h
create mode 100644
be/src/vec/exec/format/table/iceberg/unbound_partition_spec.cpp
create mode 100644
be/src/vec/exec/format/table/iceberg/unbound_partition_spec.h
delete mode 100644 be/src/vec/exec/join/vhash_join_node.cpp
delete mode 100644 be/src/vec/exec/join/vjoin_node_base.cpp
delete mode 100644 be/src/vec/exec/join/vjoin_node_base.h
delete mode 100644 be/src/vec/exec/join/vnested_loop_join_node.cpp
delete mode 100644 be/src/vec/exec/join/vnested_loop_join_node.h
delete mode 100644 be/src/vec/exec/scan/group_commit_scan_node.cpp
delete mode 100644 be/src/vec/exec/scan/group_commit_scan_node.h
delete mode 100644 be/src/vec/exec/scan/new_es_scan_node.cpp
delete mode 100644 be/src/vec/exec/scan/new_es_scan_node.h
delete mode 100644 be/src/vec/exec/scan/new_file_scan_node.cpp
delete mode 100644 be/src/vec/exec/scan/new_file_scan_node.h
delete mode 100644 be/src/vec/exec/scan/new_jdbc_scan_node.cpp
delete mode 100644 be/src/vec/exec/scan/new_jdbc_scan_node.h
delete mode 100644 be/src/vec/exec/scan/new_odbc_scan_node.cpp
delete mode 100644 be/src/vec/exec/scan/new_odbc_scan_node.h
delete mode 100644 be/src/vec/exec/scan/new_odbc_scanner.cpp
delete mode 100644 be/src/vec/exec/scan/new_odbc_scanner.h
delete mode 100644 be/src/vec/exec/scan/new_olap_scan_node.cpp
delete mode 100644 be/src/vec/exec/scan/new_olap_scan_node.h
delete mode 100644 be/src/vec/exec/scan/pip_scanner_context.h
delete mode 100644 be/src/vec/exec/scan/vmeta_scan_node.cpp
delete mode 100644 be/src/vec/exec/scan/vmeta_scan_node.h
delete mode 100644 be/src/vec/exec/scan/vscan_node.cpp
delete mode 100644 be/src/vec/exec/vaggregation_node.cpp
delete mode 100644 be/src/vec/exec/vanalytic_eval_node.cpp
delete mode 100644 be/src/vec/exec/vanalytic_eval_node.h
delete mode 100644 be/src/vec/exec/vassert_num_rows_node.cpp
delete mode 100644 be/src/vec/exec/vassert_num_rows_node.h
delete mode 100644 be/src/vec/exec/vdata_gen_scan_node.cpp
delete mode 100644 be/src/vec/exec/vdata_gen_scan_node.h
delete mode 100644 be/src/vec/exec/vempty_set_node.cpp
delete mode 100644 be/src/vec/exec/vempty_set_node.h
delete mode 100644 be/src/vec/exec/vexchange_node.cpp
delete mode 100644 be/src/vec/exec/vexchange_node.h
delete mode 100644 be/src/vec/exec/vmysql_scan_node.cpp
delete mode 100644 be/src/vec/exec/vmysql_scan_node.h
delete mode 100644 be/src/vec/exec/vpartition_sort_node.cpp
delete mode 100644 be/src/vec/exec/vpartition_sort_node.h
delete mode 100644 be/src/vec/exec/vrepeat_node.cpp
delete mode 100644 be/src/vec/exec/vrepeat_node.h
delete mode 100644 be/src/vec/exec/vschema_scan_node.cpp
delete mode 100644 be/src/vec/exec/vschema_scan_node.h
delete mode 100644 be/src/vec/exec/vselect_node.cpp
delete mode 100644 be/src/vec/exec/vselect_node.h
delete mode 100644 be/src/vec/exec/vset_operation_node.cpp
delete mode 100644 be/src/vec/exec/vset_operation_node.h
delete mode 100644 be/src/vec/exec/vsort_node.cpp
delete mode 100644 be/src/vec/exec/vsort_node.h
delete mode 100644 be/src/vec/exec/vtable_function_node.cpp
delete mode 100644 be/src/vec/exec/vtable_function_node.h
delete mode 100644 be/src/vec/exec/vunion_node.cpp
delete mode 100644 be/src/vec/exec/vunion_node.h
create mode 100644 be/src/vec/functions/array/function_array_split.cpp
delete mode 100644 be/src/vec/sink/async_writer_sink.h
delete mode 100644 be/src/vec/sink/group_commit_block_sink.cpp
delete mode 100644 be/src/vec/sink/group_commit_block_sink.h
delete mode 100644 be/src/vec/sink/multi_cast_data_stream_sink.h
delete mode 100644 be/src/vec/sink/vhive_table_sink.cpp
delete mode 100644 be/src/vec/sink/vhive_table_sink.h
delete mode 100644 be/src/vec/sink/vmemory_scratch_sink.cpp
delete mode 100644 be/src/vec/sink/vmemory_scratch_sink.h
delete mode 100644 be/src/vec/sink/volap_table_sink.cpp
delete mode 100644 be/src/vec/sink/volap_table_sink.h
delete mode 100644 be/src/vec/sink/volap_table_sink_v2.cpp
delete mode 100644 be/src/vec/sink/volap_table_sink_v2.h
delete mode 100644 be/src/vec/sink/vresult_file_sink.cpp
delete mode 100644 be/src/vec/sink/vresult_file_sink.h
delete mode 100644 be/src/vec/sink/vresult_sink.cpp
delete mode 100644 be/src/vec/sink/vresult_sink.h
create mode 100644 be/src/vec/sink/writer/iceberg/partition_data.h
create mode 100644 be/src/vec/sink/writer/iceberg/partition_transformers.cpp
create mode 100644 be/src/vec/sink/writer/iceberg/partition_transformers.h
create mode 100644 be/src/vec/sink/writer/iceberg/viceberg_partition_writer.cpp
create mode 100644 be/src/vec/sink/writer/iceberg/viceberg_partition_writer.h
create mode 100644 be/src/vec/sink/writer/iceberg/viceberg_table_writer.cpp
create mode 100644 be/src/vec/sink/writer/iceberg/viceberg_table_writer.h
delete mode 100644 be/test/runtime/mem_limit_test.cpp
create mode 100644 be/test/runtime/memory/mem_tracker_test.cpp
create mode 100644 be/test/runtime/memory/thread_mem_tracker_mgr_test.cpp
create mode 100644 be/test/vec/common/pod_array_test.cpp
create mode 100644
be/test/vec/exec/format/table/iceberg/partition_spec_parser_test.cpp
create mode 100644 be/test/vec/exec/format/table/iceberg/schema_parser_test.cpp
create mode 100644 be/test/vec/exec/format/table/iceberg/schema_test.cpp
create mode 100644 be/test/vec/exec/format/table/iceberg/type_test.cpp
delete mode 100644 be/test/vec/exec/vtablet_sink_test.cpp
delete mode 100644 be/test/vec/runtime/vdata_stream_test.cpp
create mode 100644 cloud/src/recycler/s3_obj_client.cpp
create mode 100644 cloud/src/recycler/s3_obj_client.h
create mode 100644
fe/fe-common/src/main/java/org/apache/doris/common/LogUtils.java
create mode 100644
fe/fe-common/src/main/java/org/apache/doris/common/credentials/CloudCredential.java
create mode 100644
fe/fe-common/src/main/java/org/apache/doris/common/credentials/CloudCredentialWithEndpoint.java
create mode 100644
fe/fe-common/src/main/java/org/apache/doris/common/credentials/DataLakeAWSCredentialsProvider.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/analysis/ColumnNullableType.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/analysis/ShowStoragePolicyUsingStmt.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/common/util/PlatformName.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/credentials/CloudCredential.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/credentials/CloudCredentialWithEndpoint.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/credentials/DataLakeAWSCredentialsProvider.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/event/DataChangeEvent.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/event/DropPartitionEvent.java
create mode 100644 fe/fe-core/src/main/java/org/apache/doris/event/Event.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/event/EventException.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/event/EventListener.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/event/EventProcessor.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/event/EventType.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/event/ReplacePartitionEvent.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/event/TableEvent.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/cost/ExpressionCostEvaluator.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/MergePercentileToArray.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/Placeholder.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/scalar/ArrayReverseSplit.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/scalar/ArraySplit.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/PlaceholderId.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/DropCatalogRecycleBinCommand.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/ExecuteCommand.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/PrepareCommand.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/qe/PreparedStatementContext.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/resource/workloadschedpolicy/WorkloadConditionQueryBeMemory.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/statistics/PartitionColumnStatistic.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/statistics/PartitionColumnStatisticBuilder.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/statistics/PartitionColumnStatisticCacheKey.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/statistics/PartitionColumnStatisticCacheLoader.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/catalog/DropMaterializedViewTest.java
delete mode 100644
fe/fe-core/src/test/java/org/apache/doris/common/util/UtilTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/datasource/property/PropertyPassThroughTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/nereids/cost/ExpressionCostEvaluatorTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/nereids/rules/rewrite/MergePercentileToArrayTest.java
create mode 100644
regression-test/data/datatype_p0/ip/test_ip_in_inverted_index.out
create mode 100644
regression-test/data/export_p0/test_export_table_with_label_retry.out
create mode 100644
regression-test/data/external_table_p0/hive/test_hive_orc_add_column.out
create mode 100644
regression-test/data/external_table_p0/iceberg/test_iceberg_filter.out
create mode 100644
regression-test/data/external_table_p0/iceberg/write/test_iceberg_write_insert.out
create mode 100644
regression-test/data/external_table_p0/iceberg/write/test_iceberg_write_partitions.out
create mode 100644
regression-test/data/external_table_p0/trino_connector/kafka/test_trino_kafka_base.out
create mode 100644
regression-test/data/inverted_index_p0/array_contains/documents-1000.json
create mode 100644
regression-test/data/inverted_index_p0/array_contains/test_char_replace_arr.out
copy
regression-test/data/{fault_injection_p0/test_no_index_null_fault_injection.out
=> inverted_index_p0/array_contains/test_compound_arr.out} (100%)
create mode 100644
regression-test/data/inverted_index_p0/array_contains/test_count_on_index_httplogs_arr.out
create mode 100644
regression-test/data/inverted_index_p0/array_contains/test_equal_on_fulltext_arr.out
copy regression-test/data/inverted_index_p0/{test_index_null.out =>
array_contains/test_index_bkd_null_arr.out} (100%)
create mode 100644
regression-test/data/inverted_index_p0/array_contains/test_index_change_2_arr.out
create mode 100644
regression-test/data/inverted_index_p0/array_contains/test_index_change_4_arr.out
create mode 100644
regression-test/data/inverted_index_p0/array_contains/test_index_change_6_arr.out
create mode 100644
regression-test/data/inverted_index_p0/array_contains/test_index_compaction_dup_keys_arr.out
create mode 100644
regression-test/data/inverted_index_p0/array_contains/test_index_compaction_null_arr.out
copy
regression-test/data/inverted_index_p0/{index_compaction/test_index_compaction_unique_keys.out
=> array_contains/test_index_compaction_unique_keys_arr.out} (100%)
create mode 100644
regression-test/data/inverted_index_p0/array_contains/test_index_compaction_with_multi_index_segments_arr.out
create mode 100644
regression-test/data/inverted_index_p0/array_contains/test_index_delete_arr.out
copy regression-test/data/{nereids_syntax_p0/type_cast.out =>
inverted_index_p0/array_contains/test_index_empty_string_arr.out} (100%)
create mode 100644
regression-test/data/inverted_index_p0/array_contains/test_index_equal_select_arr.out
create mode 100644
regression-test/data/inverted_index_p0/array_contains/test_index_match_phrase_edge_arr.out
create mode 100644
regression-test/data/inverted_index_p0/array_contains/test_index_match_regexp_arr.out
create mode 100644
regression-test/data/inverted_index_p0/array_contains/test_index_null_arr.out
create mode 100644
regression-test/data/inverted_index_p0/array_contains/test_invalid_param.out
create mode 100644
regression-test/data/inverted_index_p0/array_contains/test_inverted_index_null_arr.out
create mode 100644
regression-test/data/inverted_index_p0/array_contains/test_lowercase_arr.out
create mode 100644
regression-test/data/inverted_index_p0/array_contains/test_parser_with_none_stopwords.out
create mode 100644
regression-test/data/inverted_index_p0/test_index_rqg_bug2.out
delete mode 100644
regression-test/data/load_p0/broker_load/test_s3_load_with_load_parallelism.out
create mode 100644
regression-test/data/load_p0/stream_load/iterate_read_json.json
create mode 100644 regression-test/data/load_p0/stream_load/test_keyword.csv
create mode 100644
regression-test/data/load_p0/stream_load/test_special_key_json.json
create mode 100644
regression-test/data/load_p0/stream_load/test_stream_load_keyword.out
delete mode 100644
regression-test/data/load_p1/broker_load/test_s3_load_parallelism.out
create mode 100644
regression-test/data/load_p2/broker_load/test_s3_load_with_load_parallelism.out
create mode 100644 regression-test/data/mtmv_p0/test_commit_mtmv.out
create mode 100644 regression-test/data/mtmv_p0/test_truncate_table_mtmv.out
create mode 100644 regression-test/data/mv_p0/test_mv_mow/test_mv_mow.out
create mode 100644
regression-test/data/nereids_rules_p0/merge_percentile_to_array/merge_percentile_to_array.out
copy regression-test/data/{datatype_p0/date/test_default_current_timestamp.out
=> partition_p1/auto_partition/sql/two_instance_correctness.out} (100%)
create mode 100644
regression-test/data/query_p0/sql_functions/array_functions/test_array_split.out
create mode 100644 regression-test/plugins/plugin_must_contains.groovy
create mode 100644
regression-test/suites/backup_restore/test_backup_restore_mtmv.groovy
create mode 100644
regression-test/suites/backup_restore_p2/test_backup_restore_with_many_tables.groovy
create mode 100644
regression-test/suites/catalog_recycle_bin_p0/test_drop_catalog_recycle_bin.groovy
create mode 100644
regression-test/suites/clone_p0/test_clone_missing_version.groovy
create mode 100644
regression-test/suites/clone_p0/test_drop_clone_tablet_path_race.groovy
create mode 100644
regression-test/suites/cold_heat_separation_p2/test_show_storage_policy_using.groovy
create mode 100644
regression-test/suites/datatype_p0/ip/test_ip_in_inverted_index.groovy
create mode 100644
regression-test/suites/export_p0/test_export_table_with_label_retry.groovy
create mode 100644
regression-test/suites/external_table_p0/hive/test_hive_orc_add_column.groovy
create mode 100644
regression-test/suites/external_table_p0/iceberg/test_iceberg_filter.groovy
create mode 100644
regression-test/suites/external_table_p0/iceberg/write/test_iceberg_write_insert.groovy
create mode 100644
regression-test/suites/external_table_p0/iceberg/write/test_iceberg_write_partitions.groovy
create mode 100644
regression-test/suites/external_table_p0/trino_connector/kafka/data/trino_kafka_basic_data.csv
create mode 100644
regression-test/suites/external_table_p0/trino_connector/kafka/table_desc/test_trino_kafka_base_db.trino_kafka_basic_data.json
create mode 100644
regression-test/suites/external_table_p0/trino_connector/kafka/test_trino_kafka_base.groovy
create mode 100644
regression-test/suites/insert_p0/test_group_commit_variant.groovy
create mode 100644
regression-test/suites/insert_p2/txn_insert_concurrent_insert_duplicate.groovy
create mode 100644
regression-test/suites/insert_p2/txn_insert_concurrent_insert_ud.groovy
create mode 100644
regression-test/suites/insert_p2/txn_insert_concurrent_insert_unique.groovy
create mode 100644
regression-test/suites/insert_p2/txn_insert_concurrent_insert_update.groovy
create mode 100644
regression-test/suites/inverted_index_p0/array_contains/test_add_drop_index_with_data_arr.groovy
create mode 100644
regression-test/suites/inverted_index_p0/array_contains/test_add_drop_index_with_delete_data_arr.groovy
create mode 100644
regression-test/suites/inverted_index_p0/array_contains/test_char_replace_arr.groovy
create mode 100644
regression-test/suites/inverted_index_p0/array_contains/test_compound_arr.groovy
create mode 100644
regression-test/suites/inverted_index_p0/array_contains/test_count_on_index_httplogs_arr.groovy
create mode 100644
regression-test/suites/inverted_index_p0/array_contains/test_equal_on_fulltext_arr.groovy
create mode 100644
regression-test/suites/inverted_index_p0/array_contains/test_index_bkd_null_arr.groovy
create mode 100644
regression-test/suites/inverted_index_p0/array_contains/test_index_change_2_arr.groovy
create mode 100644
regression-test/suites/inverted_index_p0/array_contains/test_index_change_4_arr.groovy
create mode 100644
regression-test/suites/inverted_index_p0/array_contains/test_index_change_6_arr.groovy
create mode 100644
regression-test/suites/inverted_index_p0/array_contains/test_index_compaction_dup_keys_arr.groovy
create mode 100644
regression-test/suites/inverted_index_p0/array_contains/test_index_compaction_null_arr.groovy
create mode 100644
regression-test/suites/inverted_index_p0/array_contains/test_index_compaction_unique_keys_arr.groovy
create mode 100644
regression-test/suites/inverted_index_p0/array_contains/test_index_compaction_with_multi_index_segments_arr.groovy
create mode 100644
regression-test/suites/inverted_index_p0/array_contains/test_index_delete_arr.groovy
create mode 100644
regression-test/suites/inverted_index_p0/array_contains/test_index_empty_string_arr.groovy
create mode 100644
regression-test/suites/inverted_index_p0/array_contains/test_index_equal_select_arr.groovy
create mode 100644
regression-test/suites/inverted_index_p0/array_contains/test_index_match_phrase_edge_arr.groovy
create mode 100644
regression-test/suites/inverted_index_p0/array_contains/test_index_match_regexp_arr.groovy
create mode 100644
regression-test/suites/inverted_index_p0/array_contains/test_index_null_arr.groovy
create mode 100644
regression-test/suites/inverted_index_p0/array_contains/test_invalid_param.groovy
create mode 100644
regression-test/suites/inverted_index_p0/array_contains/test_inverted_index_null_arr.groovy
create mode 100644
regression-test/suites/inverted_index_p0/array_contains/test_lowercase_arr.groovy
create mode 100644
regression-test/suites/inverted_index_p0/array_contains/test_parser_with_none_stopwords.groovy
create mode 100644
regression-test/suites/inverted_index_p0/test_index_rqg_bug2.groovy
delete mode 100644
regression-test/suites/load_p0/broker_load/test_s3_load_with_load_parallelism.groovy
create mode 100644
regression-test/suites/load_p0/stream_load/test_stream_load_keyword.groovy
delete mode 100644
regression-test/suites/load_p1/broker_load/test_s3_load_parallelism.groovy
create mode 100644
regression-test/suites/load_p2/broker_load/test_s3_load_with_load_parallelism.groovy
create mode 100644 regression-test/suites/load_p2/test_large_data_by_rpc.groovy
create mode 100644 regression-test/suites/mtmv_p0/test_commit_mtmv.groovy
create mode 100644
regression-test/suites/mtmv_p0/test_storage_medium_mtmv.groovy
create mode 100644
regression-test/suites/mtmv_p0/test_truncate_table_mtmv.groovy
create mode 100644 regression-test/suites/mv_p0/test_mv_mow/test_mv_mow.groovy
delete mode 100644
regression-test/suites/mv_p0/test_mv_useless/mow_invalid.groovy
create mode 100644
regression-test/suites/nereids_rules_p0/adjust_nullable/set_operation.groovy
create mode 100644
regression-test/suites/nereids_rules_p0/column_pruning/window_column_pruning.groovy
create mode 100644
regression-test/suites/nereids_rules_p0/merge_percentile_to_array/merge_percentile_to_array.groovy
create mode 100644
regression-test/suites/partition_p0/dynamic_partition/test_dynamic_partition_failed.groovy
create mode 100644
regression-test/suites/partition_p1/auto_partition/sql/two_instance_correctness.groovy
create mode 100644
regression-test/suites/query_p0/sql_functions/array_functions/test_array_split.groovy
delete mode 100644
regression-test/suites/schema_change_p0/test_unique_schema_key_change_modify1.groovy
create mode 100644
regression-test/suites/statistics/test_partition_stats.groovy
delete mode 100644
regression-test/suites/statistics/test_show_partition_stats.groovy
delete mode 100644
regression-test/suites/workload_manager_p1/conf/mixed_query_test_conf.groovy
delete mode 100644
regression-test/suites/workload_manager_p1/mixed_query_test.groovy
create mode 100644 thirdparty/patches/libevent-1532.patch
create mode 100644
thirdparty/patches/libevent-enable_keepalive_accepted_socket.patch
create mode 100755 tools/find_libjvm.sh
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
