This is an automated email from the ASF dual-hosted git repository.
kakachen pushed a change to branch nested_column_prune
in repository https://gitbox.apache.org/repos/asf/doris.git
discard 3fdc1f955d9 Reduce logs in release mode
discard d234ae2df72 code refactoring for external table.
discard 49d056e4cdd refresh cases out
discard fe7873a379a fix decimalv2's precision and scale
discard 5119904664c remove debug log
discard 05c28879f0b fix: missed pruned type with map/array types
discard 6572e52de83 Clean external table implementation's code.
discard 4f4b423b6ad use selected column ids for parquet merge io ranges.
discard 6fdbdbd020d update orc thirdparty lib commit.
discard 9efa81907b5 should not prune project nested column if the bottom table
can not prune, e.g. hive(textfile)
discard b2a048deff9 fix npe
discard 0279d9c5758 Fix some tests's bugs of external table.
discard 37beb93061f only support prune olapTable/iceberg/hive(parquet/orc)
discard cd95b8eed1d fix be ut
discard bbc61b4cdab support push project in aggregate/join
discard 4000802e5fb fix topN late materialzation access_path.
discard 9b00e6fa107 fix case error in BE
discard b7f17ea8820 fix "Index 2 out of bounds for length 2" when use tvf
discard 3dfbdea9e40 use slot->col_name_lower_case().
discard 28ba969f5b5 Fix parquet_thrift_test unit test.
discard 1e5c9614055 [Feature](catalog) Prune nested columns for hive/iceberg
parquet and orc reader.
discard 6ccc8ca260a support prune nested columns
add ba6cc42281f [Enhancement](regexp) Support zero-width assertions in
some regexp functions (#57643)
add 37f2bb590cf Revert "[debug](exchange) checks on both the sending and
receiving sides of the exchange." (#57740)
add 9ef1b91af25 [Improvement](block) remove Block::row_same_bit (#57522)
add 4104f3a25f7 [Fix](mysql) Disable renegotiation during TLS (#57631)
add b4fd3d9efda [chore](function) remove useless function set (#57590)
add 2facd9edc87 [fix](csv reader) fix wrong result when escape same as
enclose (#57632)
add f4304a33e9b [ut](be) add some ut test case for varbinary (#57070)
add 4ca9942407b [refine](expr)Mark expr's execution function as const.
(#57668)
add 7adc3be15ae [opt](catalog) Reduce the memory footprint of Column
(#57401)
add 030531a5888 [Refactor](exec) remove unless time var in block (#57737)
add ea8caee7bbe [improvement](BeConfig)Add Be config to control the public
or private endpoint (#57580)
add ed8ccaffb1e [fix](broker-load) Fix the COLUMNS FROM PATH feature
(#57309)
add f12f15ce529 [fix](delete) Delete should count down latch and clear an
agent task when failed (#57428)
add 13190cacbed [chore](log) modify wal_manager log level (#57760)
add 58b9123dead [feat](web)Adapted for Spring Boot 3 and Jetty 12 (#57710)
add 63a2e9622d8 [fix](filecache) fix stack-use-after-return when
submit_download_tasks (#57724)
add a33530863c5 [optimize](nereids) explain plan process print change
point (#57772)
add ab0ef01f22d [fix](case) catch profile HTTP request failure to avoid
flaky test (#57678)
add 891d626d5cb [fix](iceberg) fix querying iceberg partition table error
(#57571)
add 3582f868856 [fix](catalog) max_meta_object_cache_num config must > 0
(#57742)
add bb04d4da7b0 [fix](catalog) fix iceberg catalog NPE issue (#57696)
add dfbaed91833 [fix](jdbc) Set the correct jdbctable id for query tvf
(#57745)
add 0b654e042d3 [chore](function) let implicit castable signature be pure
nereids (#57641)
add 7397b1def38 [chore](dep)upgrade protobuf (#57784)
add bbded3ab346 [opt](scheduler) Improve Graceful Shutdown Behavior for BE
and FE, and Optimize Query Retry During BE Shutdown (#56601)
add 03228afc2bd [refactor](type) refactor mysql output , rename
write_column_to_mysql (#57634)
add f6f5df3ec33 [fix](planner) Optimize local shuffle with bucket shuffle
join (#57768)
add 2fa7275c3f5 [fix](sql cache) inverted_index_max_expansions should be
affectQueryResult (#57769)
add 080367da844 [fix](nereids)keep NamedExpression's name unchanged when
create an Alias in BindSink (#57670)
add 0066aaa7dc0 [chore](dep)upgrade hbase (#57810)
add 5e140cf85b3 [feat](snapshot) support snapshot chain compactor (#57801)
add a32376d906e [chore](cast) simplify CastExpr's code (#57775)
add dfe888e7630 [opt](mtmv) Support window function rewrite when
materialized view contains window function (#55066)
add 54d3a631bbf [fix](mtmv) Fix nested mv rewritten fail when bottom mv is
partitioned (#57558)
add c185cde367b [fix](mtmv) Fix rewrite fail when join conjuncts eliminate
and group by key eliminate both happened (#55674)
add 958c5aa1b18 [feature](mtmv) Materialized view can participate in
transparent query rewrite even when data changes occur in its non-partitioned
base tables (#56745)
add a284e2a5279 [fix](mtmv)Restricting the use of external views to create
MTMV (#57759)
add a2b15eff7e0 [debug](p0) add debug log for unstable p0 test case
(#57757)
add 80211771717 [fix](job) fix show job and task offset info (#57736)
add c6cb536c437 [fix](mtmv)Fix multi pct multi partition cols should
throws exception (#57713)
add 9433adea8a8 [chore](planner) remove useless code in PlanNode (#57764)
add bb73892fa7f [chore](type) remove useless code in type (#57823)
add ad4fe3d5329 [chore](cloud) Correct spelling succeed instance to
successor instance (#57746)
add 6b647f77234 [fix](nereids) When explain delete stmt need skip prune
predicate (#57817)
add 62087a23f07 [Chore](nereids) remove LabelName (#57835)
add b985364613f [feat](iceberg) Implement Iceberg `rewrite_data_files`
action for table optimization and compaction (#56413)
add 3a0cc6a5acf [fix](cloud)Not log stack information by Peer read (#57816)
add c9b2141dd9f [enhance](cloud) Get tablet load stats keys in batch
(#57820)
add 371cafabb8f [fix](cloud) Make versioned tablet stats and rowset meta
compatible with non-versioned (#57809)
add 5737fa3c0b3 [enhancement](filecache) add readsize and hitsize metrics
to file_cache statistics and be vars (#52212)
add c930b5350d3 [fix](restore) Fix null pointer crash in
commit_restore_job versioned write (#57844)
add 53ebb5cffbc [Feature] Support Doris Catalog (#55274)
add 009c3b552a3 [opt](cloud) optimize the logic of creating warmup
job(#57752)
add d64910d0600 [chore] (audit) generate digest string for
LogicalPlanAdapter (#57847)
add 1ad5626dc8e [opt](arm) Improve performance of vstring functions with
SVE intrinsics (#57507)
add 978cd44680d [Enhancement] support simple sql function for
factorial(from Hive) (#57144)
add 2804f3bff7a [chore](sensitive) Hide accessor and secret key for
create/alter storage vault (#57528)
add 70af5f0ca57 [chore](cloud) write data_rowset_ref_count_key during
create tablet (#57837)
add c80dd345c1c [Fix](Olap)Fixs uninitialized group_data_size in
CompactionSampleInfo (#57523)
add 3c32b854486 [fix](cloud) Do not recycle operation log during
WRITE_ONLY (#57848)
add ecdad9cc4aa [fix](mtmv) Fix multi materialized view input trigger
refresh fail when exists alias (#57681)
add 20302fe3038 [improve](ann index)Accumulate multiple small batches
before training (#57623)
add 162c78dd7d9 [refine](function) Remove unused code from the function
framework and optimize the default NULL-handling logic. (#57811)
add 6584a990f7b [feature](inverted index) add multi position PhraseQuery
support to search function (#57588)
add 0a4ae5d75c3 [fix] fix jdk image missing problem (#57695)
add 6438d029f04 [chore](log) return error to caller when get_tablet in
cloud mode (#57520)
add bb302d0472a [chore](thirdparty) Update unstable gnu mirror (#56075)
add 7a8804f8ba3 [fix](cloud) Fix read peer file cache some bugs (#57838)
add a4b96c17b13 [chore](Expr) clear expr code (#57855)
add 757075eab2b [feat](snapshot) check delete bitmap version (#57907)
add c4230a8b9a8 [enhancement](memory) add jvm information into memory
profile page in BE (#57917)
add b84d7268da9 [Chore](job) remove unless code for streaming job (#57873)
add 6241fa215de [optimize](coordinator) broadcast finished rpc to be when
query/load finished (#57739)
add e75df83c9fc [chore](function) remove useless trait
PropagateNullableOnDateV2LikeArgs (#57920)
add 40588432a7e [Feature](partition) Support new partition recycle
mechanism (#57013)
add 14a2496c58a [Bug](udf) fix the _udf_close_workers thread pool not stop
when exit (#57918)
add 7620b9ab7e0 [Fix](job) add callback for streaming job when load image
(#57852)
add 9b818dc991c [chore](ci) exclude cases to avoid ubsan fail (#57942)
add 19ba4c66baf [fix](nereids) Fix the int type coercion to big int
wrongly when handle user set var (#52689)
add 21029ec72d6 [opt](lazy topn) set row id's unique id when new it
(#57867)
add 6997929b2fd [test](ngram) make ngram index test case robust (#57943)
add 1a24dabb212 [fix](broker load) Fix strict_mode in broker load not work
(#57825)
add c2c2f893338 [chore](test)Use JUnit BOM to Manage Versions Consistently
(#57931)
add c549bcf1071 [bugfix](jvmmetrics) some object is nullptr if java
support is not enabled (#57953)
add 71d706ed653 [chore](plan) remove ColumnStats and mem layout on tuple
(#57912)
add fe602e63480 [opt](agent-task) Add a daemon thread to clean up agent
tasks on dead BEs (#57591)
add 6da808e9ce7 [fix](thirdparth)adjust thrift patch match rule (#57693)
add 996a5002076 [fix](memtable) add check when memtable init aggregate
function to avoid core dump (#57934)
add 6b706300dbd [chore](ci) add retry to reduce compile fail of regression
framework (#57954)
add acc1bcf9e4b [fix](nereids)PushDownJoinOnAssertNumRows create invalid
LogicalProject node (#57804)
add 4cbae8a97f7 [enhancement](filecache) fine-grained cache space
observation (#57783)
add 2eda198a7bd [chore] (array-function) add alias array_append for
function array_pushback (#57914)
add 5d4f6d73537 [fix](fe-ut) Fix FE-UT RewriteDataFilePlannerTest (#57971)
add 911cf072f06 [Fix](feut) try to fix MissingInvocation of
refreshAndGetDiskInfo (#57968)
add add48d609db [fix](nereids) inferPredicate should not pullup predicates
from mark join's right child (#57712)
add cba14df782b (fix)(nerieds) fix union all with tvf throw exception
(#57889)
add 12d2aaa9eaf [opt](mtmv) Support materialized view query rewriting by
join eliminate when mv use more table than query (#56266)
add 4ce3e6fc1ea [fix](job) fix transaction timeout do not match routine
load task adaptive timeout (#57967)
add d990c5ee95f [fix](coordinator) fix "PStatus: Finished" (#57985)
add 16310051df5 [fix](function)The json_remove function did not initialize
nullmap. (#57995)
add 2b808cb0b13 [chore] Opt index scan cases (#57951)
add a8b79961f46 [Enhancement](function) The date_add function supports
DAY_HOUR as timeunit (#57506)
add 9b61da470f4 [Chore](nereids) remove TypeDef (#57834)
add e10e29d77f8 [opt](nereids) Add planWithUnknownColumnStats to
QueryState to prevent queries from analysis tasks from polluting the column
stats cache. (#57850)
add b7c287062a3 [ut](ldap)Change the implementation of LdapClientTest to
solve the problem of pipeline out failure (#57994)
add cff54664751 [chore](dep)Cleanup: Remove local K8s Ambari deployment
and obsolete dependencies (#57979)
add cef5f098006 [Exec](Func) fix condition cache not effective with range
(#57830)
add 351e5683bc8 [fix] remove useless const_cast and explain const_cast(2)
(#56427)
add 8d570c83436 [fix][arrow-flight-sql]Move ctx.clear() to the end of
close method (#57858)
add 7b48041f674 [optimize](paimon): Avoid redundant hadoop conf setting in
each split (#57950)
add 4017fc72868 [fix](executor) Fix time-sharing task executor don't
re-schedule when exceeded concurrency limit case. (#57833)
add 621a6a7e92e [fix](proc) fix schema proc row data (#57962)
add 2350f61d2c2 [fix](nereids)from_unixtime support decimalv3 literal as
its first argument when used in fold constant rule (#57363)
add d1f036d7533 [fix]handle loading iceberg partitions by auth (#57988)
add 64d1c0f93a1 [feat](snapshot) recycle versioned rowsets should remove
delete bitmap keys (#57908)
add 7e8701b0f14 [opt](nereids)estimate num_nulls for outer join (#57863)
add 08b701fe80a [fix](hive) Fix partition path scheme mismatch when
inserting into Hive partitioned tables on object storage (#57973)
add 503e738a63d [chore] (streamload) add enable stream load profile config
(#57919)
add 8aa1a7489c3 [feat](catalog)Support OneLake (#57933)
add 5cb1af2dc20 [feat](snapshot) reference_instance_id is not set in
compaction operation log (#58023)
add 68ab7cdc745 [feature](tvf) Support HTTP Table Valued Function (#57242)
add e25a4e06a47 [fix](test) Support azure config for recycler case
(#58026)
add c75467ca813 [fix](case) Increase the robustness of some docker case
(#57991)
add 915b33769c4 [fix](grace-stop) remove heartbeat condition (#58019)
add 44d6d786fcc [Bug](scan) fix topn_next fail (#57895)
add 560fa46b051 [fix](mow) skip agg delete bitmap v1 to ms when v2 (#57990)
add c84ce29871e [fix](mow) check delete bitmap version (#57989)
add 2c99d0728d5 [Chore](scan) set enable_task_executor_in_internal_table
conf's default value to true. (#58017)
add 4d2bbe99b00 [thirdparty](patch) Support a better way to get offset in
cctz (#57999)
add 8e0f77a4cac [fix](mtmx) Fix coredump because common expr push down to
scannode which belongs to an agg materialized view (#58038)
add 37a8ca8323a [enhance](job) terminate streaming task execute threads
promptly when idle (#58041)
add 14cf14fb9d2 [fix](metacache) Fix query hang caused by Caffeine cache
deadlock in external table queries (#57856)
add eb012df4dc5 [minor](status) Regard `FINISHED` as a normal status
instead of an er… (#57986)
add bcaaf60fa30 [Enhancement](nereids) Replace IndexDef with
IndexDefinition (#57886)
add 6deeab8f9f1 [fix](core)Provide add_blocks interface for
VDataStreamRecvr (#57888)
add 758f1673da3 [regression-test](variant) use s3load instead of stream
load in variant_p2 test (#56795)
add d4287941c51 [Refactor](block) remove all index_by_name usage (#57860)
add b96e3ced3c3 [Enhancement](nereids) implement CopyProperties in nereids
(#57976)
add ca1346dce60 [fix](mtmv) Fix partition trace fail when create partition
mv with view (#57958)
add 893a1e92399 [fix](nereids) Fix aggregate source repeat output is
different from child repeat (#57840)
add bd374aa7051 [Fix](filecache) Avoid read peer in local mode (#58063)
add 81b2d2bde57 [remove](sync JOB) Remove the code about SYNC JOB (#58008)
add 842b29eec40 [feat](cloud) support blob_get_range (#58086)
add af71949eb56 [refactor](exec) expr execution now returns a Column
directly instead of inserting into a Block (#57486)
add 7e0157bf95f [feature](jsonb) json type support group by and distinct
(#57679)
add 9f9a3e4cd19 [fix](filecache) fix out-of-range exception when external
query (#58082)
add 56fd3c6fa21 [chore](memtable) use doris::Exception replace
std::runtime_error (#58098)
add e8694055758 [chore](memtable) remove memtable memory limiter useless
logic (#58097)
add 5ebf22dee50 [enhancement](filecache) add more debug info via
VLOG_DEBUG (#58043)
add 3898b1e1476 [test](sqlcache)fix sql cache test case (#58079)
add 093dc7b8afc [opt](mtmv) Support limit or topN rewrite when
materialized view contains limit or topN (#55194)
add abc105b94d6 [Enhancement](ai) relax the matching restriction of some
AI-Functions (#58077)
add 48fcb2a6d17 [fix] remove useless const_cast and explain const_cast(1)
(#55943)
add 2c94ebd1001 [fix](jdbc) Skip the default creation check logic (#57983)
add ad1ad7fea91 [performance](agg) support count push agg in no null
column (#58103)
add d20275358b6 [enhancement](wg) print error code when delete cgroup path
failed (#58126)
add 1d74a0ea609 [chore](cloud) support large operation log (#58100)
add 4019f96648c [Improvement](exchange) avoid hash shuffle when partition
type is bucket shuffle and only one instance (#58068)
add eae1b88df9f [feat](tvf) support huggingface with http tvf (#58049)
add 0c2fa23d6ac [feat](storage) introduce backpressure algorithm to
control version number (part II) (#57654)
add 6f193832410 [improvement](error msg) prioritize retaining the error
URL and first_error_msg when truncation occurs (#56759)
add 88da8212593 [test](regression) Add regressionTest for using token in
broker load (#57544)
add ad4e61f0f9c [feat](case when) Push upper function into case when
branch (#56941)
add 86b9973e2bf [test](inverted index) remove regression case for inverted
index cache, use ut instead (#58137)
add a3ae7cbbb71 [improve](cloud) commit_partition splits into multi txns
(#58109)
add 9bae71b2690 [fix](snapshot) Disable chunked encoding for s3 client
(#58078)
add 1a8d71e2773 [fix](filecache) fix file cache microbench DCHECK problems
(#58005)
add bdc6e386d10 [fix](cloud) avoid broadcast remote read in topn query
(#58044)
add 0d1c7c3d7e7 [fix](cloud) Fix cloud drop tablet tasks pile up (#58131)
add 712ac5ad29f [feat](snapshot) cannot drop instance if has snapshots;
drop instance should remove snapshot_reference_key (#58111)
add 9392c1c1d3d [fix](metric) fix backend metrics missing in fe (#57935)
add a7ec6d66d5d [Test](client) Add s3 storage client test sketelon (#57456)
add e7c43f4c02b [fix](file cache) Fix inverted index cache not cleaned on
compaction failure (#58141)
add 30f4100eb63 [Fix](exec) some error in condition cache because cache
digest error (#58115)
add 972eef8f5eb [test](bloom filter) enhance bloom filter regression case
for recover fe configs (#58148)
add b9b325e1c1e [test](ip) fix ip_cidr_search case (#58151)
add a0e13069e08 [fix](case) fix group commit case (#58142)
add 1c3ed1185d5 [Chore](build) copy jeprof to output/be/bin (#58128)
add 58662209efc [opt](ann index) Introduce IDSelectorRoaring for a faster
roaring_to_faiss_selector (#58095)
add 09fc3fd2d45 [feat](Ann Index)Support create index and build index for
ANN (#55586)
add dd4ec3375d0 [fix](nereids)where clause should only contains key column
when create mv on unique table (#57915)
add 288127a8c9d [fix](nereids)allow sync mv has same columns as base table
when it has where clause (#58105)
add ba3bb0ffed9 [Improvement](exchange) optimize for channel_add_rows
(#57997)
add 1c272bc7c0c [fix](iceberg) only enable dynamic partition pruning for
identity partitions in Iceberg (#58033)
add fe7938dc29d [feat](nereids) simplify range support not equal / is null
/ is not null (#57537)
add ec0bdc7f56d [fix](paimon-catalog)Fix OSS access when using DLS
endpoint (#58099)
add adb8074d114 [fix](load) Fix undefined behavior in the JSON reader
caused by multiple calls to the get_string method (#58107)
add 8d674feac02 Revert "[Chore](scan) set
enable_task_executor_in_internal_table conf's default value to true." (#58144)
add 81aa5bea1a7 [opt](outfile) support compressed csv with names and types
in outfile (#58090)
add f8efb29a9ce [feat](load) enable stream load record to audit log system
table (#57530)
add 7118f1c8010 [fix](hive) refresh hive cache when inconsistency (#58074)
add f4dc9aed04d [refactor](index scan) Rename inverted index status to
index status (#57911)
add 85024ac00a1 [Improvement](exchange) avoid random partition send empty
block (#58175)
add 9f5b4c63d0b [opt](staticstis) use count(1) for rowCount when scan full
table (#58153)
add 112a299e881 [fix](auth)Fixed an issue where the wrong catalog was used
when checking permissions with use cluster (#58084)
add b6214553757 [fix](cloud) Fix the issue where it takes a long time to
come alive on first boot (#58152)
add bd4b70746c2 [fix](datatype)Fix unaligned memory access in
read_column_from_arrow (#58002)
add e4130d17892 [fix](merge into) result wrong when on clause contains
null safe equal (#58091)
add 159462ed528 [enhancement](tde) add configuration for TDE key rotation
(#58171)
add 68ced0d67ec [feat](snapshot) fix AdminCreateClusterSnapshotCommand
(#58119)
add eb5a8e203c1 [Improvement](block) remove all block::get_by_name usage
(#58124)
add 18e0ed156ca [fix](test) fix query39 due to merge multiple affected PRs
(#58210)
add 5d7a278dd09 [minor](refactor) Remove unused function (#58183)
add 7d0cae4f3ea [fix](float) Fix float field `to_string` (#58176)
add 636657a5abe [fix](cloud) Fixed uneven tablet performance during
upgrades from older versions (#58135)
add 4174ff00399 [test](schema-change) Minimal case for schema change index
out of bound when delete predicate column dropped (#57861)
add fb8c54092c4 [fix](datatype)Fix for unaligned memory in arrow MapArray
parsing. (#58248)
add 703badd9367 [Fix](agg) fix push agg op in nullable column before
projection (#58234)
add a5f36a18cc1 [fix](paimon) Align incremental query behavior with Spark
Paimon for single snapshot scenario (#58239)
add ada30740189 [bugfix](memory) should count memory when cancel query is
called (#58252)
add f50cf6b3e42 [feature](reader) Optimize Complex Type Column Reading
with Column Pruning.
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (3fdc1f955d9)
\
N -- N -- N refs/heads/nested_column_prune (f50cf6b3e42)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.gitmodules | 2 +-
{fe/.idea => .idea}/icon.svg | 0
.idea/vcs.xml | 34 +
be/benchmark/benchmark_vstring_function.hpp | 105 +
be/src/agent/cgroup_cpu_ctl.cpp | 7 +-
be/src/agent/task_worker_pool.cpp | 15 +-
be/src/agent/task_worker_pool.h | 2 +-
be/src/cloud/cloud_backend_service.cpp | 9 +-
be/src/cloud/cloud_internal_service.cpp | 25 +-
be/src/cloud/cloud_meta_mgr.cpp | 29 +-
be/src/cloud/cloud_meta_mgr.h | 6 +-
be/src/cloud/cloud_storage_engine.cpp | 4 +-
be/src/cloud/cloud_storage_engine.h | 2 +-
be/src/cloud/cloud_tablet.cpp | 3 +
be/src/cloud/cloud_tablet_mgr.cpp | 29 +-
be/src/cloud/cloud_warm_up_manager.cpp | 4 +-
be/src/cloud/config.cpp | 2 +
be/src/cloud/config.h | 3 +
be/src/common/config.cpp | 7 +
be/src/common/config.h | 12 +
be/src/common/consts.h | 2 -
be/src/common/status.h | 5 +-
be/src/exec/decompressor.cpp | 2 +-
be/src/exec/olap_common.h | 10 +-
be/src/exec/rowid_fetcher.cpp | 19 +-
be/src/exec/schema_scanner.cpp | 3 +
.../schema_file_cache_info_scanner.cpp | 189 ++
.../schema_file_cache_info_scanner.h | 34 +-
be/src/exec/table_connector.cpp | 2 +-
be/src/exprs/hybrid_set.h | 17 +-
be/src/geo/geo_types.cpp | 6 +-
be/src/geo/geo_types.h | 2 +-
be/src/http/action/health_action.cpp | 27 +-
be/src/http/action/http_stream.cpp | 17 +-
be/src/http/action/stream_load.cpp | 17 +-
be/src/http/http_client.cpp | 6 +
be/src/http/http_client.h | 9 +
be/src/http/http_common.h | 1 +
be/src/io/cache/block_file_cache.cpp | 64 +-
be/src/io/cache/block_file_cache.h | 8 +-
be/src/io/cache/block_file_cache_downloader.cpp | 1 +
be/src/io/cache/block_file_cache_factory.h | 3 +
be/src/io/cache/cached_remote_file_reader.cpp | 21 +-
be/src/io/cache/file_cache_common.cpp | 4 +
be/src/io/cache/peer_file_cache_reader.cpp | 21 +-
be/src/io/cache/peer_file_cache_reader.h | 5 +-
be/src/io/file_factory.cpp | 17 +
be/src/io/fs/azure_obj_storage_client.cpp | 2 +-
be/src/io/fs/file_system.h | 10 +-
be/src/io/fs/http_file_reader.cpp | 422 ++++
be/src/io/fs/http_file_reader.h | 87 +
be/src/io/fs/http_file_system.cpp | 88 +
be/src/io/fs/http_file_system.h | 104 +
be/src/io/fs/s3_file_reader.cpp | 4 +
be/src/io/fs/s3_obj_storage_client.cpp | 10 +-
be/src/io/tools/file_cache_microbench.cpp | 62 +-
be/src/olap/base_tablet.cpp | 7 +-
be/src/olap/compaction.cpp | 7 +
be/src/olap/comparison_predicate.h | 18 +-
be/src/olap/delta_writer.cpp | 10 +-
be/src/olap/delta_writer.h | 4 +
be/src/olap/in_list_predicate.h | 15 +-
be/src/olap/iterators.h | 39 +-
be/src/olap/memtable.cpp | 5 +
be/src/olap/memtable_memory_limiter.cpp | 4 -
be/src/olap/merger.cpp | 2 +-
be/src/olap/partial_update_info.cpp | 28 +-
be/src/olap/push_handler.cpp | 4 +-
be/src/olap/rowset/beta_rowset_reader.cpp | 56 +-
be/src/olap/rowset/beta_rowset_reader.h | 49 +-
be/src/olap/rowset/rowset_reader.h | 8 +-
.../segment_v2/ann_index/ann_index_writer.cpp | 35 +-
.../rowset/segment_v2/ann_index/ann_index_writer.h | 7 +
.../segment_v2/ann_index/faiss_ann_index.cpp | 60 +-
be/src/olap/rowset/segment_v2/column_writer.cpp | 8 +-
be/src/olap/rowset/segment_v2/column_writer.h | 2 +-
.../olap/rowset/segment_v2/index_file_writer.cpp | 14 +
be/src/olap/rowset/segment_v2/index_file_writer.h | 1 +
be/src/olap/rowset/segment_v2/index_writer.cpp | 114 +-
.../inverted_index/query/query_helper.cpp | 5 +
.../segment_v2/inverted_index/query/query_helper.h | 2 +
.../segment_v2/inverted_index/query_v2/doc_set.h | 106 +-
.../inverted_index/query_v2/intersection.cpp | 36 +-
.../inverted_index/query_v2/nullable_scorer.h | 77 +
.../query_v2/phrase_query/multi_phrase_query.h | 71 +
.../query_v2/phrase_query/multi_phrase_weight.h | 113 +
.../query_v2/phrase_query/phrase_query.h | 30 +-
.../query_v2/phrase_query/phrase_scorer.cpp | 3 +-
.../query_v2/phrase_query/phrase_weight.h | 67 +-
.../query_v2/phrase_query/postings_with_offset.h | 4 -
.../query_v2/postings/loaded_postings.cpp | 136 ++
.../query_v2/postings/loaded_postings.h | 55 +
.../query_v2/regexp_query/regexp_query.h | 3 +-
.../query_v2/regexp_query/regexp_weight.cpp | 29 +-
.../query_v2/regexp_query/regexp_weight.h | 5 +-
.../segment_v2/inverted_index/query_v2/scorer.h | 11 +-
.../inverted_index/query_v2/segment_postings.h | 71 +-
.../inverted_index/query_v2/union/simple_union.cpp | 143 ++
.../inverted_index/query_v2/union/simple_union.h | 60 +
.../segment_v2/inverted_index/query_v2/weight.h | 31 +
.../query_v2/wildcard_query/wildcard_query.h | 3 +-
.../query_v2/wildcard_query/wildcard_weight.h | 8 +-
be/src/olap/rowset/segment_v2/segment_iterator.cpp | 88 +-
be/src/olap/rowset/segment_v2/segment_iterator.h | 18 +-
be/src/olap/schema_change.cpp | 18 +-
be/src/olap/schema_change.h | 3 +
be/src/olap/storage_engine.cpp | 2 +-
be/src/olap/storage_engine.h | 4 +-
be/src/olap/tablet.cpp | 7 +-
be/src/olap/tablet_meta.cpp | 2 +-
be/src/olap/tablet_schema.cpp | 4 +-
be/src/olap/tablet_schema.h | 3 +-
be/src/olap/task/index_builder.cpp | 166 +-
be/src/olap/task/index_builder.h | 2 +-
be/src/olap/wal/wal_manager.cpp | 2 +-
be/src/pipeline/exec/exchange_sink_buffer.cpp | 3 +-
be/src/pipeline/exec/exchange_sink_operator.cpp | 87 +-
be/src/pipeline/exec/materialization_opertor.cpp | 5 +-
be/src/pipeline/exec/operator.cpp | 6 +-
.../exec/partitioned_aggregation_sink_operator.cpp | 2 +-
.../partitioned_aggregation_source_operator.cpp | 2 +-
.../exec/partitioned_hash_join_probe_operator.cpp | 13 +-
.../exec/partitioned_hash_join_sink_operator.cpp | 4 +-
be/src/pipeline/exec/scan_operator.cpp | 48 +-
be/src/pipeline/exec/scan_operator.h | 2 +-
be/src/pipeline/exec/schema_scan_operator.cpp | 4 +-
be/src/pipeline/exec/schema_scan_operator.h | 5 +
be/src/pipeline/exec/spill_sort_sink_operator.cpp | 4 +-
be/src/pipeline/shuffle/writer.cpp | 43 +-
be/src/pipeline/shuffle/writer.h | 8 +-
be/src/runtime/descriptors.cpp | 14 +
be/src/runtime/descriptors.h | 9 +
be/src/runtime/exec_env.cpp | 11 +-
be/src/runtime/exec_env.h | 11 +-
be/src/runtime/exec_env_init.cpp | 14 +
be/src/runtime/fragment_mgr.cpp | 1 +
be/src/runtime/load_stream.cpp | 44 +
be/src/runtime/load_stream.h | 6 +
be/src/runtime/memory/memory_profile.cpp | 36 +-
be/src/runtime/memory/memory_profile.h | 4 +
be/src/runtime/process_profile.h | 2 +-
be/src/runtime/runtime_state.cpp | 7 +-
be/src/runtime/runtime_state.h | 6 +
.../runtime/stream_load/stream_load_recorder.cpp | 35 +-
be/src/runtime/stream_load/stream_load_recorder.h | 6 +-
.../stream_load/stream_load_recorder_manager.cpp | 325 +++
.../stream_load/stream_load_recorder_manager.h | 98 +
be/src/runtime/tablets_channel.cpp | 4 +-
be/src/runtime/thread_context.cpp | 15 -
be/src/runtime/thread_context.h | 20 +-
.../runtime/workload_management/resource_context.h | 2 -
.../arrow_flight/arrow_flight_batch_reader.cpp | 5 +-
be/src/service/doris_main.cpp | 3 +
be/src/service/internal_service.cpp | 18 +-
be/src/util/disk_info_mac.cpp | 1 +
be/src/util/encryption_util.cpp | 2 +-
be/src/util/jni-util.cpp | 1 +
be/src/util/jsonb_document.h | 26 +-
be/src/util/jvm_metrics.cpp | 4 +
be/src/util/lru_multi_cache.h | 2 +-
be/src/util/lru_multi_cache.inline.h | 4 +-
be/src/util/mysql_row_buffer.h | 3 +
be/src/util/proto_util.h | 23 +-
be/src/util/runtime_profile.cpp | 6 +-
be/src/util/runtime_profile.h | 4 +-
be/src/util/s3_util.h | 1 +
be/src/util/simd/vstring_function.h | 47 +
be/src/util/slice.h | 2 +-
be/src/util/threadpool.cpp | 2 +
be/src/util/url_parser.cpp | 30 +-
.../aggregate_functions/aggregate_function_sort.h | 8 +-
be/src/vec/common/hash_table/hash_key_type.h | 3 +-
be/src/vec/common/sort/heap_sorter.h | 1 -
be/src/vec/core/block.cpp | 230 +-
be/src/vec/core/block.h | 87 +-
be/src/vec/core/field.cpp | 83 +-
be/src/vec/core/sort_block.cpp | 9 +-
be/src/vec/core/sort_description.h | 11 +-
be/src/vec/data_types/data_type_decimal.cpp | 10 +-
be/src/vec/data_types/data_type_factory.cpp | 5 +-
.../vec/data_types/serde/data_type_array_serde.cpp | 32 +-
.../vec/data_types/serde/data_type_array_serde.h | 12 +-
.../data_types/serde/data_type_bitmap_serde.cpp | 16 +-
.../vec/data_types/serde/data_type_bitmap_serde.h | 12 +-
.../serde/data_type_date_or_datetime_serde.cpp | 8 +-
.../serde/data_type_date_or_datetime_serde.h | 9 +-
.../serde/data_type_datetimev2_serde.cpp | 23 +-
.../data_types/serde/data_type_datetimev2_serde.h | 12 +-
.../data_types/serde/data_type_datev2_serde.cpp | 24 +-
.../vec/data_types/serde/data_type_datev2_serde.h | 12 +-
.../data_types/serde/data_type_decimal_serde.cpp | 22 +-
.../vec/data_types/serde/data_type_decimal_serde.h | 12 +-
.../vec/data_types/serde/data_type_hll_serde.cpp | 14 +-
be/src/vec/data_types/serde/data_type_hll_serde.h | 12 +-
.../vec/data_types/serde/data_type_ipv4_serde.cpp | 16 +-
be/src/vec/data_types/serde/data_type_ipv4_serde.h | 12 +-
.../vec/data_types/serde/data_type_ipv6_serde.cpp | 16 +-
be/src/vec/data_types/serde/data_type_ipv6_serde.h | 12 +-
.../vec/data_types/serde/data_type_jsonb_serde.cpp | 16 +-
.../vec/data_types/serde/data_type_jsonb_serde.h | 12 +-
.../vec/data_types/serde/data_type_map_serde.cpp | 30 +-
be/src/vec/data_types/serde/data_type_map_serde.h | 12 +-
.../vec/data_types/serde/data_type_nothing_serde.h | 12 +-
.../data_types/serde/data_type_nullable_serde.cpp | 16 +-
.../data_types/serde/data_type_nullable_serde.h | 12 +-
.../data_types/serde/data_type_number_serde.cpp | 27 +-
.../vec/data_types/serde/data_type_number_serde.h | 12 +-
.../serde/data_type_quantilestate_serde.h | 12 +-
be/src/vec/data_types/serde/data_type_serde.h | 25 +-
.../data_types/serde/data_type_string_serde.cpp | 23 +-
.../vec/data_types/serde/data_type_string_serde.h | 12 +-
.../data_types/serde/data_type_struct_serde.cpp | 18 +-
.../vec/data_types/serde/data_type_struct_serde.h | 12 +-
.../vec/data_types/serde/data_type_time_serde.cpp | 16 +-
be/src/vec/data_types/serde/data_type_time_serde.h | 12 +-
.../data_types/serde/data_type_varbinary_serde.cpp | 16 +-
.../data_types/serde/data_type_varbinary_serde.h | 12 +-
.../data_types/serde/data_type_variant_serde.cpp | 16 +-
.../vec/data_types/serde/data_type_variant_serde.h | 12 +-
.../vec/exec/format/arrow/arrow_stream_reader.cpp | 9 +-
.../file_reader/new_plain_text_line_reader.cpp | 4 +-
be/src/vec/exec/format/json/new_json_reader.cpp | 35 +-
be/src/vec/exec/format/json/new_json_reader.h | 20 +
be/src/vec/exec/format/orc/vorc_reader.cpp | 87 +-
.../exec/format/parquet/vparquet_group_reader.cpp | 95 +-
be/src/vec/exec/format/table/equality_delete.cpp | 42 +-
be/src/vec/exec/format/table/iceberg_reader.cpp | 22 +-
be/src/vec/exec/format/table/paimon_jni_reader.cpp | 16 +-
.../vec/exec/format/table/remote_doris_reader.cpp | 133 ++
be/src/vec/exec/format/table/remote_doris_reader.h | 70 +
be/src/vec/exec/format/wal/wal_reader.cpp | 4 +-
be/src/vec/exec/jni_connector.cpp | 4 +-
be/src/vec/exec/scan/file_scanner.cpp | 35 +-
be/src/vec/exec/scan/olap_scanner.cpp | 16 +-
be/src/vec/exec/scan/scanner.cpp | 17 +-
be/src/vec/exec/scan/scanner_context.cpp | 10 +-
be/src/vec/exec/scan/scanner_context.h | 2 +-
be/src/vec/exprs/lambda_function/lambda_function.h | 6 +-
.../lambda_function/varray_filter_function.cpp | 42 +-
.../exprs/lambda_function/varray_map_function.cpp | 94 +-
be/src/vec/exprs/vbitmap_predicate.cpp | 25 +-
be/src/vec/exprs/vbitmap_predicate.h | 3 +-
be/src/vec/exprs/vbloom_predicate.cpp | 37 +-
be/src/vec/exprs/vbloom_predicate.h | 3 +-
be/src/vec/exprs/vcase_expr.cpp | 32 +-
be/src/vec/exprs/vcase_expr.h | 5 +-
be/src/vec/exprs/vcast_expr.cpp | 81 +-
be/src/vec/exprs/vcast_expr.h | 10 +-
be/src/vec/exprs/vcolumn_ref.h | 10 +-
be/src/vec/exprs/vcompound_pred.h | 111 +-
be/src/vec/exprs/vdirect_in_predicate.h | 39 +-
be/src/vec/exprs/vectorized_fn_call.cpp | 55 +-
be/src/vec/exprs/vectorized_fn_call.h | 12 +-
be/src/vec/exprs/vexpr.cpp | 44 +-
be/src/vec/exprs/vexpr.h | 36 +-
be/src/vec/exprs/vexpr_context.cpp | 11 +-
be/src/vec/exprs/vexpr_context.h | 63 +-
be/src/vec/exprs/vin_predicate.cpp | 33 +-
be/src/vec/exprs/vin_predicate.h | 3 +-
be/src/vec/exprs/vinfo_func.cpp | 7 +-
be/src/vec/exprs/vinfo_func.h | 3 +-
be/src/vec/exprs/virtual_slot_ref.cpp | 24 +-
be/src/vec/exprs/virtual_slot_ref.h | 3 +-
be/src/vec/exprs/vlambda_function_call_expr.h | 5 +-
be/src/vec/exprs/vlambda_function_expr.h | 9 +-
be/src/vec/exprs/vliteral.cpp | 7 +-
be/src/vec/exprs/vliteral.h | 3 +-
be/src/vec/exprs/vmatch_predicate.cpp | 28 +-
be/src/vec/exprs/vmatch_predicate.h | 3 +-
be/src/vec/exprs/vruntimefilter_wrapper.cpp | 19 +-
be/src/vec/exprs/vruntimefilter_wrapper.h | 21 +-
be/src/vec/exprs/vsearch.cpp | 15 +-
be/src/vec/exprs/vsearch.h | 3 +-
be/src/vec/exprs/vslot_ref.cpp | 22 +-
be/src/vec/exprs/vslot_ref.h | 5 +-
be/src/vec/exprs/vtopn_pred.h | 39 +-
be/src/vec/functions/ai/ai_functions.h | 38 +-
.../functions/array/function_array_pushback.cpp | 1 +
be/src/vec/functions/cast/cast_to_string.h | 2 +-
be/src/vec/functions/cast/cast_to_variant.h | 2 +-
be/src/vec/functions/cast/function_cast.cpp | 2 +-
be/src/vec/functions/function.cpp | 65 +-
be/src/vec/functions/function.h | 30 +-
.../function_date_or_datetime_computation.cpp | 3 +
.../function_date_or_datetime_computation.h | 66 +
be/src/vec/functions/function_helpers.cpp | 8 -
be/src/vec/functions/function_java_udf.cpp | 18 +-
be/src/vec/functions/function_java_udf.h | 3 -
be/src/vec/functions/function_jsonb.cpp | 2 +-
be/src/vec/functions/function_regexp.cpp | 216 +-
be/src/vec/functions/function_rpc.h | 2 +-
be/src/vec/functions/function_search.cpp | 39 +-
be/src/vec/functions/like.cpp | 26 +-
be/src/vec/functions/like.h | 4 +
be/src/vec/functions/math.cpp | 69 +
be/src/vec/olap/block_reader.cpp | 11 +-
be/src/vec/olap/block_reader.h | 2 -
be/src/vec/olap/vcollect_iterator.cpp | 28 +-
be/src/vec/olap/vcollect_iterator.h | 14 +-
be/src/vec/olap/vertical_block_reader.cpp | 1 -
be/src/vec/olap/vgeneric_iterators.cpp | 12 +-
be/src/vec/olap/vgeneric_iterators.h | 15 +-
be/src/vec/runtime/vcsv_transformer.cpp | 19 +-
be/src/vec/runtime/vdata_stream_mgr.cpp | 51 +-
be/src/vec/runtime/vdata_stream_recvr.cpp | 147 +-
be/src/vec/runtime/vdata_stream_recvr.h | 81 +-
be/src/vec/sink/load_stream_stub.cpp | 45 +
be/src/vec/sink/load_stream_stub.h | 10 +
be/src/vec/sink/varrow_flight_result_writer.cpp | 8 +-
be/src/vec/sink/vdata_stream_sender.cpp | 5 +-
be/src/vec/sink/vtablet_block_convertor.cpp | 4 -
.../sink/writer/iceberg/viceberg_table_writer.cpp | 12 +-
.../sink/writer/iceberg/viceberg_table_writer.h | 3 +
be/src/vec/sink/writer/vtablet_writer.cpp | 3 +-
be/src/vec/sink/writer/vtablet_writer_v2.cpp | 22 +
be/src/vec/sink/writer/vtablet_writer_v2.h | 3 +
be/src/vec/sink/writer/vwal_writer.cpp | 3 +-
be/src/vec/spill/spill_reader.cpp | 4 +-
be/src/vec/spill/spill_writer.cpp | 3 +-
be/test/ai/ai_function_test.cpp | 154 ++
be/test/exprs/mock_vexpr.h | 6 +-
be/test/exprs/virtual_slot_ref_test.cpp | 21 +-
be/test/http/http_client_test.cpp | 4 +-
be/test/io/cache/block_file_cache_test.cpp | 8 +
be/test/io/client/s3_file_system_test.cpp | 1895 ++++++++++++++++
be/test/olap/collection_statistics_test.cpp | 19 +-
be/test/olap/index_builder_test.cpp | 369 ++-
be/test/olap/ordered_data_compaction_test.cpp | 2 +-
be/test/olap/rowid_conversion_test.cpp | 2 +-
.../inverted_index/query_v2/doc_set_test.cpp | 709 ++++--
.../query_v2/loaded_postings_test.cpp | 296 +++
..._query_test.cpp => multi_phrase_query_test.cpp} | 466 +++-
.../inverted_index/query_v2/phrase_query_test.cpp | 128 +-
.../query_v2/segment_postings_test.cpp | 361 ++-
.../inverted_index/query_v2/simple_union_test.cpp | 568 +++++
.../segment_v2/inverted_index_file_writer_test.cpp | 49 +
.../segment_v2/inverted_index_reader_test.cpp | 116 +
be/test/olap/segcompaction_mow_test.cpp | 5 +-
be/test/olap/segcompaction_test.cpp | 15 +-
.../olap/vector_search/ann_index_writer_test.cpp | 144 ++
.../olap/vector_search/faiss_vector_index_test.cpp | 50 +
be/test/olap/wal/wal_reader_writer_test.cpp | 10 +-
be/test/pipeline/exec/vdata_stream_recvr_test.cpp | 3 +-
.../operator/materialization_shared_state_test.cpp | 18 +-
be/test/scan/scanner_context_test.cpp | 5 +-
be/test/testutil/mock/mock_data_stream_sender.h | 10 +-
be/test/testutil/mock/mock_fn_call.h | 2 +-
be/test/testutil/mock/mock_in_expr.h | 2 +-
be/test/testutil/mock/mock_slot_ref.h | 2 +-
be/test/util/simd/vstring_function_test.cpp | 461 ++++
be/test/vec/columns/column_varbinary_test.cpp | 79 +-
be/test/vec/common/string_view_test.cpp | 97 +
be/test/vec/core/block_test.cpp | 188 +-
be/test/vec/data_types/common_data_type_test.h | 7 +-
.../vec/data_types/data_type_varbinary_test.cpp | 122 +-
.../serde/data_type_serde_datetime_v2_test.cpp | 66 +-
.../serde/data_type_serde_decimal_test.cpp | 48 +-
.../data_types/serde/data_type_serde_map_test.cpp | 182 ++
.../serde/data_type_serde_number_test.cpp | 53 +-
.../serde/data_type_serde_string_test.cpp | 129 +-
.../serde/data_type_serde_struct_test.cpp | 162 ++
.../vec/exec/format/parquet/parquet_read_lines.cpp | 6 +-
be/test/vec/exec/orc/orc_read_lines.cpp | 6 +-
be/test/vec/exec/vgeneric_iterators_test.cpp | 12 +-
be/test/vec/exprs/try_cast_expr_test.cpp | 23 +-
.../exprs/vexpr_evalute_inverted_index_test.cpp | 11 +-
be/test/vec/exprs/vsearch_expr_test.cpp | 32 +-
be/test/vec/function/function_math_test.cpp | 28 +
be/test/vec/function/function_time_test.cpp | 35 +
be/test/vec/olap/vertical_compaction_test.cpp | 12 +-
build.sh | 2 +-
cloud/src/common/util.cpp | 10 +-
cloud/src/meta-service/meta_service.cpp | 53 +-
cloud/src/meta-service/meta_service.h | 7 +-
cloud/src/meta-service/meta_service_helper.h | 3 +-
cloud/src/meta-service/meta_service_job.cpp | 45 +-
cloud/src/meta-service/meta_service_partition.cpp | 99 +-
cloud/src/meta-service/meta_service_resource.cpp | 100 +-
.../src/meta-service/meta_service_tablet_stats.cpp | 84 +
cloud/src/meta-service/meta_service_tablet_stats.h | 20 +
cloud/src/meta-service/meta_service_txn.cpp | 85 +-
cloud/src/meta-service/txn_lazy_committer.cpp | 25 +-
cloud/src/meta-store/blob_message.cpp | 178 ++
cloud/src/meta-store/blob_message.h | 111 +
cloud/src/meta-store/clone_chain_reader.cpp | 80 +
cloud/src/meta-store/clone_chain_reader.h | 11 +
cloud/src/meta-store/keys.cpp | 155 +-
cloud/src/meta-store/keys.h | 26 +-
cloud/src/meta-store/meta_reader.cpp | 84 +-
cloud/src/meta-store/meta_reader.h | 11 +
cloud/src/recycler/recycler.cpp | 19 +
cloud/src/recycler/recycler.h | 8 +-
cloud/src/recycler/recycler_operation_log.cpp | 84 +-
cloud/src/recycler/snapshot_chain_compactor.cpp | 141 +-
cloud/src/recycler/snapshot_data_migrator.cpp | 1 +
cloud/src/resource-manager/resource_manager.cpp | 14 +-
cloud/test/CMakeLists.txt | 3 +
cloud/test/blob_message_test.cpp | 274 +++
cloud/test/clone_chain_reader_test.cpp | 147 ++
cloud/test/document_message_test.cpp | 23 +-
cloud/test/meta_reader_test.cpp | 149 ++
cloud/test/meta_service_operation_log_test.cpp | 66 +-
cloud/test/recycler_operation_log_test.cpp | 66 +-
cloud/test/recycler_test.cpp | 87 +-
cloud/test/resource_test.cpp | 6 +-
contrib/apache-orc | 2 +-
dist/LICENSE-dist.txt | 4 -
docker/runtime/doris-compose/Dockerfile | 2 +-
.../create_preinstalled_scripts/iceberg/run19.sql | 108 +-
.../create_preinstalled_scripts/iceberg/run21.sql | 152 ++
fe/be-java-extensions/avro-scanner/pom.xml | 8 +-
fe/be-java-extensions/hadoop-deps/pom.xml | 10 +-
fe/be-java-extensions/hadoop-hudi-scanner/pom.xml | 39 +-
.../iceberg-metadata-scanner/pom.xml | 10 +-
fe/be-java-extensions/java-udf/pom.xml | 3 +-
fe/be-java-extensions/jdbc-scanner/pom.xml | 3 +-
fe/be-java-extensions/max-compute-scanner/pom.xml | 8 +-
fe/be-java-extensions/paimon-scanner/pom.xml | 14 +-
fe/be-java-extensions/preload-extensions/pom.xml | 9 +-
.../trino-connector-scanner/pom.xml | 3 +-
fe/check/checkstyle/checkstyle.xml | 6 +
fe/fe-common/pom.xml | 27 -
.../java/org/apache/doris/catalog/ArrayType.java | 56 +-
.../java/org/apache/doris/catalog/MapType.java | 67 +-
.../org/apache/doris/catalog/MultiRowType.java | 79 -
.../org/apache/doris/catalog/PrimitiveType.java | 559 +----
.../java/org/apache/doris/catalog/ScalarType.java | 199 --
.../java/org/apache/doris/catalog/StructField.java | 11 -
.../java/org/apache/doris/catalog/StructType.java | 138 --
.../org/apache/doris/catalog/TemplateType.java | 72 -
.../main/java/org/apache/doris/catalog/Type.java | 912 +-------
.../org/apache/doris/catalog/VariantField.java | 7 -
.../java/org/apache/doris/catalog/VariantType.java | 37 +-
.../main/java/org/apache/doris/common/Config.java | 39 +-
fe/fe-core/pom.xml | 102 +-
.../antlr4/org/apache/doris/nereids/DorisLexer.g4 | 1 +
.../antlr4/org/apache/doris/nereids/DorisParser.g4 | 2 +-
.../src/main/java/org/apache/doris/DorisFE.java | 52 +-
.../main/java/org/apache/doris/alter/Alter.java | 7 +-
.../doris/alter/MaterializedViewHandler.java | 4 +-
.../java/org/apache/doris/alter/RollupJobV2.java | 1 -
.../apache/doris/alter/SchemaChangeHandler.java | 68 +-
.../org/apache/doris/alter/SchemaChangeJobV2.java | 3 +-
.../org/apache/doris/analysis/AggregateInfo.java | 4 +-
.../org/apache/doris/analysis/AnalyticExpr.java | 359 ---
.../org/apache/doris/analysis/AnalyticWindow.java | 107 -
.../org/apache/doris/analysis/ArithmeticExpr.java | 158 +-
.../org/apache/doris/analysis/ArrayLiteral.java | 8 -
.../apache/doris/analysis/BetweenPredicate.java | 16 +-
.../org/apache/doris/analysis/BinaryPredicate.java | 161 +-
.../doris/analysis/BitmapFilterPredicate.java | 64 -
.../org/apache/doris/analysis/BoolLiteral.java | 1 -
.../java/org/apache/doris/analysis/BrokerDesc.java | 2 +-
.../apache/doris/analysis/BuildIndexClause.java | 27 +-
.../java/org/apache/doris/analysis/CaseExpr.java | 34 +-
.../java/org/apache/doris/analysis/CastExpr.java | 142 +-
.../java/org/apache/doris/analysis/ColumnDef.java | 260 +--
.../org/apache/doris/analysis/ColumnRefExpr.java | 5 -
.../apache/doris/analysis/CompoundPredicate.java | 70 +-
.../apache/doris/analysis/CreateIndexClause.java | 13 +-
.../org/apache/doris/analysis/DateLiteral.java | 12 -
.../apache/doris/analysis/DefaultValueExpr.java | 5 -
.../org/apache/doris/analysis/DescriptorTable.java | 150 +-
.../org/apache/doris/analysis/EncryptKeyRef.java | 3 -
.../main/java/org/apache/doris/analysis/Expr.java | 631 +-----
.../apache/doris/analysis/ExprSubstitutionMap.java | 155 +-
.../org/apache/doris/analysis/FunctionArgsDef.java | 82 -
.../apache/doris/analysis/FunctionCallExpr.java | 298 +--
.../org/apache/doris/analysis/GroupByClause.java | 31 +-
.../doris/analysis/GroupingFunctionCallExpr.java | 64 -
.../org/apache/doris/analysis/GroupingInfo.java | 8 +-
.../org/apache/doris/analysis/InPredicate.java | 82 -
.../java/org/apache/doris/analysis/IndexDef.java | 356 ---
.../org/apache/doris/analysis/IsNullPredicate.java | 37 -
.../java/org/apache/doris/analysis/JobName.java | 47 -
.../java/org/apache/doris/analysis/LabelName.java | 81 -
.../doris/analysis/LambdaFunctionCallExpr.java | 37 +-
.../apache/doris/analysis/LambdaFunctionExpr.java | 18 -
.../org/apache/doris/analysis/LikePredicate.java | 27 -
.../org/apache/doris/analysis/LiteralExpr.java | 136 --
.../org/apache/doris/analysis/LoadColumnsInfo.java | 151 --
.../org/apache/doris/analysis/MatchPredicate.java | 111 +-
.../org/apache/doris/analysis/NullLiteral.java | 6 -
.../org/apache/doris/analysis/OrderByElement.java | 16 -
.../org/apache/doris/analysis/PlaceHolderExpr.java | 17 -
.../java/org/apache/doris/analysis/Predicate.java | 59 -
.../org/apache/doris/analysis/SchemaTableType.java | 2 +
.../org/apache/doris/analysis/SearchPredicate.java | 4 +-
.../org/apache/doris/analysis/SlotDescriptor.java | 144 +-
.../java/org/apache/doris/analysis/SlotRef.java | 88 +-
.../java/org/apache/doris/analysis/SortInfo.java | 76 -
.../org/apache/doris/analysis/StorageBackend.java | 3 +-
.../org/apache/doris/analysis/StructLiteral.java | 7 -
.../doris/analysis/TimestampArithmeticExpr.java | 63 -
.../org/apache/doris/analysis/TryCastExpr.java | 34 +-
.../org/apache/doris/analysis/TupleDescriptor.java | 262 +--
.../doris/analysis/TupleIsNullPredicate.java | 154 --
.../java/org/apache/doris/analysis/TypeDef.java | 359 ---
.../org/apache/doris/analysis/VariableExpr.java | 5 -
.../org/apache/doris/analysis/VirtualSlotRef.java | 5 -
.../org/apache/doris/backup/BackupHandler.java | 2 +-
.../java/org/apache/doris/backup/RestoreJob.java | 6 +-
.../doris/catalog/AccessPrivilegeWithCols.java | 2 +-
.../apache/doris/catalog/AggregateFunction.java | 219 --
.../doris/catalog/BuiltinAggregateFunctions.java | 185 +-
.../doris/catalog/BuiltinScalarFunctions.java | 4 +-
.../doris/catalog/BuiltinTableValuedFunctions.java | 4 +-
.../main/java/org/apache/doris/catalog/Column.java | 84 +-
.../java/org/apache/doris/catalog/ColumnStats.java | 176 --
.../java/org/apache/doris/catalog/Database.java | 4 -
.../main/java/org/apache/doris/catalog/Env.java | 56 +-
.../java/org/apache/doris/catalog/Function.java | 263 +--
.../org/apache/doris/catalog/FunctionRegistry.java | 2 +-
.../java/org/apache/doris/catalog/FunctionSet.java | 2020 -----------------
.../apache/doris/catalog/FunctionTypeDeducers.java | 88 -
.../org/apache/doris/catalog/FunctionUtil.java | 11 +-
.../apache/doris/catalog/GlobalFunctionMgr.java | 5 -
.../main/java/org/apache/doris/catalog/Index.java | 37 +-
.../org/apache/doris/catalog/InternalSchema.java | 176 +-
.../doris/catalog/InternalSchemaInitializer.java | 8 +-
.../java/org/apache/doris/catalog/JdbcTable.java | 2 +-
.../java/org/apache/doris/catalog/KeysType.java | 3 +-
.../main/java/org/apache/doris/catalog/MTMV.java | 31 +-
.../java/org/apache/doris/catalog/OlapTable.java | 58 +-
.../java/org/apache/doris/catalog/SchemaTable.java | 11 +
.../java/org/apache/doris/catalog/TableIf.java | 3 +-
.../org/apache/doris/catalog/TableProperty.java | 20 +
.../doris/clone/DynamicPartitionScheduler.java | 84 +-
.../apache/doris/cloud/CacheHotspotManager.java | 11 +-
.../cloud/alter/CloudSchemaChangeHandler.java | 14 +
.../apache/doris/cloud/catalog/CloudReplica.java | 8 +-
.../doris/cloud/catalog/CloudTabletRebalancer.java | 4 +-
.../cloud/datasource/CloudInternalCatalog.java | 2 +-
.../apache/doris/cloud/load/CloudLoadManager.java | 2 +-
.../apache/doris/cloud/storage/DefaultRemote.java | 5 +-
.../doris/cloud/system/CloudSystemInfoService.java | 8 +-
.../transaction/CloudGlobalTransactionMgr.java | 8 +
.../java/org/apache/doris/common/CacheFactory.java | 22 +-
.../java/org/apache/doris/common/FeConstants.java | 26 -
.../org/apache/doris/common/NamedArguments.java | 1 +
.../main/java/org/apache/doris/common/Status.java | 5 +
.../doris/common/cache/NereidsSqlCacheManager.java | 2 +-
.../doris/common/proc/IndexSchemaProcNode.java | 3 +-
.../doris/common/proc/PartitionsProcDir.java | 2 +-
.../apache/doris/common/util/ClassLoaderUtils.java | 2 +-
.../doris/common/util/DynamicPartitionUtil.java | 8 +-
.../org/apache/doris/common/util/ExprUtil.java | 2 +-
.../org/apache/doris/common/util/JsonUtil.java | 12 +
.../org/apache/doris/common/util/LocationPath.java | 12 +
.../apache/doris/common/util/PropertyAnalyzer.java | 22 +-
.../apache/doris/datasource/CatalogFactory.java | 4 +
.../apache/doris/datasource/CatalogProperty.java | 2 +-
.../apache/doris/datasource/ExternalCatalog.java | 5 +-
.../apache/doris/datasource/ExternalDatabase.java | 5 +-
.../apache/doris/datasource/ExternalScanNode.java | 5 +-
.../doris/datasource/ExternalSchemaCache.java | 2 +-
.../org/apache/doris/datasource/ExternalTable.java | 2 +-
.../doris/datasource/FederationBackendPolicy.java | 2 +-
.../org/apache/doris/datasource/FileGroupInfo.java | 13 +-
.../apache/doris/datasource/FileQueryScanNode.java | 16 +-
.../org/apache/doris/datasource/FileScanNode.java | 6 +-
.../apache/doris/datasource/InitCatalogLog.java | 1 +
.../apache/doris/datasource/InitDatabaseLog.java | 1 +
.../apache/doris/datasource/InternalCatalog.java | 36 +-
.../apache/doris/datasource/TableFormatType.java | 3 +-
.../doris/RemoteDorisCompatibleRestClient.java | 137 ++
.../doris/RemoteDorisExternalCatalog.java | 202 ++
.../doris/RemoteDorisExternalDatabase.java | 36 +
.../datasource/doris/RemoteDorisExternalTable.java | 89 +
.../datasource/doris/RemoteDorisRestClient.java | 299 +++
.../doris/source/RemoteDorisScanNode.java | 323 +++
.../datasource/doris/source/RemoteDorisSource.java | 88 +
.../datasource/doris/source/RemoteDorisSplit.java | 53 +
.../doris/datasource/es/source/EsScanNode.java | 3 +-
.../doris/datasource/hive/HMSExternalCatalog.java | 8 +-
.../doris/datasource/hive/HMSExternalTable.java | 43 +-
.../doris/datasource/hive/HMSTransaction.java | 7 +-
.../apache/doris/datasource/hive/HiveDlaTable.java | 10 +-
.../doris/datasource/hive/HiveMetaStoreCache.java | 13 +-
.../doris/datasource/hive/IcebergDlaTable.java | 11 +-
.../doris/datasource/hive/source/HiveScanNode.java | 7 +-
.../apache/doris/datasource/hudi/HudiUtils.java | 8 +-
.../hudi/source/HudiCachedFsViewProcessor.java | 2 +-
.../hudi/source/HudiCachedMetaClientProcessor.java | 1 -
.../hudi/source/HudiCachedPartitionProcessor.java | 2 +-
.../doris/datasource/hudi/source/HudiScanNode.java | 3 +-
.../datasource/iceberg/IcebergExternalTable.java | 11 +-
.../datasource/iceberg/IcebergMetadataCache.java | 8 +-
.../datasource/iceberg/IcebergNereidsUtils.java | 478 ++++
.../datasource/iceberg/IcebergPartitionInfo.java | 6 +-
.../datasource/iceberg/IcebergTransaction.java | 171 +-
.../doris/datasource/iceberg/IcebergUtils.java | 73 +-
.../iceberg/action/BaseIcebergAction.java | 12 +-
.../action/IcebergCherrypickSnapshotAction.java | 5 +-
.../action/IcebergExecuteActionFactory.java | 14 +-
.../action/IcebergExpireSnapshotsAction.java | 6 +-
.../iceberg/action/IcebergFastForwardAction.java | 5 +-
.../action/IcebergRewriteDataFilesAction.java | 93 +-
.../action/IcebergRollbackToSnapshotAction.java | 5 +-
.../action/IcebergRollbackToTimestampAction.java | 5 +-
.../action/IcebergSetCurrentSnapshotAction.java | 5 +-
.../iceberg/rewrite/RewriteDataFileExecutor.java | 213 ++
.../iceberg/rewrite/RewriteDataFilePlanner.java | 362 +++
.../iceberg/rewrite/RewriteDataGroup.java | 112 +
.../iceberg/rewrite/RewriteGroupTask.java | 266 +++
.../datasource/iceberg/rewrite/RewriteResult.java | 111 +
.../datasource/iceberg/source/IcebergScanNode.java | 62 +-
.../doris/datasource/jdbc/JdbcExternalCatalog.java | 2 +-
.../doris/datasource/jdbc/source/JdbcScanNode.java | 25 +-
.../lakesoul/source/LakeSoulScanNode.java | 3 +-
.../maxcompute/source/MaxComputeScanNode.java | 9 +-
.../doris/datasource/metacache/MetaCache.java | 6 +-
.../doris/datasource/odbc/source/OdbcScanNode.java | 19 +-
.../datasource/paimon/PaimonMetadataCache.java | 4 +-
.../apache/doris/datasource/paimon/PaimonUtil.java | 5 +-
.../datasource/paimon/source/PaimonScanNode.java | 29 +-
.../property/constants/MCProperties.java | 2 -
.../property/constants/RemoteDorisProperties.java | 50 +
.../property/metastore/IcebergRestProperties.java | 4 +-
.../property/storage/AzureProperties.java | 117 +-
.../property/storage/AzurePropertyUtils.java | 13 +-
.../datasource/property/storage/COSProperties.java | 1 +
.../datasource/property/storage/GCSProperties.java | 1 +
.../property/storage/HdfsProperties.java | 2 +-
.../{LocalProperties.java => HttpProperties.java} | 71 +-
.../property/storage/LocalProperties.java | 2 +-
.../property/storage/MinioProperties.java | 1 +
.../datasource/property/storage/OBSProperties.java | 1 +
.../property/storage/OSSHdfsProperties.java | 40 +-
.../datasource/property/storage/OSSProperties.java | 18 +-
.../datasource/property/storage/S3Properties.java | 1 +
.../property/storage/StorageProperties.java | 28 +-
.../property/storage/exception/AzureAuthType.java} | 9 +-
.../source/TrinoConnectorScanNode.java | 7 +-
.../datasource/tvf/source/MetadataScanNode.java | 4 +-
.../doris/datasource/tvf/source/TVFScanNode.java | 3 +-
.../doris/deploy/impl/AmbariDeployManager.java | 309 ---
.../apache/doris/deploy/impl/K8sDeployManager.java | 413 ----
.../java/org/apache/doris/fs/FileSystemType.java | 3 +-
.../java/org/apache/doris/fs/SchemaTypeMapper.java | 5 +-
.../fs/TransactionScopeCachingDirectoryLister.java | 2 +-
.../java/org/apache/doris/httpv2/HttpServer.java | 28 +-
.../httpv2/config/HttpToHttpsJettyConfig.java | 22 +-
.../apache/doris/httpv2/config/WebConfigurer.java | 31 +-
.../config/WebServerFactoryCustomizerConfig.java | 2 +-
.../doris/httpv2/controller/BaseController.java | 7 +-
.../doris/httpv2/controller/ConfigController.java | 4 +-
.../httpv2/controller/HardwareInfoController.java | 2 +-
.../doris/httpv2/controller/HelpController.java | 2 +-
.../doris/httpv2/controller/LogController.java | 2 +-
.../doris/httpv2/controller/LoginController.java | 4 +-
.../doris/httpv2/controller/LogoutController.java | 7 +-
.../doris/httpv2/controller/SessionController.java | 2 +-
.../doris/httpv2/controller/SystemController.java | 2 +-
.../doris/httpv2/entity/ResponseEntityBuilder.java | 5 +
.../doris/httpv2/interceptor/AuthInterceptor.java | 21 +-
.../httpv2/interceptor/ServletTraceIterceptor.java | 18 +-
.../doris/httpv2/meta/ColocateMetaService.java | 4 +-
.../org/apache/doris/httpv2/meta/MetaService.java | 4 +-
.../apache/doris/httpv2/rest/BackendsAction.java | 4 +-
.../doris/httpv2/rest/BootstrapFinishAction.java | 5 +-
.../apache/doris/httpv2/rest/CancelLoadAction.java | 5 +-
.../doris/httpv2/rest/CheckDecommissionAction.java | 4 +-
.../doris/httpv2/rest/CheckWalSizeAction.java | 4 +-
.../apache/doris/httpv2/rest/ConnectionAction.java | 4 +-
.../apache/doris/httpv2/rest/CopyIntoAction.java | 4 +-
.../apache/doris/httpv2/rest/DebugPointAction.java | 5 +-
.../doris/httpv2/rest/ExtraBasepathAction.java | 5 +-
.../doris/httpv2/rest/FeVersionInfoAction.java | 4 +-
.../apache/doris/httpv2/rest/GetDdlStmtAction.java | 4 +-
.../apache/doris/httpv2/rest/GetLogFileAction.java | 4 +-
.../doris/httpv2/rest/GetSmallFileAction.java | 4 +-
.../doris/httpv2/rest/GetStreamLoadState.java | 5 +-
.../org/apache/doris/httpv2/rest/HealthAction.java | 16 +-
.../org/apache/doris/httpv2/rest/LoadAction.java | 4 +-
.../apache/doris/httpv2/rest/MetaInfoAction.java | 4 +-
.../doris/httpv2/rest/MetaReplayerCheckAction.java | 4 +-
.../apache/doris/httpv2/rest/MetricsAction.java | 4 +-
.../apache/doris/httpv2/rest/MinidumpAction.java | 5 +-
.../apache/doris/httpv2/rest/ProfileAction.java | 4 +-
.../apache/doris/httpv2/rest/QueryStatsAction.java | 4 +-
.../doris/httpv2/rest/RestApiStatusCode.java | 3 +-
.../doris/httpv2/rest/RestBaseController.java | 4 +-
.../apache/doris/httpv2/rest/RowCountAction.java | 4 +-
.../apache/doris/httpv2/rest/SetConfigAction.java | 4 +-
.../org/apache/doris/httpv2/rest/ShowAction.java | 4 +-
.../doris/httpv2/rest/StmtExecutionAction.java | 4 +-
.../doris/httpv2/rest/StorageTypeCheckAction.java | 4 +-
.../doris/httpv2/rest/TableQueryPlanAction.java | 4 +-
.../doris/httpv2/rest/TableRowCountAction.java | 4 +-
.../doris/httpv2/rest/TableSchemaAction.java | 56 +-
.../org/apache/doris/httpv2/rest/UploadAction.java | 4 +-
.../doris/httpv2/rest/manager/ClusterAction.java | 4 +-
.../doris/httpv2/rest/manager/HttpUtils.java | 71 +-
.../doris/httpv2/rest/manager/NodeAction.java | 4 +-
.../httpv2/rest/manager/QueryProfileAction.java | 4 +-
.../rest/response/GsonSchemaResponse.java} | 20 +-
.../httpv2/restv2/AddStoragePolicyAction.java | 4 +-
.../doris/httpv2/restv2/ESCatalogAction.java | 4 +-
.../apache/doris/httpv2/restv2/ImportAction.java | 4 +-
.../doris/httpv2/restv2/MetaInfoActionV2.java | 4 +-
.../doris/httpv2/restv2/StatisticAction.java | 4 +-
.../doris/httpv2/util/StatementSubmitter.java | 2 +-
.../util/streamresponse/JsonStreamResponse.java | 2 +-
.../util/streamresponse/StreamResponseInf.java | 3 +-
.../org/apache/doris/job/base/AbstractJob.java | 2 +-
.../doris/job/executor/DispatchTaskHandler.java | 2 +-
.../doris/job/extensions/insert/InsertJob.java | 2 +-
.../doris/job/extensions/insert/InsertTask.java | 2 +
.../insert/streaming/StreamingInsertJob.java | 10 +-
.../insert/streaming/StreamingInsertTask.java | 3 +-
.../apache/doris/job/extensions/mtmv/MTMVTask.java | 9 +-
.../org/apache/doris/job/manager/JobManager.java | 5 +-
.../java/org/apache/doris/job/offset/Offset.java | 2 +
.../org/apache/doris/job/offset/s3/S3Offset.java | 10 +-
.../job/offset/s3/S3SourceOffsetProvider.java | 30 +-
.../apache/doris/job/scheduler/JobScheduler.java | 2 +-
.../job/scheduler/StreamingTaskScheduler.java | 24 +-
.../org/apache/doris/journal/JournalEntity.java | 7 +-
.../org/apache/doris/load/StreamLoadHandler.java | 2 +-
.../org/apache/doris/load/loadv2/BulkLoadJob.java | 4 +-
.../apache/doris/load/loadv2/LoadLoadingTask.java | 1 +
.../org/apache/doris/load/loadv2/LoadManager.java | 6 +-
.../routineload/AbstractDataSourceProperties.java | 2 +-
.../load/routineload/KafkaRoutineLoadJob.java | 4 +-
.../doris/load/routineload/KafkaTaskInfo.java | 7 +-
.../doris/load/routineload/RoutineLoadManager.java | 2 +-
.../kafka/KafkaDataSourceProperties.java | 2 +-
.../org/apache/doris/load/sync/SyncFailMsg.java | 36 -
.../java/org/apache/doris/load/sync/SyncJob.java | 115 -
.../doris/load/sync/canal/CanalDestination.java | 30 -
.../apache/doris/load/sync/canal/CanalSyncJob.java | 36 -
.../java/org/apache/doris/master/MasterImpl.java | 30 +-
.../org/apache/doris/master/ReportHandler.java | 2 +-
.../org/apache/doris/mtmv/MTMVPartitionInfo.java | 22 +-
.../org/apache/doris/mtmv/MTMVPartitionUtil.java | 4 +-
.../java/org/apache/doris/mtmv/MTMVPlanUtil.java | 48 +-
.../org/apache/doris/mtmv/MTMVPropertyUtil.java | 8 +
.../doris/mtmv/MTMVRefreshPartitionSnapshot.java | 2 +-
.../org/apache/doris/mtmv/MTMVRefreshSnapshot.java | 2 +-
.../MTMVRelatedPartitionDescTransferGenerator.java | 2 +-
.../java/org/apache/doris/mtmv/MTMVRelation.java | 2 +-
.../org/apache/doris/mtmv/MTMVRelationManager.java | 2 +-
.../org/apache/doris/mtmv/MTMVRewriteUtil.java | 19 +-
.../main/java/org/apache/doris/mtmv/MTMVUtil.java | 2 +-
.../java/org/apache/doris/mysql/MysqlChannel.java | 38 +-
.../doris/mysql/authenticate/ldap/LdapClient.java | 4 +-
.../org/apache/doris/mysql/privilege/Auth.java | 2 +-
.../doris/mysql/privilege/GlobalPrivTable.java | 2 +-
.../org/apache/doris/mysql/privilege/Role.java | 2 +-
.../apache/doris/mysql/privilege/UserManager.java | 2 +-
.../doris/mysql/privilege/UserRoleManager.java | 2 +-
.../org/apache/doris/nereids/NereidsPlanner.java | 29 +-
.../org/apache/doris/nereids/StatementContext.java | 21 +
.../doris/nereids/analyzer/UnboundRelation.java | 2 +-
.../doris/nereids/glue/LogicalPlanAdapter.java | 11 +-
.../glue/translator/ExpressionTranslator.java | 11 +-
.../glue/translator/PhysicalPlanTranslator.java | 82 +-
.../glue/translator/PlanTranslatorContext.java | 7 -
.../glue/translator/RuntimeFilterTranslator.java | 3 +-
.../doris/nereids/jobs/executor/Rewriter.java | 17 +-
.../jobs/joinorder/hypergraph/HyperGraph.java | 6 +-
.../joinorder/hypergraph/bitmap/LongBitmap.java | 5 +
.../joinorder/hypergraph/node/StructInfoNode.java | 74 +-
.../jobs/rewrite/BottomUpVisitorRewriteJob.java | 4 +-
.../nereids/jobs/rewrite/CustomRewriteJob.java | 4 +-
.../nereids/jobs/rewrite/PlanTreeRewriteJob.java | 12 +-
.../nereids/jobs/rewrite/RewriteJobContext.java | 5 +
.../jobs/rewrite/RootPlanTreeRewriteJob.java | 1 +
.../jobs/rewrite/TopDownVisitorRewriteJob.java | 4 +-
.../doris/nereids/load/NereidsFileGroupInfo.java | 13 +-
.../doris/nereids/load/NereidsLoadTaskInfo.java | 3 +
.../nereids/load/NereidsRoutineLoadTaskInfo.java | 15 +-
.../nereids/load/NereidsStreamLoadPlanner.java | 7 +-
.../doris/nereids/parser/LogicalPlanBuilder.java | 5 +-
.../parser/LogicalPlanBuilderForEncryption.java | 26 +-
.../apache/doris/nereids/parser/NereidsParser.java | 2 +-
.../post/materialize/LazyMaterializeTopN.java | 11 +-
.../post/materialize/LazySlotPruning.java | 12 +-
.../org/apache/doris/nereids/rules/RuleSet.java | 18 +
.../org/apache/doris/nereids/rules/RuleType.java | 41 +-
.../nereids/rules/analysis/BindExpression.java | 2 +-
.../doris/nereids/rules/analysis/BindRelation.java | 3 +-
.../doris/nereids/rules/analysis/BindSink.java | 4 +-
.../rules/analysis/DatetimeFunctionBinder.java | 3 +
.../nereids/rules/analysis/UserAuthentication.java | 2 +-
.../mv/AbstractMaterializedViewAggregateRule.java | 6 +-
.../mv/AbstractMaterializedViewJoinRule.java | 4 +-
.../AbstractMaterializedViewLimitOrTopNRule.java | 75 +
.../mv/AbstractMaterializedViewRule.java | 191 +-
.../mv/AbstractMaterializedViewScanRule.java | 9 +
.../mv/AbstractMaterializedViewWindowRule.java | 124 +
.../rules/exploration/mv/HyperGraphComparator.java | 120 +-
.../mv/InitMaterializationContextHook.java | 5 +-
.../mv/LogicalCompatibilityContext.java | 51 +-
.../exploration/mv/MaterializationContext.java | 2 +-
.../mv/MaterializedViewLimitAggregateRule.java | 91 +
.../mv/MaterializedViewLimitJoinRule.java | 93 +
.../mv/MaterializedViewLimitScanRule.java | 88 +
.../mv/MaterializedViewTopNAggregateRule.java | 96 +
.../mv/MaterializedViewTopNJoinRule.java | 101 +
.../mv/MaterializedViewTopNScanRule.java | 90 +
.../exploration/mv/MaterializedViewUtils.java | 73 +-
.../mv/MaterializedViewWindowAggregateRule.java | 87 +
.../mv/MaterializedViewWindowJoinRule.java | 97 +
.../mv/MaterializedViewWindowScanRule.java | 96 +
.../rules/exploration/mv/PartitionCompensator.java | 43 +-
.../mv/PartitionIncrementMaintainer.java | 17 +-
.../mv/PreMaterializedViewRewriter.java | 32 +-
.../nereids/rules/exploration/mv/Predicates.java | 57 +-
.../rules/exploration/mv/RelatedTableInfo.java | 6 +-
.../nereids/rules/exploration/mv/StructInfo.java | 333 ++-
.../exploration/mv/mapping/ExpressionMapping.java | 10 +-
.../rules/exploration/mv/mapping/SlotMapping.java | 2 +-
.../rules/expression/ExpressionOptimization.java | 2 +
.../rules/expression/ExpressionRuleType.java | 1 +
.../nereids/rules/expression/check/CheckCast.java | 4 +-
.../nereids/rules/expression/rules/AddMinMax.java | 70 +-
.../rules/expression/rules/ConditionRewrite.java | 10 +-
.../expression/rules/NullSafeEqualToEqual.java | 26 +-
.../expression/rules/PartitionRangeExpander.java | 2 +-
.../expression/rules/PushIntoCaseWhenBranch.java | 219 ++
.../rules/expression/rules/RangeInference.java | 1446 +++++++++---
.../rules/expression/rules/SimplifyRange.java | 81 +-
.../rules/implementation/AggregateStrategies.java | 110 +-
.../LogicalWindowToPhysicalWindow.java | 2 +-
.../nereids/rules/rewrite/InferPredicates.java | 6 +
.../rules/rewrite/PruneFileScanPartition.java | 2 +-
.../nereids/rules/rewrite/PruneOlapScanTablet.java | 2 +-
.../nereids/rules/rewrite/PullUpPredicates.java | 19 +-
.../rules/rewrite/PushDownFilterThroughWindow.java | 5 +-
.../rules/rewrite/PushDownJoinOnAssertNumRows.java | 18 +-
.../rewrite/PushDownVectorTopNIntoOlapScan.java | 2 +-
.../rules/rewrite/QueryPartitionCollector.java | 2 +
.../doris/nereids/stats/ExpressionEstimation.java | 2 +-
.../apache/doris/nereids/stats/JoinEstimation.java | 38 +-
.../doris/nereids/stats/StatsCalculator.java | 5 +-
.../trees/copier/LogicalPlanDeepCopier.java | 8 +-
.../doris/nereids/trees/expressions/Not.java | 6 +-
.../trees/expressions/TimestampArithmetic.java | 20 +-
.../functions/ExplicitlyCastableSignature.java | 5 +-
.../functions/ImplicitlyCastableSignature.java | 20 +-
.../PropagateNullableOnDateOrTimeLikeV2Args.java | 39 -
.../functions/executable/DateTimeArithmetic.java | 8 +
.../executable/DateTimeExtractAndTransform.java | 28 +-
.../functions/executable/NumericArithmetic.java | 13 +
.../expressions/functions/scalar/DateDiff.java | 4 +-
.../scalar/{DaySecondAdd.java => DayHourAdd.java} | 24 +-
.../expressions/functions/scalar/DayName.java | 4 +-
.../expressions/functions/scalar/DayOfMonth.java | 4 +-
.../expressions/functions/scalar/DayOfWeek.java | 4 +-
.../expressions/functions/scalar/DayOfYear.java | 4 +-
.../expressions/functions/scalar/DaySecondAdd.java | 4 +-
.../expressions/functions/scalar/DaysAdd.java | 4 +-
.../expressions/functions/scalar/DaysDiff.java | 4 +-
.../expressions/functions/scalar/DaysSub.java | 4 +-
.../scalar/{ToDays.java => Factorial.java} | 29 +-
.../trees/expressions/functions/scalar/Hour.java | 4 +-
.../expressions/functions/scalar/HoursAdd.java | 4 +-
.../expressions/functions/scalar/HoursDiff.java | 4 +-
.../expressions/functions/scalar/HoursSub.java | 4 +-
.../expressions/functions/scalar/LastDay.java | 4 +-
.../functions/scalar/MicroSecondTimestamp.java | 4 +-
.../functions/scalar/MicroSecondsAdd.java | 4 +-
.../functions/scalar/MicroSecondsDiff.java | 4 +-
.../functions/scalar/MicroSecondsSub.java | 4 +-
.../expressions/functions/scalar/Microsecond.java | 4 +-
.../functions/scalar/MilliSecondTimestamp.java | 4 +-
.../functions/scalar/MilliSecondsAdd.java | 4 +-
.../functions/scalar/MilliSecondsDiff.java | 4 +-
.../functions/scalar/MilliSecondsSub.java | 4 +-
.../trees/expressions/functions/scalar/Minute.java | 4 +-
.../expressions/functions/scalar/MinutesAdd.java | 4 +-
.../expressions/functions/scalar/MinutesDiff.java | 4 +-
.../expressions/functions/scalar/MinutesSub.java | 4 +-
.../trees/expressions/functions/scalar/Month.java | 4 +-
.../expressions/functions/scalar/MonthName.java | 4 +-
.../expressions/functions/scalar/MonthsAdd.java | 4 +-
.../functions/scalar/MonthsBetween.java | 4 +-
.../expressions/functions/scalar/MonthsDiff.java | 4 +-
.../expressions/functions/scalar/MonthsSub.java | 4 +-
.../expressions/functions/scalar/Quarter.java | 4 +-
.../expressions/functions/scalar/QuartersAdd.java | 4 +-
.../expressions/functions/scalar/QuartersDiff.java | 4 +-
.../expressions/functions/scalar/QuartersSub.java | 4 +-
.../trees/expressions/functions/scalar/Second.java | 4 +-
.../functions/scalar/SecondTimestamp.java | 4 +-
.../expressions/functions/scalar/SecondsAdd.java | 4 +-
.../expressions/functions/scalar/SecondsDiff.java | 4 +-
.../expressions/functions/scalar/SecondsSub.java | 4 +-
.../trees/expressions/functions/scalar/Time.java | 4 +-
.../expressions/functions/scalar/TimeDiff.java | 4 +-
.../expressions/functions/scalar/Timestamp.java | 4 +-
.../trees/expressions/functions/scalar/ToDays.java | 4 +-
.../expressions/functions/scalar/ToMonday.java | 4 +-
.../trees/expressions/functions/scalar/Week.java | 4 +-
.../expressions/functions/scalar/WeekOfYear.java | 4 +-
.../expressions/functions/scalar/Weekday.java | 4 +-
.../expressions/functions/scalar/WeeksAdd.java | 4 +-
.../expressions/functions/scalar/WeeksDiff.java | 4 +-
.../expressions/functions/scalar/WeeksSub.java | 4 +-
.../expressions/functions/scalar/WidthBucket.java | 10 +-
.../trees/expressions/functions/scalar/Year.java | 4 +-
.../expressions/functions/scalar/YearOfWeek.java | 4 +-
.../expressions/functions/scalar/YearWeek.java | 4 +-
.../expressions/functions/scalar/YearsAdd.java | 4 +-
.../expressions/functions/scalar/YearsDiff.java | 4 +-
.../expressions/functions/scalar/YearsSub.java | 4 +-
.../trees/expressions/functions/table/Http.java | 57 +
.../functions/table/TableValuedFunction.java | 5 +-
.../expressions/literal/DateTimeV2Literal.java | 38 +
.../trees/expressions/literal/Interval.java | 1 +
.../expressions/visitor/ScalarFunctionVisitor.java | 10 +
.../visitor/TableValuedFunctionVisitor.java | 5 +
.../doris/nereids/trees/plans/AbstractPlan.java | 12 +-
.../apache/doris/nereids/trees/plans/FakePlan.java | 2 +-
.../org/apache/doris/nereids/trees/plans/Plan.java | 6 +-
.../plans/commands/AdminCheckTabletsCommand.java | 2 +-
.../AdminCreateClusterSnapshotCommand.java | 13 +-
.../plans/commands/AlterRoutineLoadCommand.java | 2 +-
.../plans/commands/AlterStorageVaultCommand.java | 7 +-
.../plans/commands/CreateFunctionCommand.java | 2 +-
.../commands/CreateMaterializedViewCommand.java | 3 +-
.../trees/plans/commands/CreateStageCommand.java | 4 +-
.../plans/commands/CreateTableLikeCommand.java | 2 +-
.../trees/plans/commands/ExecuteActionCommand.java | 1 +
.../trees/plans/commands/ExplainCommand.java | 3 +
.../commands/GrantResourcePrivilegeCommand.java | 4 +-
.../plans/commands/GrantTablePrivilegeCommand.java | 4 +-
.../nereids/trees/plans/commands/LoadCommand.java | 18 +-
.../commands/RevokeResourcePrivilegeCommand.java | 4 +-
.../commands/RevokeTablePrivilegeCommand.java | 4 +-
.../plans/commands/ShowLoadWarningsCommand.java | 2 +-
.../trees/plans/commands/ShowTypeCastCommand.java | 16 -
.../commands/alter/AlterRepositoryCommand.java | 2 +-
.../trees/plans/commands/info/BuildIndexOp.java | 32 +-
.../trees/plans/commands/info/CopyIntoInfo.java | 13 +-
.../plans/commands/info}/CopyIntoProperties.java | 35 +-
.../trees/plans/commands/info}/CopyProperties.java | 53 +-
.../trees/plans/commands/info/CreateIndexOp.java | 12 +-
.../trees/plans/commands/info/CreateTableInfo.java | 4 +-
.../commands/info/DistributionDescriptor.java | 2 +-
.../trees/plans/commands/info/IndexDefinition.java | 180 +-
.../trees/plans/commands/info/LabelNameInfo.java | 9 -
.../commands/info/MTMVPartitionDefinition.java | 30 +-
.../commands/info/ModifyTablePropertiesOp.java | 14 +
.../plans/commands/info/PartitionTableInfo.java | 69 +
.../trees/plans/commands/info/RefreshMTMVInfo.java | 2 +-
.../plans/commands/info}/StageProperties.java | 68 +-
.../commands/insert/AbstractInsertExecutor.java | 6 +-
.../insert/BaseExternalTableInsertExecutor.java | 12 +-
.../commands/insert/DictionaryInsertExecutor.java | 13 +-
.../commands/insert/IcebergRewriteExecutor.java | 60 +
.../commands/insert/InsertIntoTableCommand.java | 2 +-
.../insert/InsertOverwriteTableCommand.java | 2 +-
.../trees/plans/commands/insert/InsertUtils.java | 100 +-
.../insert/OlapGroupCommitInsertExecutor.java | 13 +-
.../plans/commands/insert/OlapInsertExecutor.java | 18 +-
.../plans/commands/insert/RewriteTableCommand.java | 326 +++
.../plans/commands/merge/MergeIntoCommand.java | 7 +-
.../plans/commands/use/UseCloudClusterCommand.java | 12 +-
.../job/UnassignedScanBucketOlapTableJob.java | 62 +-
.../trees/plans/logical/LogicalAggregate.java | 7 +
.../trees/plans/logical/LogicalCheckPolicy.java | 2 +-
.../trees/plans/logical/LogicalSetOperation.java | 21 +-
.../nereids/trees/plans/logical/LogicalView.java | 2 +-
.../nereids/trees/plans/logical/LogicalWindow.java | 9 +
.../plans/visitor/ExpressionLineageReplacer.java | 7 +
.../apache/doris/nereids/types/AggStateType.java | 4 +-
.../org/apache/doris/nereids/types/DataType.java | 4 -
.../org/apache/doris/nereids/util/DateUtils.java | 5 +-
.../apache/doris/nereids/util/ExpressionUtils.java | 57 +-
.../doris/nereids/util/FrontendConjunctsUtils.java | 2 +-
.../org/apache/doris/nereids/util/PlanUtils.java | 9 +-
.../doris/nereids/util/TypeCoercionUtils.java | 14 +-
.../java/org/apache/doris/persist/EditLog.java | 6 -
.../org/apache/doris/persist/OperationType.java | 2 +
.../org/apache/doris/persist/gson/GsonUtils.java | 14 +-
.../org/apache/doris/planner/AggregationNode.java | 31 +-
.../org/apache/doris/planner/AnalyticEvalNode.java | 63 +-
.../apache/doris/planner/AssertNumRowsNode.java | 11 +-
.../java/org/apache/doris/planner/CTEScanNode.java | 3 +-
.../org/apache/doris/planner/DataGenScanNode.java | 3 +-
.../org/apache/doris/planner/DataStreamSink.java | 5 +-
.../doris/planner/DistributedPlanColocateRule.java | 32 -
.../org/apache/doris/planner/EmptySetNode.java | 3 +-
.../org/apache/doris/planner/ExchangeNode.java | 28 +-
.../org/apache/doris/planner/FileLoadScanNode.java | 3 +-
.../apache/doris/planner/GroupCommitScanNode.java | 4 +-
.../doris/planner/HashDistributionPruner.java | 2 +-
.../org/apache/doris/planner/HashJoinNode.java | 159 +-
.../apache/doris/planner/JoinCostEvaluation.java | 152 --
.../org/apache/doris/planner/JoinNodeBase.java | 40 +-
.../apache/doris/planner/MaterializationNode.java | 3 +-
.../apache/doris/planner/NestedLoopJoinNode.java | 39 +-
.../org/apache/doris/planner/OlapScanNode.java | 67 +-
.../apache/doris/planner/PartitionSortNode.java | 4 +-
.../org/apache/doris/planner/PlanFragment.java | 12 +-
.../java/org/apache/doris/planner/PlanNode.java | 140 +-
.../org/apache/doris/planner/PlannerContext.java | 32 -
.../java/org/apache/doris/planner/RepeatNode.java | 33 +-
.../org/apache/doris/planner/ResultFileSink.java | 3 -
.../org/apache/doris/planner/RuntimeFilter.java | 5 -
.../java/org/apache/doris/planner/ScanNode.java | 16 +-
.../org/apache/doris/planner/SchemaScanNode.java | 10 +-
.../java/org/apache/doris/planner/SelectNode.java | 8 +-
.../org/apache/doris/planner/SetOperationNode.java | 24 +-
.../java/org/apache/doris/planner/SortNode.java | 62 +-
.../apache/doris/planner/TableFunctionNode.java | 3 +-
.../java/org/apache/doris/planner/UnionNode.java | 3 +-
.../doris/planner/normalize/PredicateToRange.java | 20 -
.../doris/plugin/audit/AuditStreamLoader.java | 1 +
.../org/apache/doris/qe/AbstractJobProcessor.java | 18 +
.../java/org/apache/doris/qe/ConnectContext.java | 4 +-
.../java/org/apache/doris/qe/JobProcessor.java | 2 +
.../org/apache/doris/qe/NereidsCoordinator.java | 4 +
.../main/java/org/apache/doris/qe/QueryState.java | 10 +
.../java/org/apache/doris/qe/ResultReceiver.java | 3 +
.../java/org/apache/doris/qe/SessionVariable.java | 29 +
.../java/org/apache/doris/qe/SimpleScheduler.java | 2 +-
.../java/org/apache/doris/qe/StmtExecutor.java | 21 +-
.../org/apache/doris/qe/cache/CacheAnalyzer.java | 2 +-
.../org/apache/doris/qe/runtime/LoadProcessor.java | 25 +-
.../qe/runtime/MultiFragmentsPipelineTask.java | 25 +-
.../apache/doris/qe/runtime/QueryProcessor.java | 34 +-
.../doris/qe/runtime/ThriftPlansBuilder.java | 2 +-
.../apache/doris/service/FrontendServiceImpl.java | 9 +-
.../arrowflight/FlightSqlConnectProcessor.java | 2 +-
.../apache/doris/statistics/AggStatsDerive.java | 87 -
.../doris/statistics/AnalyticEvalStatsDerive.java | 41 -
.../apache/doris/statistics/BaseAnalysisTask.java | 1 +
.../apache/doris/statistics/BaseStatsDerive.java | 152 --
.../org/apache/doris/statistics/DeriveFactory.java | 65 -
.../doris/statistics/EmptySetStatsDerive.java | 31 -
.../doris/statistics/ExchangeStatsDerive.java | 34 -
.../doris/statistics/HashJoinStatsDerive.java | 254 ---
.../org/apache/doris/statistics/Histogram.java | 2 +-
.../doris/statistics/HistogramCacheLoader.java | 2 +-
.../apache/doris/statistics/MysqlStatsDerive.java | 33 -
.../statistics/NestedLoopJoinStatsDerive.java | 48 -
.../apache/doris/statistics/OlapAnalysisTask.java | 2 +
.../doris/statistics/OlapScanStatsDerive.java | 93 -
.../apache/doris/statistics/SelectStatsDerive.java | 35 -
.../apache/doris/statistics/StatisticalType.java | 61 -
.../apache/doris/statistics/StatisticsCache.java | 2 +-
.../apache/doris/statistics/StatisticsCleaner.java | 2 +-
.../doris/statistics/StatsRecursiveDerive.java | 60 -
.../doris/statistics/TableFunctionStatsDerive.java | 34 -
.../doris/statistics/util/StatisticsUtil.java | 2 +-
.../main/java/org/apache/doris/system/Backend.java | 52 +-
.../java/org/apache/doris/system/HeartbeatMgr.java | 38 +-
.../org/apache/doris/system/SystemInfoService.java | 40 +-
.../tablefunction/FileTableValuedFunction.java | 3 +
.../org/apache/doris/tablefunction/HFUtils.java | 755 +++++++
.../tablefunction/HttpTableValuedFunction.java | 106 +
.../tablefunction/JdbcQueryTableValueFunction.java | 4 +-
.../doris/tablefunction/MetadataGenerator.java | 6 +-
.../PartitionsTableValuedFunction.java | 2 +-
.../doris/tablefunction/TableValuedFunctionIf.java | 4 +-
.../apache/doris/task/AgentTaskCleanupDaemon.java | 83 +
.../java/org/apache/doris/task/AgentTaskQueue.java | 13 +
.../org/apache/doris/task/CreateReplicaTask.java | 2 +-
.../main/java/org/apache/doris/task/PushTask.java | 14 +-
.../doris/transaction/DatabaseTransactionMgr.java | 2 +-
.../doris/transaction/PublishVersionDaemon.java | 2 +-
.../hadoop/hive/metastore/HiveMetaStoreClient.java | 2 +-
.../org/apache/doris/alter/CloudIndexTest.java | 31 +-
.../org/apache/doris/alter/IndexChangeJobTest.java | 80 +-
.../doris/alter/SchemaChangeHandlerTest.java | 6 +-
.../apache/doris/alter/SchemaChangeJobV2Test.java | 3 +-
.../doris/analysis/PartitionExprUtilTest.java | 3 +-
.../analysis/VarBinaryLiteralAnalysisTest.java | 93 +
.../org/apache/doris/catalog/ColumnStatTest.java | 87 -
.../org/apache/doris/catalog/ColumnTypeTest.java | 188 +-
.../org/apache/doris/catalog/FunctionSetTest.java | 70 -
.../doris/catalog/IndexPropertiesOrderTest.java | 10 +-
.../java/org/apache/doris/catalog/IndexTest.java | 20 +-
.../catalog/InternalSchemaInitializerTest.java | 8 +-
.../org/apache/doris/catalog/OlapTableTest.java | 4 +-
.../apache/doris/common/CacheBulkLoaderTest.java | 2 +-
.../org/apache/doris/common/CacheFactoryTest.java | 6 +-
.../doris/common/proc/IndexSchemaProcNodeTest.java | 2 +
.../doris/common/proc/IndexesProcNodeTest.java | 2 +-
.../apache/doris/common/util/LocationPathTest.java | 17 +
.../doris/RemoteDorisCompatibleRestClientTest.java | 72 +
.../doris/RemoteDorisRestClientTest.java | 108 +
.../iceberg/IcebergNereidsUtilsTest.java | 1004 +++++++++
.../doris/datasource/iceberg/IcebergUtilsTest.java | 8 +-
.../rewrite/RewriteDataFilePlannerTest.java | 1165 ++++++++++
.../metacache/MetaCacheDeadlockTest.java | 114 +
.../paimon/source/PaimonScanNodeTest.java | 52 +-
.../property/storage/AzurePropertiesTest.java | 47 +-
.../property/storage/AzurePropertyUtilsTest.java | 16 +
.../property/storage/OSSHdfsPropertiesTest.java | 67 +-
.../property/storage/OSSPropertiesTest.java | 2 +-
.../doris/deploy/AmbariDeployManagerTest.java | 101 -
.../apache/doris/deploy/K8sDeployManagerTest.java | 67 -
.../external/hms/MetastoreEventFactoryTest.java | 2 +-
.../load/routineload/KafkaRoutineLoadJobTest.java | 2 -
.../routineload/RoutineLoadTaskSchedulerTest.java | 2 -
.../org/apache/doris/mtmv/MTMVPlanUtilTest.java | 9 +-
.../apache/doris/mtmv/MTMVRelationManagerTest.java | 2 +-
.../java/org/apache/doris/mtmv/MTMVTaskTest.java | 2 +-
.../doris/mysql/MysqlSerializerVarbinaryTest.java | 143 ++
.../mysql/authenticate/ldap/LdapClientTest.java | 114 +-
.../translator/PhysicalPlanTranslatorTest.java | 2 +-
.../jobs/joinorder/hypergraph/HyperGraphTest.java | 25 +-
.../apache/doris/nereids/mv/PredicatesTest.java | 190 ++
.../apache/doris/nereids/mv/StructInfoTest.java | 367 +++
.../doris/nereids/parser/EncryptSQLTest.java | 15 +-
.../nereids/privileges/TestCheckPrivileges.java | 2 +-
.../nereids/rules/analysis/GenerateFunction.java | 1644 --------------
.../rules/exploration/mv/BuildStructInfoTest.java | 107 +
.../exploration/mv/HyperGraphComparatorTest.java | 47 +-
.../exploration/mv/PartitionColumnTraceTest.java | 88 +-
.../rules/expression/ExpressionRewriteTest.java | 22 +-
.../expression/ExpressionRewriteTestHelper.java | 10 +
.../nereids/rules/expression/FoldConstantTest.java | 25 +
.../rules/expression/SimplifyRangeTest.java | 252 ++-
.../rules/PushIntoCaseWhenBranchTest.java | 92 +
.../LogicalWindowToPhysicalWindowTest.java | 2 +-
.../nereids/rules/rewrite/InferPredicatesTest.java | 54 +
.../rewrite/PushDownJoinOnAssertNumRowsTest.java | 22 +
.../org/apache/doris/nereids/stats/DummyPlan.java | 2 +-
.../trees/copier/LogicalPlanDeepCopierTest.java | 76 +-
.../nereids/trees/expressions/FunctionTest.java | 161 --
.../DateTimeExtractAndTransformTest.java | 18 +
.../AdminCreateClusterSnapshotCommandTest.java | 48 +-
.../trees/plans/commands/BackupCommandTest.java | 2 +-
.../plans/commands/CreateStageCommandTest.java | 4 +-
.../trees/plans/commands/LoadCommandTest.java | 6 +-
.../trees/plans/commands/RestoreCommandTest.java | 2 +-
.../commands/UseCloudClusterCommandTest.java} | 29 +-
.../plans/commands/insert/InsertUtilsTest.java | 220 ++
.../plans/commands/merge/MergeIntoCommandTest.java | 11 +-
.../doris/nereids/util/ExpressionUtilsTest.java | 46 +
.../persist/TableAddOrDropColumnsInfoTest.java | 4 +-
.../doris/planner/HashDistributionPrunerTest.java | 2 +-
.../doris/planner/JoinCostEvaluationTest.java | 130 --
.../org/apache/doris/planner/OlapScanNodeTest.java | 10 +-
.../doris/planner/QueryCacheNormalizerTest.java | 2 +-
.../java/org/apache/doris/qe/LocalShuffleTest.java | 2 +-
.../doris/qe/LocalShuffleWithBucketJoinTest.java | 86 +
.../org/apache/doris/qe/OlapQueryCacheTest.java | 1 -
.../doris/statistics/OlapAnalysisTaskTest.java | 5 +
.../apache/doris/tablefunction/HFUtilsTest.java | 416 ++++
.../doris/utframe/DemoMultiBackendsTest.java | 3 +-
fe/hive-udf/pom.xml | 14 +
fe/pom.xml | 206 +-
gensrc/proto/cloud.proto | 3 +-
gensrc/proto/internal_service.proto | 2 +
gensrc/script/Makefile | 10 +-
gensrc/script/doris_builtins_functions.py | 2370 --------------------
gensrc/script/gen_builtins_functions.py | 241 --
gensrc/thrift/Descriptors.thrift | 8 +
gensrc/thrift/PaloInternalService.thrift | 5 +
gensrc/thrift/PlanNodes.thrift | 14 +
gensrc/thrift/Status.thrift | 3 +
gensrc/thrift/Types.thrift | 4 +-
pytest/lib/palo_client.py | 68 -
pytest/lib/palo_job.py | 36 -
regression-test/conf/regression-conf.groovy | 1 +
.../test_s3_load_with_tmp_token.out} | 8 +-
.../test_backup_restore_retention_count.out} | 12 +-
.../json/json_group_by_and_distinct.out | 25 +
.../data/export_p0/test_outfile_csv_compress.out | 240 ++
.../action/test_iceberg_execute_actions.out | 3 -
.../action/test_iceberg_rewrite_data_files.out | 70 +
...iceberg_rewrite_data_files_where_conditions.out | 38 +
..._runtime_filter_partition_pruning_transform.out | 108 +
.../iceberg/test_iceberg_transform_partitions.out | 5 +
.../info_schema_db/test_info_schema_db.out | 12 +-
.../jdbc/test_query_tvf_cross_catalog.out} | 11 +-
.../external_table_p0/paimon/paimon_incr_read.out | 14 +
.../test_remote_doris_all_types_select.out | 16 +
.../test_remote_doris_all_types_show.out | 76 +
.../remote_doris/test_remote_doris_refresh.out | 16 +
.../remote_doris/test_remote_doris_statistics.out | 20 +
.../external_table_p0/tvf/test_catalogs_tvf.out | 4 +-
.../data/external_table_p0/tvf/test_http_tvf.out | 176 ++
.../test_ip_cidr_search_with_inverted_index.out | 9 +-
.../data/load_p0/merge_into/test_merge_into.out | 4 +
.../stream_load/enclose_with_same_escape.csv | 2 +
.../stream_load/load_json_with_jsonpath.out | 10 +
.../test_csv_with_enclose_and_escape.out | 2 +
.../stream_load/test_load_with_jsonpath2.json | 20 +
.../data/mtmv_p0/test_hive_pct_mtmv.out | 34 +
.../data/mtmv_p0/test_multi_pct_mtmv.out | 12 +
.../test_multi_pct_null_partition_mtmv.out} | 10 +-
.../adjust_nullable/test_subquery_nullable.out | 2 +-
.../push_down_filter_other_condition.out | 2 +-
.../mv/agg_variety/join_conjuncts_eliminate.out | 11 +
.../mv/availability/consistency_relaxed_tables.out | 19 +
.../mv/derive_mv_operative/derive_mv_oprative.out} | 20 +-
.../data/nereids_rules_p0/mv/topN/topN_rewrite.out | 107 +
.../mv/window/window_above_aggregate.out | 271 +++
.../mv/window/window_above_join.out | 453 ++++
.../mv/window/window_above_scan.out | 349 +++
.../partition_prune/list_prune_predicate.out | 12 +
.../predicate_infer/infer_predicate.out | 10 +-
.../data/nereids_syntax_p0/mv/where/k123/k123.out | 4 +
.../auto_partition/test_auto_new_recycle.out | 21 +
.../data/query_p0/cache/condition_cache.out | 14 +
.../array_functions/test_array_functions.out | 132 ++
.../test_dateadd_with_other_timeunit.out | 64 +
.../test_math_unary_always_nullable.out | 38 +
.../test_string_function_regexp.out | 56 +-
.../adjust_virtual_slot_nullable.out | 8 +-
...st_materialized_view_common_expr_push_down.out} | 6 +-
.../test_non_overlap_seg_heavy_sc.out | 5 +-
.../tpcds_sf100/noStatsRfPrune/query21.out | 2 +-
.../tpcds_sf100/noStatsRfPrune/query34.out | 2 +-
.../tpcds_sf100/noStatsRfPrune/query39.out | 2 +-
.../tpcds_sf100/noStatsRfPrune/query47.out | 2 +-
.../tpcds_sf100/noStatsRfPrune/query53.out | 2 +-
.../tpcds_sf100/noStatsRfPrune/query57.out | 2 +-
.../tpcds_sf100/noStatsRfPrune/query63.out | 2 +-
.../tpcds_sf100/noStatsRfPrune/query73.out | 2 +-
.../tpcds_sf100/noStatsRfPrune/query89.out | 2 +-
.../tpcds_sf100/no_stats_shape/query21.out | 2 +-
.../tpcds_sf100/no_stats_shape/query34.out | 2 +-
.../tpcds_sf100/no_stats_shape/query39.out | 2 +-
.../tpcds_sf100/no_stats_shape/query47.out | 2 +-
.../tpcds_sf100/no_stats_shape/query53.out | 2 +-
.../tpcds_sf100/no_stats_shape/query57.out | 2 +-
.../tpcds_sf100/no_stats_shape/query63.out | 2 +-
.../tpcds_sf100/no_stats_shape/query73.out | 2 +-
.../tpcds_sf100/no_stats_shape/query89.out | 2 +-
.../shape_check/tpcds_sf100/rf_prune/query21.out | 2 +-
.../shape_check/tpcds_sf100/rf_prune/query34.out | 18 +-
.../shape_check/tpcds_sf100/rf_prune/query39.out | 2 +-
.../shape_check/tpcds_sf100/rf_prune/query47.out | 2 +-
.../shape_check/tpcds_sf100/rf_prune/query53.out | 2 +-
.../shape_check/tpcds_sf100/rf_prune/query57.out | 2 +-
.../shape_check/tpcds_sf100/rf_prune/query63.out | 2 +-
.../shape_check/tpcds_sf100/rf_prune/query73.out | 2 +-
.../shape_check/tpcds_sf100/rf_prune/query89.out | 2 +-
.../data/shape_check/tpcds_sf100/shape/query21.out | 2 +-
.../data/shape_check/tpcds_sf100/shape/query34.out | 20 +-
.../data/shape_check/tpcds_sf100/shape/query39.out | 2 +-
.../data/shape_check/tpcds_sf100/shape/query47.out | 2 +-
.../data/shape_check/tpcds_sf100/shape/query53.out | 2 +-
.../data/shape_check/tpcds_sf100/shape/query57.out | 2 +-
.../data/shape_check/tpcds_sf100/shape/query63.out | 2 +-
.../data/shape_check/tpcds_sf100/shape/query73.out | 2 +-
.../data/shape_check/tpcds_sf100/shape/query89.out | 2 +-
.../data/shape_check/tpcds_sf1000/hint/query21.out | 2 +-
.../data/shape_check/tpcds_sf1000/hint/query34.out | 2 +-
.../data/shape_check/tpcds_sf1000/hint/query39.out | 2 +-
.../data/shape_check/tpcds_sf1000/hint/query47.out | 2 +-
.../data/shape_check/tpcds_sf1000/hint/query53.out | 2 +-
.../data/shape_check/tpcds_sf1000/hint/query57.out | 2 +-
.../data/shape_check/tpcds_sf1000/hint/query63.out | 2 +-
.../data/shape_check/tpcds_sf1000/hint/query73.out | 2 +-
.../data/shape_check/tpcds_sf1000/hint/query89.out | 2 +-
.../shape_check/tpcds_sf1000/shape/query21.out | 2 +-
.../shape_check/tpcds_sf1000/shape/query34.out | 12 +-
.../shape_check/tpcds_sf1000/shape/query39.out | 2 +-
.../shape_check/tpcds_sf1000/shape/query47.out | 2 +-
.../shape_check/tpcds_sf1000/shape/query53.out | 2 +-
.../shape_check/tpcds_sf1000/shape/query57.out | 2 +-
.../shape_check/tpcds_sf1000/shape/query63.out | 2 +-
.../shape_check/tpcds_sf1000/shape/query73.out | 2 +-
.../shape_check/tpcds_sf1000/shape/query89.out | 2 +-
.../shape_check/tpcds_sf10t_orc/shape/query21.out | 2 +-
.../shape_check/tpcds_sf10t_orc/shape/query34.out | 2 +-
.../shape_check/tpcds_sf10t_orc/shape/query39.out | 2 +-
.../shape_check/tpcds_sf10t_orc/shape/query47.out | 2 +-
.../shape_check/tpcds_sf10t_orc/shape/query53.out | 2 +-
.../shape_check/tpcds_sf10t_orc/shape/query57.out | 2 +-
.../shape_check/tpcds_sf10t_orc/shape/query63.out | 2 +-
.../shape_check/tpcds_sf10t_orc/shape/query73.out | 2 +-
.../shape_check/tpcds_sf10t_orc/shape/query89.out | 2 +-
regression-test/framework/pom.xml | 14 +-
.../doris/regression/suite/SuiteCluster.groovy | 6 +-
.../apache/doris/regression/suite/Syncer.groovy | 37 +-
regression-test/pipeline/common/doris-utils.sh | 4 +-
.../pipeline/external/conf/regression-conf.groovy | 7 +
.../pipeline/p0/conf/regression-conf.groovy | 1 +
.../plugins/cloud_recycler_plugin.groovy | 198 +-
.../suites/ann_index_p0/ann_index_only_scan.groovy | 6 +-
.../ann_index_only_scan_distance_expr.groovy | 5 +-
.../ann_index_only_scan_metric_direction.groovy | 5 +-
.../ann_index_p0/build_ann_index_test.groovy | 112 +
.../ann_index_p0/create_ann_index_test.groovy | 23 +-
.../ann_index_p0/product_quantization.groovy | 2 +-
.../auth_p0/test_s3_load_with_tmp_token.groovy | 188 ++
.../test_backup_restore_retention_count.groovy | 99 +
.../bloom_filter_p0/test_bloom_filter.groovy | 71 +-
.../cloud_p0/balance/test_balance_warm_up.groovy | 2 +
.../cloud_p0/cache/test_file_cache_info.groovy | 130 ++
.../cloud_p0/cache/test_topn_broadcast.groovy | 148 ++
.../cloud_p0/multi_cluster/test_auto_start.groovy | 1 +
.../test_cloud_add_backend_heartbeat.groovy | 52 +
.../query_retry/test_retry_no_scan_node.groovy | 80 +
.../suites/cloud_p0/recycler/test_checker.groovy | 58 +-
.../test_clean_tablet_when_drop_force_table.groovy | 17 +-
.../test_clean_tablet_when_rebalance.groovy | 3 +-
.../json/json_group_by_and_distinct.groovy | 112 +
.../fault_tolerance_nereids.groovy | 2 +-
.../manual-partitioning.md.groovy | 5 +-
.../export_p0/test_outfile_csv_compress.groovy | 139 +-
.../action/test_iceberg_execute_actions.groovy | 7 -
.../action/test_iceberg_rewrite_data_files.groovy | 495 ++++
...rewrite_data_files_expression_conversion.groovy | 430 ++++
...berg_rewrite_data_files_where_conditions.groovy | 152 ++
...ntime_filter_partition_pruning_transform.groovy | 98 +
.../test_iceberg_transform_partitions.groovy | 8 +
.../test_iceberg_write_transform_partitions.groovy | 6 +
.../jdbc/test_mysql_jdbc_catalog.groovy | 10 +-
.../jdbc/test_oracle_jdbc_catalog.groovy | 34 +-
.../jdbc/test_query_tvf_cross_catalog.groovy | 73 +
.../paimon/paimon_incr_read.groovy | 13 +-
.../test_remote_doris_all_types_select.groovy | 172 ++
.../test_remote_doris_all_types_show.groovy | 168 ++
.../remote_doris/test_remote_doris_catalog.groovy | 68 +
.../remote_doris/test_remote_doris_predict.groovy | 150 ++
.../remote_doris/test_remote_doris_refresh.groovy | 128 ++
.../test_remote_doris_statistics.groovy | 105 +
.../test_remote_doris_table_stats.groovy | 99 +
.../external_table_p0/tvf/test_http_tvf.groovy | 248 ++
.../tvf/test_tvf_topn_lazy_mat.groovy | 4 +-
.../external_table_p0/tvf/test_tvf_view.groovy | 36 +-
.../hive_on_hms_and_dlf.groovy | 113 +
.../test_delete_from_timeout.groovy | 10 +
.../test_inverted_index_cache.groovy | 145 --
.../test_load_back_pressure_version.groovy | 23 +-
.../test_sc_fail_when_be_down.groovy | 64 +
.../test_sc_success_when_be_down.groovy | 56 +
.../index_p0/test_ngram_bloomfilter_index.groovy | 2 +-
.../test_ngram_bloomfilter_index_change.groovy | 213 +-
.../insert_overwrite_auto_detect.groovy | 6 +-
.../test_iot_overwrite_and_create.groovy | 10 +-
.../insert_p0/test_error_msg_truncate.groovy | 150 ++
.../suites/insert_p0/test_insert_docs_demo.groovy | 2 +-
...ommit_http_stream_lineitem_schema_change.groovy | 2 +
...ommit_stream_load_lineitem_schema_change.groovy | 2 +
.../inverted_index_p0/test_array_index2.groovy | 11 +-
.../test_inverted_index_io_timer.groovy | 48 +-
.../test_ip_cidr_search_with_inverted_index.groovy | 4 +-
.../streaming_job/test_streaming_insert_job.groovy | 22 +-
.../test_streaming_insert_job_alter.groovy | 2 +-
.../test_streaming_insert_job_crud.groovy | 7 +-
.../test_streaming_insert_job_offset.groovy | 8 +-
...st_streaming_job_alter_offset_restart_fe.groovy | 12 +-
.../test_streaming_job_restart_fe.groovy | 8 +-
.../test_broker_load_strict_mode.groovy | 110 +
.../broker_load/test_load_data_from_path.groovy | 205 ++
.../load_p0/merge_into/test_merge_into.groovy | 6 +-
.../stream_load/load_json_with_jsonpath.groovy | 87 +-
.../test_csv_with_enclose_and_escape.groovy | 9 +
.../stream_load/test_stream_load_audit_log.groovy | 83 +
.../test_stream_load_include_where_expr.groovy | 2 +-
.../test_tvf_strict_mode_and_filter_ratio.groovy | 2 +-
.../suites/manager/test_manager_interface_1.groovy | 2 +-
.../suites/mtmv_p0/test_hive_mtmv.groovy | 12 +
.../suites/mtmv_p0/test_hive_pct_mtmv.groovy | 161 ++
.../suites/mtmv_p0/test_multi_pct_mtmv.groovy | 27 +
.../test_multi_pct_null_partition_mtmv.groovy | 121 +
.../mtmv_p0/test_partition_refresh_mtmv.groovy | 15 +-
.../suites/mv_p0/count_star/count_star.groovy | 2 +-
.../test_uniq_mv_useless.groovy | 16 +
.../mv_p0/where/k123_nereids/k123_nereids.groovy | 2 +-
.../suites/nereids_function_p0/load.groovy | 5 +
.../nereids_p0/cache/mtmv_with_sql_cache.groovy | 17 +-
.../create_table/test_ctas_auto_partition.groovy | 2 +-
.../explain/test_pushdown_explain.groovy | 84 +
.../fold_constant_numeric_arithmatic.groovy | 10 +
.../show/test_nereids_show_typecast.groovy | 29 -
.../test_forbid_unknown_col_stats.groovy | 2 +-
.../mv/agg_on_none_agg/agg_on_none_agg.groovy | 6 +-
.../mv/agg_variety/join_conjuncts_eliminate.groovy | 130 ++
.../availability/consistency_relaxed_tables.groovy | 262 +++
.../derive_mv_operative/derive_mv_oprative.groovy | 72 +
.../null_aware_left_anti_join.groovy | 4 +-
.../join_elim_line_pattern.groovy | 558 +++++
.../join_elim_star_pattern.groovy | 592 +++++
.../nereids_rules_p0/mv/limit/limit_rewrite.groovy | 663 ++++++
.../mv/nested_mtmv/nested_mtmv.groovy | 85 +
.../mv/pre_rewrite/limit/query_with_limit.groovy | 248 +-
.../nereids_rules_p0/mv/topN/topN_rewrite.groovy | 690 ++++++
.../nereids_rules_p0/mv/tpch/mv_tpch_test.groovy | 13 +-
.../mv/window/window_above_aggregate.groovy | 1821 +++++++++++++++
.../mv/window/window_above_join.groovy | 1591 +++++++++++++
.../mv/window/window_above_scan.groovy | 1329 +++++++++++
.../partition_prune/list_prune_predicate.groovy | 39 +
.../partition_prune/one_key_range_part_test.groovy | 4 +-
.../one_key_range_part_update_test.groovy | 2 +-
.../push_down_join_on_assert_num_rows.groovy | 130 ++
.../distribute/colocate_union_numbers.groovy | 72 -
.../nereids_syntax_p0/distribute/union_tvf.groovy | 10 +-
.../nereids_syntax_p0/mv/where/k123/k123.groovy | 11 +
.../auto_partition/test_auto_new_recycle.groovy | 185 ++
.../test_auto_partition_behavior.groovy | 4 +-
.../test_auto_range_partition.groovy | 30 +-
.../test_list_partition_datatype.groovy | 2 +-
.../test_multi_column_partition.groovy | 2 +-
.../suites/query_p0/cache/condition_cache.groovy | 23 +
.../suites/query_p0/set/test_user_var.groovy | 53 +
.../array_functions/test_array_functions.groovy | 13 +
.../test_dateadd_with_other_timeunit.groovy | 58 +
.../test_math_unary_always_nullable.groovy | 43 +
.../test_string_function_regexp.groovy | 52 +-
.../adjust_virtual_slot_nullable.groovy | 2 +-
..._materialized_view_common_expr_push_down.groovy | 76 +
.../test_non_overlap_seg_heavy_sc.groovy | 52 +-
regression-test/suites/variant_p0/desc.groovy | 22 +-
regression-test/suites/variant_p0/load.groovy | 12 +-
.../variant_p0/predefine/variant_hirachinal.groovy | 4 +-
.../suites/variant_p0/variant_hirachinal.groovy | 4 +-
.../suites/variant_p0/variant_with_rowstore.groovy | 4 +-
regression-test/suites/variant_p2/load.groovy | 4 +-
.../create/test_create_vault_with_kerberos.groovy | 2 +-
run-be-ut.sh | 2 +-
thirdparty/download-thirdparty.sh | 7 +-
thirdparty/patches/cctz-lookup-offset.patch | 172 ++
thirdparty/vars.sh | 8 +-
1417 files changed, 48662 insertions(+), 25447 deletions(-)
copy {fe/.idea => .idea}/icon.svg (100%)
create mode 100644 .idea/vcs.xml
create mode 100644 be/benchmark/benchmark_vstring_function.hpp
create mode 100644
be/src/exec/schema_scanner/schema_file_cache_info_scanner.cpp
rename fe/fe-core/src/main/java/org/apache/doris/statistics/PlanStats.java =>
be/src/exec/schema_scanner/schema_file_cache_info_scanner.h (54%)
create mode 100644 be/src/io/fs/http_file_reader.cpp
create mode 100644 be/src/io/fs/http_file_reader.h
create mode 100644 be/src/io/fs/http_file_system.cpp
create mode 100644 be/src/io/fs/http_file_system.h
create mode 100644
be/src/olap/rowset/segment_v2/inverted_index/query_v2/nullable_scorer.h
create mode 100644
be/src/olap/rowset/segment_v2/inverted_index/query_v2/phrase_query/multi_phrase_query.h
create mode 100644
be/src/olap/rowset/segment_v2/inverted_index/query_v2/phrase_query/multi_phrase_weight.h
create mode 100644
be/src/olap/rowset/segment_v2/inverted_index/query_v2/postings/loaded_postings.cpp
create mode 100644
be/src/olap/rowset/segment_v2/inverted_index/query_v2/postings/loaded_postings.h
create mode 100644
be/src/olap/rowset/segment_v2/inverted_index/query_v2/union/simple_union.cpp
create mode 100644
be/src/olap/rowset/segment_v2/inverted_index/query_v2/union/simple_union.h
create mode 100644 be/src/runtime/stream_load/stream_load_recorder_manager.cpp
create mode 100644 be/src/runtime/stream_load/stream_load_recorder_manager.h
create mode 100644 be/src/vec/exec/format/table/remote_doris_reader.cpp
create mode 100644 be/src/vec/exec/format/table/remote_doris_reader.h
create mode 100644 be/test/io/client/s3_file_system_test.cpp
create mode 100644
be/test/olap/rowset/segment_v2/inverted_index/query_v2/loaded_postings_test.cpp
copy
be/test/olap/rowset/segment_v2/inverted_index/query_v2/{phrase_query_test.cpp
=> multi_phrase_query_test.cpp} (56%)
create mode 100644
be/test/olap/rowset/segment_v2/inverted_index/query_v2/simple_union_test.cpp
create mode 100644 be/test/util/simd/vstring_function_test.cpp
create mode 100644 be/test/vec/data_types/serde/data_type_serde_map_test.cpp
create mode 100644 be/test/vec/data_types/serde/data_type_serde_struct_test.cpp
create mode 100644 cloud/test/blob_message_test.cpp
create mode 100644
docker/thirdparties/docker-compose/iceberg/scripts/create_preinstalled_scripts/iceberg/run21.sql
delete mode 100644
fe/fe-common/src/main/java/org/apache/doris/catalog/MultiRowType.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/analysis/AnalyticExpr.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/analysis/BitmapFilterPredicate.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/analysis/FunctionArgsDef.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/analysis/GroupingFunctionCallExpr.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/analysis/IndexDef.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/analysis/JobName.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/analysis/LabelName.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/analysis/LoadColumnsInfo.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/analysis/TupleIsNullPredicate.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/analysis/TypeDef.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/catalog/ColumnStats.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/catalog/FunctionTypeDeducers.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/doris/RemoteDorisCompatibleRestClient.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/doris/RemoteDorisExternalCatalog.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/doris/RemoteDorisExternalDatabase.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/doris/RemoteDorisExternalTable.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/doris/RemoteDorisRestClient.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/doris/source/RemoteDorisScanNode.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/doris/source/RemoteDorisSource.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/doris/source/RemoteDorisSplit.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/iceberg/IcebergNereidsUtils.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/iceberg/rewrite/RewriteDataFileExecutor.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/iceberg/rewrite/RewriteDataFilePlanner.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/iceberg/rewrite/RewriteDataGroup.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/iceberg/rewrite/RewriteGroupTask.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/iceberg/rewrite/RewriteResult.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/property/constants/RemoteDorisProperties.java
copy
fe/fe-core/src/main/java/org/apache/doris/datasource/property/storage/{LocalProperties.java
=> HttpProperties.java} (50%)
copy fe/fe-core/src/main/java/org/apache/doris/{load/sync/DataSyncJobType.java
=> datasource/property/storage/exception/AzureAuthType.java} (87%)
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/deploy/impl/AmbariDeployManager.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/deploy/impl/K8sDeployManager.java
rename
fe/fe-core/src/main/java/org/apache/doris/{statistics/AssertNumRowsStatsDerive.java
=> httpv2/rest/response/GsonSchemaResponse.java} (76%)
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/load/sync/SyncFailMsg.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/load/sync/SyncJob.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/load/sync/canal/CanalDestination.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/load/sync/canal/CanalSyncJob.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/exploration/mv/AbstractMaterializedViewLimitOrTopNRule.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/exploration/mv/AbstractMaterializedViewWindowRule.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/exploration/mv/MaterializedViewLimitAggregateRule.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/exploration/mv/MaterializedViewLimitJoinRule.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/exploration/mv/MaterializedViewLimitScanRule.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/exploration/mv/MaterializedViewTopNAggregateRule.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/exploration/mv/MaterializedViewTopNJoinRule.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/exploration/mv/MaterializedViewTopNScanRule.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/exploration/mv/MaterializedViewWindowAggregateRule.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/exploration/mv/MaterializedViewWindowJoinRule.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/exploration/mv/MaterializedViewWindowScanRule.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/rules/PushIntoCaseWhenBranch.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/PropagateNullableOnDateOrTimeLikeV2Args.java
copy
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/scalar/{DaySecondAdd.java
=> DayHourAdd.java} (78%)
copy
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/scalar/{ToDays.java
=> Factorial.java} (72%)
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/table/Http.java
rename fe/fe-core/src/main/java/org/apache/doris/{analysis =>
nereids/trees/plans/commands/info}/CopyIntoProperties.java (84%)
rename fe/fe-core/src/main/java/org/apache/doris/{analysis =>
nereids/trees/plans/commands/info}/CopyProperties.java (87%)
rename fe/fe-core/src/main/java/org/apache/doris/{analysis =>
nereids/trees/plans/commands/info}/StageProperties.java (87%)
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/insert/IcebergRewriteExecutor.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/insert/RewriteTableCommand.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/planner/DistributedPlanColocateRule.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/planner/JoinCostEvaluation.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/planner/PlannerContext.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/statistics/AggStatsDerive.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/statistics/AnalyticEvalStatsDerive.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/statistics/BaseStatsDerive.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/statistics/DeriveFactory.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/statistics/EmptySetStatsDerive.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/statistics/ExchangeStatsDerive.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/statistics/HashJoinStatsDerive.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/statistics/MysqlStatsDerive.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/statistics/NestedLoopJoinStatsDerive.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/statistics/OlapScanStatsDerive.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/statistics/SelectStatsDerive.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/statistics/StatisticalType.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/statistics/StatsRecursiveDerive.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/statistics/TableFunctionStatsDerive.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/tablefunction/HFUtils.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/tablefunction/HttpTableValuedFunction.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/task/AgentTaskCleanupDaemon.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/analysis/VarBinaryLiteralAnalysisTest.java
delete mode 100644
fe/fe-core/src/test/java/org/apache/doris/catalog/ColumnStatTest.java
delete mode 100644
fe/fe-core/src/test/java/org/apache/doris/catalog/FunctionSetTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/datasource/doris/RemoteDorisCompatibleRestClientTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/datasource/doris/RemoteDorisRestClientTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/datasource/iceberg/IcebergNereidsUtilsTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/datasource/iceberg/rewrite/RewriteDataFilePlannerTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/datasource/metacache/MetaCacheDeadlockTest.java
delete mode 100644
fe/fe-core/src/test/java/org/apache/doris/deploy/AmbariDeployManagerTest.java
delete mode 100644
fe/fe-core/src/test/java/org/apache/doris/deploy/K8sDeployManagerTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/mysql/MysqlSerializerVarbinaryTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/nereids/mv/PredicatesTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/nereids/mv/StructInfoTest.java
delete mode 100644
fe/fe-core/src/test/java/org/apache/doris/nereids/rules/analysis/GenerateFunction.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/nereids/rules/expression/rules/PushIntoCaseWhenBranchTest.java
delete mode 100644
fe/fe-core/src/test/java/org/apache/doris/nereids/trees/expressions/FunctionTest.java
copy
fe/fe-core/src/test/java/org/apache/doris/nereids/trees/{copier/LogicalPlanDeepCopierTest.java
=> plans/commands/UseCloudClusterCommandTest.java} (52%)
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/nereids/trees/plans/commands/insert/InsertUtilsTest.java
delete mode 100644
fe/fe-core/src/test/java/org/apache/doris/planner/JoinCostEvaluationTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/qe/LocalShuffleWithBucketJoinTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/tablefunction/HFUtilsTest.java
delete mode 100644 gensrc/script/doris_builtins_functions.py
delete mode 100755 gensrc/script/gen_builtins_functions.py
copy regression-test/data/{schema_change_p0/test_non_overlap_seg_heavy_sc.out
=> auth_p0/test_s3_load_with_tmp_token.out} (82%)
copy regression-test/data/{load_p0/stream_load/load_json_with_jsonpath.out =>
backup_restore/test_backup_restore_retention_count.out} (58%)
create mode 100644
regression-test/data/datatype_p0/json/json_group_by_and_distinct.out
create mode 100644
regression-test/data/external_table_p0/iceberg/action/test_iceberg_rewrite_data_files.out
create mode 100644
regression-test/data/external_table_p0/iceberg/action/test_iceberg_rewrite_data_files_where_conditions.out
copy regression-test/data/{schema_change_p0/test_non_overlap_seg_heavy_sc.out
=> external_table_p0/jdbc/test_query_tvf_cross_catalog.out} (53%)
create mode 100644
regression-test/data/external_table_p0/remote_doris/test_remote_doris_all_types_select.out
create mode 100644
regression-test/data/external_table_p0/remote_doris/test_remote_doris_all_types_show.out
create mode 100644
regression-test/data/external_table_p0/remote_doris/test_remote_doris_refresh.out
create mode 100644
regression-test/data/external_table_p0/remote_doris/test_remote_doris_statistics.out
create mode 100644 regression-test/data/external_table_p0/tvf/test_http_tvf.out
create mode 100644
regression-test/data/load_p0/stream_load/enclose_with_same_escape.csv
create mode 100644
regression-test/data/load_p0/stream_load/test_load_with_jsonpath2.json
create mode 100644 regression-test/data/mtmv_p0/test_hive_pct_mtmv.out
rename
regression-test/data/{nereids_syntax_p0/distribute/colocate_union_numbers.out
=> mtmv_p0/test_multi_pct_null_partition_mtmv.out} (75%)
create mode 100644
regression-test/data/nereids_rules_p0/mv/agg_variety/join_conjuncts_eliminate.out
create mode 100644
regression-test/data/nereids_rules_p0/mv/availability/consistency_relaxed_tables.out
copy regression-test/data/{load_p0/stream_load/load_json_with_jsonpath.out =>
nereids_rules_p0/mv/derive_mv_operative/derive_mv_oprative.out} (50%)
create mode 100644
regression-test/data/nereids_rules_p0/mv/topN/topN_rewrite.out
create mode 100644
regression-test/data/nereids_rules_p0/mv/window/window_above_aggregate.out
create mode 100644
regression-test/data/nereids_rules_p0/mv/window/window_above_join.out
create mode 100644
regression-test/data/nereids_rules_p0/mv/window/window_above_scan.out
create mode 100644
regression-test/data/partition_p0/auto_partition/test_auto_new_recycle.out
copy regression-test/data/{schema_change_p0/test_non_overlap_seg_heavy_sc.out
=> rollup_p0/test_materialized_view_common_expr_push_down.out} (86%)
create mode 100644
regression-test/suites/ann_index_p0/build_ann_index_test.groovy
create mode 100644
regression-test/suites/auth_p0/test_s3_load_with_tmp_token.groovy
create mode 100644
regression-test/suites/backup_restore/test_backup_restore_retention_count.groovy
create mode 100644
regression-test/suites/cloud_p0/cache/test_file_cache_info.groovy
create mode 100644
regression-test/suites/cloud_p0/cache/test_topn_broadcast.groovy
create mode 100644
regression-test/suites/cloud_p0/node_mgr/test_cloud_add_backend_heartbeat.groovy
create mode 100644
regression-test/suites/cloud_p0/query_retry/test_retry_no_scan_node.groovy
create mode 100644
regression-test/suites/datatype_p0/json/json_group_by_and_distinct.groovy
create mode 100644
regression-test/suites/external_table_p0/iceberg/action/test_iceberg_rewrite_data_files.groovy
create mode 100644
regression-test/suites/external_table_p0/iceberg/action/test_iceberg_rewrite_data_files_expression_conversion.groovy
create mode 100644
regression-test/suites/external_table_p0/iceberg/action/test_iceberg_rewrite_data_files_where_conditions.groovy
create mode 100644
regression-test/suites/external_table_p0/jdbc/test_query_tvf_cross_catalog.groovy
create mode 100644
regression-test/suites/external_table_p0/remote_doris/test_remote_doris_all_types_select.groovy
create mode 100644
regression-test/suites/external_table_p0/remote_doris/test_remote_doris_all_types_show.groovy
create mode 100644
regression-test/suites/external_table_p0/remote_doris/test_remote_doris_catalog.groovy
create mode 100644
regression-test/suites/external_table_p0/remote_doris/test_remote_doris_predict.groovy
create mode 100644
regression-test/suites/external_table_p0/remote_doris/test_remote_doris_refresh.groovy
create mode 100644
regression-test/suites/external_table_p0/remote_doris/test_remote_doris_statistics.groovy
create mode 100644
regression-test/suites/external_table_p0/remote_doris/test_remote_doris_table_stats.groovy
create mode 100644
regression-test/suites/external_table_p0/tvf/test_http_tvf.groovy
delete mode 100644
regression-test/suites/fault_injection_p0/test_inverted_index_cache.groovy
create mode 100644
regression-test/suites/fault_injection_p0/test_sc_fail_when_be_down.groovy
create mode 100644
regression-test/suites/fault_injection_p0/test_sc_success_when_be_down.groovy
create mode 100644
regression-test/suites/insert_p0/test_error_msg_truncate.groovy
create mode 100644
regression-test/suites/load_p0/broker_load/test_broker_load_strict_mode.groovy
create mode 100644
regression-test/suites/load_p0/broker_load/test_load_data_from_path.groovy
create mode 100644
regression-test/suites/load_p0/stream_load/test_stream_load_audit_log.groovy
create mode 100644 regression-test/suites/mtmv_p0/test_hive_pct_mtmv.groovy
create mode 100644
regression-test/suites/mtmv_p0/test_multi_pct_null_partition_mtmv.groovy
delete mode 100644
regression-test/suites/nereids_p0/show/test_nereids_show_typecast.groovy
create mode 100644
regression-test/suites/nereids_rules_p0/mv/agg_variety/join_conjuncts_eliminate.groovy
create mode 100644
regression-test/suites/nereids_rules_p0/mv/availability/consistency_relaxed_tables.groovy
create mode 100644
regression-test/suites/nereids_rules_p0/mv/derive_mv_operative/derive_mv_oprative.groovy
create mode 100644
regression-test/suites/nereids_rules_p0/mv/join_elim_p_f_key/join_elim_line_pattern.groovy
create mode 100644
regression-test/suites/nereids_rules_p0/mv/join_elim_p_f_key/join_elim_star_pattern.groovy
create mode 100644
regression-test/suites/nereids_rules_p0/mv/limit/limit_rewrite.groovy
create mode 100644
regression-test/suites/nereids_rules_p0/mv/topN/topN_rewrite.groovy
create mode 100644
regression-test/suites/nereids_rules_p0/mv/window/window_above_aggregate.groovy
create mode 100644
regression-test/suites/nereids_rules_p0/mv/window/window_above_join.groovy
create mode 100644
regression-test/suites/nereids_rules_p0/mv/window/window_above_scan.groovy
create mode 100644
regression-test/suites/nereids_rules_p0/push_down_join_on_assertnumrows/push_down_join_on_assert_num_rows.groovy
delete mode 100644
regression-test/suites/nereids_syntax_p0/distribute/colocate_union_numbers.groovy
rename
fe/fe-core/src/main/java/org/apache/doris/load/sync/DataSyncJobType.java =>
regression-test/suites/nereids_syntax_p0/distribute/union_tvf.groovy (88%)
create mode 100644
regression-test/suites/partition_p0/auto_partition/test_auto_new_recycle.groovy
create mode 100644
regression-test/suites/rollup_p0/test_materialized_view_common_expr_push_down.groovy
create mode 100644 thirdparty/patches/cctz-lookup-offset.patch
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
