This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a change to branch auto-pick-58426-branch-4.0
in repository https://gitbox.apache.org/repos/asf/doris.git
discard 9543f0fd967 [fix](mv) Fix stats unknown when calc sync mv plan
statistics (#58426)
add 17c8748c310 branch-4.0: [Improvement](hash) add int96 int104 to hash
method #58770 (#58902)
add 18054665cfc branch-4.0: [chore](security) Add ssrf check for create
stage #58585 (#58875)
add 99ea4a8f9fc branch-4.0: [opt](hive) use binary search to prune hive
partitions #58877 (#58921)
add 5ac57dd85a7 branch-4.0: [Chore](profile) adjust some profile #58861
(#58956)
add 017011d464f [test](doris catalog)Add more test cases for doris catalog
(#58784) (#58831) (#58891)
add bd6dad23957 [test](variable)add up and down test case of variables
persistence (#58818)
add 0f22ae9b27c [opt](filecache) schema change adaptive to file cache
(#58620)
add c11e585e996 branch-4.0: [feat](es) support es flatten type #58762
(#58794)
add 6997c87b017 branch-4.0: [bugfix](memory_profile) should split memtable
memory from task's memory profile #58908 (#58979)
add b37b8b63eb2 branch4.0: [feature](variable) enables views, materialized
views, generated columns, and alias functions to persist session variables
#58031 (#58821)
add aa9c1894d3f branch-4.0: [opt](jvm) add more --add-opens #58936 (#58984)
add 9ac212e1730 branch-4.0: [opt](agg) Adjust the parameters of
StreamingHtMinReductionEntry #58938 (#58986)
add 43f4cb862a4 branch-4.0: [fix](inverted index) fix incorrect variable
usage in phrase query building #58952 (#58992)
add 3b5f9b762d0 branch-4.0: [chore](ci) exclude test_insert_error_url to
avoid stop grace fail #58985 (#58987)
add 863f11b27ae Revert "[test](variable)add up and down test case of
variables persistence" (#59038)
add dd8d33da18b branch-4.0: [fix](ann search)
l2_distance_approximate/inner_product_approximate should not be deterministic
#58988 (#59001)
add 3255d1798c5 branch-4.0: [fix](test) Fix output result for
test_non_overlap_seg_heavy_sc #58805 (#59032)
add 6518277ea76 branch-4.0: [fix](json) Replace invalid JSONB with default
JSONB null value #59007 (#59022)
add 63f3d2e6923 branch-4.0: [Fix](fe) Fix NPE in
`OlapTableSink.createPaloNodesInfo` due to concurrent drop backend #58999
(#59057)
add 0c5f8e0a22d branch-4.0: [fix](close) Release resource once fragment
was cancelled #58909 (#58971)
add 900d828f0c0 branch-4.0: [fix](cloud)Not log stack information by Peer
read #57816 (#58996)
add 21fa64b8581 branch-4.0 [Feature](Segment) Support external meta V3
with common format (#59042)
add c80c3a85de3 branch-4.0: [test](mtmv) Inject stats for complex mv to
make test case stable #58876 (#58966)
add b0b6a9e9888 branch-4.0: [fix](skew_join)should make skewExpr and
skewValues have same datatype #58941 (#59004)
add 1e3e2ee1e19 branch-4.0: [Enhancement](insert-overwrite) Make clean
garbage partition interval of InsertOverwriteManager adjustable #58733 (#58925)
add 10666f21b9c branch-4.0: [fix](mtmv) Fix query rewrite by mv fail when
both use cte and group sets #57741 (#58994)
add b6865b83e44 branch-4.0: [case](profile) rm useless profile_test case
#58569 (#59081)
add b20bb56b1d2 branch-4.0: [fix](delete) Fix missing `prepare_rowset` for
delete job #59089 (#59097)
add 6cd317ae584 branch-4.0: [fix](parser) Fix FROM DUAL incorrectly
matching table names starting with dual #59003 (#59021)
add 698b28ac319 branch-4.0: [Fix](Iam-role)Ensure StorageProperties list
remains ordered when auto-loading default HDFS #58968 (#59072)
add 99afe889052 branch-4.0: [fix](enforcer) shuffle if has continuous
project or filter on cte consumer #58964 (#59046)
add 4a41ca3cb88 branch-4.0:[test](mv) Remove 'set enable_stats=false' in
sync mv regression test to make mv regression test case stable #58827 (#59030)
add 471f74704a3 branch-4.0: [enhance](job) support adaptive batch param
for routine load job (#58846)
add 916fad0c778 branch-4.0: [fix](profile) Fix the issue about load
channel profile #58787 (#59094)
add 9d7eaddfc36 branch-4.0: [fix](filecache) fix warm up cancel failure
when BE is down #58035 (#59059)
add f7a52e49539 branch-4.0: [opt](inverted index) release dir memory after
index file writer close #59031 (#59050)
add f06b46484ee branch-4.0: [chore](catalog)Remove the HMS-related parts
from be-extension. #58280 (#59070)
add 63fc74083fe branch-4.0: [check](column)Columns must be created only
via their corresponding create methods. #59087 (#59114)
add c06d94fed52 branch-4.0: [fix](mtmv) Fix mv rewrite failed when mv is
rewritten by LimitAggToTopNAgg but query is not #58974 (#59061)
add 92915f3252b branch-4.0: [fix](case)fix create sync mv failed in test
case #58834 (#58959)
add eb2352aee99 branch-4.0:[fix](variable persist) fix regression test
test_mtmv #58869 (#59034)
add 47108329dbe branch-4.0: [feature](QueryPlanAction)add sql received
from table query plan action into audit log #58739 (#59041)
add 6ebca2788f5 branch-4.0: [opt](nereids) Add planWithUnknownColumnStats
to QueryState to prevent queries from analysis tasks from polluting the column
stats cache. #57850 (#58741)
add 88beb1a2ed8 branch-4.0: [fix](catalog) Fix lastUpdateTime not updated
on refresh and add scheduled refresh logs #58997 (#59108)
add 61aa893e418 branch-4.0: [Opt](tbl_priv) Optimize PrivTable search and
insert with ordered lookup #58828 (#59029)
add 7b131947fa3 branch-4.0: [enhance](iceberg) Doris Iceberg Scan Metrics
Integration #59010 (#59095)
add f34ee31303f branch-4.0: [feat](load) enable stream load record to
audit log system table (#57530) (#58849)
add 742bd041417 [feature](inverted index) Add inverted index NORMALIZER
support (#59112)
add 41e95bd0089 branch-4.0: [fix](rbo) use wrong child's output of set
operation #59105 (#59131)
add 4006befd3ec [fix](mv) Fix stats unknown when calc sync mv plan
statistics (#58426)
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (9543f0fd967)
\
N -- N -- N refs/heads/auto-pick-58426-branch-4.0 (4006befd3ec)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
be/src/cloud/cloud_cumulative_compaction.cpp | 2 +-
be/src/cloud/cloud_delete_task.cpp | 8 +-
be/src/cloud/cloud_schema_change_job.cpp | 48 +-
be/src/cloud/cloud_schema_change_job.h | 3 +-
be/src/cloud/cloud_tablet.cpp | 2 +-
be/src/cloud/pb_convert.cpp | 4 +
be/src/common/config.cpp | 10 +
be/src/common/config.h | 8 +
be/src/http/action/http_stream.cpp | 17 +-
be/src/http/action/stream_load.cpp | 17 +-
be/src/http/http_client.cpp | 6 +
be/src/http/http_client.h | 2 +
be/src/http/http_common.h | 1 +
be/src/io/cache/cached_remote_file_reader.cpp | 2 +-
be/src/io/cache/peer_file_cache_reader.cpp | 13 +-
be/src/olap/inverted_index_parser.cpp | 16 +-
be/src/olap/inverted_index_parser.h | 8 +-
be/src/olap/memtable.cpp | 6 +-
.../olap/rowset/segment_v2/binary_prefix_page.cpp | 3 +-
.../rowset/segment_v2/column_meta_accessor.cpp | 218 ++
.../olap/rowset/segment_v2/column_meta_accessor.h | 93 +
be/src/olap/rowset/segment_v2/column_reader.cpp | 28 +-
be/src/olap/rowset/segment_v2/column_reader.h | 18 +-
.../olap/rowset/segment_v2/column_reader_cache.cpp | 182 +-
.../olap/rowset/segment_v2/column_reader_cache.h | 37 +-
be/src/olap/rowset/segment_v2/column_writer.h | 3 +-
.../rowset/segment_v2/external_col_meta_util.cpp | 199 ++
.../rowset/segment_v2/external_col_meta_util.h | 91 +
.../olap/rowset/segment_v2/index_file_writer.cpp | 1 +
.../inverted_index/analysis_factory_mgr.cpp | 7 +
.../inverted_index/analyzer/analyzer.cpp | 6 +-
.../analyzer/custom_analyzer_config.h | 1 -
.../char_filter/icu_normalizer_char_filter.cpp | 91 +
.../char_filter/icu_normalizer_char_filter.h | 51 +
.../icu_normalizer_char_filter_factory.h | 119 +
.../normalizer/custom_normalizer.cpp | 110 +
.../inverted_index/normalizer/custom_normalizer.h | 73 +
.../normalizer/custom_normalizer_config.cpp | 49 +
.../custom_normalizer_config.h} | 40 +-
.../token_filter/icu_normalizer_filter.cpp | 72 +
.../token_filter/icu_normalizer_filter.h | 34 +-
.../token_filter/icu_normalizer_filter_factory.h | 105 +
.../rowset/segment_v2/inverted_index_writer.cpp | 4 +-
be/src/olap/rowset/segment_v2/segment.cpp | 108 +-
be/src/olap/rowset/segment_v2/segment.h | 25 +-
be/src/olap/rowset/segment_v2/segment_iterator.cpp | 33 +-
be/src/olap/rowset/segment_v2/segment_iterator.h | 2 +-
be/src/olap/rowset/segment_v2/segment_writer.cpp | 12 +-
.../segment_v2/variant/variant_column_reader.cpp | 688 +++-
.../segment_v2/variant/variant_column_reader.h | 122 +-
.../variant/variant_column_writer_impl.cpp | 34 +-
.../variant/variant_column_writer_impl.h | 5 +-
.../segment_v2/variant/variant_ext_meta_writer.cpp | 199 ++
.../segment_v2/variant/variant_ext_meta_writer.h | 99 +
.../variant/variant_external_meta_reader.cpp | 238 ++
.../variant/variant_external_meta_reader.h | 95 +
.../rowset/segment_v2/vertical_segment_writer.cpp | 14 +-
be/src/olap/schema_change.h | 1 +
be/src/olap/tablet_manager.cpp | 3 +-
be/src/olap/tablet_meta.cpp | 30 +-
be/src/olap/tablet_meta.h | 7 +-
be/src/olap/tablet_schema.cpp | 12 +-
be/src/olap/tablet_schema.h | 15 +-
be/src/pipeline/common/agg_utils.h | 10 +
be/src/pipeline/common/distinct_agg_utils.h | 10 +
be/src/pipeline/common/join_utils.h | 9 +
be/src/pipeline/common/partition_sort_utils.h | 10 +
be/src/pipeline/common/set_utils.h | 10 +
be/src/pipeline/exec/analytic_sink_operator.h | 8 +-
.../distinct_streaming_aggregation_operator.cpp | 2 +-
.../pipeline/exec/join/process_hash_table_probe.h | 1 +
.../exec/join/process_hash_table_probe_impl.h | 2 +
be/src/pipeline/pipeline_fragment_context.cpp | 16 +-
be/src/pipeline/pipeline_task.cpp | 1 -
be/src/runtime/exec_env.h | 3 +
be/src/runtime/exec_env_init.cpp | 7 +
be/src/runtime/index_policy/index_policy_mgr.cpp | 82 +-
be/src/runtime/index_policy/index_policy_mgr.h | 21 +-
be/src/runtime/memory/memory_profile.cpp | 24 +-
be/src/runtime/memory/memory_profile.h | 5 +-
be/src/runtime/primitive_type.h | 129 +-
be/src/runtime/runtime_state.h | 4 -
.../runtime/stream_load/stream_load_recorder.cpp | 35 +-
be/src/runtime/stream_load/stream_load_recorder.h | 6 +-
.../stream_load/stream_load_recorder_manager.cpp | 325 ++
.../stream_load/stream_load_recorder_manager.h | 98 +
be/src/util/jsonb_document.cpp | 38 +
be/src/util/jsonb_document.h | 25 -
.../vec/aggregate_functions/aggregate_function.h | 1 -
.../aggregate_function_ai_agg.cpp | 2 +-
.../aggregate_function_approx_count_distinct.cpp | 4 +-
.../aggregate_function_array_agg.cpp | 1 +
.../aggregate_functions/aggregate_function_avg.cpp | 36 +-
.../aggregate_functions/aggregate_function_avg.h | 33 +-
.../aggregate_function_bitmap.cpp | 5 +-
.../aggregate_function_bitmap_agg.cpp | 1 +
.../aggregate_function_collect.cpp | 1 +
.../aggregate_function_corr.cpp | 2 +
.../aggregate_function_count.cpp | 5 +-
.../aggregate_function_count_by_enum.cpp | 1 +
.../aggregate_function_covar.cpp | 2 +
.../aggregate_function_distinct.cpp | 5 +-
.../aggregate_function_foreach.cpp | 10 +-
.../aggregate_function_foreachv2.cpp | 11 +-
.../aggregate_function_group_array_set_op.cpp | 8 +-
.../aggregate_function_group_concat.cpp | 1 +
.../aggregate_function_histogram.cpp | 1 +
.../aggregate_function_kurtosis.cpp | 1 +
.../aggregate_function_linear_histogram.cpp | 1 +
.../aggregate_functions/aggregate_function_map.cpp | 1 +
.../aggregate_function_map_v2.cpp | 1 +
.../aggregate_function_min_max.cpp | 7 +-
.../aggregate_function_min_max.h | 5 +-
.../aggregate_function_min_max_by.h | 1 +
.../aggregate_function_orthogonal_bitmap.cpp | 1 +
.../aggregate_function_percentile.cpp | 8 +-
.../aggregate_function_percentile_reservoir.cpp | 1 +
.../aggregate_function_product.h | 11 +-
.../aggregate_function_quantile_state.cpp | 4 +-
.../aggregate_function_quantile_state.h | 4 +-
.../aggregate_function_reader_first_last.h | 41 +-
.../aggregate_function_regr_union.cpp | 1 +
.../aggregate_functions/aggregate_function_sem.cpp | 12 +-
.../aggregate_function_sequence_match.cpp | 1 +
.../aggregate_function_simple_factory.h | 14 +-
.../aggregate_function_skew.cpp | 1 +
.../aggregate_function_stddev.cpp | 16 +-
.../aggregate_functions/aggregate_function_sum.cpp | 18 +-
.../aggregate_functions/aggregate_function_sum.h | 51 +-
.../aggregate_function_topn.cpp | 3 +
.../aggregate_function_uniq.cpp | 1 +
.../aggregate_function_uniq_distribute_key.cpp | 1 +
.../aggregate_function_window.cpp | 4 +-
.../aggregate_function_window_funnel.cpp | 1 +
be/src/vec/aggregate_functions/helpers.h | 72 +
be/src/vec/columns/column_varbinary.h | 4 +-
be/src/vec/columns/column_variant.h | 3 +
be/src/vec/columns/column_vector.h | 10 +
be/src/vec/common/hash_table/hash.h | 21 +
be/src/vec/common/hash_table/hash_key_type.h | 6 +
be/src/vec/common/schema_util.cpp | 25 +-
be/src/vec/common/uint128.h | 19 +
be/src/vec/core/call_on_type_index.h | 162 +
be/src/vec/data_types/data_type_agg_state.h | 20 +-
be/src/vec/exec/scan/file_scanner.cpp | 5 +-
be/src/vec/exprs/vcast_expr.cpp | 5 +-
be/src/vec/exprs/vectorized_agg_fn.cpp | 10 +-
be/src/vec/exprs/vectorized_fn_call.cpp | 3 +-
be/src/vec/exprs/vin_predicate.cpp | 5 +-
be/src/vec/exprs/vmatch_predicate.cpp | 7 +-
be/src/vec/exprs/vtopn_pred.h | 4 +-
.../functions/array/function_array_aggregation.cpp | 491 ++-
.../vec/functions/array/function_array_cum_sum.cpp | 131 +-
be/src/vec/functions/comparison_equal_for_null.cpp | 10 +-
be/src/vec/functions/complex_dict_hash_map.h | 8 +
be/src/vec/functions/function.h | 73 +-
.../function_date_or_datetime_computation.h | 16 +-
be/src/vec/functions/function_ifnull.h | 3 +-
be/src/vec/functions/function_search.cpp | 9 +-
be/src/vec/functions/function_string.cpp | 1 +
be/src/vec/functions/function_string.h | 146 +
be/src/vec/functions/function_tokenize.cpp | 5 +-
be/src/vec/functions/match.cpp | 6 +-
be/src/vec/functions/nullif.cpp | 10 +-
be/src/vec/functions/simple_function_factory.h | 32 +-
be/src/vec/sink/vdata_stream_sender.cpp | 4 -
be/test/ai/aggregate_function_ai_agg_test.cpp | 2 +-
be/test/olap/date_bloom_filter_test.cpp | 19 +-
be/test/olap/delete_bitmap_calculator_test.cpp | 13 +
be/test/olap/inverted_index_parser_test.cpp | 17 +
.../segment_v2/column_meta_accessor_test.cpp | 705 ++++
.../rowset/segment_v2/column_reader_cache_test.cpp | 47 +-
.../segment_v2/external_col_meta_util_test.cpp | 663 ++++
.../inverted_index/ananlyzer/analyzer_test.cpp | 16 +-
.../icu_normalizer_char_filter_factory_test.cpp | 268 ++
.../icu_normalizer_filter_factory_test.cpp | 295 ++
.../segment_v2/inverted_index/util/reader_test.cpp | 2 +-
be/test/olap/rowset/segment_v2/mock/mock_segment.h | 6 -
.../variant_column_writer_reader_test.cpp | 272 +-
be/test/olap/tablet_schema_index_test.cpp | 6 +-
.../pipeline/common/distinct_agg_utils_test.cpp | 10 +
be/test/pipeline/common/set_utils_test.cpp | 8 +
.../operator/analytic_sink_operator_test.cpp | 35 +-
.../operator/streaming_agg_operator_test.cpp | 5 +-
be/test/testutil/mock/mock_agg_fn_evaluator.cpp | 21 +-
be/test/testutil/mock/mock_agg_fn_evaluator.h | 4 +-
.../vec/aggregate_functions/agg_array_agg_test.cpp | 11 +-
be/test/vec/aggregate_functions/agg_avg_test.cpp | 3 +-
be/test/vec/aggregate_functions/agg_bit_test.cpp | 9 +-
.../vec/aggregate_functions/agg_bitmap_test.cpp | 10 +-
.../aggregate_functions/agg_bool_union_test.cpp | 18 +-
.../vec/aggregate_functions/agg_collect_test.cpp | 14 +-
be/test/vec/aggregate_functions/agg_corr_test.cpp | 6 +-
be/test/vec/aggregate_functions/agg_count_test.cpp | 3 +-
.../vec/aggregate_functions/agg_function_test.h | 5 +-
.../agg_group_array_intersect_test.cpp | 8 +-
.../vec/aggregate_functions/agg_histogram_test.cpp | 2 +-
.../agg_linear_histogram_test.cpp | 4 +-
.../aggregate_functions/agg_min_max_by_test.cpp | 2 +-
.../vec/aggregate_functions/agg_min_max_test.cpp | 8 +-
.../vec/aggregate_functions/agg_replace_test.cpp | 4 +-
be/test/vec/aggregate_functions/agg_test.cpp | 45 +-
.../aggregate_functions/vec_count_by_enum_test.cpp | 2 +-
.../vec/aggregate_functions/vec_retention_test.cpp | 2 +-
.../vec_sequence_match_test.cpp | 10 +-
.../aggregate_functions/vec_window_funnel_test.cpp | 2 +-
be/test/vec/common/schema_util_rowset_test.cpp | 130 +-
.../vec/function/function_dict_get_many_test.cpp | 4 +-
be/test/vec/function/function_dict_get_test.cpp | 4 +-
be/test/vec/function/function_match_test.cpp | 4 +-
be/test/vec/function/function_string_test.cpp | 91 +
.../vec/function/simple_function_factory_test.cpp | 4 +-
be/test/vec/jsonb/jsonb_document_test.cpp | 15 +
conf/be.conf | 2 +-
conf/fe.conf | 2 +-
.../scripts/data/test_flatten_data.json | 26 +
.../elasticsearch/scripts/es_init.sh | 6 +
.../scripts/index/es7_test_flatten.json | 25 +
fe/be-java-extensions/avro-scanner/pom.xml | 21 +-
fe/be-java-extensions/hadoop-hudi-scanner/pom.xml | 35 +-
.../iceberg-metadata-scanner/pom.xml | 1 +
fe/be-java-extensions/java-common/pom.xml | 84 +
fe/be-java-extensions/java-udf/pom.xml | 7 +-
fe/be-java-extensions/jdbc-scanner/pom.xml | 3 +-
fe/be-java-extensions/max-compute-scanner/pom.xml | 17 +-
fe/be-java-extensions/paimon-scanner/pom.xml | 25 +-
fe/be-java-extensions/preload-extensions/pom.xml | 96 +-
.../main/java/org/apache/doris/preload/README.md | 217 --
.../hadoop/hive/metastore/HiveMetaStoreClient.java | 3630 --------------------
.../hadoop/hive/metastore/HiveVersionUtil.java | 84 -
.../src/main/resources/package-proj.xml | 37 -
.../trino-connector-scanner/pom.xml | 8 +-
.../main/java/org/apache/doris/common/Config.java | 305 +-
fe/fe-core/pom.xml | 5 +
.../antlr4/org/apache/doris/nereids/DorisLexer.g4 | 4 +-
.../antlr4/org/apache/doris/nereids/DorisParser.g4 | 8 +-
.../src/main/java/org/apache/doris/DorisFE.java | 7 +-
.../main/java/org/apache/doris/alter/Alter.java | 12 +-
.../org/apache/doris/alter/AlterJobV2Factory.java | 9 +-
.../org/apache/doris/alter/CloudRollupJobV2.java | 8 +-
.../apache/doris/alter/CloudSchemaChangeJobV2.java | 2 +-
.../doris/alter/MaterializedViewHandler.java | 20 +-
.../java/org/apache/doris/alter/RollupJobV2.java | 15 +-
.../apache/doris/alter/SchemaChangeHandler.java | 14 +-
.../org/apache/doris/alter/SchemaChangeJobV2.java | 6 +-
.../java/org/apache/doris/analysis/IndexDef.java | 6 +-
.../apache/doris/analysis/InvertedIndexUtil.java | 86 +-
.../org/apache/doris/analysis/MVColumnItem.java | 3 +-
.../org/apache/doris/analysis/MatchPredicate.java | 8 +-
.../org/apache/doris/catalog/AliasFunction.java | 27 +
.../doris/catalog/BuiltinScalarFunctions.java | 2 +
.../main/java/org/apache/doris/catalog/Column.java | 34 +-
.../java/org/apache/doris/catalog/Database.java | 7 +-
.../main/java/org/apache/doris/catalog/Env.java | 5 +-
.../org/apache/doris/catalog/FunctionUtil.java | 37 +-
.../apache/doris/catalog/GlobalFunctionMgr.java | 4 +-
.../main/java/org/apache/doris/catalog/Index.java | 14 +-
.../org/apache/doris/catalog/InternalSchema.java | 3 +-
.../main/java/org/apache/doris/catalog/MTMV.java | 70 +-
.../doris/catalog/MaterializedIndexMeta.java | 108 +-
.../java/org/apache/doris/catalog/OlapTable.java | 10 +-
.../org/apache/doris/catalog/OlapTableFactory.java | 10 +
.../org/apache/doris/catalog/RefreshManager.java | 13 +-
.../main/java/org/apache/doris/catalog/View.java | 18 +-
.../org/apache/doris/cloud/CloudWarmUpJob.java | 75 +-
.../apache/doris/cloud/backup/CloudRestoreJob.java | 3 +-
.../cloud/common/util/CloudPropertyAnalyzer.java | 1 -
.../cloud/datasource/CloudInternalCatalog.java | 15 +-
.../cache/NereidsSortedPartitionsCacheManager.java | 39 +-
.../doris/common/profile/SummaryProfile.java | 3 +
.../apache/doris/common/util/PropertyAnalyzer.java | 7 +-
.../org/apache/doris/datasource/CatalogMgr.java | 19 +-
.../apache/doris/datasource/CatalogProperty.java | 30 +-
.../apache/doris/datasource/ExternalCatalog.java | 1 +
.../org/apache/doris/datasource/ExternalTable.java | 14 +
.../org/apache/doris/datasource/es/EsUtil.java | 1 +
.../doris/datasource/hive/HMSExternalTable.java | 15 +
.../doris/datasource/hive/HiveMetaStoreCache.java | 143 +-
.../datasource/iceberg/IcebergExternalCatalog.java | 5 +-
.../iceberg/profile/IcebergMetricsReporter.java | 167 +
.../datasource/iceberg/source/IcebergScanNode.java | 4 +-
.../datasource/paimon/PaimonExternalCatalog.java | 2 +-
.../property/storage/StorageProperties.java | 23 +-
.../doris/httpv2/rest/TableQueryPlanAction.java | 119 +-
.../ICUNormalizerCharFilterValidator.java | 75 +
.../ICUNormalizerTokenFilterValidator.java | 64 +
.../org/apache/doris/indexpolicy/IndexPolicy.java | 7 +-
.../apache/doris/indexpolicy/IndexPolicyMgr.java | 159 +-
.../doris/indexpolicy/IndexPolicyTypeEnum.java | 3 +-
.../insertoverwrite/InsertOverwriteManager.java | 6 +-
.../doris/load/routineload/KafkaTaskInfo.java | 26 +-
.../load/routineload/RoutineLoadTaskInfo.java | 2 +-
.../main/java/org/apache/doris/mtmv/MTMVCache.java | 26 +-
.../java/org/apache/doris/mtmv/MTMVPlanUtil.java | 6 +-
.../apache/doris/mysql/privilege/PrivTable.java | 35 +-
.../org/apache/doris/nereids/SqlCacheContext.java | 2 +-
.../org/apache/doris/nereids/StatementContext.java | 10 +-
.../glue/translator/ExpressionTranslator.java | 6 +
.../glue/translator/PhysicalPlanTranslator.java | 46 +-
.../doris/nereids/jobs/executor/Rewriter.java | 2 +
.../doris/nereids/load/NereidsLoadTaskInfo.java | 3 +
.../nereids/load/NereidsRoutineLoadTaskInfo.java | 15 +-
.../apache/doris/nereids/memo/StructInfoMap.java | 13 +-
.../doris/nereids/minidump/MinidumpUtils.java | 4 +-
.../doris/nereids/parser/LogicalPlanBuilder.java | 48 +-
.../post/CommonSubExpressionCollector.java | 18 +-
.../properties/ChildrenPropertiesRegulator.java | 39 +
.../doris/nereids/rules/analysis/BindRelation.java | 22 +-
.../doris/nereids/rules/analysis/BindSink.java | 66 +-
.../nereids/rules/analysis/CollectRelation.java | 11 +-
.../nereids/rules/analysis/NormalizeAggregate.java | 12 +-
.../nereids/rules/analysis/NormalizeRepeat.java | 48 +-
.../rules/analysis/SessionVarGuardRewriter.java | 149 +
.../mv/AbstractMaterializedViewAggregateRule.java | 6 +-
.../mv/InitMaterializationContextHook.java | 30 +-
.../mv/MaterializedViewFilterAggregateRule.java | 7 +-
...MaterializedViewFilterProjectAggregateRule.java | 5 +-
...MaterializedViewProjectFilterAggregateRule.java | 14 +-
.../expression/ExpressionBottomUpRewriter.java | 12 +-
.../rules/expression/ExpressionRuleType.java | 4 +-
.../nereids/rules/expression/MergeGuardExpr.java | 52 +
.../rules/SimplifyComparisonPredicate.java | 6 +-
.../expression/rules/SortedPartitionRanges.java | 44 +
.../implementation/SplitAggWithoutDistinct.java | 53 +-
.../doris/nereids/rules/rewrite/ColumnPruning.java | 2 +-
.../rules/rewrite/DistinctAggregateRewriter.java | 22 +-
.../rules/rewrite/EliminateEmptyRelation.java | 94 +-
.../nereids/rules/rewrite/MergeAggregate.java | 46 +-
.../rules/rewrite/PruneFileScanPartition.java | 12 +-
.../doris/nereids/rules/rewrite/SkewJoin.java | 9 +-
.../doris/nereids/stats/StatsCalculator.java | 5 +-
.../doris/nereids/trees/expressions/Add.java | 2 +-
.../doris/nereids/trees/expressions/CaseWhen.java | 2 +-
.../doris/nereids/trees/expressions/Divide.java | 2 +-
.../doris/nereids/trees/expressions/Mod.java | 2 +-
.../doris/nereids/trees/expressions/Multiply.java | 2 +-
.../trees/expressions/NeedSessionVarGuard.java} | 27 +-
.../trees/expressions/SessionVarGuardExpr.java | 147 +
.../doris/nereids/trees/expressions/Subtract.java | 2 +-
.../trees/expressions/WindowExpression.java | 2 +-
.../functions/ComputePrecisionForSum.java | 6 +
.../trees/expressions/functions/agg/Avg.java | 4 +-
.../functions/agg/MultiDistinctSum.java | 3 +-
.../functions/agg/MultiDistinctSum0.java | 3 +-
.../trees/expressions/functions/agg/Sum.java | 3 +-
.../trees/expressions/functions/agg/Sum0.java | 3 +-
.../functions/executable/NumericArithmetic.java | 18 +-
.../expressions/functions/scalar/ArrayAvg.java | 3 +-
.../expressions/functions/scalar/ArrayCumSum.java | 4 +-
.../expressions/functions/scalar/ArrayMax.java | 3 +-
.../expressions/functions/scalar/ArrayMin.java | 3 +-
.../expressions/functions/scalar/ArrayProduct.java | 3 +-
.../expressions/functions/scalar/ArraySum.java | 3 +-
.../expressions/functions/scalar/Coalesce.java | 3 +-
.../functions/scalar/GreatestLeast.java | 3 +-
.../trees/expressions/functions/scalar/If.java | 3 +-
.../functions/scalar/InnerProductApproximate.java | 5 +
.../functions/scalar/L2DistanceApproximate.java | 5 +
.../trees/expressions/functions/scalar/NullIf.java | 3 +-
.../trees/expressions/functions/scalar/Nvl.java | 3 +-
...tanceApproximate.java => UnicodeNormalize.java} | 44 +-
.../trees/expressions/functions/udf/AliasUdf.java | 30 +-
.../expressions/functions/udf/AliasUdfBuilder.java | 64 +-
.../expressions/literal/DecimalV3Literal.java | 19 +-
.../nereids/trees/expressions/literal/Literal.java | 2 +-
.../expressions/visitor/ExpressionVisitor.java | 5 +
.../expressions/visitor/ScalarFunctionVisitor.java | 5 +
.../apache/doris/nereids/trees/plans/PlanType.java | 3 +
.../nereids/trees/plans/algebra/Aggregate.java | 29 +
.../plans/commands/CreateFunctionCommand.java | 5 +-
.../commands/CreateIndexNormalizerCommand.java | 67 +
.../commands/CreateMaterializedViewCommand.java | 5 +-
.../trees/plans/commands/CreateStageCommand.java | 3 +
.../plans/commands/DropIndexNormalizerCommand.java | 63 +
.../plans/commands/ShowIndexNormalizerCommand.java | 61 +
.../plans/commands/info/ColumnDefinition.java | 13 +-
.../trees/plans/commands/info/CreateMTMVInfo.java | 9 +-
.../trees/plans/commands/info/CreateTableInfo.java | 14 +
.../trees/plans/commands/info/IndexDefinition.java | 14 +-
.../plans/commands/info/ShowCreateMTMVInfo.java | 3 +-
.../trees/plans/visitor/CommandVisitor.java | 18 +
.../apache/doris/nereids/types/DecimalV2Type.java | 2 +-
.../apache/doris/nereids/types/DecimalV3Type.java | 24 +-
.../org/apache/doris/nereids/util/PlanUtils.java | 42 +
.../org/apache/doris/persist/AlterViewInfo.java | 20 +-
.../org/apache/doris/planner/OlapTableSink.java | 3 +
.../doris/plugin/audit/AuditStreamLoader.java | 1 +
.../java/org/apache/doris/qe/AuditLogHelper.java | 2 +-
.../apache/doris/qe/AutoCloseSessionVariable.java | 63 +
.../org/apache/doris/qe/ConnectContextUtil.java | 8 +
.../main/java/org/apache/doris/qe/QueryState.java | 10 +
.../java/org/apache/doris/qe/SessionVariable.java | 182 +-
.../main/java/org/apache/doris/qe/VariableMgr.java | 87 +-
.../apache/doris/statistics/BaseAnalysisTask.java | 1 +
.../org/apache/doris/task/CreateReplicaTask.java | 1 -
.../doris/alter/MaterializedViewHandlerTest.java | 7 +-
.../org/apache/doris/alter/RollupJobV2Test.java | 2 +-
.../org/apache/doris/catalog/CreateViewTest.java | 5 +-
.../org/apache/doris/catalog/DatabaseTest.java | 1 +
.../doris/datasource/RefreshCatalogTest.java | 13 +-
.../property/storage/AzurePropertiesTest.java | 4 +-
.../property/storage/COSPropertiesTest.java | 7 +-
.../property/storage/OBSPropertyTest.java | 8 +-
.../property/storage/OSSPropertiesTest.java | 7 +-
.../property/storage/S3PropertiesTest.java | 3 +-
.../ChildrenPropertiesRegulatorTest.java | 167 +
.../expression/ExpressionRewriteTestHelper.java | 3 +-
.../doris/nereids/rules/rewrite/PrepareTest.java | 83 +
.../trees/expressions/VariablePersistTest.java | 215 ++
.../plans/commands/merge/MergeIntoCommandTest.java | 42 +-
.../apache/doris/persist/AlterViewInfoTest.java | 3 +-
.../org/apache/doris/qe/OlapQueryCacheTest.java | 9 +-
.../org/apache/doris/qe/SessionVariablesTest.java | 3 +-
fe/pom.xml | 5 +
gensrc/proto/olap_file.proto | 6 +
gensrc/proto/segment_v2.proto | 29 +-
gensrc/thrift/AgentService.thrift | 7 +-
gensrc/thrift/Exprs.thrift | 2 +-
.../agg_state/decimalv3/test_decimalv3.out | 17 +
.../data/empty_relation/eliminate_empty.out | 17 +-
.../external_table_p0/es/test_es_flatten_type.out | 13 +
...est_query_remote_doris_as_olap_table_select.out | 24 +
.../analyzer/test_custom_normalizer.out | 15 +
.../analyzer/test_unicode_normalize.out | 19 +
.../data/inverted_index_p0/test_pinyin_phrase.out | 18 +
.../variables_persist/test_alias_function.out | 37 +
.../variables_persist/test_array_agg_view.out | 41 +
.../variables_persist/test_generated_column.out | 25 +
.../nereids_p0/variables_persist/test_mtmv.out | 21 +
.../nereids_p0/variables_persist/test_sync_mv.out | 16 +
.../variables_persist/test_view_var_persist.out | 216 ++
.../variables_persist/use_view_create_mv.out | 11 +
.../infer_predicate/pull_up_predicate_set_op.out | 4 +-
.../mv/grouping_sets/grouping_sets.out | 60 +
regression-test/data/query_p0/dual/dual.out | 15 +
.../test_load_channel_profile.out} | 3 +-
.../test_non_overlap_seg_heavy_sc.out | 2 +-
.../ext_meta/test_storage_format_v2_1.out | 21 +
.../test_variant_external_meta_concurrent.out | 29 +
.../test_variant_external_meta_edge_cases.out | 174 +
.../test_variant_external_meta_integration.out | 120 +
.../test_variant_external_meta_with_sparse.out | 197 ++
.../data/variant_p0/query_subcolumns.out | 211 ++
.../nonConcurrent/conf/regression-conf.groovy | 1 +
.../agg_state/decimalv3/test_decimalv3.groovy | 66 +
...test_export_table_with_materialized_view.groovy | 2 +-
.../es/test_es_flatten_type.groovy | 50 +
.../test_jdbc_catalog_refresh_update_time.groovy | 104 +
..._query_remote_doris_as_olap_table_select.groovy | 104 +
.../hudi/hudi_mtmv/test_hudi_rewrite_mtmv.groovy | 82 +-
.../analyzer/test_custom_normalizer.groovy | 131 +
.../analyzer/test_unicode_normalize.groovy | 79 +
.../inverted_index_p0/test_pinyin_phrase.groovy | 73 +
.../test_routine_load_adaptive_param.groovy | 91 +
.../stream_load/test_stream_load_audit_log.groovy | 83 +
.../suites/manager/test_manager_interface_1.groovy | 2 +-
.../agg_have_dup_base/agg_have_dup_base.groovy | 10 -
.../mv_p0/agg_state/test_agg_state_max_by.groovy | 10 +-
.../suites/mv_p0/case_ignore/case_ignore.groovy | 12 +-
.../suites/mv_p0/count_star/count_star.groovy | 14 -
.../suites/mv_p0/dis_26495/dis_26495.groovy | 4 -
.../suites/mv_p0/k1ap2spa/k1ap2spa.groovy | 3 -
.../mv_p0/k1s2m3_auto_inc/k1s2m3_auto_inc.groovy | 5 +-
.../multi_agg_with_same_slot.groovy | 11 -
.../mv_p0/multi_slot_k123p/multi_slot_k123p.groovy | 5 +-
.../multi_slot_k1a2p2ap3p.groovy | 4 -
.../multi_slot_k1a2p2ap3ps.groovy | 6 -
.../multi_slot_k1p2ap3p/multi_slot_k1p2ap3p.groovy | 4 -
.../multi_slot_k1p2ap3ps.groovy | 4 -
.../multi_slot_multi_mv/multi_slot_multi_mv.groovy | 34 -
.../mv_ignore_predicate/mv_ignore_predicate.groovy | 15 -
.../mv_p0/mv_percentile/mv_percentile.groovy | 15 -
.../suites/mv_p0/mv_with_view/mv_with_view.groovy | 22 -
.../suites/mv_p0/null_insert/null_insert.groovy | 8 +-
.../mv_p0/routine_load_hll/routine_load_hll.groovy | 4 -
.../ssb/multiple_no_where/multiple_no_where.groovy | 1 -
.../mv_p0/ssb/multiple_ssb/multiple_ssb.groovy | 47 -
.../suites/mv_p0/ssb/q_4_1_r1/q_4_1_r1.groovy | 14 +-
.../mv_p0/sum_divede_count/sum_devide_count.groovy | 11 -
.../suites/mv_p0/test_28741/test_28741.groovy | 4 -
.../test_approx_count_distinct.groovy | 6 -
.../suites/mv_p0/test_base/test_base.groovy | 6 +-
.../mv_p0/test_casewhen/test_casewhen.groovy | 4 -
.../suites/mv_p0/test_doc_e4/test_doc_e4.groovy | 10 -
.../test_dup_group_by_mv_abs.groovy | 5 -
.../test_dup_group_by_mv_plus.groovy | 5 -
.../mv_p0/test_dup_mv_abs/test_dup_mv_abs.groovy | 20 -
.../mv_p0/test_dup_mv_bin/test_dup_mv_bin.groovy | 15 -
.../test_dup_mv_bitmap_hash.groovy | 9 -
.../test_dup_mv_expr_priority.groovy | 7 -
.../mv_p0/test_dup_mv_plus/test_dup_mv_plus.groovy | 18 -
.../test_dup_mv_repeat/test_dup_mv_repeat.groovy | 7 +-
.../mv_p0/test_dup_mv_year/test_dup_mv_year.groovy | 8 -
.../test_insert_multi/test_insert_multi.groovy | 5 -
.../suites/mv_p0/test_mv_dp/test_mv_dp.groovy | 19 +-
.../suites/mv_p0/test_mv_mow/test_mv_mow.groovy | 4 -
.../suites/mv_p0/test_ndv/test_ndv.groovy | 7 -
.../suites/mv_p0/test_nvl/test_nvl.groovy | 13 -
.../suites/mv_p0/test_o2/test_o2.groovy | 4 -
.../suites/mv_p0/test_substr/test_substr.groovy | 5 -
.../mv_p0/test_tbl_name/test_tbl_name.groovy | 14 -
.../mv_p0/test_upper_alias/test_upper_alias.groovy | 13 -
.../test_user_activity/test_user_activity.groovy | 7 +-
regression-test/suites/mv_p0/unique/unique.groovy | 9 +-
.../testAggQueryOnAggMV10.groovy | 7 -
.../testAggQueryOnAggMV11.groovy | 8 -
.../testAggQueryOnAggMV2.groovy | 7 -
.../testAggQueryOnAggMV3.groovy | 10 -
.../testAggQuqeryOnAggMV5.groovy | 7 -
.../testAggQuqeryOnAggMV6.groovy | 7 -
.../testAggQuqeryOnAggMV7.groovy | 7 -
.../testAggTableCountDistinctInBitmapType.groovy | 10 -
.../testAggregateMVCalcAggFunctionQuery.groovy | 6 -
.../testBitmapUnionInQuery.groovy | 7 -
.../testCountDistinctToBitmap.groovy | 13 -
.../testIncorrectMVRewriteInSubquery.groovy | 8 -
.../testIncorrectRewriteCountDistinct.groovy | 7 -
.../testJoinOnLeftProjectToJoin.groovy | 6 -
.../mv_p0/ut/testNDVToHll/testNDVToHll.groovy | 9 -
.../ut/testProjectionMV1/testProjectionMV1.groovy | 13 +-
.../ut/testProjectionMV2/testProjectionMV2.groovy | 9 -
.../ut/testProjectionMV3/testProjectionMV3.groovy | 9 -
.../ut/testProjectionMV4/testProjectionMV4.groovy | 7 -
.../testSingleMVMultiUsage.groovy | 10 -
.../mv_p0/ut/testSubQuery/testSubQuery.groovy | 5 -
.../ut/testUnionDistinct/testUnionDistinct.groovy | 12 +-
.../suites/mv_p0/where/k123/k123.groovy | 17 -
.../suites/mv_p0/where/mvljc/mvljc.groovy | 8 -
.../nereids_p0/variables_persist/load.groovy | 44 +
.../variables_persist/test_alias_function.groovy | 80 +
.../variables_persist/test_array_agg_view.groovy | 77 +
.../variables_persist/test_generated_column.groovy | 147 +
.../nereids_p0/variables_persist/test_mtmv.groovy | 246 ++
.../variables_persist/test_sync_mv.groovy | 117 +
.../variables_persist/test_view_var_persist.groovy | 525 +++
.../variables_persist/use_view_create_mv.groovy | 68 +
.../agg_optimize_when_uniform.groovy | 4 +
.../mv/grouping_sets/grouping_sets.groovy | 168 +-
.../mv/nested/nested_materialized_view.groovy | 1252 ++++++-
.../single_table_without_aggregate.groovy | 1 -
.../nereids_rules_p0/skew_join/skew_join.groovy | 137 +
.../suites/nereids_syntax_p0/advance_mv.groovy | 10 -
.../suites/nereids_syntax_p0/cte.groovy | 5 +
.../mv/aggregate/agg_sync_mv.groovy | 1 -
.../nereids_syntax_p0/mv/newMv/case_ignore.groovy | 7 -
.../mv/newMv/dup_gb_mv_abs.groovy | 8 -
.../mv/newMv/dup_gb_mv_plus.groovy | 8 -
.../nereids_syntax_p0/mv/newMv/dup_mv_abs.groovy | 16 -
.../nereids_syntax_p0/mv/newMv/dup_mv_bin.groovy | 15 -
.../mv/newMv/dup_mv_bm_hash.groovy | 6 -
.../nereids_syntax_p0/mv/newMv/dup_mv_plus.groovy | 15 -
.../nereids_syntax_p0/mv/newMv/dup_mv_year.groovy | 8 -
.../nereids_syntax_p0/mv/newMv/multi_slot1.groovy | 5 -
.../nereids_syntax_p0/mv/newMv/multi_slot2.groovy | 10 -
.../nereids_syntax_p0/mv/newMv/multi_slot3.groovy | 5 -
.../nereids_syntax_p0/mv/newMv/multi_slot4.groovy | 6 -
.../nereids_syntax_p0/mv/newMv/multi_slot5.groovy | 5 -
.../nereids_syntax_p0/mv/newMv/multi_slot6.groovy | 3 -
.../nereids_syntax_p0/mv/newMv/mv_with_view.groovy | 22 -
.../nereids_syntax_p0/mv/newMv/rollback1.groovy | 2 -
.../nereids_syntax_p0/mv/newMv/single_slot.groovy | 8 -
.../mv/newMv/sum_devide_count.groovy | 11 -
.../nereids_syntax_p0/mv/newMv/unique_mv.groovy | 6 -
.../nereids_syntax_p0/mv/ut/MVMultiUsage.groovy | 12 -
.../suites/nereids_syntax_p0/mv/ut/MVWithAs.groovy | 5 -
.../nereids_syntax_p0/mv/ut/aggCDInBitmap.groovy | 9 -
.../nereids_syntax_p0/mv/ut/aggMVCalcAggFun.groovy | 5 -
.../nereids_syntax_p0/mv/ut/aggOnAggMV1.groovy | 7 -
.../nereids_syntax_p0/mv/ut/aggOnAggMV10.groovy | 8 -
.../nereids_syntax_p0/mv/ut/aggOnAggMV11.groovy | 9 -
.../nereids_syntax_p0/mv/ut/aggOnAggMV2.groovy | 6 -
.../nereids_syntax_p0/mv/ut/aggOnAggMV3.groovy | 7 -
.../nereids_syntax_p0/mv/ut/aggOnAggMV6.groovy | 8 -
.../nereids_syntax_p0/mv/ut/aggOnAggMV7.groovy | 7 -
.../nereids_syntax_p0/mv/ut/bitmapUnionIn.groovy | 8 -
.../nereids_syntax_p0/mv/ut/incMVReInSub.groovy | 8 -
.../nereids_syntax_p0/mv/ut/incRewriteCD.groovy | 8 -
.../mv/ut/joinOnCalcToJoin.groovy | 7 -
.../mv/ut/joinOnLeftPToJoin.groovy | 6 -
.../suites/nereids_syntax_p0/mv/ut/onStar.groovy | 1 -
.../nereids_syntax_p0/mv/ut/onlyGroupBy.groovy | 4 -
.../nereids_syntax_p0/mv/ut/orderByOnPView.groovy | 8 -
.../nereids_syntax_p0/mv/ut/projectMV1.groovy | 7 -
.../nereids_syntax_p0/mv/ut/projectMV2.groovy | 10 -
.../nereids_syntax_p0/mv/ut/projectMV3.groovy | 10 -
.../nereids_syntax_p0/mv/ut/projectMV4.groovy | 10 -
.../suites/nereids_syntax_p0/mv/ut/subQuery.groovy | 15 -
.../suites/nereids_syntax_p0/mv/ut/unionDis.groovy | 12 -
.../suites/nereids_syntax_p0/rollup/agg.groovy | 3 -
.../nereids_syntax_p0/rollup/agg_date.groovy | 4 -
.../suites/nereids_syntax_p0/rollup/date.groovy | 11 +-
.../suites/nereids_syntax_p0/rollup/hll/hll.groovy | 5 -
.../hll_with_light_sc/hll_with_light_sc.groovy | 5 -
.../mv/aggregate/agg_sync_mv.groovy | 1 -
regression-test/suites/query_p0/dual/dual.groovy | 30 +
.../query_p0/system/test_table_properties.groovy | 9 +-
.../query_profile/test_load_channel_profile.groovy | 139 +
.../suites/query_profile/test_profile.groovy | 83 -
.../suites/rollup_p0/test_rollup_agg.groovy | 6 -
.../test_filecache_with_alter_table.groovy | 234 ++
...test_show_create_table_and_views_nereids.groovy | 3 +-
.../load.groovy | 14 +
.../ext_meta/test_storage_format_v2_1.groovy | 98 +
.../test_variant_external_meta_concurrent.groovy | 307 ++
.../test_variant_external_meta_edge_cases.groovy | 296 ++
.../test_variant_external_meta_integration.groovy | 355 ++
.../test_variant_external_meta_with_sparse.groovy | 251 ++
...est_variant_compaction_with_sparse_limit.groovy | 6 +-
.../suites/variant_p0/query_subcolumns.groovy | 94 +
609 files changed, 18336 insertions(+), 7748 deletions(-)
create mode 100644 be/src/olap/rowset/segment_v2/column_meta_accessor.cpp
create mode 100644 be/src/olap/rowset/segment_v2/column_meta_accessor.h
create mode 100644 be/src/olap/rowset/segment_v2/external_col_meta_util.cpp
create mode 100644 be/src/olap/rowset/segment_v2/external_col_meta_util.h
create mode 100644
be/src/olap/rowset/segment_v2/inverted_index/char_filter/icu_normalizer_char_filter.cpp
create mode 100644
be/src/olap/rowset/segment_v2/inverted_index/char_filter/icu_normalizer_char_filter.h
create mode 100644
be/src/olap/rowset/segment_v2/inverted_index/char_filter/icu_normalizer_char_filter_factory.h
create mode 100644
be/src/olap/rowset/segment_v2/inverted_index/normalizer/custom_normalizer.cpp
create mode 100644
be/src/olap/rowset/segment_v2/inverted_index/normalizer/custom_normalizer.h
create mode 100644
be/src/olap/rowset/segment_v2/inverted_index/normalizer/custom_normalizer_config.cpp
copy
be/src/olap/rowset/segment_v2/inverted_index/{analyzer/custom_analyzer_config.h
=> normalizer/custom_normalizer_config.h} (59%)
create mode 100644
be/src/olap/rowset/segment_v2/inverted_index/token_filter/icu_normalizer_filter.cpp
rename
regression-test/suites/datatype_p0/agg_state/decimal_sum/test_decimal_sum.groovy
=>
be/src/olap/rowset/segment_v2/inverted_index/token_filter/icu_normalizer_filter.h
(55%)
create mode 100644
be/src/olap/rowset/segment_v2/inverted_index/token_filter/icu_normalizer_filter_factory.h
create mode 100644
be/src/olap/rowset/segment_v2/variant/variant_ext_meta_writer.cpp
create mode 100644
be/src/olap/rowset/segment_v2/variant/variant_ext_meta_writer.h
create mode 100644
be/src/olap/rowset/segment_v2/variant/variant_external_meta_reader.cpp
create mode 100644
be/src/olap/rowset/segment_v2/variant/variant_external_meta_reader.h
create mode 100644 be/src/runtime/stream_load/stream_load_recorder_manager.cpp
create mode 100644 be/src/runtime/stream_load/stream_load_recorder_manager.h
create mode 100644 be/test/olap/rowset/segment_v2/column_meta_accessor_test.cpp
create mode 100644
be/test/olap/rowset/segment_v2/external_col_meta_util_test.cpp
create mode 100644
be/test/olap/rowset/segment_v2/inverted_index/char_filter/icu_normalizer_char_filter_factory_test.cpp
create mode 100644
be/test/olap/rowset/segment_v2/inverted_index/token_filter/icu_normalizer_filter_factory_test.cpp
create mode 100644
docker/thirdparties/docker-compose/elasticsearch/scripts/data/test_flatten_data.json
create mode 100644
docker/thirdparties/docker-compose/elasticsearch/scripts/index/es7_test_flatten.json
delete mode 100644
fe/be-java-extensions/preload-extensions/src/main/java/org/apache/doris/preload/README.md
delete mode 100644
fe/be-java-extensions/preload-extensions/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
delete mode 100644
fe/be-java-extensions/preload-extensions/src/main/java/org/apache/hadoop/hive/metastore/HiveVersionUtil.java
delete mode 100644
fe/be-java-extensions/preload-extensions/src/main/resources/package-proj.xml
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/iceberg/profile/IcebergMetricsReporter.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/indexpolicy/ICUNormalizerCharFilterValidator.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/indexpolicy/ICUNormalizerTokenFilterValidator.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/analysis/SessionVarGuardRewriter.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/MergeGuardExpr.java
copy
fe/fe-core/src/main/java/org/apache/doris/{indexpolicy/IndexPolicyTypeEnum.java
=> nereids/trees/expressions/NeedSessionVarGuard.java} (55%)
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/SessionVarGuardExpr.java
copy
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/scalar/{L2DistanceApproximate.java
=> UnicodeNormalize.java} (62%)
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/CreateIndexNormalizerCommand.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/DropIndexNormalizerCommand.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/ShowIndexNormalizerCommand.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/qe/AutoCloseSessionVariable.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/nereids/properties/ChildrenPropertiesRegulatorTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/nereids/rules/rewrite/PrepareTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/nereids/trees/expressions/VariablePersistTest.java
create mode 100644
regression-test/data/datatype_p0/agg_state/decimalv3/test_decimalv3.out
create mode 100644
regression-test/data/external_table_p0/es/test_es_flatten_type.out
create mode 100644
regression-test/data/inverted_index_p0/analyzer/test_custom_normalizer.out
create mode 100644
regression-test/data/inverted_index_p0/analyzer/test_unicode_normalize.out
create mode 100644
regression-test/data/inverted_index_p0/test_pinyin_phrase.out
create mode 100644
regression-test/data/nereids_p0/variables_persist/test_alias_function.out
create mode 100644
regression-test/data/nereids_p0/variables_persist/test_array_agg_view.out
create mode 100644
regression-test/data/nereids_p0/variables_persist/test_generated_column.out
create mode 100644
regression-test/data/nereids_p0/variables_persist/test_mtmv.out
create mode 100644
regression-test/data/nereids_p0/variables_persist/test_sync_mv.out
create mode 100644
regression-test/data/nereids_p0/variables_persist/test_view_var_persist.out
create mode 100644
regression-test/data/nereids_p0/variables_persist/use_view_create_mv.out
rename
regression-test/data/{datatype_p0/agg_state/decimal_sum/test_decimal_sum.out =>
query_profile/test_load_channel_profile.out} (84%)
create mode 100644
regression-test/data/variant_p0/ext_meta/test_storage_format_v2_1.out
create mode 100644
regression-test/data/variant_p0/ext_meta/test_variant_external_meta_concurrent.out
create mode 100644
regression-test/data/variant_p0/ext_meta/test_variant_external_meta_edge_cases.out
create mode 100644
regression-test/data/variant_p0/ext_meta/test_variant_external_meta_integration.out
create mode 100644
regression-test/data/variant_p0/ext_meta/test_variant_external_meta_with_sparse.out
create mode 100644 regression-test/data/variant_p0/query_subcolumns.out
create mode 100644
regression-test/suites/datatype_p0/agg_state/decimalv3/test_decimalv3.groovy
create mode 100644
regression-test/suites/external_table_p0/es/test_es_flatten_type.groovy
create mode 100644
regression-test/suites/external_table_p0/jdbc/test_jdbc_catalog_refresh_update_time.groovy
create mode 100644
regression-test/suites/inverted_index_p0/analyzer/test_custom_normalizer.groovy
create mode 100644
regression-test/suites/inverted_index_p0/analyzer/test_unicode_normalize.groovy
create mode 100644
regression-test/suites/inverted_index_p0/test_pinyin_phrase.groovy
create mode 100644
regression-test/suites/load_p0/routine_load/test_routine_load_adaptive_param.groovy
create mode 100644
regression-test/suites/load_p0/stream_load/test_stream_load_audit_log.groovy
create mode 100644
regression-test/suites/nereids_p0/variables_persist/load.groovy
create mode 100644
regression-test/suites/nereids_p0/variables_persist/test_alias_function.groovy
create mode 100644
regression-test/suites/nereids_p0/variables_persist/test_array_agg_view.groovy
create mode 100644
regression-test/suites/nereids_p0/variables_persist/test_generated_column.groovy
create mode 100644
regression-test/suites/nereids_p0/variables_persist/test_mtmv.groovy
create mode 100644
regression-test/suites/nereids_p0/variables_persist/test_sync_mv.groovy
create mode 100644
regression-test/suites/nereids_p0/variables_persist/test_view_var_persist.groovy
create mode 100644
regression-test/suites/nereids_p0/variables_persist/use_view_create_mv.groovy
create mode 100644
regression-test/suites/query_profile/test_load_channel_profile.groovy
delete mode 100644 regression-test/suites/query_profile/test_profile.groovy
create mode 100644
regression-test/suites/schema_change/test_filecache_with_alter_table.groovy
create mode 100644
regression-test/suites/variant_p0/ext_meta/test_storage_format_v2_1.groovy
create mode 100644
regression-test/suites/variant_p0/ext_meta/test_variant_external_meta_concurrent.groovy
create mode 100644
regression-test/suites/variant_p0/ext_meta/test_variant_external_meta_edge_cases.groovy
create mode 100644
regression-test/suites/variant_p0/ext_meta/test_variant_external_meta_integration.groovy
create mode 100644
regression-test/suites/variant_p0/ext_meta/test_variant_external_meta_with_sparse.groovy
create mode 100644 regression-test/suites/variant_p0/query_subcolumns.groovy
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
