This is an automated email from the ASF dual-hosted git repository.
morningman pushed a change to branch auto-pick-59401-branch-4.0
in repository https://gitbox.apache.org/repos/asf/doris.git
discard 45a05547f8f [regression](hudi) Impl new Hudi Docker environment
(#59401)
add aae5957dfa2 [Chore](query) add _query_ctx_map_delay_delete (#59262)
(#56911) (#59638)
add c992307c331 branch-4.0: [fix](pipeline)Fix set_operation not correctly
setting shuffled_operator #59293 (#59702)
add 0cf97f451bd branch-4.0: [fix](agg) fix rule merge_aggregate #59629
(#59663)
add 658c14e4d26 branch-4.0: [cloud](fdb) Fix missing backup agent in
fdb.conf of fdb_ctl.sh #59641 (#59753)
add 681d0b890fc branch-4.0: [Opt](cloud) cache table version for cloud
mode #59339 (#59442)
add cb4d81366a8 branch-4.0: [fix](parquet)fix hudi parquet read
hoodie.datasource.write.drop.partition.columns prop table cause be core. #58532
(#59749)
add f6ebcb83600 [chore](be) remove all LRU Cache default value (#59556)
(#59751)
add 00503bc74ea branch-4.0: [optimize](parquet-reader) Optimize
performance by parquet bloom filter. (#59727)
add 2ded0048952 [fix](fe) add compatible code for LocalTablet and
LocalReplica (#59771)
add 05c063d7324 branch-4.0: [feature](inverted index) Implement es-like
boolean query #58545 (#59766)
add b4d26a34add branch-4.0: [Fix](Hdfs)Fix Kerberos auth config detection
and auto-enable hive.metastore.sasl #59748 (#59763)
add e3943ef952f [Bug](runtime-filter) use rf_lock to lock
local_state.conjuncts read operator #59688 (#59770)
add a33cc788609 branch-4.0: [Fix](Streamingjob) Fix the issue of
synchronization failure under empty tables #59735 (#59790)
add 3940c799ce5 branch-4.0: [fix](profile) sort out parquet reader profile
(#58895) (#59789)
add 16b404a48fc branch-4.0: [bug](parquet) fix parquet type not handle
float16 type #58528 (#58630)
add 396c09362fb branch-4.0: [Feature](func) Support function
QUANTILE_STATE_TO/FROM_BASE64 #59664 (#59746)
add 93aa5f4a549 branch-4.0: [Exec](profile) add max row count one backend
in materialization profile #59728 (#59758)
add ba970b560b1 branch-4.0: [opt](memory) lazily create column writer when
flush memtable for partial update (#59772)
add e773e767127 branch-4.0: [Fix](function) fix skip null value for some
date functions #59616 (#59694)
add d285daf9b66 branch-4.0: [fix](cloud) Txn lazy committer update the txn
commit op log min_versionstamp #59448 (#59711)
add a1ceaf027d3 branch-4.0: [fix](recycler) Add recycle state for rs meta
to avoid data loss #58459 (#59765)
add 0e0bbae04e0 branch-4.0: [fix](cloud) Fix brpc::Server::ClearServices()
SEGV #59717 (#59803)
add 98adabe98f7 branch-4.0: [Fix](Streamingjob) fix show task error info
when task timeout #59784 (#59807)
add 2eac5174c94 branch-4.0: [refact](inverted index) use inverted index
context for query #58981 (#59595)
add 8f2348bc7eb branch-4.0: [chore](tools) Update TPC-DS DDLs #59756
(#59802)
add 3cd69346d3a [fix](set) Fix correctness caused by set operation
(#59589) (#59801)
add 2852d845490 branch-4.0: [fix](inverted index) fix BM25 LENGTH_TABLE
using byte4_to_int for correct norm decoding #59713 (#59812)
add e8975b441cc branch-4.0: [Chore](Notice)Update Notice year to 2026
#59822 (#59824)
add da6a88722d9 branch-4.0: [behaviorchange](segmentcache) disable segment
cache prune default #59670 (#59825)
add 3347892cadf branch-4.0: [refine](type) Ensure that the internal
encoding types in Doris are trivially copyable. #59736 (#59805)
add 31f0d7b86f3 branch-4.0: [add](test)Add a test case for parquet reader
lazy read struct column. #59777 (#59811)
add d41fe6b2b39 branch-4.0: [Fix](Streaming) fix get remote meta failed to
pause streaming job #59760 (#59808)
add 9efe793ad3e branch-4.0: [opt](varbinary) add conf control mapping
iceberg uuid (#59726) (#59764)
add f988f65f45c branch-4.0: [chore](cloud) Change enable_packed_file
default to false and add fuzzy config #59773 (#59817)
add 0174967dfb9 [branch-4.0]Pick predicates-related PR (#59804)
add a42b16a6a1a branch-4.0: [fix](auth)fix some ldap bug #58541 (#59826)
add d53be9fef84 branch-4.0: [fix](parquet) Fix struct column reading error
when all queried fields are missing after schema evolution #59586 (#59839)
add d2023938ae1 branch-4.0: [fix](cloud) Fix memory leak in
CloudTxnDeleteBitmapCache for empty rowsets #59710 (#59819)
add f8aed3b5547 branch-4.0: [Fix](StreamingJob) fix postgres consumer data
in multi backend #59798 (#59841)
add fef71d4cb05 branch-4.0: [feature](search) introduce lucene bool mode
for search function #59394 (#59745)
add 7a50c7dc0aa branch-4.0: [fix](regression-case) Fix case
test_s3_load_with_where #59744 (#59829)
add 170baba9e4f branch-4.0: [enhancement](cloud) improve FE RPC retry and
MetaService connection handling #59698 (#59849)
add 194de6a6791 branch-4.0: [feat](hms)Enabling Kerberos authentication
for HMS automatically enables the SASL transport #59783 (#59842)
add ff175d9eeab branch-4.0: [fix](variant) variant type rely on light
schema change #59712 (#59855)
add d556b6e2ee4 branch-4.0: [Fix](StreamingJob) fix create table issues
when create streaming job #59828 (#59853)
add 43e309e9fc5 branch-4.0: [feature](posexplode) support multi args
#58990 (#59875)
add 555e561574c branch-4.0: [fix](hudi) Disable nested column pruning for
HUDI tables to avoid potential issues (#59860) (#59869)
add 443e00ad155 [fix](fe) add compatible code for CloudTablet (#59815)
add 5a76dbf2332 [fix](profile) Fix the issues of too many writer profile
results (#59863)
add 19414edeff7 branch-4.0: [fix](inverted index) fix incorrect supported
types output in tokenizer error message #59666 (#59865)
add 53749edd244 branch-4.0: [Fix](Streamingjob) fix streaming job auto
resume clear error #59868 (#59894)
add 076c08b67f4 [refine](expr) Use the new execute interface of expr in
some places. … (#59879)
add 16e4ec61991 branch-4.0: [fix](local shuffle) fix set operation compute
wrong data because not set AggregationNode.isColocate #59840 (#59882)
add 12e0e1ec046 branch-4.0: [Fix](StreamingJob) fix the first split task
scheduled and fe restart remainsplit relay problem #59883 (#59902)
add e5786112c48 branch-4.0: [fix](project push down) union and its
children's output are mis order #58765 (#59886)
add b084f3b6716 branch-4.0: [chore](tools) split TPC-DS' multi sql in one
file #58691 (#59913)
add 13b157ce286 branch-4.0: [Fix](partition) fix concurrent visit of
partition items #59848 (#59891)
add 173bb2f8731 [regression](hudi) Impl new Hudi Docker environment
(#59401)
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (45a05547f8f)
\
N -- N -- N refs/heads/auto-pick-59401-branch-4.0 (173bb2f8731)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
NOTICE.txt | 2 +-
.../cloud/cloud_engine_calc_delete_bitmap_task.cpp | 19 +-
be/src/cloud/cloud_rowset_builder.cpp | 9 +
be/src/cloud/cloud_rowset_writer.cpp | 1 +
be/src/cloud/cloud_schema_change_job.cpp | 1 +
be/src/cloud/cloud_tablet_mgr.cpp | 4 +-
be/src/cloud/cloud_txn_delete_bitmap_cache.cpp | 43 +-
be/src/cloud/cloud_txn_delete_bitmap_cache.h | 13 +
be/src/cloud/config.cpp | 4 +-
be/src/cloud/pb_convert.cpp | 24 +
be/src/common/config.cpp | 4 +-
be/src/exec/olap_common.h | 262 +--
be/src/exec/olap_utils.h | 53 +-
be/src/exprs/bitmapfilter_predicate.h | 4 +-
be/src/exprs/create_predicate_function.h | 45 +-
be/src/io/fs/buffered_reader.cpp | 3 -
be/src/io/fs/buffered_reader.h | 18 -
be/src/io/fs/tracing_file_reader.h | 1 +
be/src/olap/accept_null_predicate.h | 30 +-
be/src/olap/bitmap_filter_predicate.h | 25 +-
be/src/olap/block_column_predicate.h | 15 +-
be/src/olap/bloom_filter_predicate.h | 23 +-
be/src/olap/collection_statistics.cpp | 59 +-
be/src/olap/column_predicate.h | 83 +-
be/src/olap/compaction.cpp | 1 +
be/src/olap/comparison_predicate.h | 100 +-
be/src/olap/delete_handler.cpp | 488 +++++-
be/src/olap/delete_handler.h | 21 +-
be/src/olap/delta_writer_v2.cpp | 8 +-
be/src/olap/in_list_predicate.h | 257 ++-
be/src/olap/inverted_index_parser.h | 28 +-
be/src/olap/iterators.h | 5 +-
be/src/olap/like_column_predicate.cpp | 4 +-
be/src/olap/like_column_predicate.h | 31 +-
be/src/olap/null_predicate.cpp | 5 +-
be/src/olap/null_predicate.h | 25 +-
be/src/olap/page_cache.h | 11 +-
be/src/olap/predicate_creator.cpp | 135 ++
be/src/olap/predicate_creator.h | 471 +++---
be/src/olap/push_handler.cpp | 8 +-
be/src/olap/push_handler.h | 1 +
be/src/olap/rowset/rowset_meta.h | 4 +
be/src/olap/rowset/rowset_reader_context.h | 4 +-
be/src/olap/rowset/rowset_writer_context.h | 2 +
be/src/olap/rowset/segment_v2/bloom_filter.h | 9 +-
be/src/olap/rowset/segment_v2/column_reader.cpp | 14 +-
be/src/olap/rowset/segment_v2/column_reader.h | 28 +-
.../inverted_index/analyzer/analyzer.cpp | 57 +-
.../segment_v2/inverted_index/analyzer/analyzer.h | 4 +-
.../inverted_index/query_v2/all_query/all_query.h | 112 ++
.../query_v2/boolean_query/boolean_query_builder.h | 84 +
.../query_v2/{operator.h => boolean_query/occur.h} | 2 +-
.../query_v2/boolean_query/occur_boolean_query.h | 75 +
.../boolean_query/occur_boolean_weight.cpp | 287 ++++
.../query_v2/boolean_query/occur_boolean_weight.h | 88 +
.../query_v2/{ => boolean_query}/operator.h | 0
.../{boolean_query.h => operator_boolean_query.h} | 41 +-
...{boolean_weight.h => operator_boolean_weight.h} | 12 +-
.../inverted_index/query_v2/disjunction_scorer.cpp | 129 ++
.../inverted_index/query_v2/disjunction_scorer.h | 66 +
.../segment_v2/inverted_index/query_v2/doc_set.h | 4 +
.../inverted_index/query_v2/exclude_scorer.cpp | 86 +
.../{union/simple_union.h => exclude_scorer.h} | 42 +-
.../inverted_index/query_v2/intersection.cpp | 206 ++-
.../inverted_index/query_v2/intersection.h | 20 +-
.../query_v2/phrase_query/multi_phrase_weight.h | 3 +-
.../query_v2/phrase_query/phrase_scorer.cpp | 16 +-
.../query_v2/phrase_query/phrase_scorer.h | 7 +-
.../query_v2/phrase_query/phrase_weight.h | 6 +-
.../query_v2/postings/loaded_postings.cpp | 3 +-
.../query_v2/regexp_query/regexp_weight.cpp | 2 +-
.../inverted_index/query_v2/reqopt_scorer.h | 80 +
.../inverted_index/query_v2/segment_postings.h | 195 ++-
.../segment_v2/inverted_index/query_v2/size_hint.h | 48 +
.../query_v2/term_query/term_scorer.h | 5 +-
.../query_v2/term_query/term_weight.h | 23 +-
.../query_v2/union/buffered_union.cpp | 259 +++
.../inverted_index/query_v2/union/buffered_union.h | 65 +
.../inverted_index/query_v2/union/simple_union.cpp | 2 +-
.../inverted_index/query_v2/union/simple_union.h | 5 +
.../segment_v2/inverted_index/query_v2/weight.h | 20 +-
.../inverted_index/similarity/bm25_similarity.cpp | 2 +-
.../segment_v2/inverted_index/util/tiny_set.h | 34 +-
.../olap/rowset/segment_v2/inverted_index_cache.h | 17 +-
.../rowset/segment_v2/inverted_index_iterator.cpp | 3 +-
.../rowset/segment_v2/inverted_index_iterator.h | 6 +
.../rowset/segment_v2/inverted_index_reader.cpp | 21 +-
.../olap/rowset/segment_v2/inverted_index_reader.h | 12 +-
.../rowset/segment_v2/inverted_index_writer.cpp | 30 +-
.../olap/rowset/segment_v2/inverted_index_writer.h | 4 +-
be/src/olap/rowset/segment_v2/segment.cpp | 35 +-
be/src/olap/rowset/segment_v2/segment.h | 1 +
be/src/olap/rowset/segment_v2/segment_iterator.cpp | 73 +-
be/src/olap/rowset/segment_v2/segment_iterator.h | 22 +-
.../rowset/segment_v2/vertical_segment_writer.cpp | 34 +-
be/src/olap/schema_cache.h | 4 +-
be/src/olap/segment_loader.h | 9 +-
be/src/olap/shared_predicate.h | 70 +-
be/src/olap/storage_engine.h | 4 +-
be/src/olap/tablet_column_object_pool.h | 5 +-
be/src/olap/tablet_meta.cpp | 4 +-
be/src/olap/tablet_reader.cpp | 118 +-
be/src/olap/tablet_reader.h | 26 +-
be/src/olap/tablet_schema_cache.h | 5 +-
be/src/olap/txn_manager.h | 6 +-
be/src/pipeline/exec/analytic_sink_operator.cpp | 7 +-
be/src/pipeline/exec/dict_sink_operator.cpp | 18 +-
be/src/pipeline/exec/file_scan_operator.cpp | 25 +-
be/src/pipeline/exec/file_scan_operator.h | 24 +-
be/src/pipeline/exec/materialization_opertor.cpp | 13 +
be/src/pipeline/exec/materialization_opertor.h | 8 +
be/src/pipeline/exec/mock_scan_operator.h | 44 +-
be/src/pipeline/exec/olap_scan_operator.cpp | 125 +-
be/src/pipeline/exec/olap_scan_operator.h | 38 +-
be/src/pipeline/exec/operator.cpp | 12 +-
.../pipeline/exec/partition_sort_sink_operator.cpp | 13 +-
.../pipeline/exec/partition_sort_sink_operator.h | 1 -
be/src/pipeline/exec/repeat_operator.cpp | 12 +-
be/src/pipeline/exec/scan_operator.cpp | 1082 ++++++-------
be/src/pipeline/exec/scan_operator.h | 149 +-
be/src/pipeline/exec/set_probe_sink_operator.h | 2 +
be/src/pipeline/exec/set_sink_operator.h | 2 +
be/src/pipeline/exec/union_sink_operator.h | 7 +-
be/src/pipeline/pipeline_fragment_context.cpp | 14 +-
be/src/pipeline/pipeline_fragment_context.h | 3 +-
be/src/pipeline/query_cache/query_cache.h | 4 +-
be/src/runtime/fragment_mgr.cpp | 11 +
be/src/runtime/fragment_mgr.h | 4 +-
be/src/runtime/load_channel_mgr.h | 5 +-
be/src/runtime/memory/lru_cache_policy.h | 8 +-
be/src/runtime/query_context.cpp | 9 +-
be/src/runtime/runtime_predicate.cpp | 194 +--
be/src/runtime/runtime_predicate.h | 18 +-
.../runtime/stream_load/stream_load_executor.cpp | 23 +-
.../runtime_filter_consumer_helper.cpp | 11 +
.../runtime_filter_consumer_helper.h | 4 +
be/src/runtime_filter/runtime_filter_mgr.cpp | 24 -
be/src/runtime_filter/runtime_filter_mgr.h | 5 +-
be/src/service/point_query_executor.cpp | 3 +-
be/src/service/point_query_executor.h | 5 +-
be/src/util/hash_util.hpp | 10 +
be/src/util/obj_lru_cache.cpp | 3 +-
be/src/util/thrift_rpc_helper.cpp | 52 +-
be/src/util/thrift_rpc_helper.h | 11 +
.../aggregate_function_sequence_match.h | 5 +-
.../aggregate_function_window_funnel.h | 1 -
be/src/vec/common/pod_array.h | 5 +
be/src/vec/common/sort/sorter.cpp | 12 +-
be/src/vec/core/types.h | 6 +
be/src/vec/exec/format/avro/avro_jni_reader.cpp | 6 +-
be/src/vec/exec/format/avro/avro_jni_reader.h | 4 +-
be/src/vec/exec/format/generic_reader.cpp | 252 ---
be/src/vec/exec/format/generic_reader.h | 20 -
be/src/vec/exec/format/jni_reader.cpp | 6 +-
be/src/vec/exec/format/jni_reader.h | 6 +-
be/src/vec/exec/format/json/new_json_reader.cpp | 15 +-
be/src/vec/exec/format/orc/vorc_reader.cpp | 12 +-
.../parquet/parquet_block_split_bloom_filter.cpp | 134 ++
.../parquet/parquet_block_split_bloom_filter.h | 82 +
.../exec/format/parquet/parquet_column_convert.cpp | 5 +
.../exec/format/parquet/parquet_column_convert.h | 84 +
.../{parquet_pred_cmp.h => parquet_predicate.h} | 76 +-
be/src/vec/exec/format/parquet/schema_desc.cpp | 10 +-
.../parquet/vparquet_column_chunk_reader.cpp | 14 +-
.../format/parquet/vparquet_column_chunk_reader.h | 17 +-
.../exec/format/parquet/vparquet_column_reader.cpp | 18 +-
.../exec/format/parquet/vparquet_column_reader.h | 82 +-
.../exec/format/parquet/vparquet_group_reader.cpp | 18 +-
.../exec/format/parquet/vparquet_group_reader.h | 4 +-
.../exec/format/parquet/vparquet_page_index.cpp | 2 +-
.../exec/format/parquet/vparquet_page_reader.cpp | 9 +-
.../vec/exec/format/parquet/vparquet_page_reader.h | 9 +-
be/src/vec/exec/format/parquet/vparquet_reader.cpp | 373 +++--
be/src/vec/exec/format/parquet/vparquet_reader.h | 48 +-
be/src/vec/exec/format/table/hive_reader.cpp | 10 +-
be/src/vec/exec/format/table/hive_reader.h | 6 +-
be/src/vec/exec/format/table/hudi_jni_reader.cpp | 6 +-
be/src/vec/exec/format/table/hudi_jni_reader.h | 4 +-
be/src/vec/exec/format/table/hudi_reader.cpp | 10 +-
be/src/vec/exec/format/table/hudi_reader.h | 6 +-
be/src/vec/exec/format/table/iceberg_reader.cpp | 20 +-
be/src/vec/exec/format/table/iceberg_reader.h | 6 +-
.../format/table/iceberg_sys_table_jni_reader.cpp | 5 +-
.../format/table/iceberg_sys_table_jni_reader.h | 3 +-
.../vec/exec/format/table/lakesoul_jni_reader.cpp | 5 +-
be/src/vec/exec/format/table/lakesoul_jni_reader.h | 3 +-
.../exec/format/table/max_compute_jni_reader.cpp | 6 +-
.../vec/exec/format/table/max_compute_jni_reader.h | 4 +-
be/src/vec/exec/format/table/paimon_jni_reader.cpp | 6 +-
be/src/vec/exec/format/table/paimon_jni_reader.h | 4 +-
be/src/vec/exec/format/table/paimon_reader.h | 10 +-
.../format/table/paimon_sys_table_jni_reader.cpp | 6 +-
.../format/table/paimon_sys_table_jni_reader.h | 4 +-
.../format/table/trino_connector_jni_reader.cpp | 5 +-
.../exec/format/table/trino_connector_jni_reader.h | 3 +-
be/src/vec/exec/jni_connector.cpp | 25 +-
be/src/vec/exec/jni_connector.h | 6 +-
be/src/vec/exec/scan/file_scanner.cpp | 109 +-
be/src/vec/exec/scan/file_scanner.h | 6 +-
be/src/vec/exec/scan/meta_scanner.cpp | 4 +-
be/src/vec/exec/scan/olap_scanner.cpp | 46 +-
be/src/vec/exec/scan/olap_scanner.h | 10 +-
be/src/vec/exec/scan/scanner.cpp | 3 +-
.../vec/exprs/table_function/vexplode_bitmap.cpp | 6 +-
.../exprs/table_function/vexplode_json_object.cpp | 6 +-
be/src/vec/exprs/vexpr.cpp | 9 +-
be/src/vec/exprs/vexpr.h | 3 +-
be/src/vec/exprs/vexpr_context.cpp | 12 +
be/src/vec/exprs/vexpr_context.h | 23 +-
be/src/vec/exprs/vmatch_predicate.cpp | 36 +-
be/src/vec/exprs/vmatch_predicate.h | 6 +-
be/src/vec/exprs/vruntimefilter_wrapper.h | 1 +
be/src/vec/exprs/vtopn_pred.h | 3 +
be/src/vec/functions/array/function_array_index.h | 4 +-
.../vec/functions/array/function_arrays_overlap.h | 5 +-
be/src/vec/functions/function.h | 8 +-
.../function_date_or_datetime_computation.cpp | 8 +-
.../function_date_or_datetime_computation.h | 148 +-
.../function_date_or_datetime_to_string.cpp | 49 +-
be/src/vec/functions/function_ip.h | 1 +
be/src/vec/functions/function_multi_match.cpp | 4 +-
be/src/vec/functions/function_multi_match.h | 1 +
.../vec/functions/function_needs_to_handle_null.h | 6 +
be/src/vec/functions/function_quantile_state.cpp | 127 ++
be/src/vec/functions/function_search.cpp | 80 +-
be/src/vec/functions/function_search.h | 3 +-
be/src/vec/functions/function_tokenize.cpp | 48 +-
be/src/vec/functions/function_tokenize.h | 4 +-
be/src/vec/functions/functions_comparison.h | 4 +-
be/src/vec/functions/in.h | 6 +-
be/src/vec/functions/is_not_null.h | 1 +
be/src/vec/functions/is_null.h | 1 +
be/src/vec/functions/match.cpp | 128 +-
be/src/vec/functions/match.h | 25 +-
be/src/vec/olap/olap_data_convertor.cpp | 8 +-
be/src/vec/olap/olap_data_convertor.h | 2 +
be/src/vec/runtime/timestamptz_value.cpp | 2 -
be/src/vec/runtime/vdatetime_value.h | 13 +
be/test/olap/block_column_predicate_test.cpp | 515 ++++--
be/test/olap/date_bloom_filter_test.cpp | 124 +-
be/test/olap/delete_handler_test.cpp | 46 +-
be/test/olap/inverted_index_parser_test.cpp | 34 +-
be/test/olap/lru_cache_test.cpp | 4 +-
.../rowset/segment_v2/index_reader_helper_test.cpp | 3 +-
.../inverted_index/ananlyzer/analyzer_test.cpp | 126 +-
.../boolean_query/boolean_query_builder_test.cpp | 387 +++++
.../inverted_index/query_v2/boolean_query_test.cpp | 31 +-
.../query_v2/buffered_union_test.cpp | 684 ++++++++
.../query_v2/disjunction_scorer_test.cpp | 389 +++++
.../query_v2/exclude_scorer_test.cpp | 569 +++++++
.../inverted_index/query_v2/intersection_test.cpp | 94 +-
.../query_v2/occur_boolean_query_test.cpp | 707 ++++++++
.../inverted_index/query_v2/reqopt_scorer_test.cpp | 540 +++++++
.../query_v2/segment_postings_test.cpp | 226 +--
.../similarity/bm25_similarity_test.cpp | 28 +-
.../segment_v2/inverted_index/util/reader_test.cpp | 20 +-
be/test/olap/wal/wal_manager_test.cpp | 4 +-
.../operator/scan_normalize_predicate_test.cpp | 416 ++---
be/test/testutil/mock/mock_in_expr.h | 2 +-
be/test/vec/columns/pod_array_type_test.cpp | 66 +
.../vec/exec/format/parquet/parquet_expr_test.cpp | 813 +++++++---
.../vec/exec/format/parquet/parquet_read_lines.cpp | 3 +-
.../exec/format/parquet/parquet_reader_test.cpp | 23 +-
.../format/parquet/parquet_statistics_test.cpp | 2 +-
.../exec/format/table/hive/hive_reader_test.cpp | 3 +-
.../format/table/iceberg/iceberg_reader_test.cpp | 3 +-
be/test/vec/exec/vfile_scanner_exception_test.cpp | 4 +-
be/test/vec/function/function_ip_test.cpp | 7 +-
be/test/vec/function/function_is_null_test.cpp | 16 +-
be/test/vec/function/function_match_test.cpp | 19 +-
be/test/vec/function/function_multi_match_test.cpp | 2 +-
.../vec/function/function_quantile_state_test.cpp | 216 +++
be/test/vec/function/function_search_test.cpp | 445 +++++-
be/test/vec/function/function_test_util.cpp | 11 +
cloud/src/common/config.h | 4 +
cloud/src/meta-service/meta_service.cpp | 94 +-
cloud/src/meta-service/meta_service.h | 12 +
cloud/src/meta-service/meta_service_job.cpp | 87 +-
cloud/src/meta-service/meta_service_txn.cpp | 21 +-
cloud/src/meta-service/txn_lazy_committer.cpp | 35 +
cloud/src/recycler/recycler.cpp | 460 ++++--
cloud/src/recycler/recycler.h | 31 +
cloud/test/meta_server_test.cpp | 2 +
cloud/test/meta_service_job_test.cpp | 1378 +++++++++++++++-
cloud/test/meta_service_test.cpp | 72 +-
cloud/test/recycler_test.cpp | 7 +-
cloud/test/rpc_kv_bvar_test.cpp | 30 +-
cloud/test/txn_lazy_commit_test.cpp | 425 +----
contrib/clucene | 2 +-
dist/NOTICE-dist.txt | 2 +-
.../scripts/create_preinstalled_scripts/run85.hql | 15 +
.../parquet_bloom_filter/create_table.hql | 39 +
.../multi_catalog/parquet_bloom_filter/data.tar.gz | Bin 0 -> 2735 bytes
.../parquet_bloom_filter/generate_data.script | 44 +
.../run.sh | 0
.../test_parquet_lazy_read_struct/example.parquet | Bin 0 -> 729024 bytes
.../create_preinstalled_scripts/iceberg/run24.sql | 151 ++
.../main/java/org/apache/doris/common/Config.java | 2 +-
.../antlr4/org/apache/doris/nereids/DorisParser.g4 | 2 +-
.../apache/doris/alter/SchemaChangeHandler.java | 11 +
.../org/apache/doris/analysis/SearchPredicate.java | 24 +
.../doris/catalog/BuiltinScalarFunctions.java | 4 +
.../catalog/BuiltinTableGeneratingFunctions.java | 3 +-
.../java/org/apache/doris/catalog/OlapTable.java | 41 +-
.../org/apache/doris/catalog/PartitionInfo.java | 4 +-
.../doris/clone/DynamicPartitionScheduler.java | 19 +-
.../apache/doris/cloud/catalog/CloudTablet.java | 26 +-
.../apache/doris/cloud/rpc/MetaServiceProxy.java | 35 +-
.../property/metastore/HMSBaseProperties.java | 2 +
.../property/storage/HdfsProperties.java | 1 +
.../doris/httpv2/rest/StreamingJobAction.java | 3 +
.../apache/doris/indexpolicy/IndexPolicyMgr.java | 2 +-
.../org/apache/doris/job/common/FailureReason.java | 7 +-
.../insert/streaming/StreamingInsertJob.java | 26 +-
.../streaming/StreamingJobSchedulerTask.java | 1 -
.../insert/streaming/StreamingMultiTblTask.java | 49 +-
.../job/offset/jdbc/JdbcSourceOffsetProvider.java | 97 +-
.../apache/doris/job/util/StreamingJobUtils.java | 46 +-
.../doris/mysql/authenticate/ldap/LdapClient.java | 16 +-
.../mysql/authenticate/ldap/LdapUserInfo.java | 3 +-
.../org/apache/doris/mysql/privilege/Auth.java | 16 +-
.../glue/translator/PhysicalPlanTranslator.java | 12 +-
.../nereids/rules/rewrite/MergeAggregate.java | 1 +
.../nereids/rules/rewrite/PushDownProject.java | 168 +-
.../functions/generator/PosExplode.java | 56 +-
.../functions/generator/PosExplodeOuter.java | 55 +-
.../functions/scalar/QuantileStateFromBase64.java | 71 +
.../functions/scalar/QuantileStateToBase64.java | 69 +
.../trees/expressions/functions/scalar/Search.java | 60 +-
.../functions/scalar/SearchDslParser.java | 761 ++++++++-
.../expressions/visitor/ScalarFunctionVisitor.java | 10 +
.../plans/commands/refresh/RefreshLdapCommand.java | 10 +-
.../trees/plans/logical/LogicalFileScan.java | 6 +
.../org/apache/doris/persist/gson/GsonUtils.java | 4 +-
.../java/org/apache/doris/qe/SessionVariable.java | 17 +
.../doris/alter/SchemaChangeHandlerTest.java | 41 +
.../org/apache/doris/catalog/OlapTableTest.java | 122 ++
.../doris/cloud/rpc/MetaServiceProxyTest.java | 127 ++
.../property/storage/HdfsPropertiesTest.java | 6 +-
.../doris/job/util/StreamingJobUtilsTest.java | 240 +++
.../mysql/authenticate/ldap/LdapUserInfoTest.java | 14 +-
.../org/apache/doris/mysql/privilege/AuthTest.java | 25 +-
.../nereids/rules/rewrite/MergeAggregateTest.java | 124 ++
.../nereids/rules/rewrite/PushDownProjectTest.java | 192 +++
.../functions/generator/PosExplodeOuterTest.java | 73 +
.../functions/generator/PosExplodeTest.java | 73 +
.../functions/scalar/SearchDslParserTest.java | 348 ++++
.../cdcclient/controller/ClientController.java | 21 +
.../cdcclient/service/PipelineCoordinator.java | 72 +-
.../doris/cdcclient/sink/DorisBatchStreamLoad.java | 13 +
.../source/reader/JdbcIncrementalSourceReader.java | 10 +-
.../source/reader/mysql/MySqlSourceReader.java | 7 +-
.../reader/postgres/PostgresSourceReader.java | 8 +-
gensrc/proto/olap_file.proto | 6 +
gensrc/proto/segment_v2.proto | 1 +
gensrc/thrift/Exprs.thrift | 14 +-
gensrc/thrift/PaloInternalService.thrift | 3 +
gensrc/thrift/parquet.thrift | 526 ++++--
.../hive/test_parquet_bloom_filter.out | 123 ++
.../hive/test_parquet_nested_types.out | 42 +
.../test_iceberg_struct_schema_evolution.out | 161 ++
.../tvf/test_hdfs_parquet_group0.out | 38 +-
.../tvf/test_hdfs_parquet_group6.out | 20 +-
.../tvf/test_hdfs_tvf_float16.out | 19 +
.../paimon/test_paimon_hms_catalog.out | 12 +
.../test_index_lowercase_fault_injection.out | 2 +-
.../streaming_job/cdc/test_streaming_mysql_job.out | 2 +
.../cdc/test_streaming_mysql_job_all_type.out | 2 +-
.../{Map.out => nereids_scalar_fn_map.out} | 8 +
.../data/nereids_p0/join/test_outer_join.out | 5 +
.../sql_functions/table_function/posexplode.out | 159 +-
.../test_quantile_state_function.out | 37 +
regression-test/data/search/test_search_escape.out | 46 +
.../data/search/test_search_lucene_mode.out | 86 +
.../suites/auth_p0/test_refresh_ldap_auth.groovy | 56 +
.../hive/test_parquet_bloom_filter.groovy | 60 +
.../hive/test_parquet_nested_types.groovy | 10 +
.../test_iceberg_struct_schema_evolution.groovy | 194 +++
.../test_information_schema_timezone.groovy | 1 -
.../kerberos/test_single_hive_kerberos.groovy | 1 -
.../kerberos/test_two_hive_kerberos.groovy | 2 -
.../tvf/test_hdfs_parquet_group6.groovy | 15 +
.../tvf/test_hdfs_tvf_float16.groovy | 46 +
.../paimon/test_paimon_hms_catalog.groovy | 11 +-
.../hive_on_hms_and_dlf.groovy | 10 +-
.../iceberg_on_hms_and_filesystem_and_dlf.groovy | 15 +-
...test_index_policy_unsupported_type_error.groovy | 28 +-
.../cdc/test_streaming_mysql_job.groovy | 17 +-
.../cdc/test_streaming_mysql_job_errormsg.groovy | 111 ++
.../cdc/test_streaming_mysql_job_priv.groovy | 44 +-
.../cdc/test_streaming_postgres_job_priv.groovy | 52 +-
.../broker_load/test_s3_load_with_where.groovy | 2 +-
.../{Map.groovy => nereids_scalar_fn_map.groovy} | 21 +
.../suites/nereids_p0/join/test_outer_join.groovy | 29 +-
.../test_date_function_v2.groovy | 4 +-
.../sql_functions/table_function/posexplode.groovy | 49 +
.../test_quantile_state_function.groovy | 82 +
.../query_profile/test_writer_profile.groovy | 194 +++
.../test_search_default_field_operator.groovy | 48 +-
.../suites/search/test_search_escape.groovy | 189 +++
.../test_search_inverted_is_null_pushdown.groovy | 74 +
.../suites/search/test_search_lucene_mode.groovy | 250 +++
tools/fdb/fdb_ctl.sh | 3 +-
tools/tpcds-tools/bin/run-tpcds-queries.sh | 51 +-
tools/tpcds-tools/ddl/create-tpcds-tables-sf1.sql | 1157 +++++++-------
.../tpcds-tools/ddl/create-tpcds-tables-sf100.sql | 1171 +++++++-------
.../tpcds-tools/ddl/create-tpcds-tables-sf1000.sql | 1682 +++++++------------
.../ddl/create-tpcds-tables-sf10000.sql | 1688 +++++++-------------
tools/tpcds-tools/queries/sf1/query14.sql | 106 --
.../{sf1000/query14.sql => sf1/query14_1.sql} | 101 --
tools/tpcds-tools/queries/sf1/query23.sql | 55 -
.../{sf1000/query23.sql => sf1/query23_1.sql} | 49 -
tools/tpcds-tools/queries/sf1/query24.sql | 52 -
tools/tpcds-tools/queries/sf1/query24_1.sql | 52 +
tools/tpcds-tools/queries/sf1/query39.sql | 26 -
.../queries/sf1/{query39.sql => query39_1.sql} | 25 -
tools/tpcds-tools/queries/sf100/query14.sql | 106 --
.../queries/sf100/{query14.sql => query14_1.sql} | 101 --
tools/tpcds-tools/queries/sf100/query23.sql | 55 -
.../queries/sf100/{query23.sql => query23_1.sql} | 49 -
tools/tpcds-tools/queries/sf100/query24.sql | 52 -
tools/tpcds-tools/queries/sf100/query24_1.sql | 52 +
tools/tpcds-tools/queries/sf100/query39.sql | 26 -
.../queries/sf100/{query39.sql => query39_1.sql} | 24 -
tools/tpcds-tools/queries/sf1000/query14.sql | 106 --
.../queries/sf1000/{query14.sql => query14_1.sql} | 101 --
tools/tpcds-tools/queries/sf1000/query23.sql | 55 -
.../queries/sf1000/{query23.sql => query23_1.sql} | 49 -
tools/tpcds-tools/queries/sf1000/query24.sql | 52 -
tools/tpcds-tools/queries/sf1000/query24_1.sql | 52 +
tools/tpcds-tools/queries/sf1000/query39.sql | 25 -
.../queries/sf1000/{query39.sql => query39_1.sql} | 24 -
tools/tpcds-tools/queries/sf10000/query14.sql | 106 --
.../queries/sf10000/{query14.sql => query14_1.sql} | 101 --
tools/tpcds-tools/queries/sf10000/query23.sql | 55 -
.../queries/sf10000/{query23.sql => query23_1.sql} | 49 -
tools/tpcds-tools/queries/sf10000/query24.sql | 52 -
.../queries/sf10000/{query24.sql => query24_1.sql} | 52 -
tools/tpcds-tools/queries/sf10000/query39.sql | 26 -
.../queries/sf10000/{query39.sql => query39_1.sql} | 25 -
440 files changed, 22084 insertions(+), 10679 deletions(-)
create mode 100644 be/src/olap/predicate_creator.cpp
create mode 100644
be/src/olap/rowset/segment_v2/inverted_index/query_v2/all_query/all_query.h
create mode 100644
be/src/olap/rowset/segment_v2/inverted_index/query_v2/boolean_query/boolean_query_builder.h
copy be/src/olap/rowset/segment_v2/inverted_index/query_v2/{operator.h =>
boolean_query/occur.h} (87%)
create mode 100644
be/src/olap/rowset/segment_v2/inverted_index/query_v2/boolean_query/occur_boolean_query.h
create mode 100644
be/src/olap/rowset/segment_v2/inverted_index/query_v2/boolean_query/occur_boolean_weight.cpp
create mode 100644
be/src/olap/rowset/segment_v2/inverted_index/query_v2/boolean_query/occur_boolean_weight.h
copy be/src/olap/rowset/segment_v2/inverted_index/query_v2/{ =>
boolean_query}/operator.h (100%)
rename
be/src/olap/rowset/segment_v2/inverted_index/query_v2/boolean_query/{boolean_query.h
=> operator_boolean_query.h} (62%)
rename
be/src/olap/rowset/segment_v2/inverted_index/query_v2/boolean_query/{boolean_weight.h
=> operator_boolean_weight.h} (95%)
create mode 100644
be/src/olap/rowset/segment_v2/inverted_index/query_v2/disjunction_scorer.cpp
create mode 100644
be/src/olap/rowset/segment_v2/inverted_index/query_v2/disjunction_scorer.h
create mode 100644
be/src/olap/rowset/segment_v2/inverted_index/query_v2/exclude_scorer.cpp
copy
be/src/olap/rowset/segment_v2/inverted_index/query_v2/{union/simple_union.h =>
exclude_scorer.h} (53%)
create mode 100644
be/src/olap/rowset/segment_v2/inverted_index/query_v2/reqopt_scorer.h
create mode 100644
be/src/olap/rowset/segment_v2/inverted_index/query_v2/size_hint.h
create mode 100644
be/src/olap/rowset/segment_v2/inverted_index/query_v2/union/buffered_union.cpp
create mode 100644
be/src/olap/rowset/segment_v2/inverted_index/query_v2/union/buffered_union.h
delete mode 100644 be/src/vec/exec/format/generic_reader.cpp
create mode 100644
be/src/vec/exec/format/parquet/parquet_block_split_bloom_filter.cpp
create mode 100644
be/src/vec/exec/format/parquet/parquet_block_split_bloom_filter.h
rename be/src/vec/exec/format/parquet/{parquet_pred_cmp.h =>
parquet_predicate.h} (84%)
create mode 100644
be/test/olap/rowset/segment_v2/inverted_index/query_v2/boolean_query/boolean_query_builder_test.cpp
create mode 100644
be/test/olap/rowset/segment_v2/inverted_index/query_v2/buffered_union_test.cpp
create mode 100644
be/test/olap/rowset/segment_v2/inverted_index/query_v2/disjunction_scorer_test.cpp
create mode 100644
be/test/olap/rowset/segment_v2/inverted_index/query_v2/exclude_scorer_test.cpp
create mode 100644
be/test/olap/rowset/segment_v2/inverted_index/query_v2/occur_boolean_query_test.cpp
create mode 100644
be/test/olap/rowset/segment_v2/inverted_index/query_v2/reqopt_scorer_test.cpp
create mode 100644 be/test/vec/columns/pod_array_type_test.cpp
create mode 100644 be/test/vec/function/function_quantile_state_test.cpp
create mode 100644
docker/thirdparties/docker-compose/hive/scripts/create_preinstalled_scripts/run85.hql
create mode 100644
docker/thirdparties/docker-compose/hive/scripts/data/multi_catalog/parquet_bloom_filter/create_table.hql
create mode 100644
docker/thirdparties/docker-compose/hive/scripts/data/multi_catalog/parquet_bloom_filter/data.tar.gz
create mode 100644
docker/thirdparties/docker-compose/hive/scripts/data/multi_catalog/parquet_bloom_filter/generate_data.script
copy
docker/thirdparties/docker-compose/hive/scripts/data/multi_catalog/{hive_text_complex_type
=> parquet_bloom_filter}/run.sh (100%)
create mode 100644
docker/thirdparties/docker-compose/hive/scripts/preinstalled_data/parquet_table/test_parquet_lazy_read_struct/example.parquet
create mode 100644
docker/thirdparties/docker-compose/iceberg/scripts/create_preinstalled_scripts/iceberg/run24.sql
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/scalar/QuantileStateFromBase64.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/scalar/QuantileStateToBase64.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/cloud/rpc/MetaServiceProxyTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/job/util/StreamingJobUtilsTest.java
copy be/src/olap/rowset/segment_v2/inverted_index/query_v2/operator.h =>
fe/fe-core/src/test/java/org/apache/doris/mysql/authenticate/ldap/LdapUserInfoTest.java
(73%)
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/nereids/rules/rewrite/MergeAggregateTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/nereids/rules/rewrite/PushDownProjectTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/nereids/trees/expressions/functions/generator/PosExplodeOuterTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/nereids/trees/expressions/functions/generator/PosExplodeTest.java
create mode 100644
regression-test/data/external_table_p0/hive/test_parquet_bloom_filter.out
create mode 100644
regression-test/data/external_table_p0/iceberg/test_iceberg_struct_schema_evolution.out
create mode 100644
regression-test/data/external_table_p0/tvf/test_hdfs_tvf_float16.out
rename regression-test/data/nereids_function_p0/scalar_function/{Map.out =>
nereids_scalar_fn_map.out} (99%)
create mode 100644
regression-test/data/query_p0/sql_functions/quantile_state_functions/test_quantile_state_function.out
create mode 100644 regression-test/data/search/test_search_escape.out
create mode 100644 regression-test/data/search/test_search_lucene_mode.out
create mode 100644 regression-test/suites/auth_p0/test_refresh_ldap_auth.groovy
create mode 100644
regression-test/suites/external_table_p0/hive/test_parquet_bloom_filter.groovy
create mode 100644
regression-test/suites/external_table_p0/iceberg/test_iceberg_struct_schema_evolution.groovy
create mode 100644
regression-test/suites/external_table_p0/tvf/test_hdfs_tvf_float16.groovy
rename be/src/olap/rowset/segment_v2/inverted_index/query_v2/operator.h =>
regression-test/suites/inverted_index_p0/test_index_policy_unsupported_type_error.groovy
(50%)
create mode 100644
regression-test/suites/job_p0/streaming_job/cdc/test_streaming_mysql_job_errormsg.groovy
rename regression-test/suites/nereids_function_p0/scalar_function/{Map.groovy
=> nereids_scalar_fn_map.groovy} (97%)
create mode 100644
regression-test/suites/query_p0/sql_functions/quantile_state_functions/test_quantile_state_function.groovy
create mode 100644
regression-test/suites/query_profile/test_writer_profile.groovy
create mode 100644 regression-test/suites/search/test_search_escape.groovy
create mode 100644
regression-test/suites/search/test_search_inverted_is_null_pushdown.groovy
create mode 100644 regression-test/suites/search/test_search_lucene_mode.groovy
copy tools/tpcds-tools/queries/{sf1000/query14.sql => sf1/query14_1.sql} (52%)
copy tools/tpcds-tools/queries/{sf1000/query23.sql => sf1/query23_1.sql} (54%)
create mode 100644 tools/tpcds-tools/queries/sf1/query24_1.sql
copy tools/tpcds-tools/queries/sf1/{query39.sql => query39_1.sql} (50%)
copy tools/tpcds-tools/queries/sf100/{query14.sql => query14_1.sql} (52%)
copy tools/tpcds-tools/queries/sf100/{query23.sql => query23_1.sql} (54%)
create mode 100644 tools/tpcds-tools/queries/sf100/query24_1.sql
copy tools/tpcds-tools/queries/sf100/{query39.sql => query39_1.sql} (50%)
copy tools/tpcds-tools/queries/sf1000/{query14.sql => query14_1.sql} (52%)
copy tools/tpcds-tools/queries/sf1000/{query23.sql => query23_1.sql} (54%)
create mode 100644 tools/tpcds-tools/queries/sf1000/query24_1.sql
copy tools/tpcds-tools/queries/sf1000/{query39.sql => query39_1.sql} (50%)
copy tools/tpcds-tools/queries/sf10000/{query14.sql => query14_1.sql} (52%)
copy tools/tpcds-tools/queries/sf10000/{query23.sql => query23_1.sql} (54%)
copy tools/tpcds-tools/queries/sf10000/{query24.sql => query24_1.sql} (50%)
copy tools/tpcds-tools/queries/sf10000/{query39.sql => query39_1.sql} (50%)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
