This is an automated email from the ASF dual-hosted git repository.
morningman pushed a change to branch branch-catalog-spi
in repository https://gitbox.apache.org/repos/asf/doris.git
discard 5c240dc7a34 [feat](connector) P3 hudi connector hardening + test
baseline + dispatch design (hybrid, T02-T08) (#64143)
omit 63fb7e9fec7 [feat](connector) P2 migrate trino-connector to catalog
SPI (T01-T13) (#64096)
omit efbbef6be2b [P1-T03-T05] route plugin-driven scans first in nereids
translator (#63641)
omit e5dce49618b [feat](connector) P0 SPI baseline + DDL/Partition + import
gate (T03-T27) (#63582)
omit b921778a648 [doc](connector) add project tracking system for catalog
SPI migration
add 691189aaad6 [fix](fe) Prevent unsafe runtime filter pushdown through
outer joins (#64102)
add 6e37a86a361 [fix](variant) Avoid mutating shared variant columns
(#64092)
add 98a7a7e9548 [improvement](fe) Preload external table metadata before
internal table lock in mixed queries (#64035)
add 2471d75d8f5 [Enhancement](be) Fix small_file_mgr to support HTTPS when
FE runs in HTTPS-only mode (#63918)
add c27fd0ba968 [refactor](be) Add operator IO wrappers (#64139)
add 6ac210bfc3d [improvement](fe) TopN lazy materialization support
struct/variant nested column pruning (#63736)
add dc7b50ce99e [fix](be) Avoid unsigned underflow in JSON modify path
(#63579)
add 3ab5d8b245d [opt](variant) Reduce sparse variant parse memory (#63970)
add e2aaee4d0ed [fix](serde) match STRUCT sub-fields by name when loading
JSON (#64011)
add 98119b95dfb [fix](variant) return raw string for element_at on
scalar-string variant (#64103)
add 20d68fee949 [fix](ann-index) Fix ANN IVF/PQ recall, avoid init-time
large ANN build-buffer reservation, and skip ANN index build for segments with
insufficient rows. (#64082)
add 236f8666029 [optimize](be) optimize floating fmod fast path (#64161)
add 20c58e46d4f [opt](memory) add BE-level cache for load tablet schemas
(#63508)
add 3ccb4b81e6a [Optimization](array) Avoid materializing const array
columns in element_at (#64175)
add 15add0fc4c4 [fix](partition_prune) Avoid incorrect convert_tz
partition pruning across DST transitions (#63853)
add 5d97b29034a [improvement](be) Optimize count on nullable column
(#64166)
add 62dc78fbb00 [fix](logstash) Replace HttpClient5 async with HttpClient4
sync to fix CircularRedirectException (1.2.0 -> 1.2.1) (#63181)
add 2867e9c0d21 [enhance](struct) Merge struct_element into element_at
(#64027)
add 11a038d87d1 [fix](be) Stop extra operator work after cancellation
(#64077)
add 441f28ae76c [fix](cloud) Fix double assignment in recycler_service.cpp
that cause… (#64168)
add ba6a4439eb5 [improvement](fe) Balance runtime filter coordinator
selection (#64130)
add fee3bab76b5 [fix](test) replace removed struct_element with element_at
in regression suites (#64204)
add 4f8ac0954ac [fix](ci) rm some infos (#63228)
add 79226c65326 [fix](fe) Fix datediff folding for zero date (#64084)
add 8255f94bc5f [fix](nereids) Guard UniqueFunction in multiple
filter/topn pushdown rules (#62742)
add f7eac4e7401 [opt](cloud) cache cluster id per query and drop redundant
locks on getBackendId hot path (#63636)
add ab930cd39b7 [chore](build) Update doris-skills submodule (#64240)
add 08b37d52121 [opt](recycler) Optimize rowset recycling for recycler
(#63295)
new f0e14452c20 [doc](connector) add project tracking system for catalog
SPI migration
new 765cb2546dd [feat](connector) P0 SPI baseline + DDL/Partition + import
gate (T03-T27) (#63582)
new 68ba1cce66e [P1-T03-T05] route plugin-driven scans first in nereids
translator (#63641)
new c6d897baf74 [feat](connector) P2 migrate trino-connector to catalog
SPI (T01-T13) (#64096)
new 3dfd72a3de8 [feat](connector) P3 hudi connector hardening + test
baseline + dispatch design (hybrid, T02-T08) (#64143)
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (5c240dc7a34)
\
N -- N -- N refs/heads/branch-catalog-spi (3dfd72a3de8)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 5 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
be/benchmark/benchmark_fmod.hpp | 442 ++++++++
be/benchmark/benchmark_main.cpp | 1 +
be/src/common/config.cpp | 16 +-
be/src/common/config.h | 7 +-
be/src/core/data_type/data_type_struct.cpp | 8 +-
be/src/core/data_type/data_type_variant.cpp | 42 +-
.../data_type_serde/data_type_struct_serde.cpp | 73 +-
be/src/exec/common/variant_util.cpp | 60 +-
.../exec/exchange/local_exchange_sink_operator.cpp | 2 +-
.../exec/exchange/local_exchange_sink_operator.h | 2 +-
.../exchange/local_exchange_source_operator.cpp | 2 +-
.../exec/exchange/local_exchange_source_operator.h | 2 +-
be/src/exec/operator/aggregation_sink_operator.cpp | 2 +-
be/src/exec/operator/aggregation_sink_operator.h | 2 +-
.../exec/operator/aggregation_source_operator.cpp | 2 +-
be/src/exec/operator/aggregation_source_operator.h | 2 +-
be/src/exec/operator/analytic_sink_operator.cpp | 7 +-
be/src/exec/operator/analytic_sink_operator.h | 4 +-
be/src/exec/operator/analytic_source_operator.cpp | 3 +-
be/src/exec/operator/analytic_source_operator.h | 2 +-
be/src/exec/operator/blackhole_sink_operator.cpp | 2 +-
be/src/exec/operator/blackhole_sink_operator.h | 2 +-
.../bucketed_aggregation_sink_operator.cpp | 2 +-
.../operator/bucketed_aggregation_sink_operator.h | 2 +-
.../bucketed_aggregation_source_operator.cpp | 2 +-
.../bucketed_aggregation_source_operator.h | 2 +-
be/src/exec/operator/cache_sink_operator.cpp | 2 +-
be/src/exec/operator/cache_sink_operator.h | 2 +-
be/src/exec/operator/cache_source_operator.cpp | 2 +-
be/src/exec/operator/cache_source_operator.h | 2 +-
be/src/exec/operator/datagen_operator.cpp | 2 +-
be/src/exec/operator/datagen_operator.h | 2 +-
be/src/exec/operator/dict_sink_operator.cpp | 2 +-
be/src/exec/operator/dict_sink_operator.h | 2 +-
be/src/exec/operator/empty_set_operator.cpp | 2 +-
be/src/exec/operator/empty_set_operator.h | 2 +-
be/src/exec/operator/exchange_sink_operator.cpp | 2 +-
be/src/exec/operator/exchange_sink_operator.h | 2 +-
be/src/exec/operator/exchange_source_operator.cpp | 2 +-
be/src/exec/operator/exchange_source_operator.h | 2 +-
.../operator/group_commit_block_sink_operator.cpp | 2 +-
.../operator/group_commit_block_sink_operator.h | 2 +-
.../exec/operator/group_commit_scan_operator.cpp | 2 +-
be/src/exec/operator/group_commit_scan_operator.h | 2 +-
be/src/exec/operator/hashjoin_build_sink.cpp | 2 +-
be/src/exec/operator/hashjoin_build_sink.h | 2 +-
be/src/exec/operator/hive_table_sink_operator.h | 2 +-
.../exec/operator/iceberg_delete_sink_operator.h | 2 +-
be/src/exec/operator/iceberg_merge_sink_operator.h | 2 +-
be/src/exec/operator/iceberg_table_sink_operator.h | 2 +-
be/src/exec/operator/jdbc_table_sink_operator.cpp | 2 +-
be/src/exec/operator/jdbc_table_sink_operator.h | 2 +-
.../operator/local_merge_sort_source_operator.cpp | 2 +-
.../operator/local_merge_sort_source_operator.h | 2 +-
.../exec/operator/maxcompute_table_sink_operator.h | 2 +-
.../exec/operator/memory_scratch_sink_operator.cpp | 2 +-
.../exec/operator/memory_scratch_sink_operator.h | 2 +-
be/src/exec/operator/mock_operator.h | 2 +-
be/src/exec/operator/mock_scan_operator.h | 2 +-
.../exec/operator/multi_cast_data_stream_sink.cpp | 2 +-
be/src/exec/operator/multi_cast_data_stream_sink.h | 2 +-
.../operator/multi_cast_data_stream_source.cpp | 4 +-
.../exec/operator/multi_cast_data_stream_source.h | 2 +-
.../operator/nested_loop_join_build_operator.cpp | 8 +-
.../operator/nested_loop_join_build_operator.h | 2 +-
be/src/exec/operator/olap_table_sink_operator.h | 2 +-
be/src/exec/operator/olap_table_sink_v2_operator.h | 2 +-
be/src/exec/operator/operator.cpp | 6 +-
be/src/exec/operator/operator.h | 23 +-
.../exec/operator/partition_sort_sink_operator.cpp | 2 +-
.../exec/operator/partition_sort_sink_operator.h | 2 +-
.../operator/partition_sort_source_operator.cpp | 4 +-
.../exec/operator/partition_sort_source_operator.h | 2 +-
.../partitioned_aggregation_sink_operator.cpp | 5 +-
.../partitioned_aggregation_sink_operator.h | 2 +-
.../partitioned_aggregation_source_operator.cpp | 6 +-
.../partitioned_aggregation_source_operator.h | 2 +-
.../partitioned_hash_join_probe_operator.cpp | 6 +-
.../partitioned_hash_join_probe_operator.h | 2 +-
.../partitioned_hash_join_sink_operator.cpp | 3 +-
.../operator/partitioned_hash_join_sink_operator.h | 2 +-
.../exec/operator/rec_cte_anchor_sink_operator.h | 2 +-
be/src/exec/operator/rec_cte_scan_operator.h | 2 +-
be/src/exec/operator/rec_cte_sink_operator.h | 2 +-
be/src/exec/operator/rec_cte_source_operator.h | 2 +-
be/src/exec/operator/result_file_sink_operator.cpp | 2 +-
be/src/exec/operator/result_file_sink_operator.h | 2 +-
be/src/exec/operator/result_sink_operator.cpp | 2 +-
be/src/exec/operator/result_sink_operator.h | 2 +-
be/src/exec/operator/scan_operator.cpp | 2 +-
be/src/exec/operator/scan_operator.h | 4 +-
be/src/exec/operator/schema_scan_operator.cpp | 2 +-
be/src/exec/operator/schema_scan_operator.h | 2 +-
be/src/exec/operator/set_probe_sink_operator.cpp | 3 +-
be/src/exec/operator/set_probe_sink_operator.h | 2 +-
be/src/exec/operator/set_sink_operator.cpp | 2 +-
be/src/exec/operator/set_sink_operator.h | 2 +-
be/src/exec/operator/set_source_operator.cpp | 3 +-
be/src/exec/operator/set_source_operator.h | 2 +-
be/src/exec/operator/sort_sink_operator.cpp | 2 +-
be/src/exec/operator/sort_sink_operator.h | 2 +-
be/src/exec/operator/sort_source_operator.cpp | 2 +-
be/src/exec/operator/sort_source_operator.h | 2 +-
.../operator/spill_iceberg_table_sink_operator.cpp | 3 +-
.../operator/spill_iceberg_table_sink_operator.h | 2 +-
be/src/exec/operator/spill_sort_sink_operator.cpp | 6 +-
be/src/exec/operator/spill_sort_sink_operator.h | 2 +-
.../exec/operator/spill_sort_source_operator.cpp | 5 +-
be/src/exec/operator/spill_sort_source_operator.h | 2 +-
be/src/exec/operator/spill_utils.h | 2 +
be/src/exec/operator/tvf_table_sink_operator.h | 2 +-
be/src/exec/operator/union_sink_operator.cpp | 2 +-
be/src/exec/operator/union_sink_operator.h | 2 +-
be/src/exec/operator/union_source_operator.cpp | 2 +-
be/src/exec/operator/union_source_operator.h | 2 +-
be/src/exec/pipeline/pipeline_task.cpp | 3 +-
be/src/exprs/aggregate/aggregate_function_count.h | 23 +-
.../exprs/function/array/function_array_element.h | 232 +++-
be/src/exprs/function/cast/cast_to_variant.h | 68 +-
be/src/exprs/function/fmod_fast.cpp | 160 +++
be/src/exprs/function/fmod_fast.h | 41 +
be/src/exprs/function/function_jsonb.cpp | 5 +-
be/src/exprs/function/function_struct_element.cpp | 148 ---
be/src/exprs/function/function_variant_element.cpp | 9 +
be/src/exprs/function/modulo.cpp | 68 +-
be/src/exprs/function/simple_function_factory.h | 2 -
be/src/load/delta_writer/delta_writer_v2.cpp | 42 +-
be/src/runtime/small_file_mgr.cpp | 46 +-
be/src/storage/index/ann/ann_index_writer.cpp | 119 +-
be/src/storage/index/ann/ann_index_writer.h | 20 +-
be/src/storage/index/ann/faiss_ann_index.cpp | 3 +-
be/src/storage/rowset_builder.cpp | 38 +-
be/src/storage/segment/segment_writer.cpp | 4 +-
.../segment/variant/variant_column_writer_impl.cpp | 514 +++++++--
.../segment/variant/variant_column_writer_impl.h | 6 +
be/src/storage/tablet/tablet_schema_cache.cpp | 88 ++
be/src/storage/tablet/tablet_schema_cache.h | 17 +
be/src/storage/tablet_info.h | 2 +-
be/test/core/column/column_variant_test.cpp | 57 +
.../data_type_serde_struct_test.cpp | 103 ++
be/test/exec/operator/agg_operator_test.cpp | 2 +-
.../exec/operator/analytic_sink_operator_test.cpp | 66 +-
.../nested_loop_join_build_operator_test.cpp | 141 +++
.../operator/partition_sort_sink_operator_test.cpp | 4 +-
.../partitioned_aggregation_sink_operator_test.cpp | 28 +-
...artitioned_aggregation_source_operator_test.cpp | 48 +
.../operator/partitioned_aggregation_test_helper.h | 4 +-
.../partitioned_hash_join_probe_operator_test.cpp | 33 +
.../partitioned_hash_join_sink_operator_test.cpp | 26 +
.../operator/partitioned_hash_join_test_helper.h | 4 +-
.../exec/operator/query_cache_operator_test.cpp | 4 +-
be/test/exec/operator/sort_operator_test.cpp | 4 +-
.../operator/spill_sort_sink_operator_test.cpp | 79 ++
.../operator/spill_sort_source_operator_test.cpp | 20 +
be/test/exec/operator/spill_sort_test_helper.h | 2 +-
.../exec/operator/streaming_agg_operator_test.cpp | 4 +-
.../exec/operator/table_function_operator_test.cpp | 4 +-
be/test/exprs/aggregate/agg_count_test.cpp | 21 +
.../function/cast/function_variant_cast_test.cpp | 57 +
.../exprs/function/function_array_element_test.cpp | 230 ++++
...est.cpp => function_element_at_struct_test.cpp} | 60 +-
be/test/exprs/function/function_fmod_fast_test.cpp | 353 ++++++
be/test/exprs/function/function_jsonb_test.cpp | 24 +
.../function/function_variant_element_test.cpp | 38 +
.../storage/index/ann/ann_index_writer_test.cpp | 677 +++++-------
.../segment/variant_column_writer_reader_test.cpp | 952 ++++++++++++++++
be/test/storage/segment/variant_util_test.cpp | 276 ++++-
.../storage/tablet/tablet_schema_cache_test.cpp | 125 +++
be/test/testutil/mock/mock_operators.h | 4 +-
be/test/util/profile_spec_test.cpp | 4 +-
cloud/src/common/config.h | 4 +
cloud/src/recycler/recycler.cpp | 363 ++++--
cloud/src/recycler/recycler.h | 13 +-
cloud/src/recycler/recycler_service.cpp | 2 +-
cloud/test/recycler_test.cpp | 438 +++++++-
.../create_preinstalled_scripts/iceberg/run24.sql | 4 +-
doris-skills | 2 +-
extension/logstash/README.md | 50 +-
extension/logstash/lib/logstash/outputs/doris.rb | 89 +-
extension/logstash/logstash-output-doris.gemspec | 4 +-
.../java/org/apache/doris/alter/AlterJobV2.java | 2 +-
.../doris/catalog/BuiltinScalarFunctions.java | 2 -
.../java/org/apache/doris/catalog/TableIf.java | 14 +
.../apache/doris/cloud/catalog/CloudReplica.java | 125 +--
.../apache/doris/cloud/catalog/CloudTablet.java | 15 +-
.../doris/cloud/system/CloudSystemInfoService.java | 93 +-
.../doris/common/profile/SummaryProfile.java | 29 +-
.../apache/doris/datasource/ExternalCatalog.java | 15 +
.../apache/doris/datasource/ExternalDatabase.java | 15 +
.../datasource/PluginDrivenExternalTable.java | 25 +
.../doris/datasource/hive/HMSExternalTable.java | 12 +
.../datasource/iceberg/IcebergExternalTable.java | 10 +
.../datasource/paimon/PaimonExternalTable.java | 10 +
.../org/apache/doris/nereids/CascadesContext.java | 6 +-
.../nereids/ExternalMetadataPreloadResult.java | 65 ++
.../doris/nereids/ExternalTablePreloadInfo.java | 55 +
.../org/apache/doris/nereids/NereidsPlanner.java | 30 +-
.../org/apache/doris/nereids/StatementContext.java | 56 +
.../doris/nereids/exceptions/CastException.java | 4 +-
.../doris/nereids/jobs/executor/Rewriter.java | 5 +-
.../executor/TableCollectAndHookInitializer.java | 26 +-
.../post/RuntimeFilterPushDownVisitor.java | 52 +-
.../nereids/processor/post/TopnFilterContext.java | 16 +-
.../post/materialize/LazyMaterializeTopN.java | 190 ++--
.../post/materialize/LazySlotPruning.java | 15 +-
.../post/materialize/MaterializeProbeVisitor.java | 39 +-
.../org/apache/doris/nereids/rules/RuleType.java | 2 +
.../nereids/rules/analysis/BindExpression.java | 8 +-
.../nereids/rules/analysis/CollectRelation.java | 5 +
.../nereids/rules/analysis/ExpressionAnalyzer.java | 7 +-
.../rules/analysis/PreloadExternalMetadata.java | 134 +++
.../mv/PreMaterializedViewRewriter.java | 1 +
.../rules/expression/ExpressionNormalization.java | 4 +-
.../rules/expression/ExpressionRuleType.java | 2 +-
...eStructElement.java => NormalizeElementAt.java} | 31 +-
.../expression/rules/StringEmptyToLengthRule.java | 2 +-
.../rewrite/AccessPathExpressionCollector.java | 51 +-
.../rules/rewrite/AccessPathPlanCollector.java | 4 +-
.../rewrite/AddProjectForVolatileExpression.java | 105 +-
.../nereids/rules/rewrite/InferPredicates.java | 13 +
.../nereids/rules/rewrite/NestedColumnPruning.java | 2 +-
.../rules/rewrite/OperativeColumnDerive.java | 20 +-
...ProjectOtherJoinConditionForNestedLoopJoin.java | 8 +
.../rules/rewrite/PullUpProjectExprUnderTopN.java | 718 ++++++++++++
.../rules/rewrite/PullUpProjectUnderLimit.java | 11 +-
.../rules/rewrite/PullUpProjectUnderTopN.java | 11 +-
.../PushDownFilterThroughPartitionTopN.java | 8 +-
.../rewrite/PushDownFilterThroughSetOperation.java | 82 +-
.../rules/rewrite/PushDownFilterThroughWindow.java | 13 +-
.../rules/rewrite/PushDownJoinOtherCondition.java | 8 +-
.../rules/rewrite/PushDownTopNThroughJoin.java | 82 +-
.../rules/rewrite/PushDownUnnestInProject.java | 10 +-
.../functions/executable/DateTimeArithmetic.java | 16 +-
.../executable/DateTimeExtractAndTransform.java | 2 +-
.../expressions/functions/scalar/ConvertTz.java | 72 +-
.../expressions/functions/scalar/ElementAt.java | 70 +-
.../expressions/functions/scalar/FromUnixtime.java | 56 +-
.../functions/scalar/StructElement.java | 117 --
.../expressions/visitor/ScalarFunctionVisitor.java | 5 -
.../plans/physical/PhysicalLazyMaterialize.java | 13 +-
.../physical/PhysicalLazyMaterializeFileScan.java | 10 +-
.../physical/PhysicalLazyMaterializeOlapScan.java | 10 +-
.../physical/PhysicalLazyMaterializeTVFScan.java | 10 +-
.../org/apache/doris/nereids/util/DateUtils.java | 15 +
.../org/apache/doris/nereids/util/JoinUtils.java | 4 +
.../apache/doris/planner/MaterializationNode.java | 2 +-
.../org/apache/doris/planner/OlapScanNode.java | 26 +-
.../org/apache/doris/planner/OlapTableSink.java | 17 +-
.../java/org/apache/doris/planner/PlanNode.java | 8 +-
.../java/org/apache/doris/qe/SessionVariable.java | 26 +
.../qe/runtime/RuntimeFiltersThriftBuilder.java | 29 +-
.../apache/doris/service/FrontendServiceImpl.java | 35 +-
.../cloud/system/CloudSystemInfoServiceTest.java | 15 +
.../doris/common/profile/SummaryProfileTest.java | 19 +-
.../PluginDrivenExternalTableEngineTest.java | 72 +-
.../apache/doris/nereids/NereidsPlannerTest.java | 82 ++
.../apache/doris/nereids/StatementContextTest.java | 559 ++++++++++
.../nereids/postprocess/RuntimeFilterTest.java | 19 +
.../nereids/postprocess/TopNRuntimeFilterTest.java | 36 +
.../materialize/MaterializeProbeVisitorTest.java | 110 ++
.../nereids/rules/expression/FoldConstantTest.java | 6 +
.../rules/StringEmptyToLengthRuleTest.java | 14 +-
.../AddProjectForVolatileExpressionTest.java | 149 ++-
.../rewrite/FindHashConditionForJoinTest.java | 22 +
.../rules/rewrite/OperativeColumnDeriveTest.java | 35 +-
.../nereids/rules/rewrite/PartitionPrunerTest.java | 1 -
.../doris/nereids/rules/rewrite/PrepareTest.java | 2 +-
.../rules/rewrite/PruneNestedColumnTest.java | 150 +--
.../rewrite/PullUpProjectExprUnderTopNTest.java | 1166 ++++++++++++++++++++
.../rewrite/PushDownFilterThroughWindowTest.java | 40 +
.../rewrite/PushDownJoinOtherConditionTest.java | 29 +
.../rules/rewrite/PushDownUnnestInProjectTest.java | 4 +-
.../functions/scalar/ConvertTzTest.java | 173 +++
.../functions/scalar/FromUnixtimeTest.java | 99 ++
.../apache/doris/nereids/util/JoinUtilsTest.java | 19 +
.../org/apache/doris/qe/SessionVariablesTest.java | 12 +
.../runtime/RuntimeFiltersThriftBuilderTest.java | 114 ++
.../data/ann_index_p0/ivf_on_disk_index_test.out | 4 +
.../ivf_pq_full_buffer_train_recall.out} | 4 +-
.../ivf_pq_recall.out} | 9 +-
...est_runtime_filter_outer_join_nullable_side.out | 17 +
.../test_struct_field_align_ordered.json | 7 +
.../test_struct_field_align_swapped.json | 7 +
.../data/doc/sql-manual/StructNullsafe.out | 6 +-
.../external_table_p0/hive/test_complex_types.out | 8 +-
.../data/load_p0/stream_load/test_stream_load.out | 6 +-
.../stream_load/test_stream_load_move_memtable.out | 6 +-
.../column_pruning/topn_expr_pullup.out | 76 ++
.../topn_lazy_nested_column_pruning.out | 39 +
.../extend_infer_equal_predicate.out | 10 +-
.../infer_set_operator_distinct.out | 30 +-
.../partition_prune/test_convert_tz.out} | 5 +-
.../add_project_for_unique_function.out | 13 +-
...nfer_predicates_set_op_with_unique_function.out | 47 +
...join_condition_for_nlj_with_unique_function.out | 36 +
...wn_filter_through_join_with_unique_function.out | 9 +
...through_partition_topn_with_unique_function.out | 17 +
..._through_set_operation_with_unique_function.out | 74 ++
..._filter_through_window_with_unique_function.out | 23 +
...n_join_other_condition_with_unique_function.out | 53 +
.../cast_function/test_cast_struct.out | 2 +-
.../struct_functions/test_struct_functions.out | 43 +-
.../data/shape_check/clickbench/query36.out | 8 +-
.../tpcds_sf100/noStatsRfPrune/query17.out | 8 +-
.../tpcds_sf100/noStatsRfPrune/query54.out | 8 +-
.../tpcds_sf100/noStatsRfPrune/query58.out | 165 +--
.../tpcds_sf100/noStatsRfPrune/query59.out | 55 +-
.../tpcds_sf100/noStatsRfPrune/query61.out | 4 +-
.../tpcds_sf100/noStatsRfPrune/query83.out | 153 +--
.../tpcds_sf100/noStatsRfPrune/query84.out | 40 +-
.../tpcds_sf100/no_stats_shape/query17.out | 8 +-
.../tpcds_sf100/no_stats_shape/query54.out | 8 +-
.../tpcds_sf100/no_stats_shape/query58.out | 165 +--
.../tpcds_sf100/no_stats_shape/query59.out | 55 +-
.../tpcds_sf100/no_stats_shape/query61.out | 4 +-
.../tpcds_sf100/no_stats_shape/query83.out | 153 +--
.../tpcds_sf100/no_stats_shape/query84.out | 40 +-
.../shape_check/tpcds_sf100/rf_prune/query17.out | 8 +-
.../shape_check/tpcds_sf100/rf_prune/query54.out | 8 +-
.../shape_check/tpcds_sf100/rf_prune/query58.out | 165 +--
.../shape_check/tpcds_sf100/rf_prune/query59.out | 55 +-
.../shape_check/tpcds_sf100/rf_prune/query61.out | 4 +-
.../shape_check/tpcds_sf100/rf_prune/query83.out | 153 +--
.../shape_check/tpcds_sf100/rf_prune/query84.out | 42 +-
.../data/shape_check/tpcds_sf100/shape/query17.out | 8 +-
.../data/shape_check/tpcds_sf100/shape/query54.out | 8 +-
.../data/shape_check/tpcds_sf100/shape/query58.out | 165 +--
.../data/shape_check/tpcds_sf100/shape/query59.out | 55 +-
.../data/shape_check/tpcds_sf100/shape/query61.out | 4 +-
.../data/shape_check/tpcds_sf100/shape/query83.out | 153 +--
.../data/shape_check/tpcds_sf100/shape/query84.out | 42 +-
.../tpcds_sf1000/bs_downgrade_shape/query54.out | 8 +-
.../tpcds_sf1000/bs_downgrade_shape/query61.out | 4 +-
.../data/shape_check/tpcds_sf1000/hint/query17.out | 8 +-
.../data/shape_check/tpcds_sf1000/hint/query54.out | 8 +-
.../data/shape_check/tpcds_sf1000/hint/query58.out | 165 +--
.../data/shape_check/tpcds_sf1000/hint/query59.out | 55 +-
.../data/shape_check/tpcds_sf1000/hint/query61.out | 4 +-
.../data/shape_check/tpcds_sf1000/hint/query84.out | 42 +-
.../shape_check/tpcds_sf1000/shape/query17.out | 8 +-
.../shape_check/tpcds_sf1000/shape/query54.out | 8 +-
.../shape_check/tpcds_sf1000/shape/query58.out | 165 +--
.../shape_check/tpcds_sf1000/shape/query59.out | 55 +-
.../shape_check/tpcds_sf1000/shape/query61.out | 4 +-
.../shape_check/tpcds_sf1000/shape/query83.out | 153 +--
.../shape_check/tpcds_sf1000/shape/query84.out | 42 +-
.../tpcds_sf1000_nopkfk/shape/query17.out | 8 +-
.../tpcds_sf1000_nopkfk/shape/query54.out | 8 +-
.../tpcds_sf1000_nopkfk/shape/query58.out | 165 +--
.../tpcds_sf1000_nopkfk/shape/query59.out | 55 +-
.../tpcds_sf1000_nopkfk/shape/query61.out | 4 +-
.../tpcds_sf1000_nopkfk/shape/query83.out | 153 +--
.../tpcds_sf1000_nopkfk/shape/query84.out | 42 +-
.../shape_check/tpcds_sf10t_orc/shape/query17.out | 8 +-
.../shape_check/tpcds_sf10t_orc/shape/query54.out | 8 +-
.../shape_check/tpcds_sf10t_orc/shape/query58.out | 165 +--
.../shape_check/tpcds_sf10t_orc/shape/query59.out | 55 +-
.../shape_check/tpcds_sf10t_orc/shape/query61.out | 4 +-
.../shape_check/tpcds_sf10t_orc/shape/query83.out | 153 +--
.../shape_check/tpcds_sf10t_orc/shape/query84.out | 40 +-
.../data/variant_p0/sql/select_from_value.out | 2 +-
regression-test/pipeline/common/doris-utils.sh | 17 +-
.../ann_index_build_min_segment_rows.groovy | 66 ++
.../ann_index_p0/ivf_on_disk_index_test.groovy | 16 +-
.../ivf_pq_full_buffer_train_recall.groovy | 68 ++
.../suites/ann_index_p0/ivf_pq_recall.groovy | 85 ++
..._runtime_filter_outer_join_nullable_side.groovy | 79 ++
.../complex_types/test_pruned_columns.groovy | 16 +-
.../complex_types/test_struct_field_align.groovy | 91 ++
.../one_level_nestedtypes_with_s3data.groovy | 18 +-
.../nested_types/query/sql/nested_with_join.sql | 8 +-
.../three_level_nestedtypes_with_s3data.groovy | 86 +-
.../two_level_nestedtypes_with_s3data.groovy | 34 +-
.../suites/dictionary_p0/test_trade_demo.groovy | 16 +-
.../suites/doc/sql-manual/StructNullsafe.groovy | 8 +-
.../basic-elements/data-types/map-md.groovy | 2 +-
.../basic-elements/data-types/struct-md.groovy | 24 +-
.../hive/test_complex_types.groovy | 4 +-
.../hive/test_hive_struct_add_column.groovy | 24 +-
.../hive/test_hive_text_complex_type.groovy | 6 +-
.../hive/test_hive_topn_lazy_mat.groovy | 10 +-
.../hive/test_orc_nested_types.groovy | 32 +-
.../hive/test_parquet_bloom_filter.groovy | 2 +-
.../hive/test_parquet_nested_types.groovy | 14 +-
.../iceberg/iceberg_complex_type.groovy | 8 +-
.../iceberg/iceberg_schema_change.groovy | 16 +-
.../iceberg/test_iceberg_full_schema_change.groovy | 64 +-
...test_iceberg_schema_change_complex_types.groovy | 4 +-
.../test_iceberg_struct_schema_evolution.groovy | 60 +-
.../paimon/paimon_timestamp_types.groovy | 4 +-
.../paimon/test_paimon_full_schema_change.groovy | 30 +-
.../tvf/test_tvf_topn_lazy_mat.groovy | 8 +-
.../hudi/test_hudi_full_schema_change.groovy | 64 +-
.../iceberg/test_hive_migrate_iceberg.groovy | 10 +-
.../nereids_function_p0/scalar_function/IP.groovy | 8 +-
.../nested_container_offset_pruning.groovy | 16 +-
.../column_pruning/null_column_pruning.groovy | 62 +-
.../string_length_column_pruning.groovy | 28 +-
.../column_pruning/topn_expr_pullup.groovy | 161 +++
.../topn_lazy_nested_column_pruning.groovy | 365 ++++++
.../month_quarter_cast_in_prune.groovy | 8 +-
.../partition_prune/test_convert_tz.groovy | 24 +-
.../add_project_for_unique_function.groovy | 1 +
...r_predicates_set_op_with_unique_function.groovy | 66 ++
...n_condition_for_nlj_with_unique_function.groovy | 58 +
...filter_through_join_with_unique_function.groovy | 10 +
...ough_partition_topn_with_unique_function.groovy | 47 +
...rough_set_operation_with_unique_function.groovy | 75 ++
...lter_through_window_with_unique_function.groovy | 56 +
...oin_other_condition_with_unique_function.groovy | 73 ++
.../nereids_syntax_p0/test_complex_type.groovy | 10 +-
.../compress_materialize/pushdown_encode.groovy | 16 +-
.../ip_functions/test_ipv4_cidr_to_range.groovy | 2 +-
.../test_ipv6_cidr_to_range_function.groovy | 4 +-
.../struct_functions/test_struct_functions.groovy | 29 +-
.../test_struct_functions_by_literal.groovy | 4 +-
.../suites/variant_p0/element_function.groovy | 28 +
.../suites/variant_p0/test_sub_path_pruning.groovy | 2 +-
418 files changed, 14924 insertions(+), 3886 deletions(-)
create mode 100644 be/benchmark/benchmark_fmod.hpp
create mode 100644 be/src/exprs/function/fmod_fast.cpp
create mode 100644 be/src/exprs/function/fmod_fast.h
delete mode 100644 be/src/exprs/function/function_struct_element.cpp
create mode 100644
be/test/exec/operator/nested_loop_join_build_operator_test.cpp
rename be/test/exprs/function/{function_struct_element_test.cpp =>
function_element_at_struct_test.cpp} (61%)
create mode 100644 be/test/exprs/function/function_fmod_fast_test.cpp
create mode 100644 be/test/storage/tablet/tablet_schema_cache_test.cpp
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/ExternalMetadataPreloadResult.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/ExternalTablePreloadInfo.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/analysis/PreloadExternalMetadata.java
rename
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/rules/{NormalizeStructElement.java
=> NormalizeElementAt.java} (69%)
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/PullUpProjectExprUnderTopN.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/scalar/StructElement.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/nereids/NereidsPlannerTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/nereids/StatementContextTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/nereids/processor/post/materialize/MaterializeProbeVisitorTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/nereids/rules/rewrite/PullUpProjectExprUnderTopNTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/nereids/trees/expressions/functions/scalar/ConvertTzTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/nereids/trees/expressions/functions/scalar/FromUnixtimeTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/qe/runtime/RuntimeFiltersThriftBuilderTest.java
copy regression-test/data/{variant_p0/sql/select_from_value.out =>
ann_index_p0/ivf_pq_full_buffer_train_recall.out} (76%)
copy regression-test/data/{variant_p0/sql/select_from_value.out =>
ann_index_p0/ivf_pq_recall.out} (53%)
create mode 100644
regression-test/data/correctness_p0/test_runtime_filter_outer_join_nullable_side.out
create mode 100644
regression-test/data/datatype_p0/complex_types/test_struct_field_align_ordered.json
create mode 100644
regression-test/data/datatype_p0/complex_types/test_struct_field_align_swapped.json
create mode 100644
regression-test/data/nereids_rules_p0/column_pruning/topn_expr_pullup.out
create mode 100644
regression-test/data/nereids_rules_p0/column_pruning/topn_lazy_nested_column_pruning.out
copy regression-test/data/{variant_p0/sql/select_from_value.out =>
nereids_rules_p0/partition_prune/test_convert_tz.out} (63%)
create mode 100644
regression-test/data/nereids_rules_p0/unique_function/infer_predicates_set_op_with_unique_function.out
create mode 100644
regression-test/data/nereids_rules_p0/unique_function/project_other_join_condition_for_nlj_with_unique_function.out
create mode 100644
regression-test/data/nereids_rules_p0/unique_function/push_down_filter_through_partition_topn_with_unique_function.out
create mode 100644
regression-test/data/nereids_rules_p0/unique_function/push_down_filter_through_set_operation_with_unique_function.out
create mode 100644
regression-test/data/nereids_rules_p0/unique_function/push_down_filter_through_window_with_unique_function.out
create mode 100644
regression-test/data/nereids_rules_p0/unique_function/push_down_join_other_condition_with_unique_function.out
create mode 100644
regression-test/suites/ann_index_p0/ann_index_build_min_segment_rows.groovy
create mode 100644
regression-test/suites/ann_index_p0/ivf_pq_full_buffer_train_recall.groovy
create mode 100644 regression-test/suites/ann_index_p0/ivf_pq_recall.groovy
create mode 100644
regression-test/suites/correctness_p0/test_runtime_filter_outer_join_nullable_side.groovy
create mode 100644
regression-test/suites/datatype_p0/complex_types/test_struct_field_align.groovy
create mode 100644
regression-test/suites/nereids_rules_p0/column_pruning/topn_expr_pullup.groovy
create mode 100644
regression-test/suites/nereids_rules_p0/column_pruning/topn_lazy_nested_column_pruning.groovy
create mode 100644
regression-test/suites/nereids_rules_p0/unique_function/infer_predicates_set_op_with_unique_function.groovy
create mode 100644
regression-test/suites/nereids_rules_p0/unique_function/project_other_join_condition_for_nlj_with_unique_function.groovy
create mode 100644
regression-test/suites/nereids_rules_p0/unique_function/push_down_filter_through_partition_topn_with_unique_function.groovy
create mode 100644
regression-test/suites/nereids_rules_p0/unique_function/push_down_filter_through_set_operation_with_unique_function.groovy
create mode 100644
regression-test/suites/nereids_rules_p0/unique_function/push_down_filter_through_window_with_unique_function.groovy
create mode 100644
regression-test/suites/nereids_rules_p0/unique_function/push_down_join_other_condition_with_unique_function.groovy
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
