This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a change to branch auto-pick-58135-branch-4.0
in repository https://gitbox.apache.org/repos/asf/doris.git
discard 6a73bc25e66 [fix](cloud) Fixed uneven tablet performance during
upgrades from older versions (#58135)
add 8b6b871cc93 branch-4.0: [fix](test) fix regression sumRewrite #58232
(#58235)
add f79c0ddaed9 [branch-4.0](pick)expr execution now returns a Column
directly instead of inserting into a Block (#58227)
add ecdc189390a branch-4.0: [Chore](build) copy jeprof to output/be/bin
#58128 (#58207)
add 97315a8b8d0 branch-4.0: [opt](staticstis) use count(1) for rowCount
when scan full table #58153 (#58190)
add 516e65215ad branch-4.0: [fix](nereids) inferPredicate should not
pullup predicates from mark join's right child #57712 (#58211)
add 2f19e95bc08 branch-4.0: [feat](load) support merge into #57044 #58091
(#58219)
add 7ce3d6a63b1 branch-4.0: [feature](regression-plugin) support version
compare plugin with cloud #56850 (#58244)
add ab3b6638005 [fix](paimon-catalog)Fix OSS access when using DLS
endpoint (#58099) (#58226)
add 66a92031a69 branch-4.0: [feat](iceberg) Implement Iceberg
`rewrite_data_files` action for table optimization and compaction (#56413
#56638) (#57871)
add e591ce6fe92 branch-4.0: [enhancement](tde) add configuration for TDE
key rotation #58171 (#58203)
add 606756848c5 branch-4.0: [fix](datatype)Fix for unaligned memory in
arrow MapArray parsing. #58248 (#58251)
add a5aa884062b branch-4.0: [fix](mtmx) Fix coredump because common expr
push down to scannode which belongs to an agg materialized view #58038 (#58096)
add 6ae3f3afe54 branch-4.0: [test](schema-change) Minimal case for schema
change index out of bound when delete predicate column dropped #57861 (#58250)
add efb3f4a5176 branch-4.0: [fix](nereids)allow sync mv has same columns
as base table when it has where clause #58105 (#58180)
add 7989173a585 branch-4.0: [Feature] Support Doris Catalog (#55274)
(#58162)
add 58f171035fa branch-4.0: [feat](storage) introduce backpressure
algorithm to control version number (part I) #57133 (#57416)
add 74560490956 branch-4.0: [bugfix](memory) should count memory when
cancel query is called #58252 (#58256)
add ee73e97f46e branch-4.0: [chore](cloud) Expose download command in
tools/fdb/fdb_ctl.sh for convenience #58257 (#58266)
add 78da6151fa3 branch-4.0: [Test](client) Add s3 storage client test
sketelon #57456 (#58164)
add 93c8bef60ab branch-4.0: [fix](jni)Add hadoop-mapreduce-client-core
dependencies #58217 (#58267)
add ef90838145c branch-4.0: [bug](test) fix the different test case with
same table name #58197 (#58260)
add 30b51a54f9b branch-4.0: [refactor](mysql output)Use to_string when
outputting plain text to MySQL. #57824 (#58264)
add 1739237db10 branch-4.0: [fix](function) Align function signatures
between backend and frontend and remove datev1 types #56807 (#58262)
add e19909e11ed branch-4.0: [chore](proc) Fix wrong keys type in show proc
index schema #58201 (#58272)
add 751b22aba44 branch-4.0: [Improvement](join) add direct mapping opt for
join #57960 (#58309)
add 2988d887b83 branch-4.0: [fix](test) remove spark load test cases
(#58270)
add 8fa5b988793 branch-4.0: [fix](window) allow constant expression
evaluation in LAG/LEAD offset parameter #58200 (#58261)
add ae5d963dbfa branch-4.0: [fix](cases) Remove badcase
test_auto_partition_with_single_replica_insert #58259 (#58308)
add a2671c4d601 branch-4.0: [fix](regression) fix
test_iceberg_optimize_actions_ddl and test_iceberg_rewrite_data_files (#58311)
add ec8678a74b9 branch-4.0: [fix](nereids) setOperator column prunning bug
#58237 (#58318)
add 5604f3079f2 branch-4.0: [fix](type) Unexpected throw exception due to
unknown file type #57755 (#58319)
add 8eb3f1b1119 branch-4.0: [fix](regression-case) Fix
test_dml_stream_load_auth, use sync function but not async #58286 (#58314)
add 1785dac51b4 branch-4.0: [Improvement](exchange) avoid hash shuffle
when partition type is bucket shuffle and only one instance #58068 (#58206)
add 0d5b1aa6da3 branch-4.0: [fix](test) Add 'waitForSchemaChangeDone'
after drop column from table #58205 (#58303)
add 97619483118 branch-4.0: [fix](test) remove spark load test cases
#58269 (#58294)
add 5e5615f27a8 branch-4.0: [Fix](agg) fix push agg op in nullable column
before projection #58234 (#58281)
add 0cb509daa26 branch-4.0:[fix](mtmv) Fix nested mv rewritten fail when
bottom mv is partitioned (#57558) (#58277)
add 458e2ef0a33 branch-4.0: [Fix](package)Revert be-extension package
management changes due to class loading issues #58255 (#58271)
add 662c6da3a55 branch-4.0: [FIX](regression-test) Fix case
test_s3_tvf_number_range, use the right file path in HDFS #58292 (#58323)
add b916953b4b8 branch-4.0: [fix](test) fix unstable check_before_quit
cases #58295 (#58321)
add c1d2ee07788 branch-4.0: [fix](blockrule)Fix concurrency issues in
SqlBlockRuleMgr after restart fe #58296 (#58326)
add f36b2389979 branch-4.0: [fix](auth)Fixed the issue nereids not check
for the existence of permission items #58282 (#58325)
add f88c44aaaeb branch-4.0: [feature](file-cache) Implement cache warm up
select functionality. (#58315)
add 00ab6551ed1 branch-4.0: [opt](function) better signature match for
time type #58193 (#58335)
add e0035e7ebb7 branch-4.0: [chore] Opt index scan cases #57951 (#58332)
add 4003e3b3367 branch-4.0: [fix](case)Fix temp table case #58299 (#58327)
add 1d964d80f7c branch-4.0: [fix](variable) support set ON and ALL to
variable #58287 (#58329)
add 3fcfd66bc65 branch-4.0: [Fix](geo) fix memory leak in geo #58004
(#58365)
add 22365ade573 [fix](cloud) Fixed uneven tablet performance during
upgrades from older versions (#58135)
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (6a73bc25e66)
\
N -- N -- N refs/heads/auto-pick-58135-branch-4.0 (22365ade573)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
be/src/common/config.cpp | 5 +
be/src/common/config.h | 5 +
be/src/exec/tablet_info.cpp | 6 +
be/src/geo/geo_types.cpp | 20 +-
be/src/geo/geo_types.h | 10 +-
be/src/geo/wkb_parse.cpp | 92 +-
be/src/geo/wkb_parse.h | 22 +-
be/src/geo/wkb_parse_ctx.h | 12 +-
be/src/geo/wkt_lex.l | 21 +-
be/src/geo/wkt_parse.cpp | 40 +-
be/src/geo/wkt_parse.h | 9 +-
be/src/geo/wkt_parse_ctx.h | 9 +-
be/src/geo/wkt_parse_type.h | 20 +-
be/src/geo/wkt_yacc.y | 175 +-
be/src/io/file_factory.h | 5 +-
be/src/io/fs/azure_obj_storage_client.cpp | 2 +-
be/src/io/fs/s3_obj_storage_client.cpp | 10 +-
be/src/olap/delta_writer.cpp | 16 +
be/src/olap/delta_writer.h | 3 +
be/src/olap/rowset_builder.h | 5 +-
be/src/pipeline/common/join_utils.h | 104 +-
be/src/pipeline/exec/blackhole_sink_operator.cpp | 112 ++
be/src/pipeline/exec/blackhole_sink_operator.h | 90 +
be/src/pipeline/exec/exchange_sink_operator.cpp | 38 +-
be/src/pipeline/exec/hashjoin_build_sink.cpp | 44 +-
be/src/pipeline/exec/hashjoin_build_sink.h | 6 +-
.../exec/join/process_hash_table_probe_impl.h | 33 +-
be/src/pipeline/exec/operator.cpp | 8 +-
.../exec/partitioned_aggregation_sink_operator.cpp | 2 +-
.../partitioned_aggregation_source_operator.cpp | 2 +-
.../exec/partitioned_hash_join_probe_operator.cpp | 15 +-
.../exec/partitioned_hash_join_sink_operator.cpp | 4 +-
be/src/pipeline/exec/spill_sort_sink_operator.cpp | 4 +-
be/src/pipeline/pipeline_fragment_context.cpp | 11 +
be/src/runtime/descriptors.cpp | 14 +
be/src/runtime/descriptors.h | 9 +
be/src/runtime/fragment_mgr.cpp | 1 +
be/src/runtime/runtime_query_statistics_mgr.cpp | 20 +-
be/src/runtime/tablets_channel.cpp | 6 +
be/src/runtime/tablets_channel.h | 1 +
be/src/runtime/thread_context.cpp | 15 -
be/src/runtime/thread_context.h | 20 +-
be/src/runtime/workload_management/io_context.h | 10 +
.../workload_management/resource_context.cpp | 1 +
be/src/service/internal_service.cpp | 25 +-
be/src/util/s3_util.h | 1 +
.../aggregate_function_map_v2.cpp | 2 -
.../aggregate_function_window.cpp | 22 -
be/src/vec/common/hash_table/hash_map_context.h | 65 +
be/src/vec/common/hash_table/join_hash_table.h | 59 +-
be/src/vec/core/block.h | 3 +
.../data_types/serde/data_type_bitmap_serde.cpp | 15 +
.../vec/data_types/serde/data_type_bitmap_serde.h | 3 +
.../vec/data_types/serde/data_type_hll_serde.cpp | 15 +
be/src/vec/data_types/serde/data_type_hll_serde.h | 3 +
.../vec/data_types/serde/data_type_jsonb_serde.cpp | 6 +
.../vec/data_types/serde/data_type_map_serde.cpp | 16 +-
.../data_types/serde/data_type_nullable_serde.cpp | 10 +
.../data_types/serde/data_type_nullable_serde.h | 2 +
.../data_types/serde/data_type_number_serde.cpp | 12 +-
.../serde/data_type_quantilestate_serde.cpp | 18 +
.../serde/data_type_quantilestate_serde.h | 3 +
be/src/vec/data_types/serde/data_type_serde.cpp | 6 +
be/src/vec/data_types/serde/data_type_serde.h | 5 +
.../vec/exec/format/table/remote_doris_reader.cpp | 130 ++
be/src/vec/exec/format/table/remote_doris_reader.h | 70 +
be/src/vec/exec/scan/file_scanner.cpp | 17 +-
be/src/vec/exec/scan/olap_scanner.cpp | 2 +
be/src/vec/exec/scan/scanner.cpp | 7 +-
be/src/vec/exprs/lambda_function/lambda_function.h | 4 +-
.../lambda_function/varray_filter_function.cpp | 40 +-
.../exprs/lambda_function/varray_map_function.cpp | 83 +-
be/src/vec/exprs/vbitmap_predicate.cpp | 25 +-
be/src/vec/exprs/vbitmap_predicate.h | 3 +-
be/src/vec/exprs/vbloom_predicate.cpp | 25 +-
be/src/vec/exprs/vbloom_predicate.h | 3 +-
be/src/vec/exprs/vcase_expr.cpp | 32 +-
be/src/vec/exprs/vcase_expr.h | 3 +-
be/src/vec/exprs/vcast_expr.cpp | 74 +-
be/src/vec/exprs/vcast_expr.h | 6 +-
be/src/vec/exprs/vcolumn_ref.h | 10 +-
be/src/vec/exprs/vcompound_pred.h | 91 +-
be/src/vec/exprs/vdirect_in_predicate.h | 39 +-
be/src/vec/exprs/vectorized_fn_call.cpp | 55 +-
be/src/vec/exprs/vectorized_fn_call.h | 12 +-
be/src/vec/exprs/vexpr.cpp | 21 +-
be/src/vec/exprs/vexpr.h | 32 +-
be/src/vec/exprs/vexpr_context.cpp | 6 +
be/src/vec/exprs/vexpr_context.h | 2 +
be/src/vec/exprs/vin_predicate.cpp | 33 +-
be/src/vec/exprs/vin_predicate.h | 3 +-
be/src/vec/exprs/vinfo_func.cpp | 8 +-
be/src/vec/exprs/vinfo_func.h | 3 +-
be/src/vec/exprs/virtual_slot_ref.cpp | 24 +-
be/src/vec/exprs/virtual_slot_ref.h | 3 +-
be/src/vec/exprs/vlambda_function_call_expr.h | 5 +-
be/src/vec/exprs/vlambda_function_expr.h | 9 +-
be/src/vec/exprs/vliteral.cpp | 8 +-
be/src/vec/exprs/vliteral.h | 3 +-
be/src/vec/exprs/vmatch_predicate.cpp | 28 +-
be/src/vec/exprs/vmatch_predicate.h | 3 +-
be/src/vec/exprs/vruntimefilter_wrapper.cpp | 19 +-
be/src/vec/exprs/vruntimefilter_wrapper.h | 3 +-
be/src/vec/exprs/vsearch.cpp | 5 +-
be/src/vec/exprs/vsearch.h | 3 +-
be/src/vec/exprs/vslot_ref.cpp | 20 +
be/src/vec/exprs/vslot_ref.h | 3 +
be/src/vec/exprs/vtopn_pred.h | 38 +-
.../vec/functions/array/function_array_apply.cpp | 4 -
be/src/vec/functions/array/function_array_index.h | 16 +-
be/src/vec/functions/array/function_array_remove.h | 11 -
be/src/vec/functions/cast/cast_to_variant.h | 2 +-
be/src/vec/functions/cast/function_cast.cpp | 2 +-
be/src/vec/functions/function.cpp | 71 +-
be/src/vec/functions/function.h | 30 +-
be/src/vec/functions/function_bit.cpp | 16 +-
be/src/vec/functions/function_bit_count.cpp | 19 +-
.../function_date_or_datetime_computation.cpp | 167 +-
.../function_date_or_datetime_computation.h | 13 +-
.../function_date_or_datetime_computation_v2.cpp | 165 --
.../function_date_or_datetime_to_string.cpp | 4 -
.../vec/functions/function_datetime_floor_ceil.cpp | 31 +-
.../function_datetime_string_to_string.cpp | 4 -
be/src/vec/functions/function_rpc.h | 2 +-
be/src/vec/functions/function_unary_arithmetic.h | 54 +-
be/src/vec/functions/functions_comparison.h | 4 -
be/src/vec/functions/functions_geo.cpp | 4 +-
be/src/vec/functions/least_greast.cpp | 14 -
be/src/vec/functions/math.cpp | 150 +-
be/src/vec/functions/round.cpp | 1 -
be/src/vec/functions/simple_function_factory.h | 2 -
be/src/vec/functions/time_of_function.cpp | 17 -
be/src/vec/functions/to_time_function.cpp | 31 -
be/src/vec/sink/vmysql_result_writer.cpp | 84 +-
.../sink/writer/iceberg/viceberg_table_writer.cpp | 12 +-
.../sink/writer/iceberg/viceberg_table_writer.h | 3 +
be/src/vec/sink/writer/vfile_result_writer.cpp | 5 +-
be/src/vec/sink/writer/vtablet_writer.cpp | 57 +
be/src/vec/sink/writer/vtablet_writer.h | 10 +
be/test/exprs/mock_vexpr.h | 2 +
be/test/exprs/virtual_slot_ref_test.cpp | 15 +
be/test/geo/geo_types_test.cpp | 459 +++--
be/test/geo/wkb_parse_test.cpp | 179 ++
be/test/geo/wkt_parse_test.cpp | 10 +-
be/test/io/client/s3_file_system_test.cpp | 1895 ++++++++++++++++++++
be/test/olap/collection_statistics_test.cpp | 5 +
.../vec/aggregate_functions/agg_collect_test.cpp | 15 +-
be/test/vec/aggregate_functions/agg_test.cpp | 161 ++
.../data_types/serde/data_type_serde_map_test.cpp | 182 ++
.../serde/data_type_serde_struct_test.cpp | 162 ++
be/test/vec/exprs/try_cast_expr_test.cpp | 21 +-
be/test/vec/exprs/vsearch_expr_test.cpp | 4 +
.../vec/function/function_array_element_test.cpp | 36 -
be/test/vec/function/function_array_index_test.cpp | 57 -
.../vec/function/function_arrays_overlap_test.cpp | 31 -
be/test/vec/function/function_geo_test.cpp | 14 +-
be/test/vec/function/function_math_test.cpp | 40 -
be/test/vec/function/function_time_test.cpp | 58 +-
be/test/vec/function/table_function_test.cpp | 24 -
build.sh | 2 +-
.../create_preinstalled_scripts/iceberg/run21.sql | 152 ++
fe/be-java-extensions/avro-scanner/pom.xml | 8 +-
fe/be-java-extensions/hadoop-deps/pom.xml | 4 +
fe/be-java-extensions/hadoop-hudi-scanner/pom.xml | 34 +-
.../iceberg-metadata-scanner/pom.xml | 10 +-
fe/be-java-extensions/java-udf/pom.xml | 3 +-
fe/be-java-extensions/jdbc-scanner/pom.xml | 3 +-
fe/be-java-extensions/max-compute-scanner/pom.xml | 8 +-
fe/be-java-extensions/paimon-scanner/pom.xml | 14 +-
fe/be-java-extensions/preload-extensions/pom.xml | 9 +-
.../trino-connector-scanner/pom.xml | 3 +-
.../main/java/org/apache/doris/common/Config.java | 14 +
.../antlr4/org/apache/doris/nereids/DorisLexer.g4 | 1 +
.../antlr4/org/apache/doris/nereids/DorisParser.g4 | 38 +-
.../doris/alter/MaterializedViewHandler.java | 4 +-
.../org/apache/doris/analysis/SchemaTableType.java | 4 +-
.../java/org/apache/doris/analysis/StmtType.java | 1 +
.../apache/doris/blockrule/SqlBlockRuleMgr.java | 18 +-
.../main/java/org/apache/doris/catalog/Column.java | 4 +
.../java/org/apache/doris/catalog/KeysType.java | 3 +-
.../java/org/apache/doris/catalog/OlapTable.java | 4 +
.../java/org/apache/doris/catalog/SchemaTable.java | 7 +
.../java/org/apache/doris/catalog/TableIf.java | 3 +-
.../org/apache/doris/common/NamedArguments.java | 1 +
.../apache/doris/common/proc/IndexInfoProcDir.java | 2 +-
.../doris/common/profile/ProfileManager.java | 3 +-
.../org/apache/doris/common/util/JsonUtil.java | 12 +
.../org/apache/doris/common/util/LocationPath.java | 3 +
.../java/org/apache/doris/common/util/Util.java | 17 +-
.../apache/doris/datasource/CatalogFactory.java | 4 +
.../apache/doris/datasource/ExternalCatalog.java | 3 +
.../apache/doris/datasource/InitCatalogLog.java | 1 +
.../apache/doris/datasource/InitDatabaseLog.java | 1 +
.../apache/doris/datasource/TableFormatType.java | 3 +-
.../doris/RemoteDorisCompatibleRestClient.java | 137 ++
.../doris/RemoteDorisExternalCatalog.java | 202 +++
.../doris/RemoteDorisExternalDatabase.java | 36 +
.../datasource/doris/RemoteDorisExternalTable.java | 89 +
.../datasource/doris/RemoteDorisRestClient.java | 299 +++
.../doris/source/RemoteDorisScanNode.java | 327 ++++
.../datasource/doris/source/RemoteDorisSource.java | 88 +
.../datasource/doris/source/RemoteDorisSplit.java | 53 +
.../datasource/iceberg/IcebergNereidsUtils.java | 478 +++++
.../datasource/iceberg/IcebergTransaction.java | 171 +-
.../iceberg/action/BaseIcebergAction.java | 22 +-
.../action/IcebergCherrypickSnapshotAction.java | 5 +-
...ctory.java => IcebergExecuteActionFactory.java} | 26 +-
.../action/IcebergExpireSnapshotsAction.java | 6 +-
.../iceberg/action/IcebergFastForwardAction.java | 5 +-
.../action/IcebergRewriteDataFilesAction.java | 93 +-
.../action/IcebergRollbackToSnapshotAction.java | 5 +-
.../action/IcebergRollbackToTimestampAction.java | 5 +-
.../action/IcebergSetCurrentSnapshotAction.java | 5 +-
.../iceberg/rewrite/RewriteDataFileExecutor.java | 213 +++
.../iceberg/rewrite/RewriteDataFilePlanner.java | 362 ++++
.../iceberg/rewrite/RewriteDataGroup.java | 112 ++
.../iceberg/rewrite/RewriteGroupTask.java | 266 +++
.../datasource/iceberg/rewrite/RewriteResult.java | 111 ++
.../datasource/iceberg/source/IcebergScanNode.java | 34 +-
.../property/constants/RemoteDorisProperties.java | 50 +
.../property/storage/OSSHdfsProperties.java | 39 +-
.../datasource/property/storage/OSSProperties.java | 13 +-
.../property/storage/StorageProperties.java | 19 +-
.../org/apache/doris/httpv2/rest/HealthAction.java | 6 +-
.../doris/httpv2/rest/TableSchemaAction.java | 52 +
.../httpv2/rest/response/GsonSchemaResponse.java | 21 +-
.../org/apache/doris/mtmv/MTMVRewriteUtil.java | 16 +-
.../org/apache/doris/nereids/StatementContext.java | 22 +
.../nereids/analyzer/UnboundBlackholeSink.java | 158 ++
.../glue/translator/PhysicalPlanTranslator.java | 15 +
.../doris/nereids/parser/LogicalPlanBuilder.java | 160 +-
.../nereids/properties/RequestPropertyDeriver.java | 8 +
.../org/apache/doris/nereids/rules/RuleSet.java | 2 +
.../org/apache/doris/nereids/rules/RuleType.java | 2 +
.../doris/nereids/rules/analysis/BindRelation.java | 1 +
.../doris/nereids/rules/analysis/BindSink.java | 29 +-
.../rules/exploration/mv/PartitionCompensator.java | 43 +-
.../rules/implementation/AggregateStrategies.java | 55 +-
...ogicalBlackholeSinkToPhysicalBlackholeSink.java | 43 +
.../doris/nereids/rules/rewrite/ColumnPruning.java | 12 +-
.../nereids/rules/rewrite/InferPredicates.java | 6 +
.../nereids/rules/rewrite/PullUpPredicates.java | 16 +
.../rules/rewrite/QueryPartitionCollector.java | 2 +
.../functions/ComputeSignatureHelper.java | 111 ++
.../expressions/functions/SearchSignature.java | 46 +-
.../expressions/functions/agg/TopNWeighted.java | 19 +-
.../functions/executable/DateTimeArithmetic.java | 13 +-
.../executable/DateTimeExtractAndTransform.java | 259 ---
.../functions/scalar/Crc32Internal.java | 26 +-
.../trees/expressions/functions/scalar/Date.java | 5 +-
.../expressions/functions/scalar/DateDiff.java | 5 +-
.../expressions/functions/scalar/DateFormat.java | 6 +-
.../expressions/functions/scalar/DayName.java | 5 +-
.../expressions/functions/scalar/DayOfMonth.java | 5 +-
.../expressions/functions/scalar/DayOfWeek.java | 5 +-
.../expressions/functions/scalar/DayOfYear.java | 5 +-
.../expressions/functions/scalar/DaysDiff.java | 8 +-
.../trees/expressions/functions/scalar/Hour.java | 2 -
.../expressions/functions/scalar/HoursDiff.java | 5 +-
.../trees/expressions/functions/scalar/Minute.java | 2 -
.../expressions/functions/scalar/MinutesDiff.java | 5 +-
.../trees/expressions/functions/scalar/Month.java | 4 +-
.../expressions/functions/scalar/MonthName.java | 5 +-
.../expressions/functions/scalar/MonthsDiff.java | 5 +-
.../expressions/functions/scalar/Quarter.java | 5 +-
.../trees/expressions/functions/scalar/Second.java | 2 -
.../expressions/functions/scalar/SecondsDiff.java | 5 +-
.../expressions/functions/scalar/Timestamp.java | 16 +-
.../trees/expressions/functions/scalar/ToDate.java | 6 +-
.../trees/expressions/functions/scalar/ToDays.java | 4 +-
.../expressions/functions/scalar/ToIso8601.java | 7 +-
.../functions/scalar/UnixTimestamp.java | 6 +-
.../trees/expressions/functions/scalar/Week.java | 6 +-
.../expressions/functions/scalar/WeekOfYear.java | 5 +-
.../expressions/functions/scalar/Weekday.java | 5 +-
.../expressions/functions/scalar/WeeksDiff.java | 5 +-
.../trees/expressions/functions/scalar/Year.java | 5 +-
.../expressions/functions/scalar/YearWeek.java | 6 +-
.../expressions/functions/scalar/YearsDiff.java | 5 +-
.../trees/expressions/functions/window/Lag.java | 2 +-
.../trees/expressions/functions/window/Lead.java | 4 +-
.../trees/expressions/literal/TimeV2Literal.java | 61 +
.../literal/format/DateTimeChecker.java | 12 +-
.../expressions/literal/format/TimeChecker.java | 71 +
.../apache/doris/nereids/trees/plans/PlanType.java | 6 +
...TableCommand.java => ExecuteActionCommand.java} | 86 +-
.../trees/plans/commands/UpdateCommand.java | 22 +-
.../BaseExecuteAction.java} | 11 +-
.../ExecuteAction.java} | 16 +-
.../ExecuteActionFactory.java} | 24 +-
.../commands/insert/AbstractInsertExecutor.java | 35 +
.../commands/insert/BlackholeInsertExecutor.java | 91 +
.../commands/insert/IcebergRewriteExecutor.java | 60 +
.../commands/insert/InsertIntoTableCommand.java | 42 +-
.../trees/plans/commands/insert/InsertUtils.java | 3 +
.../plans/commands/insert/RewriteTableCommand.java | 326 ++++
.../plans/commands/insert/WarmupSelectCommand.java | 228 +++
.../plans/commands/merge/MergeIntoCommand.java | 530 ++++++
.../plans/commands/merge/MergeMatchedClause.java | 55 +
.../commands/merge/MergeNotMatchedClause.java | 55 +
.../trees/plans/logical/LogicalBlackholeSink.java | 88 +
.../plans/physical/PhysicalBlackholeSink.java | 144 ++
.../trees/plans/visitor/CommandVisitor.java | 11 +-
.../nereids/trees/plans/visitor/SinkVisitor.java | 17 +
.../doris/nereids/util/TypeCoercionUtils.java | 16 +-
.../org/apache/doris/persist/gson/GsonUtils.java | 4 +-
.../org/apache/doris/planner/BlackholeSink.java | 61 +
.../org/apache/doris/planner/OlapScanNode.java | 6 +-
.../org/apache/doris/qe/NereidsCoordinator.java | 4 +
.../java/org/apache/doris/qe/SessionVariable.java | 20 +
.../WorkloadRuntimeStatusMgr.java | 54 +-
.../org/apache/doris/statistics/DeriveFactory.java | 1 +
.../apache/doris/statistics/OlapAnalysisTask.java | 2 +
.../apache/doris/statistics/StatisticalType.java | 3 +-
.../doris/blockrule/SqlBlockRuleMgrTest.java | 42 +
.../doris/RemoteDorisCompatibleRestClientTest.java | 72 +
.../doris/RemoteDorisRestClientTest.java | 108 ++
.../iceberg/IcebergNereidsUtilsTest.java | 1004 +++++++++++
.../rewrite/RewriteDataFilePlannerTest.java | 1165 ++++++++++++
.../property/storage/OSSHdfsPropertiesTest.java | 67 +-
.../property/storage/OSSPropertiesTest.java | 2 +-
.../external/elasticsearch/QueryBuildersTest.java | 32 +
.../doris/nereids/parser/NereidsParserTest.java | 241 ++-
.../nereids/rules/expression/FoldConstantTest.java | 95 +-
.../nereids/rules/rewrite/InferPredicatesTest.java | 54 +
.../functions/ComputeSignatureHelperTest.java | 40 +
.../expressions/literal/DateTimeLiteralTest.java | 1 -
.../literal/format/TimeCheckerTest.java | 61 +
.../plans/commands/merge/MergeIntoCommandTest.java | 959 ++++++++++
.../doris/nereids/util/TypeCoercionUtilsTest.java | 9 +-
.../doris/statistics/OlapAnalysisTaskTest.java | 5 +
gensrc/proto/data.proto | 1 +
gensrc/thrift/DataSinks.thrift | 5 +
gensrc/thrift/Descriptors.thrift | 8 +
gensrc/thrift/FrontendService.thrift | 9 +-
gensrc/thrift/PaloInternalService.thrift | 4 +
gensrc/thrift/PlanNodes.thrift | 10 +
gensrc/thrift/Types.thrift | 3 +-
regression-test/conf/regression-conf.groovy | 1 +
.../test_function_signature_all_types.out | 553 ++++++
.../sql-functions/doc_date_functions_test.out | 21 +-
...ctions.out => test_iceberg_execute_actions.out} | 3 -
.../action/test_iceberg_rewrite_data_files.out | 70 +
...iceberg_rewrite_data_files_where_conditions.out | 38 +
.../test_remote_doris_all_types_select.out | 16 +
.../test_remote_doris_all_types_show.out | 76 +
.../remote_doris/test_remote_doris_refresh.out | 16 +
.../remote_doris/test_remote_doris_statistics.out | 20 +
.../data/load_p0/merge_into/test_merge_into.out | 21 +
.../data/load_p0/spark_load/all_types1.txt | 11 -
.../data/load_p0/spark_load/all_types2.txt | 11 -
.../data/load_p0/spark_load/test_spark_load.out | 37 -
.../data/nereids_function_p0/scalar_function/S.out | 173 +-
.../data/nereids_rules_p0/sumRewrite.out | 58 +-
.../data/nereids_syntax_p0/lag_lead_signature.out | 25 +
.../data/nereids_syntax_p0/mv/where/k123/k123.out | 4 +
...t_auto_partition_with_single_replica_insert.csv | 78 -
...t_auto_partition_with_single_replica_insert.out | 161 --
...st_materialized_view_common_expr_push_down.out} | 6 +-
.../test_non_overlap_seg_heavy_sc.out | 5 +-
.../tpcds_sf100/noStatsRfPrune/query38.out | 78 +-
.../tpcds_sf100/noStatsRfPrune/query87.out | 78 +-
.../tpcds_sf100/no_stats_shape/query38.out | 78 +-
.../tpcds_sf100/no_stats_shape/query87.out | 78 +-
.../shape_check/tpcds_sf100/rf_prune/query38.out | 78 +-
.../shape_check/tpcds_sf100/rf_prune/query87.out | 78 +-
.../data/shape_check/tpcds_sf100/shape/query38.out | 78 +-
.../data/shape_check/tpcds_sf100/shape/query87.out | 78 +-
.../data/shape_check/tpcds_sf1000/hint/query38.out | 78 +-
.../data/shape_check/tpcds_sf1000/hint/query87.out | 78 +-
.../shape_check/tpcds_sf1000/shape/query38.out | 78 +-
.../shape_check/tpcds_sf1000/shape/query87.out | 78 +-
.../pipeline/external/conf/regression-conf.groovy | 7 +
.../plugins/plugin_cluster_version.groovy | 46 +-
.../suites/ann_index_p0/ann_index_only_scan.groovy | 6 +-
.../ann_index_only_scan_distance_expr.groovy | 5 +-
.../ann_index_only_scan_metric_direction.groovy | 5 +-
.../auth_call/test_dml_stream_load_auth.groovy | 21 +-
.../suites/auth_p0/test_grant_auth.groovy | 14 +
.../check_before_quit/check_before_quit.groovy | 12 +-
.../warm_up_select/test_warmup_select.groovy | 113 ++
.../test_function_signature_all_types.groovy | 1044 +++++++++++
.../sql-functions/doc_date_functions_test.groovy | 63 +
.../cache/test_hive_warmup_select.groovy | 156 ++
....groovy => test_iceberg_execute_actions.groovy} | 179 +-
.../action/test_iceberg_rewrite_data_files.groovy | 495 +++++
...rewrite_data_files_expression_conversion.groovy | 430 +++++
...berg_rewrite_data_files_where_conditions.groovy | 152 ++
.../test_remote_doris_all_types_select.groovy | 172 ++
.../test_remote_doris_all_types_show.groovy | 168 ++
.../remote_doris/test_remote_doris_catalog.groovy | 68 +
.../remote_doris/test_remote_doris_predict.groovy | 150 ++
.../remote_doris/test_remote_doris_refresh.groovy | 128 ++
.../test_remote_doris_statistics.groovy | 105 ++
.../test_remote_doris_table_stats.groovy | 99 +
.../tvf/test_s3_tvf_number_range.groovy | 16 +-
.../test_load_back_pressure_version.groovy | 46 +
.../jsonb_p0/test_jsonb_predict_is_null.groovy | 5 +
.../load_p0/merge_into/test_merge_into.groovy | 254 +++
.../load_p0/spark_load/test_spark_load.groovy | 150 --
.../nereids_function_p0/scalar_function/S.groovy | 3 +
.../explain/test_pushdown_explain.groovy | 34 +
.../mv/nested_mtmv/nested_mtmv.groovy | 85 +
.../nereids_syntax_p0/lag_lead_signature.groovy | 31 +-
.../nereids_syntax_p0/mv/where/k123/k123.groovy | 11 +
...uto_partition_with_single_replica_insert.groovy | 78 -
.../binary_functions/test_binary_function.groovy | 2 +-
.../test_binary_for_digest.groovy | 2 +-
..._materialized_view_common_expr_push_down.groovy | 76 +
.../test_non_overlap_seg_heavy_sc.groovy | 52 +-
.../suites/temp_table_p0/test_temp_table.groovy | 6 +-
run-be-ut.sh | 2 +-
tools/fdb/fdb_ctl.sh | 5 +-
413 files changed, 20836 insertions(+), 3879 deletions(-)
create mode 100644 be/src/pipeline/exec/blackhole_sink_operator.cpp
create mode 100644 be/src/pipeline/exec/blackhole_sink_operator.h
create mode 100644 be/src/vec/exec/format/table/remote_doris_reader.cpp
create mode 100644 be/src/vec/exec/format/table/remote_doris_reader.h
delete mode 100644
be/src/vec/functions/function_date_or_datetime_computation_v2.cpp
create mode 100644 be/test/geo/wkb_parse_test.cpp
create mode 100644 be/test/io/client/s3_file_system_test.cpp
create mode 100644 be/test/vec/data_types/serde/data_type_serde_map_test.cpp
create mode 100644 be/test/vec/data_types/serde/data_type_serde_struct_test.cpp
create mode 100644
docker/thirdparties/docker-compose/iceberg/scripts/create_preinstalled_scripts/iceberg/run21.sql
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/doris/RemoteDorisCompatibleRestClient.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/doris/RemoteDorisExternalCatalog.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/doris/RemoteDorisExternalDatabase.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/doris/RemoteDorisExternalTable.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/doris/RemoteDorisRestClient.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/doris/source/RemoteDorisScanNode.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/doris/source/RemoteDorisSource.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/doris/source/RemoteDorisSplit.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/iceberg/IcebergNereidsUtils.java
rename
fe/fe-core/src/main/java/org/apache/doris/datasource/iceberg/action/{IcebergOptimizeActionFactory.java
=> IcebergExecuteActionFactory.java} (84%)
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/iceberg/rewrite/RewriteDataFileExecutor.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/iceberg/rewrite/RewriteDataFilePlanner.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/iceberg/rewrite/RewriteDataGroup.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/iceberg/rewrite/RewriteGroupTask.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/iceberg/rewrite/RewriteResult.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/property/constants/RemoteDorisProperties.java
copy be/src/geo/wkt_parse_ctx.h =>
fe/fe-core/src/main/java/org/apache/doris/httpv2/rest/response/GsonSchemaResponse.java
(76%)
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/analyzer/UnboundBlackholeSink.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/implementation/LogicalBlackholeSinkToPhysicalBlackholeSink.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/literal/format/TimeChecker.java
rename
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/{OptimizeTableCommand.java
=> ExecuteActionCommand.java} (65%)
rename
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/{optimize/BaseOptimizeAction.java
=> execute/BaseExecuteAction.java} (95%)
rename
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/{optimize/OptimizeAction.java
=> execute/ExecuteAction.java} (88%)
rename
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/{optimize/OptimizeActionFactory.java
=> execute/ExecuteActionFactory.java} (75%)
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/insert/BlackholeInsertExecutor.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/insert/IcebergRewriteExecutor.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/insert/RewriteTableCommand.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/insert/WarmupSelectCommand.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/merge/MergeIntoCommand.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/merge/MergeMatchedClause.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/merge/MergeNotMatchedClause.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/logical/LogicalBlackholeSink.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/physical/PhysicalBlackholeSink.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/planner/BlackholeSink.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/blockrule/SqlBlockRuleMgrTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/datasource/doris/RemoteDorisCompatibleRestClientTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/datasource/doris/RemoteDorisRestClientTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/datasource/iceberg/IcebergNereidsUtilsTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/datasource/iceberg/rewrite/RewriteDataFilePlannerTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/nereids/trees/expressions/literal/format/TimeCheckerTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/nereids/trees/plans/commands/merge/MergeIntoCommandTest.java
create mode 100644
regression-test/data/correctness_p0/test_function_signature_all_types.out
rename
regression-test/data/external_table_p0/iceberg/action/{test_iceberg_optimize_actions.out
=> test_iceberg_execute_actions.out} (96%)
create mode 100644
regression-test/data/external_table_p0/iceberg/action/test_iceberg_rewrite_data_files.out
create mode 100644
regression-test/data/external_table_p0/iceberg/action/test_iceberg_rewrite_data_files_where_conditions.out
create mode 100644
regression-test/data/external_table_p0/remote_doris/test_remote_doris_all_types_select.out
create mode 100644
regression-test/data/external_table_p0/remote_doris/test_remote_doris_all_types_show.out
create mode 100644
regression-test/data/external_table_p0/remote_doris/test_remote_doris_refresh.out
create mode 100644
regression-test/data/external_table_p0/remote_doris/test_remote_doris_statistics.out
create mode 100644 regression-test/data/load_p0/merge_into/test_merge_into.out
delete mode 100644 regression-test/data/load_p0/spark_load/all_types1.txt
delete mode 100644 regression-test/data/load_p0/spark_load/all_types2.txt
delete mode 100644 regression-test/data/load_p0/spark_load/test_spark_load.out
create mode 100644
regression-test/data/nereids_syntax_p0/lag_lead_signature.out
delete mode 100644
regression-test/data/partition_p0/auto_partition/test_auto_partition_with_single_replica_insert.csv
delete mode 100644
regression-test/data/partition_p0/auto_partition/test_auto_partition_with_single_replica_insert.out
copy regression-test/data/{schema_change_p0/test_non_overlap_seg_heavy_sc.out
=> rollup_p0/test_materialized_view_common_expr_push_down.out} (86%)
create mode 100644
regression-test/suites/cloud_p0/warm_up_select/test_warmup_select.groovy
create mode 100644
regression-test/suites/correctness_p0/test_function_signature_all_types.groovy
create mode 100644
regression-test/suites/external_table_p0/cache/test_hive_warmup_select.groovy
rename
regression-test/suites/external_table_p0/iceberg/action/{test_iceberg_optimize_actions.groovy
=> test_iceberg_execute_actions.groovy} (76%)
create mode 100644
regression-test/suites/external_table_p0/iceberg/action/test_iceberg_rewrite_data_files.groovy
create mode 100644
regression-test/suites/external_table_p0/iceberg/action/test_iceberg_rewrite_data_files_expression_conversion.groovy
create mode 100644
regression-test/suites/external_table_p0/iceberg/action/test_iceberg_rewrite_data_files_where_conditions.groovy
create mode 100644
regression-test/suites/external_table_p0/remote_doris/test_remote_doris_all_types_select.groovy
create mode 100644
regression-test/suites/external_table_p0/remote_doris/test_remote_doris_all_types_show.groovy
create mode 100644
regression-test/suites/external_table_p0/remote_doris/test_remote_doris_catalog.groovy
create mode 100644
regression-test/suites/external_table_p0/remote_doris/test_remote_doris_predict.groovy
create mode 100644
regression-test/suites/external_table_p0/remote_doris/test_remote_doris_refresh.groovy
create mode 100644
regression-test/suites/external_table_p0/remote_doris/test_remote_doris_statistics.groovy
create mode 100644
regression-test/suites/external_table_p0/remote_doris/test_remote_doris_table_stats.groovy
create mode 100644
regression-test/suites/fault_injection_p0/test_load_back_pressure_version.groovy
create mode 100644
regression-test/suites/load_p0/merge_into/test_merge_into.groovy
delete mode 100644
regression-test/suites/load_p0/spark_load/test_spark_load.groovy
delete mode 100644
regression-test/suites/partition_p0/auto_partition/test_auto_partition_with_single_replica_insert.groovy
create mode 100644
regression-test/suites/rollup_p0/test_materialized_view_common_expr_push_down.groovy
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
