This is an automated email from the ASF dual-hosted git repository.
dataroaring pushed a change to branch auto-pick-50578-branch-3.0
in repository https://gitbox.apache.org/repos/asf/doris.git
discard c18be4075a5 [fix](iceberg)Fix the inconsistency between the data in pg
and the data in MinIO. (#50578)
add 74d60fe75b1 branch-3.0: [improvement](test) if run cloud mode, give
user usage priv on comput… #50613 (#50639)
add c7010e1a910 branch-3.0: [feat](thirdparty) Add aws dentity-management
and sts library for be and ry #49565 (#49683)
add 98c3f03d980 branch-3.0: [improve](function) support collect_list with
nested types param #47965 (#48114)
add 6bc495f5462 branch-3.0: [Bug](agg-state) fix agg state inner type not
conversioned #50418 (#50575)
add 8d939813ca1 branch-3.0: [fix](inverted index) Fix for
test_clean_stale_rs_index_file_cache case issue #50289 (#50594)
add f7f36745431 [branch-3.0](function) fix wrong floor of function
date_diff when unit less than day (#49429) (#50608)
add a1d30fc909c branch-3.0[fix](nereids) common-sub-expression opt: avoid
computing alias repeatedly (#45043) (#50359)
add a220f96a3a4 branch-3.0: [feat](binlog) add modify distribution bucket
num binlog #49894 (#50068)
add b4a80b294d1 [branch-3.0](function) Fix unexpected result of
unix_timestamp when input is out of bound (#49430) (#50610)
add 5615d7d781d branch-3.0: [fix](nereids) fix bug of
MergePercentileToArray #49773 (#50602)
add d87bf7aa437 branch-3.0: [opt](lambda) let lambda expression support
refer outer slot #44365 (#49394)
add d3e8f93ef27 branch-3.0: [fix](compaction)init
_cumulative_compaction_policies to avoid out_of_range #49767 (#50581)
add 33cbf776642 branch-3.0: [fix](nereids) do eliminate constant group by
key in normalizeagg #49589 (#50215)
add f35cc6a5c5d branch-3.0: [fix](constant fold)Make sure FE cast double
to varchar generate identical result with BE. #50425 (#50666)
add 54fb444b4e4 branch-3.0: [fix](function) fix error result when input
utf8 in url_encode, strright, append_trailing_char_if_absent #49127 (#50659)
add e9d14da87cf branch-3.0: [fix](Nereids) fix unix_timestamp #49686
(#50670)
add 663ef81ffe9 [enhancement](cloud) make file cache background task
interval configu… (#50646)
add 6862d2ed7d6 branch-3.0: [fix](planner) return explicit error msg when
falling back to old planner (#50677)
add 778bb38a7d1 branch-3.0: [fix](binlog) Acquire migration lock before
ingesting binlog #50663 (#50708)
add 7dbec0d9b2f [bugfix](becore) pipeline core when workload group
shutdown (#50743)
add 329b2995e27 branch-3.0: [fix](regression)Fix test analyze mv case.
#50701 (#50731)
add b33d1a8a0c1 branch-3.0: [enhance](mtmv)Exclude trigger table support
db and catalog #49961 (#50553)
add eaa8209ab5d branch-3.0: [fix](memory) Fix `PODArray::add_num_element`
#50756 (#50781)
add 99f4124f251 branch-3.0: [improve](udf) support list/map basic class
for array/map type #50684 (#50752)
add e3e3f8f3d08 [improvement](test) if run cloud mode, give user usage
priv on comput… (#50642)
add 6f570edd539 branch-3.0: [fix](http) remove file before downloading
#50754 (#50824)
add 1d697dcf45a branch-3.0: [improvement](regression)Add log to print jdbc
url in prepare stmt test #50711 (#50728)
add 338c256a6bc [refactor](vault) Refine create storage vault process on
FE and fix missing S3 vault root path (#50395) (#50790)
add 70a44489e23 branch-3.0: [Fix](cloud-mow) Rollup task should retry when
encouter TXN_CONFILCT in cloud mode #50705 (#50836)
add 4bc9ddcb271 branch-3.0: [Fix](recycler) Fix transaction label
recycling to prevent key cleanup failures and 'key not found' errors #50766
(#50814)
add 35084362516 branch-3.0: [fix](regress) fix join_condition #50719
(#50739)
add 6ef9a8ca330 branch-3.0: [Fix](group commit) Fix group commit null map
#50713 (#50813)
add 03c1235d9a2 branch-3.0: [chore](cloud) Remove set -eo pipefail when
start ms #50679 (#50693)
add 1b102c118b0 branch-3.0: [Opt](profile) More profile for
`OlapScanLocalState::hold_tablets` #50564 (#50725)
add f8a4636c007 branch-3.0: [feature](restore) introduce
AgentBoundedBatchTask to manage concurrent restore tasks #50740 (#50843)
add a5103823fbd branch-3.0: [feat](hive) add catalog level partition cache
property #50724 (#50761)
add bec367b719d [fix](nereids) fix fold constant return wrong scale of
datetime type (#50142) (#50717)
add aecc93c002c [fix](information_schema) fix backend_active_tasks table
only return one backend's data (#50721) (#50723)
add 60d5073f76a branch-3.0: [chore](binlog) allow reading binlogs from
follower #50714 (#50844)
add f21b1450d54 branch-3.0: [fix](storage vault) Fix two storage vault
regression cases #50703 (#50862)
add 7200d038287 branch-3.0: [fix](cooldown) allow cooldown_ttl = 0 when
altering storage policy #50830 (#50854)
add 27440b40195 branch-3.0: [Fix](regression-test) fix
test_export_max_file_size case #50795 (#50809)
add df67c7a9341 branch-3.0: [opt](jdbc scan) Add more jdbc scan profile
items #46460 (#50800)
add 59aec80398d branch-3.0: [fix](ut) fix unstable FE ut case for schema
change job #50694 (#50771)
add 58412da5ee6 branch-3.0: [fix](case) adjust
remote_fragment_exec_timeout_ms to avoid unstable … #50801 (#50806)
add 87b9d34744c [Fix](orc-reader) Fixed issue with top level struct column
having present stream failing to access repeatedly when late materialization
occurs. (#50652)
add b89cfe149cb branch-3.0: [fix](mtmv)The change to the
max_persistence_task_count configuration applies retroactively to existing MTMV
#50537 (#50626)
add 9d4ecf7c6df branch-3.0: [fix](jdbc catalog) Improve conjunct
expression handling in JdbcScanNode #50542 (#50647)
add 669f2dfc98d branch-3.0 [opt](nereids) catch all exceptions in
StatsCalculator (#49415) (#50360)
add 39542191c23 [Improve](explode) explode function support multi param
(#50793)
add 4e3b99f43e3 branch-3.0: [bugfix](nerids) align locate function
behavior with BE side #50797 (#50831)
add 76f97b19569 [fix](iceberg)Fix the inconsistency between the data in pg
and the data in MinIO. (#50578)
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (c18be4075a5)
\
N -- N -- N refs/heads/auto-pick-50578-branch-3.0 (76f97b19569)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
be/src/apache-orc | 2 +-
be/src/cloud/cloud_meta_mgr.cpp | 14 +-
be/src/cloud/cloud_tablet.h | 8 +-
be/src/cloud/cloud_tablet_mgr.cpp | 15 +-
be/src/http/http_client.cpp | 7 +
be/src/io/cache/block_file_cache.cpp | 8 +-
be/src/olap/olap_server.cpp | 1 +
be/src/olap/rowset/segment_v2/segment_iterator.cpp | 10 +-
be/src/pipeline/exec/olap_scan_operator.cpp | 36 +-
be/src/pipeline/exec/olap_scan_operator.h | 4 +
be/src/pipeline/exec/table_function_operator.cpp | 7 +-
be/src/pipeline/task_queue.cpp | 2 -
be/src/service/backend_service.cpp | 21 +-
be/src/util/datetype_cast.hpp | 8 +-
be/src/util/url_coding.cpp | 4 +-
.../aggregate_function_collect.cpp | 8 +
.../aggregate_function_collect.h | 70 ++-
.../aggregate_function_sequence_match.h | 10 +-
be/src/vec/common/pod_array.h | 6 +-
be/src/vec/exec/scan/new_jdbc_scanner.cpp | 14 +-
be/src/vec/exec/scan/new_jdbc_scanner.h | 6 +-
be/src/vec/exec/vjdbc_connector.cpp | 57 ++-
be/src/vec/exec/vjdbc_connector.h | 6 +-
be/src/vec/exprs/lambda_function/lambda_function.h | 9 +-
.../exprs/lambda_function/varray_map_function.cpp | 212 +++++++--
.../table_function/table_function_factory.cpp | 19 +-
.../exprs/table_function/table_function_factory.h | 17 +-
be/src/vec/exprs/table_function/vexplode_v2.cpp | 246 ++++++++++
.../table_function/{vexplode.h => vexplode_v2.h} | 28 +-
be/src/vec/exprs/vcolumn_ref.h | 11 +-
be/src/vec/exprs/vexpr.h | 3 +
be/src/vec/exprs/vlambda_function_call_expr.h | 1 +
.../vec/functions/array/function_array_range.cpp | 8 +-
be/src/vec/functions/date_time_transforms.h | 28 +-
.../function_date_or_datetime_computation.h | 8 +-
.../vec/functions/function_datetime_floor_ceil.cpp | 6 +-
be/src/vec/functions/function_fake.cpp | 68 ++-
be/src/vec/functions/function_string.cpp | 73 +--
be/src/vec/functions/function_string.h | 6 +-
be/src/vec/functions/function_timestamp.cpp | 3 +-
be/src/vec/functions/simple_function_factory.h | 17 +-
be/src/vec/runtime/vdatetime_value.cpp | 4 +-
be/src/vec/runtime/vdatetime_value.h | 104 +++--
be/test/io/cache/block_file_cache_test.cpp | 14 +-
be/test/vec/common/pod_array_test.cpp | 26 ++
be/test/vec/function/function_string_test.cpp | 24 +
be/test/vec/function/function_test_util.cpp | 125 +++++-
be/test/vec/function/function_test_util.h | 3 +-
be/test/vec/function/table_function_test.cpp | 163 ++++++-
be/test/vec/runtime/vdatetime_value_test.cpp | 203 +++++++++
cloud/script/start.sh | 2 -
cloud/src/recycler/recycler.cpp | 8 +-
cloud/test/recycler_test.cpp | 55 +++
.../doris/common/jni/utils/JavaUdfDataType.java | 4 +-
.../main/java/org/apache/doris/catalog/Type.java | 8 +-
.../main/java/org/apache/doris/common/Config.java | 11 +-
.../java/org/apache/doris/alter/RollupJobV2.java | 13 +-
.../org/apache/doris/alter/SchemaChangeJobV2.java | 5 +-
.../doris/analysis/CreateStorageVaultStmt.java | 13 +-
.../java/org/apache/doris/analysis/TableName.java | 25 ++
.../java/org/apache/doris/backup/RestoreJob.java | 42 +-
.../org/apache/doris/binlog/BinlogManager.java | 17 +
.../catalog/BuiltinTableGeneratingFunctions.java | 11 +
.../org/apache/doris/catalog/DistributionInfo.java | 8 +
.../main/java/org/apache/doris/catalog/Env.java | 5 +-
.../apache/doris/catalog/HashDistributionInfo.java | 15 +-
.../org/apache/doris/catalog/HdfsStorageVault.java | 74 +--
.../main/java/org/apache/doris/catalog/MTMV.java | 11 +-
.../doris/catalog/RandomDistributionInfo.java | 5 +
.../org/apache/doris/catalog/S3StorageVault.java | 35 +-
.../org/apache/doris/catalog/StorageVault.java | 23 +-
.../org/apache/doris/catalog/StorageVaultMgr.java | 10 +-
.../doris/datasource/hive/HMSExternalCatalog.java | 29 +-
.../doris/datasource/hive/HiveMetaStoreCache.java | 17 +-
.../doris/datasource/hive/source/HiveScanNode.java | 4 +
.../doris/datasource/jdbc/source/JdbcScanNode.java | 78 +++-
.../org/apache/doris/load/GroupCommitManager.java | 4 +-
.../java/org/apache/doris/master/MasterImpl.java | 4 +
.../java/org/apache/doris/mtmv/MTMVJobInfo.java | 2 +-
.../org/apache/doris/mtmv/MTMVPartitionUtil.java | 51 ++-
.../java/org/apache/doris/mtmv/MTMVService.java | 13 +-
.../org/apache/doris/nereids/analyzer/Scope.java | 11 +-
.../glue/translator/PhysicalPlanTranslator.java | 2 +-
.../doris/nereids/jobs/executor/Analyzer.java | 3 -
.../doris/nereids/jobs/executor/Rewriter.java | 2 -
.../doris/nereids/parser/LogicalPlanBuilder.java | 10 +-
.../processor/post/CommonSubExpressionOpt.java | 16 +-
.../nereids/rules/analysis/BindExpression.java | 2 +-
.../nereids/rules/analysis/ExpressionAnalyzer.java | 5 +-
.../nereids/rules/analysis/NormalizeAggregate.java | 108 ++++-
.../nereids/rules/analysis/SubExprAnalyzer.java | 11 +-
.../expression/rules/FoldConstantRuleOnFE.java | 52 ++-
.../rules/SimplifyConditionalFunction.java | 51 ++-
.../rules/rewrite/MergePercentileToArray.java | 34 +-
.../doris/nereids/stats/ExpressionEstimation.java | 22 +-
.../doris/nereids/stats/StatsCalculator.java | 24 +-
.../executable/DateTimeExtractAndTransform.java | 8 +-
.../functions/executable/StringArithmetic.java | 23 +-
.../expressions/functions/generator/Explode.java | 61 ++-
.../functions/generator/ExplodeOuter.java | 61 ++-
.../functions/generator/ExplodeVariantArray.java | 57 ++-
.../trees/expressions/literal/DoubleLiteral.java | 30 ++
.../doris/nereids/trees/plans/GroupPlan.java | 2 +-
.../plans/commands/AlterStorageVaultCommand.java | 4 +-
.../apache/doris/nereids/types/AggStateType.java | 6 +
.../doris/nereids/util/TypeCoercionUtils.java | 22 +
.../java/org/apache/doris/persist/EditLog.java | 7 +-
...leDefaultDistributionBucketNumOperationLog.java | 29 +-
.../planner/BackendPartitionedSchemaScanNode.java | 12 +-
.../apache/doris/planner/SingleNodePlanner.java | 4 +-
.../org/apache/doris/policy/StoragePolicy.java | 4 +-
.../java/org/apache/doris/qe/SessionVariable.java | 7 +
.../apache/doris/service/FrontendServiceImpl.java | 52 ++-
.../apache/doris/statistics/ColumnStatistic.java | 30 ++
.../java/org/apache/doris/task/AgentBatchTask.java | 42 +-
.../apache/doris/task/AgentBoundedBatchTask.java | 281 ++++++++++++
.../apache/doris/catalog/MetadataTableTest.java | 61 +++
.../doris/cloud/catalog/HdfsStorageVaultTest.java | 23 +-
.../datasource/jdbc/source/JdbcScanNodeTest.java | 496 +++++++++++++++++++++
.../org/apache/doris/mtmv/MTMVJobInfoTest.java | 45 ++
.../apache/doris/mtmv/MTMVPartitionUtilTest.java | 78 ++++
.../org/apache/doris/mtmv/MTMVRewriteUtilTest.java | 11 +-
.../java/org/apache/doris/mtmv/MTMVTaskTest.java | 7 +-
.../test/java/org/apache/doris/mtmv/MTMVTest.java | 31 ++
.../analysis/EliminateGroupByConstantTest.java | 165 -------
.../rules/analysis/NormalizeAggregateTest.java | 120 ++++-
.../nereids/rules/expression/FoldConstantTest.java | 20 +-
.../rules/SimplifyConditionalFunctionTest.java | 35 ++
.../rules/rewrite/MergePercentileToArrayTest.java | 13 +
.../nereids/stats/ExpressionEstimationTest.java | 9 +
.../functions/generator/ExplodeOuterTest.java | 95 ++++
.../functions/generator/ExplodeTest.java | 95 ++++
.../generator/ExplodeVariantArrayTest.java | 68 +++
.../expressions/literal/DoubleLiteralTest.java | 49 ++
gensrc/thrift/FrontendService.thrift | 5 +-
.../agg_state/decimal_sum/test_decimal_sum.out} | Bin 126 -> 130 bytes
.../hive/test_hive_meta_cache.out | Bin 0 -> 858 bytes
.../jdbc/test_oracle_jdbc_catalog.out | Bin 7894 -> 10560 bytes
.../cloud/test_cloud_mow_sync_mv.out} | Bin 419 -> 454 bytes
.../data/javaudf_p0/test_javaudf_array.out | Bin 1725 -> 1891 bytes
.../data/javaudf_p0/test_javaudf_map.out | Bin 279 -> 402 bytes
.../test_excluded_trigger_table_mtmv.out} | Bin 294 -> 333 bytes
.../datetime_functions/test_date_function.out | Bin 9900 -> 9997 bytes
.../datetime_functions/test_date_function_v2.out | Bin 0 -> 430 bytes
.../eliminate_constant_gby_key.out | Bin 0 -> 3009 bytes
.../merge_percentile_to_array.out | Bin 723 -> 2204 bytes
.../data/query_p0/aggregate/array_agg.out | Bin 8211 -> 37357 bytes
.../array_functions/test_array_functions.out | Bin 28028 -> 28005 bytes
.../test_array_map_function_with_column.out | Bin 0 -> 1083 bytes
.../sql_functions/table_function/explode.out | Bin 25904 -> 27843 bytes
.../udf/{BigintTest.java => ArrayListTest.java} | 8 +-
.../doris/udf/{BigintTest.java => MapsiTest.java} | 8 +-
.../pipeline/cloud_p0/conf/fe_custom.conf | 2 +-
regression-test/pipeline/p0/conf/fe.conf | 2 +-
.../test_clean_stale_rs_index_file_cache.groovy | 1 -
.../agg_state/decimal_sum/test_decimal_sum.groovy} | 32 +-
.../export_p2/test_export_max_file_size.groovy | 24 +-
.../export/hive_read/orc/test_hive_read_orc.groovy | 2 +-
.../hive/test_hive_meta_cache.groovy | 235 ++++++++++
.../hive/test_hive_star_qualifier.groovy | 2 +-
.../hive/test_parquet_nested_types.groovy | 8 +-
.../iceberg/iceberg_drop_rest_table.groovy | 8 +
.../jdbc/test_oracle_jdbc_catalog.groovy | 64 +++
.../cloud/test_cloud_mow_sync_mv.groovy | 115 +++++
.../suites/javaudf_p0/test_javaudf_array.groovy | 9 +
.../suites/javaudf_p0/test_javaudf_map.groovy | 25 ++
.../test_excluded_trigger_table_mtmv.groovy | 135 ++++++
.../fold_constant_date_arithmatic.groovy | 17 +
.../fold_constant_string_arithmatic.groovy | 124 +++++-
.../datetime_functions/test_date_function.groovy | 4 +
.../test_date_function_v2.groovy | 53 +++
.../suites/nereids_rules_p0/cse/cse.groovy | 54 ++-
.../eliminate_constant_gby_key.groovy | 172 +++++++
.../merge_percentile_to_array.groovy | 21 +
.../suites/nereids_syntax_p0/join_condition.groovy | 36 +-
.../suites/nereids_syntax_p0/runtime_filter.groovy | 42 +-
.../suites/prepared_stmt_p0/prepared_stmt.groovy | 1 +
.../suites/query_p0/aggregate/array_agg.groovy | 16 +
.../array_functions/test_array_map_function.groovy | 3 +-
.../test_array_map_function_with_column.groovy | 70 +++
.../datetime_functions/test_convert_tz.groovy | 3 +-
.../sql_functions/table_function/explode.groovy | 51 +++
.../suites/statistics/test_analyze_mv.groovy | 16 +
.../suites/trino_p0/constant_group_key.groovy | 4 +-
.../vault_p0/create/test_create_vault.groovy | 67 ++-
.../create/test_minio_storage_vault.groovy | 34 +-
thirdparty/build-thirdparty.sh | 4 +-
187 files changed, 5474 insertions(+), 874 deletions(-)
create mode 100644 be/src/vec/exprs/table_function/vexplode_v2.cpp
copy be/src/vec/exprs/table_function/{vexplode.h => vexplode_v2.h} (71%)
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/task/AgentBoundedBatchTask.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/catalog/MetadataTableTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/datasource/jdbc/source/JdbcScanNodeTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/mtmv/MTMVJobInfoTest.java
delete mode 100644
fe/fe-core/src/test/java/org/apache/doris/nereids/rules/analysis/EliminateGroupByConstantTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/nereids/trees/expressions/functions/generator/ExplodeOuterTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/nereids/trees/expressions/functions/generator/ExplodeTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/nereids/trees/expressions/functions/generator/ExplodeVariantArrayTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/nereids/trees/expressions/literal/DoubleLiteralTest.java
copy
regression-test/data/{correctness/test_insert_table_with_dump_nereids_memo.out
=> datatype_p0/agg_state/decimal_sum/test_decimal_sum.out} (84%)
create mode 100644
regression-test/data/external_table_p0/hive/test_hive_meta_cache.out
copy regression-test/data/{mv_p0/unique_rewrite/unique_rewrite.out =>
fault_injection_p0/cloud/test_cloud_mow_sync_mv.out} (70%)
copy regression-test/data/{backup_restore/test_backup_restore_alias.out =>
mtmv_p0/test_excluded_trigger_table_mtmv.out} (51%)
create mode 100644
regression-test/data/nereids_p0/sql_functions/datetime_functions/test_date_function_v2.out
create mode 100644
regression-test/data/nereids_rules_p0/eliminate_gby_key/eliminate_constant_gby_key.out
create mode 100644
regression-test/data/query_p0/sql_functions/array_functions/test_array_map_function_with_column.out
copy
regression-test/java-udf-src/src/main/java/org/apache/doris/udf/{BigintTest.java
=> ArrayListTest.java} (86%)
copy
regression-test/java-udf-src/src/main/java/org/apache/doris/udf/{BigintTest.java
=> MapsiTest.java} (85%)
copy regression-test/suites/{correctness/test_set_session_var_exception.groovy
=> datatype_p0/agg_state/decimal_sum/test_decimal_sum.groovy} (64%)
create mode 100644
regression-test/suites/external_table_p0/hive/test_hive_meta_cache.groovy
create mode 100644
regression-test/suites/fault_injection_p0/cloud/test_cloud_mow_sync_mv.groovy
create mode 100644
regression-test/suites/mtmv_p0/test_excluded_trigger_table_mtmv.groovy
create mode 100644
regression-test/suites/nereids_p0/sql_functions/datetime_functions/test_date_function_v2.groovy
create mode 100644
regression-test/suites/nereids_rules_p0/eliminate_gby_key/eliminate_constant_gby_key.groovy
create mode 100644
regression-test/suites/query_p0/sql_functions/array_functions/test_array_map_function_with_column.groovy
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
