This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a change to branch branch-2.1
in repository https://gitbox.apache.org/repos/asf/doris.git
from 41a67fc2182 change to 2.1.0-rc10
new e48f4f38d05 [Fix](fe-common) Fix the Pair.java code about the hidden
danger of NullPointException (#31371)
new 3451cd6c23e [fix](datetime) fix hour 24 on be (#31304)
new 48804a978a1 [Fix](group commit) Fix group commit flink error message
(#31350)
new 3b7261abe4e [Mv] check delete from column exists on mv (#31321)
new 3acffaa2053 [Feature](agg-state) support write_column_to_pb from
DataTypeFixedLengthObjectSerDe (#31171)
new f951ca2efb3 [refactor](stats) Remove useless async loader code.
(#31380)
new b1fc0ebbe71 [improvement](iceberg/paimon)support estimate row count
(#31204)
new fcea2b964e0 [Chore](materialized-view) forbid create mv have
calculations included outside aggregate functions (#31336)
new 60c002ba91e [Docs](Job)Add job Docs (#28186)
new d6f143b7d53 [doc](job)add job doc sidebars (#31319)
new bc41033b5d0 [doc](community) modify the release verify doc (#31385)
new 5d4a2d93a67 [feature](nereids) support join with joinRelation (#30909)
new 17f042b85af [opt](scanner) scan enough blocks in each scan task
(#31277)
new c9f8111ec75 [fix](function) fix unexpected be core in string search
function (#31312)
new e0cc087f379 [feature](doris compose) Add create cloud cluster (#31315)
new 90f36ee1815 [feature](function) support sequence function(alias of
array_range), enhance both to handle datetimev2 (#30823)
new 7b9e500c31a Fix cgroup cpu controller file leak (#31398)
new fbbaf74b778 [Improment](executor)Add scanbytes/scanrows condition
(#31364)
new e41d091a714 [Improvement](executor)add remote scan thread pool (#31376)
The 19 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
be/src/agent/cgroup_cpu_ctl.cpp | 10 +-
be/src/agent/cgroup_cpu_ctl.h | 5 +-
be/src/agent/workload_group_listener.cpp | 2 +-
be/src/agent/workload_sched_policy_listener.cpp | 4 +-
be/src/http/action/http_stream.cpp | 8 +-
be/src/http/action/stream_load.cpp | 8 +-
be/src/http/utils.cpp | 11 +-
be/src/http/utils.h | 2 +-
be/src/runtime/fragment_mgr.cpp | 7 -
be/src/runtime/query_context.cpp | 4 +-
be/src/runtime/query_context.h | 5 +
be/src/runtime/runtime_query_statistics_mgr.cpp | 19 ++
be/src/runtime/runtime_query_statistics_mgr.h | 8 +
be/src/runtime/task_group/task_group.cpp | 172 +++++++++-
be/src/runtime/task_group/task_group.h | 28 ++
be/src/runtime/task_group/task_group_manager.cpp | 171 +---------
be/src/runtime/task_group/task_group_manager.h | 23 +-
.../runtime/workload_management/workload_action.h | 4 +-
.../workload_management/workload_condition.h | 4 +-
.../workload_sched_policy_mgr.cpp | 5 +
be/src/vec/data_types/data_type_agg_state.h | 2 +-
.../vec/data_types/data_type_fixed_length_object.h | 2 +-
.../serde/data_type_fixedlengthobject_serde.cpp | 22 --
.../serde/data_type_fixedlengthobject_serde.h | 110 -------
.../data_types/serde/data_type_string_serde.cpp | 281 ----------------
.../vec/data_types/serde/data_type_string_serde.h | 233 +++++++++++--
be/src/vec/exec/scan/scanner_context.cpp | 152 ++++-----
be/src/vec/exec/scan/scanner_context.h | 30 +-
be/src/vec/exec/scan/scanner_scheduler.cpp | 60 ++--
be/src/vec/exec/scan/scanner_scheduler.h | 34 +-
.../vec/functions/array/function_array_range.cpp | 184 +++++++++--
.../functions/functions_multi_string_position.cpp | 15 +-
.../functions/functions_multi_string_search.cpp | 15 +-
be/src/vec/runtime/vdatetime_value.cpp | 2 +-
be/test/http/stream_load_test.cpp | 12 +-
docker/runtime/doris-compose/Dockerfile | 19 +-
docker/runtime/doris-compose/Readme.md | 31 +-
docker/runtime/doris-compose/cluster.py | 326 ++++++++++++++----
docker/runtime/doris-compose/command.py | 365 +++++++++++++++++----
docker/runtime/doris-compose/doris-compose.py | 2 +-
.../doris-compose/resource/cloud.ini.example | 25 ++
docker/runtime/doris-compose/resource/common.sh | 95 +++++-
docker/runtime/doris-compose/resource/fdb.conf | 45 +++
docker/runtime/doris-compose/resource/init_be.sh | 121 +++++--
.../runtime/doris-compose/resource/init_cloud.sh | 104 ++++++
.../resource/{common.sh => init_fdb.sh} | 65 ++--
docker/runtime/doris-compose/resource/init_fe.sh | 104 +++++-
.../community/release-and-verify/release-verify.md | 2 +-
.../sql-functions/array-functions/array-range.md | 27 +-
.../sql-functions/array-functions/sequence.md | 104 ++++++
.../sql-functions/table-functions/job.md | 96 ++++++
.../sql-functions/table-functions/tasks.md | 69 +++-
.../Data-Definition-Statements/Alter/PAUSE-JOB.md | 14 +-
.../Data-Definition-Statements/Alter/RESUME-JOB.md | 12 +-
.../Create/CREATE-JOB.md | 76 +++--
.../{Alter/STOP-JOB.md => Drop/DROP-JOB.md} | 20 +-
.../sql-reference/Show-Statements/SHOW-JOB-TASK.md | 74 -----
.../sql-reference/Show-Statements/SHOW-JOB.md | 93 ------
docs/sidebars.json | 16 +-
.../community/release-and-verify/release-verify.md | 2 +-
.../sql-functions/array-functions/array-range.md | 27 +-
.../sql-functions/array-functions/sequence.md | 102 ++++++
.../sql-functions/table-functions/job.md | 97 ++++++
.../sql-functions/table-functions/tasks.md | 65 +++-
.../Data-Definition-Statements/Alter/PAUSE-JOB.md | 10 +-
.../Data-Definition-Statements/Alter/RESUME-JOB.md | 10 +-
.../Create/CREATE-JOB.md | 48 +--
.../{Alter/STOP-JOB.md => Drop/DROP-JOB.md} | 14 +-
.../sql-reference/Show-Statements/SHOW-JOB-TASK.md | 75 -----
.../sql-reference/Show-Statements/SHOW-JOB.md | 93 ------
.../main/java/org/apache/doris/common/Config.java | 3 -
.../main/java/org/apache/doris/common/Pair.java | 14 +-
.../java/org/apache/doris/common/PairTest.java | 55 ++++
.../antlr4/org/apache/doris/nereids/DorisLexer.g4 | 2 +
.../antlr4/org/apache/doris/nereids/DorisParser.g4 | 18 +-
.../doris/analysis/CreateMaterializedViewStmt.java | 5 +
.../apache/doris/analysis/FunctionCallExpr.java | 10 +
.../doris/catalog/BuiltinScalarFunctions.java | 2 +-
.../main/java/org/apache/doris/catalog/Env.java | 6 -
.../java/org/apache/doris/catalog/FunctionSet.java | 6 +
.../publish/WorkloadActionPublishThread.java | 123 -------
.../doris/datasource/hive/HMSExternalTable.java | 2 +-
.../datasource/iceberg/IcebergExternalTable.java | 6 +
.../doris/datasource/iceberg/IcebergUtils.java | 34 ++
.../datasource/iceberg/source/IcebergScanNode.java | 8 +-
.../datasource/paimon/PaimonExternalTable.java | 17 +
.../doris/nereids/parser/LogicalPlanBuilder.java | 57 +++-
.../expression/rules/DateFunctionRewrite.java | 9 +-
.../expressions/functions/scalar/ArrayRange.java | 7 +-
.../{ArrayRange.java => ArrayRangeDayUnit.java} | 57 +---
.../{ArrayRange.java => ArrayRangeHourUnit.java} | 57 +---
.../{ArrayRange.java => ArrayRangeMinuteUnit.java} | 57 +---
.../{ArrayRange.java => ArrayRangeMonthUnit.java} | 57 +---
.../{ArrayRange.java => ArrayRangeSecondUnit.java} | 57 +---
.../{ArrayRange.java => ArrayRangeWeekUnit.java} | 57 +---
.../{ArrayRange.java => ArrayRangeYearUnit.java} | 57 +---
.../trees/expressions/literal/DateV2Literal.java | 31 +-
.../expressions/visitor/ScalarFunctionVisitor.java | 35 ++
.../trees/plans/commands/DeleteFromCommand.java | 12 +
.../resource/workloadgroup/WorkloadGroup.java | 47 ++-
.../resource/workloadgroup/WorkloadGroupMgr.java | 3 +-
.../workloadschedpolicy/WorkloadAction.java | 2 -
.../WorkloadActionMoveQueryToGroup.java | 67 ----
.../workloadschedpolicy/WorkloadCondition.java | 4 +
...tion.java => WorkloadConditionBeScanBytes.java} | 32 +-
...ition.java => WorkloadConditionBeScanRows.java} | 32 +-
.../workloadschedpolicy/WorkloadConditionMeta.java | 4 +
.../workloadschedpolicy/WorkloadMetricType.java | 2 +-
.../workloadschedpolicy/WorkloadSchedPolicy.java | 4 +-
.../WorkloadSchedPolicyMgr.java | 22 +-
.../doris/statistics/BasicAsyncCacheLoader.java | 48 ---
.../statistics/ColumnStatisticsCacheLoader.java | 9 -
.../apache/doris/statistics/StatisticsCache.java | 16 -
.../doris/statistics/util/StatisticsUtil.java | 26 --
.../doris/tablefunction/MetadataGenerator.java | 10 +-
.../WorkloadGroupsTableValuedFunction.java | 2 +
.../org/apache/doris/statistics/CacheTest.java | 2 +-
gensrc/script/doris_builtins_functions.py | 8 +-
gensrc/thrift/BackendService.thrift | 14 +-
gensrc/thrift/FrontendService.thrift | 2 +-
.../test_agg_state_avg.out} | 0
.../datatype_p0/datetimev2/test_curdate_fold.out | 57 ++++
.../datatype_p0/datetimev2/test_invalid_hour.out | 14 +
.../where_invalid/where_invalid.out | 4 +
.../nereids_function_p0/scalar_function/Array.out | 194 +++++++++++
.../data/nereids_p0/subquery/test_subquery.out | 3 +
.../array_functions/test_array_functions.out | 131 ++++++++
.../test_workload_sched_policy.out | 1 -
.../org/apache/doris/regression/suite/Suite.groovy | 17 +-
.../doris/regression/suite/SuiteCluster.groovy | 16 +-
.../test_agg_state_avg.groovy} | 13 +-
.../test_agg_state_quantile_union.groovy | 3 +
.../datatype_p0/agg_state/test_agg_state.groovy | 1 -
.../datetimev2/test_curdate_fold.groovy | 102 ++++++
.../datetimev2/test_invalid_hour.groovy | 89 +++++
.../suites/demo_p0/docker_action.groovy | 15 +
regression-test/suites/mv_p0/k1s2m3/k1s2m3.groovy | 15 +
.../where_invalid/where_invalid.groovy | 14 +
.../scalar_function/Array.groovy | 14 +
.../nereids_p0/subquery/test_subquery.groovy | 6 +
.../array_functions/test_array_functions.groovy | 84 +++++
.../test_multi_string_position.groovy | 24 ++
.../test_multi_string_search.groovy | 26 +-
.../test_workload_sched_policy.groovy | 21 +-
144 files changed, 4007 insertions(+), 2176 deletions(-)
delete mode 100644
be/src/vec/data_types/serde/data_type_fixedlengthobject_serde.cpp
delete mode 100644
be/src/vec/data_types/serde/data_type_fixedlengthobject_serde.h
delete mode 100644 be/src/vec/data_types/serde/data_type_string_serde.cpp
create mode 100644 docker/runtime/doris-compose/resource/cloud.ini.example
create mode 100644 docker/runtime/doris-compose/resource/fdb.conf
create mode 100644 docker/runtime/doris-compose/resource/init_cloud.sh
copy docker/runtime/doris-compose/resource/{common.sh => init_fdb.sh} (54%)
create mode 100644
docs/en/docs/sql-manual/sql-functions/array-functions/sequence.md
create mode 100644 docs/en/docs/sql-manual/sql-functions/table-functions/job.md
rename
docs/en/docs/sql-manual/sql-reference/Data-Definition-Statements/{Alter/STOP-JOB.md
=> Drop/DROP-JOB.md} (79%)
delete mode 100644
docs/en/docs/sql-manual/sql-reference/Show-Statements/SHOW-JOB-TASK.md
delete mode 100644
docs/en/docs/sql-manual/sql-reference/Show-Statements/SHOW-JOB.md
create mode 100644
docs/zh-CN/docs/sql-manual/sql-functions/array-functions/sequence.md
create mode 100644
docs/zh-CN/docs/sql-manual/sql-functions/table-functions/job.md
rename
docs/zh-CN/docs/sql-manual/sql-reference/Data-Definition-Statements/{Alter/STOP-JOB.md
=> Drop/DROP-JOB.md} (80%)
delete mode 100644
docs/zh-CN/docs/sql-manual/sql-reference/Show-Statements/SHOW-JOB-TASK.md
delete mode 100644
docs/zh-CN/docs/sql-manual/sql-reference/Show-Statements/SHOW-JOB.md
create mode 100644
fe/fe-common/src/test/java/org/apache/doris/common/PairTest.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/common/publish/WorkloadActionPublishThread.java
copy
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/scalar/{ArrayRange.java
=> ArrayRangeDayUnit.java} (52%)
copy
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/scalar/{ArrayRange.java
=> ArrayRangeHourUnit.java} (52%)
copy
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/scalar/{ArrayRange.java
=> ArrayRangeMinuteUnit.java} (52%)
copy
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/scalar/{ArrayRange.java
=> ArrayRangeMonthUnit.java} (52%)
copy
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/scalar/{ArrayRange.java
=> ArrayRangeSecondUnit.java} (52%)
copy
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/scalar/{ArrayRange.java
=> ArrayRangeWeekUnit.java} (52%)
copy
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/scalar/{ArrayRange.java
=> ArrayRangeYearUnit.java} (52%)
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/resource/workloadschedpolicy/WorkloadActionMoveQueryToGroup.java
copy
fe/fe-core/src/main/java/org/apache/doris/resource/workloadschedpolicy/{WorkloadCondition.java
=> WorkloadConditionBeScanBytes.java} (51%)
copy
fe/fe-core/src/main/java/org/apache/doris/resource/workloadschedpolicy/{WorkloadCondition.java
=> WorkloadConditionBeScanRows.java} (51%)
copy
regression-test/data/datatype_p0/agg_state/{quantile_union/test_agg_state_quantile_union.out
=> avg/test_agg_state_avg.out} (100%)
create mode 100644
regression-test/data/datatype_p0/datetimev2/test_curdate_fold.out
create mode 100644
regression-test/data/datatype_p0/datetimev2/test_invalid_hour.out
create mode 100644
regression-test/data/mv_p0/test_mv_useless/where_invalid/where_invalid.out
copy
regression-test/suites/datatype_p0/agg_state/{quantile_union/test_agg_state_quantile_union.groovy
=> avg/test_agg_state_avg.groovy} (74%)
create mode 100644
regression-test/suites/datatype_p0/datetimev2/test_curdate_fold.groovy
create mode 100644
regression-test/suites/datatype_p0/datetimev2/test_invalid_hour.groovy
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
