This is an automated email from the ASF dual-hosted git repository.
morningman pushed a change to branch array-type
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git.
omit 706f7ff [feature-wip](array-type) Add codes and UT for array_contains
and array_position functions (#8401)
omit 76231c2 [feature-wip][array-type] Fix compilation error. (#8422)
omit d03a93d [feature-wip][array-type] Support nested array insertion.
(#8305)
omit 386f90f [feature-wip][array-type] Array data can be loaded in stream
load. (#8368)
omit 330216d [feature-wip](array-type)support select ARRAY data type on
vectorized engine (#8217)
omit 5db9cac [feature-wip][array-type] Support ArrayLiteral in SQL. (#8089)
omit 4532e10 [feature-wip](array-type) Create table with nested array
type. (#8003)
add 37c201a [doc] fix metadata operation links to external docs (#8403)
add 68dd799 [improvement](vectorized) Support function tuple is null
(#8442)
add e403dbc [feature](user-property) Support user level exec_mem_limit
and load_mem_limit (#8365)
add e0ef9b8 [refactor](vectorized) to_bitmap(-1) return NULL instead of
return parse failed error_message (#8373)
add a76889b [improvement] Avoid print large string in error log (#8436)
add ffddebf [fix](report) fix bug that tablet may already be delete when
reporting (#8444)
add 4a38f2d [fix](transaction) Fix committed transaction couldn't be
finished when table is dropped (#8423)
add c86d469 [Refactor](storage_engine) Use std::shared_mutex to replace
RWMutex (#8387)
add e17aef9 [refactor] refactor the implement of MemTracker, and related
usage (#8322)
add 23d0e7b [Feature](proc) Support proc dir for showing tablet health
status (#8324)
add ebbe6f6 [fix](broker-load) hdfs or bos path parser not support glob
exprs (#8390)
add a467e7a [refactor][fix] small fixes and code cleanups related to
schema change (#8328)
add 316d7c6 [doc] Update hive support version (#8371)
add f3c44bc [chore][fix](librdkafka) disable librdkafka assert and update
some thirdparty (#8425)
add 5f8e948 [fix] BE crash when reporting tablet (#8453)
add 2c63fc1 [improvement](vectorized) Support BetweenPredicate enable
fold const expr (#8450)
add 5ab3a8a [typo]broker load docs (#8434)
add 392a977 [doc] Update documentation configuration parameter
`sink.batch.bytes` in flink-doris-connector (#8369)
add 705989d [improvement](VHashJoin) add probe timer (#8233)
add a4b710c [chore](dependency) fix build thirdparty errors (#8456)
add e807e8b [improvement](memory) fix olap table scan and sink memory
usage problem (#8451)
add 991dc7f [fix][routine-load] fix bug that routine load cannot cancel
task when append_data return error (#8457)
add 41a15cc [fix](vectorized) Agg/Unique not null column outer join
coredump (#8461)
new 70323db [feature-wip](array-type) Create table with nested array
type. (#8003)
new 96ae5c9 [feature-wip][array-type] Support ArrayLiteral in SQL. (#8089)
new ae4fd5c [feature-wip](array-type)support select ARRAY data type on
vectorized engine (#8217)
new 0fedefd [feature-wip][array-type] Array data can be loaded in stream
load. (#8368)
new d4caf56 [feature-wip][array-type] Support nested array insertion.
(#8305)
new 33592b9 [feature-wip][array-type] Fix compilation error. (#8422)
new 2e91573 [feature-wip](array-type) Add codes and UT for array_contains
and array_position functions (#8401)
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (706f7ff)
\
N -- N -- N refs/heads/array-type (2e91573)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 7 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
be/CMakeLists.txt | 54 +-
be/src/agent/task_worker_pool.cpp | 4 +-
be/src/agent/topic_subscriber.cpp | 6 +-
be/src/common/config.h | 23 +-
be/src/common/daemon.cpp | 11 -
be/src/exec/analytic_eval_node.cpp | 2 +-
be/src/exec/base_scanner.cpp | 2 +-
be/src/exec/broker_scan_node.cpp | 2 +-
be/src/exec/broker_scanner.cpp | 1 -
be/src/exec/broker_scanner.h | 1 -
be/src/exec/cross_join_node.cpp | 2 +-
be/src/exec/data_sink.cpp | 4 +-
be/src/exec/es/es_scroll_parser.cpp | 10 +-
be/src/exec/es_http_scanner.cpp | 2 +-
be/src/exec/es_scan_node.cpp | 5 +-
be/src/exec/except_node.cpp | 2 +-
be/src/exec/exec_node.cpp | 15 +-
be/src/exec/exec_node.h | 28 +-
be/src/exec/hash_join_node.cpp | 6 +-
be/src/exec/hash_table.cpp | 12 +-
be/src/exec/intersect_node.cpp | 2 +-
be/src/exec/json_scanner.cpp | 1 -
be/src/exec/json_scanner.h | 1 -
be/src/exec/olap_scan_node.cpp | 53 +-
be/src/exec/olap_scan_node.h | 8 +
be/src/exec/olap_scanner.cpp | 25 +-
be/src/exec/olap_scanner.h | 4 +-
be/src/exec/orc_scanner.cpp | 1 -
be/src/exec/parquet_writer.cpp | 2 +-
be/src/exec/partitioned_aggregation_node.cc | 20 +-
be/src/exec/partitioned_hash_table.cc | 9 +-
be/src/exec/set_operation_node.cpp | 2 +-
be/src/exec/tablet_info.cpp | 10 +-
be/src/exec/tablet_info.h | 2 -
be/src/exec/tablet_sink.cpp | 22 +-
be/src/exec/tablet_sink.h | 3 +
be/src/exec/union_node.cpp | 2 -
be/src/exprs/agg_fn.h | 1 -
be/src/exprs/agg_fn_evaluator.cpp | 4 +-
be/src/exprs/anyval_util.cpp | 5 +-
be/src/exprs/bitmap_function.cpp | 49 +-
be/src/exprs/bloomfilter_predicate.h | 12 +-
be/src/exprs/expr.cpp | 17 +-
be/src/exprs/expr.h | 11 +-
be/src/exprs/expr_context.cpp | 17 +-
be/src/exprs/expr_context.h | 3 +
be/src/exprs/hll_function.cpp | 2 +-
be/src/exprs/new_agg_fn_evaluator.cc | 16 +-
be/src/exprs/new_agg_fn_evaluator.h | 6 +-
be/src/gutil/strings/numbers.cc | 35 ++
be/src/gutil/strings/numbers.h | 4 +
be/src/http/action/compaction_action.cpp | 6 +-
be/src/http/action/compaction_action.h | 9 +-
be/src/http/default_path_handlers.cpp | 18 +-
be/src/olap/aggregate_func.h | 4 +-
be/src/olap/base_compaction.cpp | 5 +-
be/src/olap/base_compaction.h | 3 +-
be/src/olap/collect_iterator.h | 2 +
be/src/olap/compaction.cpp | 21 +-
be/src/olap/cumulative_compaction.cpp | 5 +-
be/src/olap/cumulative_compaction.h | 3 +-
be/src/olap/data_dir.cpp | 6 +-
be/src/olap/data_dir.h | 3 +-
be/src/olap/delta_writer.cpp | 15 +-
be/src/olap/fs/block_manager.h | 5 -
be/src/olap/fs/file_block_manager.cpp | 5 +-
be/src/olap/fs/file_block_manager.h | 5 -
be/src/olap/generic_iterators.cpp | 4 +-
be/src/olap/lru_cache.cpp | 20 +-
be/src/olap/lru_cache.h | 9 +-
be/src/olap/memtable.cpp | 2 +-
be/src/olap/memtable.h | 1 +
be/src/olap/merger.cpp | 4 +-
be/src/olap/olap_index.cpp | 3 +-
be/src/olap/olap_index.h | 1 -
be/src/olap/page_cache.cpp | 15 +-
be/src/olap/push_handler.cpp | 18 +-
be/src/olap/reader.cpp | 18 +-
be/src/olap/reader.h | 5 +-
be/src/olap/row_block.cpp | 2 +-
be/src/olap/row_block2.cpp | 2 +-
be/src/olap/rowset/alpha_rowset_reader.h | 1 -
be/src/olap/rowset/beta_rowset_reader.h | 1 -
be/src/olap/rowset/rowset.h | 1 -
be/src/olap/rowset/segment_reader.cpp | 21 +-
be/src/olap/rowset/segment_v2/binary_dict_page.cpp | 3 +-
be/src/olap/rowset/segment_v2/binary_dict_page.h | 2 -
.../olap/rowset/segment_v2/bitmap_index_reader.h | 5 +-
.../olap/rowset/segment_v2/bitmap_index_writer.cpp | 5 +-
.../rowset/segment_v2/bloom_filter_index_reader.h | 5 +-
.../segment_v2/bloom_filter_index_writer.cpp | 5 +-
be/src/olap/rowset/segment_v2/column_reader.cpp | 2 +-
be/src/olap/rowset/segment_v2/column_reader.h | 4 +-
be/src/olap/rowset/segment_v2/column_writer.h | 3 -
.../rowset/segment_v2/indexed_column_writer.cpp | 3 +-
.../olap/rowset/segment_v2/indexed_column_writer.h | 2 -
be/src/olap/rowset/segment_v2/segment.cpp | 15 +-
be/src/olap/rowset/segment_v2/segment_iterator.cpp | 6 +-
be/src/olap/rowset/segment_v2/segment_writer.cpp | 8 +-
be/src/olap/rowset/segment_v2/zone_map_index.cpp | 6 +-
be/src/olap/rowset/segment_v2/zone_map_index.h | 2 -
be/src/olap/schema_change.cpp | 331 +++++-----
be/src/olap/segment_loader.cpp | 6 +-
be/src/olap/segment_loader.h | 2 -
be/src/olap/snapshot_manager.cpp | 4 +-
be/src/olap/storage_engine.cpp | 41 +-
be/src/olap/storage_engine.h | 13 +
be/src/olap/tablet.cpp | 72 ++-
be/src/olap/tablet.h | 23 +-
be/src/olap/tablet_manager.cpp | 111 ++--
be/src/olap/tablet_manager.h | 16 +-
be/src/olap/tablet_meta.cpp | 4 +-
be/src/olap/tablet_meta.h | 3 +-
be/src/olap/tablet_schema.cpp | 8 +-
be/src/olap/tablet_schema.h | 3 +-
be/src/olap/task/engine_alter_tablet_task.cpp | 1 +
be/src/olap/task/engine_checksum_task.cpp | 5 +-
be/src/olap/task/engine_clone_task.cpp | 160 +++--
be/src/olap/task/engine_storage_migration_task.cpp | 45 +-
be/src/olap/tuple_reader.cpp | 1 -
be/src/olap/txn_manager.cpp | 30 +-
be/src/olap/txn_manager.h | 6 +-
be/src/runtime/CMakeLists.txt | 1 +
be/src/runtime/buffered_block_mgr2.cc | 47 +-
be/src/runtime/buffered_tuple_stream3.cc | 1 -
be/src/runtime/bufferpool/buffer_pool.cc | 3 +-
be/src/runtime/bufferpool/reservation_tracker.cc | 15 +-
be/src/runtime/cache/result_cache.h | 1 -
be/src/runtime/data_stream_mgr.h | 1 -
be/src/runtime/data_stream_recvr.cc | 8 +-
be/src/runtime/data_stream_sender.cpp | 6 +-
be/src/runtime/disk_io_mgr.cc | 57 +-
be/src/runtime/disk_io_mgr.h | 10 +-
be/src/runtime/exec_env.h | 23 +-
be/src/runtime/exec_env_init.cpp | 25 +-
be/src/runtime/export_sink.cpp | 5 +-
be/src/runtime/export_sink.h | 3 -
be/src/runtime/fold_constant_executor.cpp | 4 +-
be/src/runtime/fragment_mgr.cpp | 4 +-
be/src/runtime/free_pool.hpp | 2 +-
be/src/runtime/initial_reservations.cc | 4 +-
be/src/runtime/load_channel.cpp | 4 +-
be/src/runtime/load_channel_mgr.cpp | 16 +-
be/src/runtime/mem_pool.cpp | 57 +-
be/src/runtime/mem_pool.h | 66 +-
be/src/runtime/mem_tracker.cpp | 588 +++++------------
be/src/runtime/mem_tracker.h | 692 +++++++++------------
be/src/runtime/mem_tracker_task_pool.cpp | 135 ++++
be/src/runtime/mem_tracker_task_pool.h | 62 ++
be/src/runtime/memory/chunk_allocator.cpp | 35 +-
be/src/runtime/memory/chunk_allocator.h | 12 +-
be/src/runtime/memory_scratch_sink.h | 1 -
be/src/runtime/mysql_result_writer.cpp | 6 +-
be/src/runtime/mysql_table_sink.cpp | 2 +-
be/src/runtime/odbc_table_sink.cpp | 11 +-
be/src/runtime/odbc_table_sink.h | 4 +-
be/src/runtime/plan_fragment_executor.cpp | 32 +-
be/src/runtime/plan_fragment_executor.h | 1 -
be/src/runtime/result_file_sink.cpp | 7 +-
be/src/runtime/result_file_sink.h | 1 -
be/src/runtime/result_sink.h | 1 -
.../runtime/routine_load/data_consumer_group.cpp | 14 +-
be/src/runtime/row_batch.cpp | 6 +-
be/src/runtime/runtime_filter_mgr.cpp | 9 +-
be/src/runtime/runtime_filter_mgr.h | 2 +-
be/src/runtime/runtime_state.cpp | 108 +---
be/src/runtime/runtime_state.h | 40 +-
be/src/runtime/spill_sorter.cc | 2 -
be/src/runtime/tablets_channel.cpp | 4 +-
be/src/runtime/vectorized_row_batch.cpp | 2 +-
be/src/service/doris_main.cpp | 3 +
be/src/service/http_service.cpp | 2 +-
be/src/util/arrow/row_batch.cpp | 3 +-
be/src/util/arrow/row_block.cpp | 3 +-
be/src/util/doris_metrics.h | 1 +
be/src/util/mem_info.h | 2 +
be/src/util/mutex.cpp | 63 --
be/src/util/mutex.h | 96 +--
be/src/util/thread_group.h | 16 +-
be/src/vec/CMakeLists.txt | 1 +
.../aggregate_function_bitmap.h | 17 +-
.../aggregate_function_hll_union_agg.cpp | 18 +-
.../aggregate_function_hll_union_agg.h | 94 +--
.../aggregate_function_reader.cpp | 2 +-
.../aggregate_function_simple_factory.cpp | 2 +-
be/src/vec/columns/column.h | 5 +
be/src/vec/columns/column_nullable.cpp | 5 +
be/src/vec/columns/column_nullable.h | 3 +
be/src/vec/columns/column_vector.cpp | 3 +-
be/src/vec/common/string_ref.h | 3 +
be/src/vec/exec/join/vhash_join_node.cpp | 234 ++++---
be/src/vec/exec/join/vhash_join_node.h | 3 +
be/src/vec/exec/vaggregation_node.cpp | 12 +-
be/src/vec/exec/vcross_join_node.cpp | 6 +-
be/src/vec/exec/volap_scan_node.cpp | 57 +-
be/src/vec/exec/volap_scanner.cpp | 13 +-
be/src/vec/exec/volap_scanner.h | 3 +-
be/src/vec/exec/vset_operation_node.cpp | 12 +-
be/src/vec/exec/vsort_node.cpp | 4 +-
be/src/vec/exprs/vexpr.cpp | 5 +
be/src/vec/exprs/vtuple_is_null_predicate.cpp | 84 +++
.../{vslot_ref.h => vtuple_is_null_predicate.h} | 42 +-
be/src/vec/functions/function_bit.cpp | 25 +
be/src/vec/functions/function_bitmap.cpp | 187 ++++--
be/src/vec/functions/function_encryption.cpp | 2 +-
be/src/vec/functions/function_string.cpp | 4 +-
be/src/vec/olap/vcollect_iterator.cpp | 2 +-
be/src/vec/olap/vgeneric_iterators.cpp | 4 +-
be/src/vec/runtime/vdata_stream_recvr.cpp | 15 +-
be/src/vec/sink/vdata_stream_sender.cpp | 6 +-
be/src/vec/sink/vmysql_table_sink.cpp | 2 +-
be/src/vec/sink/vtablet_sink.cpp | 10 +-
be/test/exec/hash_table_test.cpp | 12 +-
be/test/exec/tablet_sink_test.cpp | 1 +
be/test/exprs/bitmap_function_test.cpp | 7 +-
be/test/exprs/bloom_filter_predicate_test.cpp | 6 +-
.../olap/bloom_filter_column_predicate_test.cpp | 2 +-
be/test/olap/cumulative_compaction_policy_test.cpp | 2 +-
be/test/olap/generic_iterators_test.cpp | 8 +-
be/test/olap/rowset/beta_rowset_test.cpp | 3 +-
be/test/runtime/mem_limit_test.cpp | 130 ++--
be/test/runtime/memory/chunk_allocator_test.cpp | 3 +-
be/test/runtime/memory_scratch_sink_test.cpp | 2 +-
be/test/runtime/test_env.cc | 7 +-
be/test/util/arrow/arrow_work_flow_test.cpp | 3 +-
be/test/vec/core/block_test.cpp | 2 +-
be/test/vec/exec/vgeneric_iterators_test.cpp | 6 +-
be/test/vec/exprs/vexpr_test.cpp | 8 +-
be/test/vec/function/function_comparison_test.cpp | 2 +-
be/test/vec/function/function_string_test.cpp | 15 +
docker/Dockerfile | 3 +
docs/.vuepress/sidebar/en.js | 1 -
docs/.vuepress/sidebar/zh-CN.js | 1 -
docs/en/administrator-guide/config/be_config.md | 18 +-
.../operation/metadata-operation.md | 2 +-
docs/en/extending-doris/flink-doris-connector.md | 2 +-
docs/en/extending-doris/hive-of-doris.md | 2 +-
.../sql-functions/bitmap-functions/to_bitmap.md | 11 +-
.../sql-functions/string-functions/coalesce.md | 62 --
.../Account Management/SET PROPERTY.md | 10 +
.../Data Manipulation/BROKER LOAD.md | 16 +-
docs/zh-CN/administrator-guide/config/be_config.md | 18 +-
.../operation/metadata-operation.md | 2 +-
.../zh-CN/extending-doris/flink-doris-connector.md | 2 +-
docs/zh-CN/extending-doris/hive-of-doris.md | 2 +-
.../sql-functions/bitmap-functions/to_bitmap.md | 8 +
.../conditional-functions/coalesce.md | 4 +-
.../sql-functions/string-functions/coalesce.md | 63 --
.../Account Management/SET PROPERTY.md | 12 +-
.../Data Manipulation/BROKER LOAD.md | 23 +-
.../apache/doris/analysis/ColumnRenameClause.java | 2 +-
.../apache/doris/analysis/CreateFunctionStmt.java | 8 +-
.../java/org/apache/doris/analysis/ExportStmt.java | 28 +-
.../java/org/apache/doris/analysis/HdfsURI.java | 83 ---
.../doris/analysis/TupleIsNullPredicate.java | 27 +-
.../apache/doris/catalog/AggregateFunction.java | 22 +-
.../java/org/apache/doris/catalog/Catalog.java | 8 +
.../java/org/apache/doris/catalog/Database.java | 19 +-
.../java/org/apache/doris/catalog/Function.java | 21 +-
.../org/apache/doris/catalog/ScalarFunction.java | 8 +-
.../main/java/org/apache/doris/catalog/Table.java | 9 +-
...nitorProcDir.java => ClusterHealthProcDir.java} | 50 +-
.../common/proc/IncompleteTabletsProcNode.java | 25 +-
.../org/apache/doris/common/proc/ProcService.java | 3 +-
.../apache/doris/common/proc/StatisticProcDir.java | 263 --------
.../doris/common/proc/StatisticProcNode.java | 151 +++++
.../doris/common/proc/TabletHealthProcDir.java | 331 ++++++++++
.../apache/doris/common/util/MetaLockUtils.java | 12 +
.../java/org/apache/doris/common/util/URI.java | 204 ++++++
.../doris/load/routineload/ScheduleRule.java | 13 +
.../mysql/privilege/CommonUserProperties.java | 22 +
.../org/apache/doris/mysql/privilege/PaloAuth.java | 18 +
.../apache/doris/mysql/privilege/UserProperty.java | 40 ++
.../doris/mysql/privilege/UserPropertyMgr.java | 22 +-
.../apache/doris/planner/SingleNodePlanner.java | 5 +-
.../main/java/org/apache/doris/qe/Coordinator.java | 23 +-
.../apache/doris/rewrite/FoldConstantsRule.java | 6 +-
.../apache/doris/service/FrontendServiceImpl.java | 2 +-
.../doris/transaction/DatabaseTransactionMgr.java | 38 +-
.../doris/transaction/PublishVersionDaemon.java | 7 +-
.../java/org/apache/doris/common/util/URITest.java | 97 +++
fe/pom.xml | 2 +-
gensrc/script/doris_builtins_functions.py | 14 +-
thirdparty/CHANGELOG.md | 7 +
thirdparty/build-thirdparty.sh | 84 +--
thirdparty/download-thirdparty.sh | 36 +-
thirdparty/patches/librdkafka-1.8.2.patch | 13 +
thirdparty/vars.sh | 87 +--
288 files changed, 3806 insertions(+), 3444 deletions(-)
create mode 100644 be/src/runtime/mem_tracker_task_pool.cpp
create mode 100644 be/src/runtime/mem_tracker_task_pool.h
create mode 100644 be/src/vec/exprs/vtuple_is_null_predicate.cpp
copy be/src/vec/exprs/{vslot_ref.h => vtuple_is_null_predicate.h} (52%)
delete mode 100644
docs/en/sql-reference/sql-functions/string-functions/coalesce.md
delete mode 100644
docs/zh-CN/sql-reference/sql-functions/string-functions/coalesce.md
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/analysis/HdfsURI.java
copy
fe/fe-core/src/main/java/org/apache/doris/common/proc/{MonitorProcDir.java =>
ClusterHealthProcDir.java} (62%)
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/common/proc/StatisticProcDir.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/common/proc/StatisticProcNode.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/common/proc/TabletHealthProcDir.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/common/util/URI.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/common/util/URITest.java
create mode 100644 thirdparty/patches/librdkafka-1.8.2.patch
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
