This is an automated email from the ASF dual-hosted git repository.
caiconghui pushed a change to branch table_lock
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git.
discard a447398 Fix dead lock bug
discard d8f4bcd fix
discard 31d2af5 fix
discard 96322e5 fix
discard 0c74eb9 remove useless db read lock and change some db read lock to
table locks
discard 2a04b58 remove useless db read lock
discard db0fd46 remove useless code of dropTableWithLock for Database
discard 29275c0 use MetadataLockUtils to lock table when finish transaction
discard 6a5d3de Add unit test for MetaLockUtils
discard 0f0f6b7 Fix save image lock
discard 175c719 Fix SparkLoadJobTest Failed
discard 37b8de7 fix unit test failed
discard 4aac0ed fix unittest failed
discard b5639ad fix conflict for table lock and db lock
discard 59c162a fix unittest failed
discard c55e6cb Change db level lock to table level lock for query
discard c4ab877 Fix unittest for AlterTest failed
discard 24c322e Add MetaLockUtils and fix some table lock level
discard 2d21230 fix
discard 29c0911 fix unit test failed for AlterTest
discard 0a851bf fix ReportHandler
discard 45e80ef fix conflict after rebasing mater
discard 7656db1 fix test
discard fccf9a0 fix DeleteHandler
discard b83d3d6 fix
discard d4c3bbe fix SchemaChangeJobV2
discard 1a8f88e fix cancel
discard 7c8271f fix RollupJobV2
discard 181de93 Fix RollupJob
discard d504f32 Fix AlterJobV2 and MaterializedViewHandler
discard 695aab7 fix Alter Class
discard d0c5344 modify .gitignore content
discard d9509ca fix merge
discard a71eedf add null check for olap_table
discard c07031b fix unit test
discard 56944df finish to replace db lock by table lock
discard 5311671 Continue to use table lock to replace db lock
discard fd59331 use table lock to replace db lock
discard f5949ba remove some unused temp code
discard 4161976 use table write lock to replace db write lock
discard 6037d93 Use table lock to replace db lock in some functions
discard 0d57571 Support table level read and write lock
add 66132d2 [Feature] Running Profile OLAP_SCAN_NODE layering and enhance
readability (#4825)
add f0e8939 [Bug] Fix DCHECK failed in group_concat (#4850)
add 74bc25f [Metrics] Add metric to monitor timeout canceled fragment
count (#4862)
add 4ccd7b8 [Bug] Rename table logic error (#4870)
add 1151a00 [Bug] Make 'LastStartTime' in backends list as the actual BE
start time (#4872)
add dd70653 [DOCS] Fix some docs typo (#4873)
add a1ae399 [Refactor] Refactor storage medium migration task process
(#4475)
add 1810f10 [Bug] Fix bug that failed to create view with complex select
stmt (#4840)
add 796f44b [Bug] Fix bug that routine load blocked with TOO_MANY_TASKS
error (#4861)
add 9786736 Revert "[FEATURE]Check date type to avoid scan all partitions
(#4756)" (#4877)
add e992310 [Profile][UT] Fix UT and remove useless profile (#4879)
add 69c422e [Bug] Fix bug #4886 and #4586 by refactoring code of method
'getDbs' (#4887)
add aca9b2d [Bug] Fix bug introduced by split RowsDelFiltered profile
(#4881)
add 18a22bd [BUG] Fix field error in information_schema.columns (#4858)
add e706a6b [Doc] Running Profile document add HASH_JOIN_NODE, etc.
(#4878)
add 2af4bc2 [Bug] Java Version BitmapValue deserialized failed when only
has 32-bit bitmap (#4884)
add 5aefd70 [Improve]modify isDecommissioned be capacity calculate rule
(#4889)
add c5e4351 [Refactor] Remove break label for readability (#4890)
add 55080ba [BUG] Fix colocate join memory limit problem (#4894)
add 448df42 [Compatibility] Add table_privileges, schema_privileges and
user_privileges tables(#4899)
add b48c768 [ComplexType] Restructure storage type to support complex
types expending (#4905)
add bba85fc Update routine-load-manual.md (#4911)
add dcca3bb Avoid duplicate column when adding slot in empty tuple (#4901)
add 6247408 [Compact]Take tablet scan frequency into consider when
selecting tablet for compaction (#4837)
add ec9da30 [New Feature]Support udf when loading data (#4863)
add 6101155 [CodeStyle]Replace tab with spaces (#4909)
add 0eda527 [Docs] Add doc of be_config.md and change some default value
of BE config (#4906)
add d1a7f1d Fix column_reader_writer_test UT (#4924)
add 64b219f Fix typo (#4923)
add 234e9b5 [Doc] Fiexed example content in bitmap_union.md (#4919)
add f445ed5 Disable the creation of segment v1 table (#4913)
add e507fcc [Enhancement] Improve list comparing performance (#4880)
add f1b57c4 [Optimize] Avoid repeated sending of common components in
Fragments (#4904)
add fb7f4c8 [Bug] fix bug that be thrift client cannot connect to fe
thrift server when fe thrift server use `TThreadedSelectorServer` model (#4908)
add 4f7c6da [Refactor] Refactor function getScanRangeLength (#4912)
add c28769c [Bug] Avoid partition prune if predicate is not with SlotRef
(#4833) (#4921)
add 8e9bbfb [Script] Check and create if the log directory not existed
before outputing message to the log file. (#4929)
add 584b33f [Bug] Fix the bug of NULL do not show in CTE statement.
(#4932)
add 37a6731 [BUG] Fix Colocate table balance bug (#4936)
add 589c1d3 Support table level read and write lock
add c2daf4a Use table lock to replace db lock in some functions
add f2ac4a6 use table write lock to replace db write lock
add e1e9893 remove some unused temp code
add 247d7f0 use table lock to replace db lock
add fc8f28c Continue to use table lock to replace db lock
add 212c816 finish to replace db lock by table lock
add b80fcec fix unit test
add 4fe56fa add null check for olap_table
add 4cfe529 fix merge
add 7467b70 modify .gitignore content
add e1817c8 fix Alter Class
add 9821229 Fix AlterJobV2 and MaterializedViewHandler
add 2461be0 Fix RollupJob
add c5c579f fix RollupJobV2
add 0105d87 fix cancel
add edd2638 fix SchemaChangeJobV2
add 9c207f0 fix
add 32731c0 fix DeleteHandler
add 347c5d9 fix test
add 805837b fix conflict after rebasing mater
add b224b51 fix ReportHandler
add 52183db fix unit test failed for AlterTest
add 6171a42 fix
add 6c62042 Add MetaLockUtils and fix some table lock level
add 65bc38c Fix unittest for AlterTest failed
add 3e2be1f Change db level lock to table level lock for query
add 56b5229 fix unittest failed
add 5e0ed1f fix conflict for table lock and db lock
add f35bdcf fix unittest failed
add f324700 fix unit test failed
add c781a92 Fix SparkLoadJobTest Failed
add c703d8e Fix save image lock
add 717af00 Add unit test for MetaLockUtils
add 2db6bfb use MetadataLockUtils to lock table when finish transaction
add 277c8ad remove useless code of dropTableWithLock for Database
add ca48518 remove useless db read lock
add 32f360e remove useless db read lock and change some db read lock to
table locks
add 3d836aa fix
add 6cede9d fix
add a359225 fix
add 220ac11 Fix dead lock bug
add 1585a01 fix unit test failed for SelectStmt
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (a447398)
\
N -- N -- N refs/heads/table_lock (1585a01)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
be/src/agent/heartbeat_server.cpp | 22 +-
be/src/agent/heartbeat_server.h | 3 +-
be/src/agent/task_worker_pool.cpp | 88 +++-
be/src/agent/task_worker_pool.h | 5 +
be/src/common/config.h | 18 +-
be/src/exec/CMakeLists.txt | 4 +
be/src/exec/csv_scan_node.cpp | 1 -
be/src/exec/es_scan_node.cpp | 1 -
be/src/exec/mysql_scan_node.cpp | 1 -
be/src/exec/odbc_scan_node.cpp | 1 -
be/src/exec/olap_scan_node.cpp | 69 +--
be/src/exec/olap_scan_node.h | 7 +
be/src/exec/olap_scanner.cpp | 3 +
be/src/exec/scan_node.cpp | 20 -
be/src/exec/scan_node.h | 35 --
be/src/exec/schema_scan_node.cpp | 1 -
be/src/exec/schema_scanner.cpp | 11 +
.../exec/schema_scanner/schema_columns_scanner.cpp | 40 +-
be/src/exec/schema_scanner/schema_helper.cpp | 33 ++
be/src/exec/schema_scanner/schema_helper.h | 18 +
.../schema_schema_privileges_scanner.cpp | 160 +++++++
...canner.h => schema_schema_privileges_scanner.h} | 17 +-
.../schema_table_privileges_scanner.cpp | 168 +++++++
...scanner.h => schema_table_privileges_scanner.h} | 17 +-
.../schema_user_privileges_scanner.cpp | 150 ++++++
..._scanner.h => schema_user_privileges_scanner.h} | 17 +-
be/src/exec/union_node.cpp | 15 +
be/src/exec/union_node.h | 6 +
be/src/exprs/aggregate_functions.cpp | 11 +-
be/src/http/ev_http_server.cpp | 3 +-
be/src/olap/CMakeLists.txt | 1 +
be/src/olap/base_tablet.cpp | 2 +
be/src/olap/base_tablet.h | 1 +
be/src/olap/collection.h | 59 +++
be/src/olap/column_block.h | 52 +--
be/src/olap/column_vector.cpp | 212 +++++++++
be/src/olap/column_vector.h | 232 ++++++++++
be/src/olap/field.h | 61 ++-
be/src/olap/olap_common.h | 8 +-
be/src/olap/push_handler.cpp | 2 +-
be/src/olap/row_block2.cpp | 26 +-
be/src/olap/row_block2.h | 16 +-
be/src/olap/rowset/beta_rowset_reader.h | 3 +-
be/src/olap/rowset/column_reader.cpp | 2 +-
be/src/olap/rowset/column_writer.cpp | 2 +-
be/src/olap/rowset/segment_v2/binary_dict_page.cpp | 20 +-
be/src/olap/rowset/segment_v2/binary_dict_page.h | 4 +-
.../olap/rowset/segment_v2/bitmap_index_reader.cpp | 9 +-
be/src/olap/rowset/segment_v2/bitshuffle_page.h | 13 +-
.../segment_v2/bloom_filter_index_reader.cpp | 9 +-
.../segment_v2/bloom_filter_index_writer.cpp | 2 +-
be/src/olap/rowset/segment_v2/column_reader.cpp | 183 +++++++-
be/src/olap/rowset/segment_v2/column_reader.h | 66 ++-
be/src/olap/rowset/segment_v2/column_writer.cpp | 260 ++++++++---
be/src/olap/rowset/segment_v2/column_writer.h | 165 +++++--
be/src/olap/rowset/segment_v2/encoding_info.cpp | 15 +
.../rowset/segment_v2/frame_of_reference_page.h | 13 +-
be/src/olap/rowset/segment_v2/page_decoder.h | 9 +
be/src/olap/rowset/segment_v2/parsed_page.h | 5 +
be/src/olap/rowset/segment_v2/plain_page.h | 15 +-
be/src/olap/rowset/segment_v2/segment.cpp | 3 +-
be/src/olap/rowset/segment_v2/segment_iterator.cpp | 2 +-
be/src/olap/rowset/segment_v2/segment_writer.cpp | 47 +-
be/src/olap/rowset/segment_v2/segment_writer.h | 2 +
be/src/olap/rowset/segment_v2/zone_map_index.cpp | 13 +-
be/src/olap/snapshot_manager.cpp | 16 +-
be/src/olap/tablet.cpp | 16 +-
be/src/olap/tablet.h | 11 +
be/src/olap/tablet_manager.cpp | 25 +-
be/src/olap/tablet_meta.cpp | 10 +-
be/src/olap/tablet_meta_manager.cpp | 2 +-
be/src/olap/tablet_schema.cpp | 29 +-
be/src/olap/tablet_schema.h | 16 +
be/src/olap/task/engine_storage_migration_task.cpp | 171 +++----
be/src/olap/task/engine_storage_migration_task.h | 22 +-
be/src/olap/types.cpp | 113 ++++-
be/src/olap/types.h | 270 ++++++++++-
be/src/runtime/client_cache.h | 13 +-
be/src/runtime/fragment_mgr.cpp | 204 ++++++---
be/src/runtime/fragment_mgr.h | 14 +-
be/src/runtime/plan_fragment_executor.cpp | 28 +-
be/src/runtime/plan_fragment_executor.h | 60 ++-
be/src/runtime/routine_load/data_consumer.cpp | 6 +-
be/src/runtime/runtime_state.cpp | 8 +-
be/src/runtime/runtime_state.h | 2 +-
be/src/runtime/test_env.cc | 2 +-
be/src/service/internal_service.cpp | 4 +-
be/src/util/doris_metrics.cpp | 1 +
be/src/util/thrift_client.h | 7 +-
be/test/olap/CMakeLists.txt | 1 +
be/test/olap/column_vector_test.cpp | 180 ++++++++
.../rowset/segment_v2/binary_dict_page_test.cpp | 16 +-
.../rowset/segment_v2/binary_plain_page_test.cpp | 13 +-
.../rowset/segment_v2/binary_prefix_page_test.cpp | 22 +-
.../rowset/segment_v2/bitshuffle_page_test.cpp | 13 +-
.../segment_v2/column_reader_writer_test.cpp | 302 ++++++++++--
.../olap/rowset/segment_v2/encoding_info_test.cpp | 4 +-
.../segment_v2/frame_of_reference_page_test.cpp | 14 +-
be/test/olap/rowset/segment_v2/plain_page_test.cpp | 14 +-
be/test/olap/rowset/segment_v2/rle_page_test.cpp | 13 +-
be/test/olap/rowset/segment_v2/segment_test.cpp | 18 +-
be/test/olap/schema_change_test.cpp | 4 +-
be/test/olap/storage_types_test.cpp | 79 +++-
be/test/runtime/buffered_block_mgr2_test.cpp | 2 -
be/test/runtime/fragment_mgr_test.cpp | 4 +-
bin/start_fe.sh | 8 +-
docs/en/administrator-guide/config/be_config.md | 160 +++++++
.../load-data/batch-delete-manual.md | 10 +-
.../load-data/broker-load-manual.md | 2 +-
.../load-data/routine-load-manual.md | 2 +-
docs/en/administrator-guide/running-profile.md | 185 ++++----
docs/zh-CN/administrator-guide/config/be_config.md | 165 ++++++-
.../load-data/batch-delete-manual.md | 9 +-
.../load-data/routine-load-manual.md | 4 +-
docs/zh-CN/administrator-guide/running-profile.md | 185 ++++----
.../sql-functions/bitmap-functions/bitmap_union.md | 2 +-
fe/fe-core/src/main/cup/sql_parser.cup | 48 +-
.../java/org/apache/doris/analysis/Analyzer.java | 2 -
.../org/apache/doris/analysis/BaseViewStmt.java | 35 +-
.../org/apache/doris/analysis/CreateViewStmt.java | 3 +-
.../main/java/org/apache/doris/analysis/Expr.java | 32 +-
.../org/apache/doris/analysis/InlineViewRef.java | 4 +-
.../java/org/apache/doris/analysis/InsertStmt.java | 5 +-
.../java/org/apache/doris/analysis/QueryStmt.java | 12 +-
.../java/org/apache/doris/analysis/SelectStmt.java | 37 +-
.../apache/doris/analysis/SetOperationStmt.java | 9 +-
.../apache/doris/analysis/ShowResourcesStmt.java | 140 +++---
.../java/org/apache/doris/analysis/SlotRef.java | 16 +-
.../org/apache/doris/analysis/StatementBase.java | 11 +
.../java/org/apache/doris/analysis/WithClause.java | 6 +-
.../java/org/apache/doris/catalog/Catalog.java | 15 +-
.../java/org/apache/doris/catalog/SchemaTable.java | 32 +-
.../apache/doris/clone/ColocateTableBalancer.java | 505 +++++++--------------
.../main/java/org/apache/doris/common/Config.java | 12 +-
.../org/apache/doris/common/FeMetaVersion.java | 4 +-
.../apache/doris/common/util/PropertyAnalyzer.java | 4 +
.../doris/common/util/QueryPlannerProfile.java | 104 +++++
.../apache/doris/common/util/RuntimeProfile.java | 4 +-
.../org/apache/doris/common/util/ToSqlContext.java | 70 +++
.../src/main/java/org/apache/doris/load/Load.java | 9 +
.../apache/doris/load/loadv2/BrokerLoadJob.java | 9 +-
.../org/apache/doris/load/loadv2/BulkLoadJob.java | 23 +-
.../apache/doris/load/loadv2/LoadLoadingTask.java | 6 +-
.../org/apache/doris/load/loadv2/LoadManager.java | 2 +-
.../doris/load/loadv2/LoadingTaskPlanner.java | 22 +-
.../org/apache/doris/load/loadv2/SparkLoadJob.java | 6 +-
.../doris/load/routineload/RoutineLoadJob.java | 16 +-
.../doris/load/routineload/RoutineLoadManager.java | 2 +-
.../load/routineload/RoutineLoadScheduler.java | 2 +-
.../load/routineload/RoutineLoadTaskScheduler.java | 2 +-
.../org/apache/doris/mysql/privilege/PaloAuth.java | 108 +++++
.../doris/mysql/privilege/PaloPrivilege.java | 14 +
.../apache/doris/mysql/privilege/PrivTable.java | 4 +
.../apache/doris/planner/AssertNumRowsNode.java | 2 +-
.../apache/doris/planner/SingleNodePlanner.java | 25 +-
.../java/org/apache/doris/qe/ConnectProcessor.java | 1 +
.../main/java/org/apache/doris/qe/Coordinator.java | 125 +++--
.../java/org/apache/doris/qe/StmtExecutor.java | 40 +-
.../java/org/apache/doris/qe/cache/CacheProxy.java | 12 +-
.../org/apache/doris/qe/cache/PartitionCache.java | 4 +-
.../doris/rewrite/BinaryPredicatesDateRule.java | 50 --
.../org/apache/doris/rpc/BackendServiceProxy.java | 16 +-
.../java/org/apache/doris/rpc/PBackendService.java | 8 +-
.../apache/doris/service/FrontendServiceImpl.java | 54 +++
.../main/java/org/apache/doris/system/Backend.java | 7 +-
.../org/apache/doris/system/BackendHbResponse.java | 8 +-
.../java/org/apache/doris/system/HeartbeatMgr.java | 4 +-
.../org/apache/doris/system/SystemInfoService.java | 2 +-
.../org/apache/doris/alter/AlterJobV2Test.java | 4 +
.../java/org/apache/doris/analysis/ExprTest.java | 25 +
.../org/apache/doris/analysis/SelectStmtTest.java | 6 +
.../apache/doris/catalog/CatalogOperationTest.java | 160 +++++++
.../doris/clone/ColocateTableBalancerTest.java | 256 +++++++++--
.../apache/doris/common/PropertyAnalyzerTest.java | 23 +
.../doris/load/loadv2/BrokerLoadJobTest.java | 56 ++-
.../apache/doris/load/loadv2/SparkLoadJobTest.java | 7 +-
.../java/org/apache/doris/planner/PlannerTest.java | 15 +
.../org/apache/doris/planner/QueryPlanTest.java | 63 ---
.../org/apache/doris/qe/ConnectProcessorTest.java | 19 +
.../java/org/apache/doris/qe/StmtExecutorTest.java | 14 +-
.../apache/doris/load/loadv2/dpp/Roaring64Map.java | 15 +-
.../doris/load/loadv2/dpp/BitmapValueTest.java | 21 +-
gensrc/proto/olap_file.proto | 2 +-
gensrc/proto/segment_v2.proto | 5 +
gensrc/thrift/AgentService.thrift | 3 +
gensrc/thrift/FrontendService.thrift | 16 +
gensrc/thrift/HeartbeatService.thrift | 1 +
gensrc/thrift/PaloInternalService.thrift | 11 +
188 files changed, 5618 insertions(+), 1757 deletions(-)
create mode 100644
be/src/exec/schema_scanner/schema_schema_privileges_scanner.cpp
copy be/src/exec/schema_scanner/{schema_tables_scanner.h =>
schema_schema_privileges_scanner.h} (72%)
create mode 100644
be/src/exec/schema_scanner/schema_table_privileges_scanner.cpp
copy be/src/exec/schema_scanner/{schema_tables_scanner.h =>
schema_table_privileges_scanner.h} (73%)
create mode 100644
be/src/exec/schema_scanner/schema_user_privileges_scanner.cpp
copy be/src/exec/schema_scanner/{schema_tables_scanner.h =>
schema_user_privileges_scanner.h} (73%)
create mode 100644 be/src/olap/collection.h
create mode 100644 be/src/olap/column_vector.cpp
create mode 100644 be/src/olap/column_vector.h
create mode 100644 be/test/olap/column_vector_test.cpp
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/common/util/QueryPlannerProfile.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/common/util/ToSqlContext.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/rewrite/BinaryPredicatesDateRule.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/catalog/CatalogOperationTest.java
mode change 100755 => 100644
fe/fe-core/src/test/java/org/apache/doris/planner/QueryPlanTest.java
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
