This is an automated email from the ASF dual-hosted git repository.
kxiao pushed a change to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/doris.git
from 296bfb3350 [Enhancement](multi-catalog) Add some checks for
ShowPartitionsStmt. (#21446)
new 09ce2ee071 [Fix](submodule) revert clucene version wrong rollback
(#21523)
new eb3d13536f [Feature](multi-catalog) support query hive-view for
nereids planner. (#21419)
new 5e0e84dc98 [fix](backup) fix show snapshot cauing mysql connection
lost (#21520)
new 4bb920040a [fix](status) Return the correct error code when clucene
error occured (#21511)
new 8c04f750f9 [regression-test](load) add streamload default value test
(#21536)
new 7be5373ed2 [enhancement](query) enable strong consistency by syncing
max journal id from master (#21205)
new 201faa450f [bugfix](RowsetIterator) use valid stats when creating
segment iterator (#21512)
new 90edaead39 [Enhancement](inverted index) make InvertedIndexReader
shared_from_this (#21381)
new 280640f1ca [Bug](row store) Fix column aggregate info lost when table
is unique model (#21506)
new 7814f06290 [Fix](ScannerScheduler) fix dead lock when shutdown
group_local_scan_thread_pool (#21553)
new 25aebd36b3 [Performance](Nereids): add
withGroupExprLogicalPropChildren to reduce new Plan (#21477)
new 2a2d828daa [profile](join) add collisions into profile (#21510)
new acb37a13fb fix: infinit loop when handle exceed limit memory (#21556)
new 1983a9cf0b [Fix](multi-catalog) Fix hadoop short circuit reading can
not enabled in some environments. (#21516)
new 3040f202a8 [Enhancement](multi-catalog) Make meta cache batch loading
concurrently. (#21471)
new d06f625a2a [feature](profile) Add important time of legacy planner to
profile (#20602)
new 92e3a8afe5 [fix](nereids) fix some planner bugs (#21533)
new e48bb09b42 [fix](sink) Fix NodeChannel add_block_closure null pointer
(#21534)
new d54b97eb57 [fix](type system) fix datetimev2 write column to arrow
(#21529)
new 726d7c326f [fix](executor) make elt / repeat smooth upgrade. (#21493)
The 20 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
be/CMakeLists.txt | 1 -
be/src/agent/be_exec_version_manager.h | 3 +-
be/src/clucene | 2 +-
be/src/olap/rowset/beta_rowset_reader.cpp | 14 +-
be/src/olap/rowset/segment_v2/column_reader.cpp | 14 +-
be/src/olap/rowset/segment_v2/column_reader.h | 4 +-
.../rowset/segment_v2/inverted_index_reader.cpp | 17 +--
.../olap/rowset/segment_v2/inverted_index_reader.h | 57 +++++---
be/src/olap/rowset/segment_v2/segment.cpp | 4 +-
be/src/olap/rowset/segment_v2/segment_iterator.cpp | 1 +
be/src/olap/task/index_builder.cpp | 8 +-
be/src/runtime/load_channel_mgr.cpp | 2 +-
be/src/vec/common/hash_table/hash_table.h | 10 +-
.../vec/common/hash_table/partitioned_hash_table.h | 8 ++
.../serde/data_type_datetimev2_serde.cpp | 4 +-
.../vec/exec/join/process_hash_table_probe_impl.h | 1 +
be/src/vec/exec/join/vhash_join_node.cpp | 4 +
be/src/vec/exec/join/vhash_join_node.h | 2 +
be/src/vec/exec/scan/scanner_scheduler.cpp | 4 +
be/src/vec/functions/function_string.cpp | 6 +-
be/src/vec/functions/function_string.h | 61 +++++++-
be/src/vec/sink/vtablet_sink.cpp | 10 +-
bin/run-fs-benchmark.sh | 8 +-
bin/start_be.sh | 11 +-
bin/start_fe.sh | 5 +-
build.sh | 2 -
conf/hdfs-site.xml | 23 ----
docs/en/docs/advanced/variables.md | 4 +
docs/en/docs/lakehouse/multi-catalog/hive.md | 2 +-
docs/zh-CN/docs/advanced/variables.md | 4 +
docs/zh-CN/docs/lakehouse/multi-catalog/hive.md | 2 +-
.../main/java/org/apache/doris/common/Config.java | 4 +-
.../java/org/apache/doris/analysis/Analyzer.java | 6 +-
.../java/org/apache/doris/analysis/ColumnDef.java | 5 +-
.../org/apache/doris/analysis/CreateTableStmt.java | 13 +-
.../java/org/apache/doris/backup/Repository.java | 4 +
.../org/apache/doris/common/ThreadPoolManager.java | 8 ++
.../doris/common/profile/SummaryProfile.java | 45 ++++++
.../apache/doris/common/util/CacheBulkLoader.java | 51 +++++++
.../doris/datasource/ExternalMetaCacheMgr.java | 12 +-
.../doris/datasource/ExternalSchemaCache.java | 15 +-
.../doris/datasource/HMSExternalCatalog.java | 5 +-
.../doris/datasource/hive/HiveMetaStoreCache.java | 153 ++++++++++-----------
.../java/org/apache/doris/nereids/PlanContext.java | 4 +-
.../nereids/analyzer/UnboundOlapTableSink.java | 7 +
.../nereids/analyzer/UnboundOneRowRelation.java | 6 +
.../doris/nereids/analyzer/UnboundRelation.java | 7 +
.../doris/nereids/analyzer/UnboundTVFRelation.java | 6 +
.../java/org/apache/doris/nereids/memo/Memo.java | 8 +-
.../nereids/pattern/GroupExpressionMatching.java | 5 +-
.../doris/nereids/rules/analysis/BindRelation.java | 21 +++
.../nereids/rules/expression/check/CheckCast.java | 12 +-
.../doris/nereids/rules/rewrite/MultiJoin.java | 6 +
.../nereids/rules/rewrite/PruneOlapScanTablet.java | 12 +-
.../apache/doris/nereids/trees/plans/FakePlan.java | 6 +
.../doris/nereids/trees/plans/GroupPlan.java | 6 +
.../org/apache/doris/nereids/trees/plans/Plan.java | 3 +
.../nereids/trees/plans/commands/Command.java | 6 +
.../trees/plans/logical/LogicalAggregate.java | 7 +
.../nereids/trees/plans/logical/LogicalApply.java | 8 ++
.../trees/plans/logical/LogicalAssertNumRows.java | 7 +
.../nereids/trees/plans/logical/LogicalCTE.java | 7 +
.../trees/plans/logical/LogicalCTEAnchor.java | 6 +
.../trees/plans/logical/LogicalCTEConsumer.java | 9 ++
.../trees/plans/logical/LogicalCTEProducer.java | 10 +-
.../trees/plans/logical/LogicalCheckPolicy.java | 6 +
.../trees/plans/logical/LogicalEmptyRelation.java | 6 +
.../nereids/trees/plans/logical/LogicalEsScan.java | 7 +
.../nereids/trees/plans/logical/LogicalExcept.java | 6 +
.../trees/plans/logical/LogicalFileScan.java | 7 +
.../nereids/trees/plans/logical/LogicalFilter.java | 6 +
.../trees/plans/logical/LogicalGenerate.java | 6 +
.../nereids/trees/plans/logical/LogicalHaving.java | 7 +
.../trees/plans/logical/LogicalIntersect.java | 6 +
.../trees/plans/logical/LogicalJdbcScan.java | 7 +
.../nereids/trees/plans/logical/LogicalJoin.java | 7 +
.../nereids/trees/plans/logical/LogicalLimit.java | 6 +
.../trees/plans/logical/LogicalOlapScan.java | 16 ++-
.../trees/plans/logical/LogicalOlapTableSink.java | 7 +
.../trees/plans/logical/LogicalOneRowRelation.java | 8 +-
.../trees/plans/logical/LogicalPartitionTopN.java | 7 +
.../trees/plans/logical/LogicalProject.java | 7 +
.../nereids/trees/plans/logical/LogicalRepeat.java | 7 +
.../trees/plans/logical/LogicalSchemaScan.java | 6 +
.../trees/plans/logical/LogicalSelectHint.java | 6 +
.../nereids/trees/plans/logical/LogicalSort.java | 7 +
.../trees/plans/logical/LogicalSubQueryAlias.java | 15 +-
.../trees/plans/logical/LogicalTVFRelation.java | 8 ++
.../nereids/trees/plans/logical/LogicalTopN.java | 6 +
.../nereids/trees/plans/logical/LogicalUnion.java | 7 +
.../nereids/trees/plans/logical/LogicalWindow.java | 26 ++--
.../nereids/trees/plans/logical/UsingJoin.java | 7 +
.../plans/physical/PhysicalAssertNumRows.java | 7 +
.../trees/plans/physical/PhysicalCTEAnchor.java | 7 +
.../trees/plans/physical/PhysicalCTEConsumer.java | 7 +
.../trees/plans/physical/PhysicalCTEProducer.java | 6 +
.../trees/plans/physical/PhysicalDistribute.java | 7 +
.../plans/physical/PhysicalEmptyRelation.java | 7 +
.../trees/plans/physical/PhysicalEsScan.java | 7 +
.../trees/plans/physical/PhysicalExcept.java | 6 +
.../trees/plans/physical/PhysicalFileScan.java | 8 ++
.../trees/plans/physical/PhysicalFilter.java | 6 +
.../trees/plans/physical/PhysicalGenerate.java | 7 +
.../plans/physical/PhysicalHashAggregate.java | 8 ++
.../trees/plans/physical/PhysicalHashJoin.java | 7 +
.../trees/plans/physical/PhysicalIntersect.java | 6 +
.../trees/plans/physical/PhysicalJdbcScan.java | 7 +
.../trees/plans/physical/PhysicalLimit.java | 6 +
.../plans/physical/PhysicalNestedLoopJoin.java | 8 ++
.../trees/plans/physical/PhysicalOlapScan.java | 9 ++
.../plans/physical/PhysicalOlapTableSink.java | 7 +
.../plans/physical/PhysicalOneRowRelation.java | 7 +
.../plans/physical/PhysicalPartitionTopN.java | 7 +
.../trees/plans/physical/PhysicalProject.java | 6 +
.../trees/plans/physical/PhysicalQuickSort.java | 6 +
.../trees/plans/physical/PhysicalRepeat.java | 7 +
.../trees/plans/physical/PhysicalSchemaScan.java | 7 +
.../physical/PhysicalStorageLayerAggregate.java | 7 +
.../trees/plans/physical/PhysicalTVFRelation.java | 9 ++
.../nereids/trees/plans/physical/PhysicalTopN.java | 7 +
.../trees/plans/physical/PhysicalUnion.java | 6 +
.../trees/plans/physical/PhysicalWindow.java | 7 +
.../doris/planner/HashDistributionPruner.java | 12 +-
.../org/apache/doris/planner/OlapScanNode.java | 3 +-
.../org/apache/doris/planner/OriginalPlanner.java | 7 +-
.../apache/doris/planner/SingleNodePlanner.java | 3 +
.../java/org/apache/doris/qe/MasterOpExecutor.java | 107 +++++++++-----
.../java/org/apache/doris/qe/SessionVariable.java | 16 +++
.../java/org/apache/doris/qe/StmtExecutor.java | 9 ++
.../apache/doris/service/FrontendServiceImpl.java | 5 +
.../apache/doris/common/CacheBulkLoaderTest.java | 84 +++++++++++
.../apache/doris/external/hms/HmsCatalogTest.java | 137 +++++++++---------
.../doris/nereids/jobs/RewriteTopDownJobTest.java | 6 +
.../doris/nereids/trees/plans/PlanOutputTest.java | 6 +
.../doris/planner/HashDistributionPrunerTest.java | 2 +-
.../org/apache/doris/planner/OlapScanNodeTest.java | 6 +-
gensrc/thrift/FrontendService.thrift | 1 +
.../data/correctness_p0/test_constant_having.out | 5 +
.../load_p0/stream_load/test_default_value.csv | 2 +
.../correctness_p0/test_constant_having.groovy | 74 ++++++++++
.../load_p0/stream_load/test_stream_load.groovy | 58 ++++++++
.../suites/nereids_p0/datatype/test_cast.groovy | 20 +++
.../suites/nereids_p0/test_prune_tablet_mv.groovy | 45 ++++++
regression-test/suites/row_store/load.groovy | 70 ++++++++++
regression-test/suites/source_p0/ddl/create.sql | 19 +++
regression-test/suites/source_p0/ddl/insert.sql | 1 +
.../suites/source_p0/doris_source.groovy | 39 ++++++
thirdparty/build-thirdparty.sh | 3 +
148 files changed, 1595 insertions(+), 359 deletions(-)
delete mode 100644 conf/hdfs-site.xml
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/common/util/CacheBulkLoader.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/common/CacheBulkLoaderTest.java
create mode 100644 regression-test/data/correctness_p0/test_constant_having.out
create mode 100644
regression-test/data/load_p0/stream_load/test_default_value.csv
create mode 100644
regression-test/suites/correctness_p0/test_constant_having.groovy
create mode 100644
regression-test/suites/nereids_p0/test_prune_tablet_mv.groovy
create mode 100644 regression-test/suites/row_store/load.groovy
create mode 100644 regression-test/suites/source_p0/ddl/create.sql
create mode 100644 regression-test/suites/source_p0/ddl/insert.sql
create mode 100644 regression-test/suites/source_p0/doris_source.groovy
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
