This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a change to branch opt_dict_perf
in repository https://gitbox.apache.org/repos/asf/doris.git
omit f2407030fb f
omit 0ba1797a59 f
omit 2004fa3c4b f
omit f9446dedf4 f
omit 6b5052e4bd try tofix profile
omit 80445655f3 [improvement](dict) cache the value of find code (#17674)
add 948654ad38 [fix](ui)format the text file of profile #17645
add 47e9217c1e [improvement](pipeline)Avoid duplicate trigger teamcity
build (#17647)
add 2739a44eaf [fix](segcompaction) heap overflow when doing segcompaction
for cancelling load(#17529)
add 6dcd791b74 [feature](struct-type) support CAST AS Struct type (#17553)
add 566d133610 [enhancement](Nereids) Refactor EliminateLimitTest and
EliminateFilterTest by match-pattern (#17631)
add 051ab7a9c6 [refactor](Nereids): refactor Join-Dependent Predicate
Duplication. (#17653)
add 3745e6c18a [fix](Nereids): order of project's logical properties is
different with that of project expression (#17648)
add 48a2fe68ad [typo](docs) Fix some display errors (#17663)
add a74ef2377f typo fix in kyuubi doc (#17672)
add 697cba9a85 [fix](broker-load) fix broker's Dockerfile (#17657)
add 8328ab69ad [Chore](Materialized-View) add some mv regression test case
(#17345)
add 9c7854f1ff [Enhancement](k8s) Add k8s yaml demo (#17281)
add db9692a114 [feature](Nereids): convert CrossJoin to InnerJoin. (#17681)
add 3231fab8c2 [feature](nereids) add unique id for groupExpression and
plan node (#17628)
add d7cb5cf3db [feature](nereids) add session var: dump_nereids_memo
(#17666)
add 692d510edb [fix](schema_hash) remove useless schema_hash param in
tablet and replica url (#17489)
add 9745ee60a7 [fix](priv) fix bug of grant priv on ctl.db.* not work
(#17612)
add 4566de1cec Enhancement in some Hyperlinks (#17654)
add d774162a53 [minor](Nereids): rename rule (#17509)
add 9b687026bd [Doc](TLS) add doc for TLS connection (#17683)
add 11fbe07221 [refactor](Nereids) Refactor all rewrite logical unit tests
by match-pattern (#17691)
add b93e553958 [enhance](Nereids): allow empty hash condition (#17699)
add a452db35da [improvement](filecache)Change the hash field of the
backend (#17499)
add 455c800405 [feature](parquet-reader) add rle bool and delta decoder to
read AWS Glue (#17112)
add 0d05e4cce0 [Improvement](multi-catalog) The interface of external
Splitter. WIP (#17390)
add 13e05c4a5d [Enhencement](stream load) add some regression test for
json format streamload (#17520)
add a651926ba9 [fix](fqdn) Add UnknownHostException handle logic in
FQDNManager to avoid that active ip could be incorrectly assigned to dead be or
dead fe (#17689)
add 54e5c71e52 [fix](planner) Fix NPE when update stats by profile
add 46dcf69644 [fix](jdbc-catalog) avoid calculate driver's md5 when
replaying edit log (#17693)
add b0d1166989 [fix](meta) fix concurrent modification exception and
potential NPE (#17602)
add 6386458498 [Refactor](exec) remove unless attr of slot ref (#17688)
add 7948c8ede2 [bug](function) Fix bug in the process of generating
template functions #17676
add 33059d92cc [docs](doc) fix faq docs (#17707)
add 47cfc81925 [fix docs] (#17634)
add 93a865c3e8 [improvement](join) Avoid reading from left child while
hash table is empty(right join) (#17655)
add a0a2809324 [Enhancement](multi-catalog) support hms event
deserialization for HDP/CDH Hive versions. (#17660)
add edb2d90852 [fix](routine load) fix ROUTINE LOAD bug,kafka commit a
lack of one(#17282) (#17291)
add 39b5682d59 [Pipeline](shared_scan_opt) Support shared scan opt in
pipeline exec engine
add 3a6c0e7867 [fix](regression) fix test_array_export and test_map_export
dir conflict #17636
add 55c42da511 [Feature](array) Support array<decimalv3> data type (#16640)
add 782001c75b [fix](planner) project should be done inside subquery
(#17630)
add 16fc3a0e22 [Chore](compile) remove some unused static on inline
function to reduce compile time (#17603)
add 5fccbac81b [fix](demo)add Sync full database for versions below doris
1.2 (#17669)
add b9fac82fb1 [fix](regression) adjust regression pipeline
config(tablet_create_timeout_second) for avoiding create partition timeout
(#17668)
add 2b31fc1472 [fix](regression) segcompaction timeout too short (#16731)
(#17565)
add be5147c32e [enhancement](feservice) catch throwable and print log for
frontend service (#17708)
add ac944e2ac1 [fix](cooldown)Fix bug for storage policy in dynamic
partition (#17665)
add c302fa2564 [Feature](array-function) Support array_pushfront function
(#17584)
new b8616a6cb2 [improvement](dict) cache the value of find code (#17674)
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (f2407030fb)
\
N -- N -- N refs/heads/opt_dict_perf (b8616a6cb2)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.github/workflows/auto_trigger_teamcity.yml | 341 ++++++---------------
README.md | 4 +-
be/src/common/signal_handler.h | 16 +-
be/src/common/status.h | 6 +-
be/src/common/utils.h | 2 +-
be/src/exec/es/es_scroll_parser.cpp | 28 +-
be/src/exec/olap_utils.h | 8 +-
be/src/exprs/create_predicate_function.h | 25 +-
be/src/exprs/runtime_filter.cpp | 50 ---
be/src/gutil/atomicops-internals-x86.h | 2 +-
be/src/gutil/basictypes.h | 2 +-
be/src/gutil/bits.h | 6 +-
be/src/gutil/casts.h | 20 +-
be/src/gutil/dynamic_annotations.h | 6 +-
be/src/gutil/gscoped_ptr.h | 6 +-
be/src/gutil/port.h | 2 +-
be/src/gutil/stl_util.h | 14 +-
be/src/gutil/strings/escaping.cc | 4 +-
be/src/gutil/strings/join.h | 13 +-
be/src/gutil/strings/split.cc | 8 +-
be/src/gutil/strings/split.h | 19 +-
be/src/gutil/strings/strip.h | 2 +-
be/src/gutil/strings/util.cc | 10 +-
be/src/gutil/strings/util.h | 4 +-
be/src/olap/delta_writer.cpp | 3 +
be/src/olap/file_helper.h | 4 +-
be/src/olap/predicate_creator.h | 19 +-
be/src/olap/rowset/beta_rowset_writer.cpp | 8 +-
be/src/olap/rowset/beta_rowset_writer.h | 5 +-
be/src/olap/rowset/rowset_writer.h | 4 +
be/src/olap/skiplist.h | 16 +-
be/src/olap/types.cpp | 2 +-
be/src/olap/types.h | 2 +-
be/src/olap/utils.cpp | 4 +
be/src/olap/utils.h | 12 +-
be/src/pipeline/exec/scan_operator.cpp | 20 +-
be/src/pipeline/pipeline_fragment_context.cpp | 3 +
be/src/pipeline/pipeline_task.cpp | 3 +
be/src/runtime/exec_env.h | 2 +-
be/src/runtime/memory/thread_mem_tracker_mgr.h | 2 +-
be/src/runtime/query_fragments_ctx.h | 7 +
.../routine_load/routine_load_task_executor.cpp | 5 +-
be/src/runtime/runtime_state.h | 6 +
be/src/service/point_query_executor.cpp | 4 +-
be/src/util/binary_cast.hpp | 2 +-
be/src/util/bit_packing.inline.h | 7 +-
be/src/util/bit_stream_utils.h | 16 +-
be/src/util/bit_stream_utils.inline.h | 92 ++++--
be/src/util/bitmap_intersect.h | 24 +-
be/src/util/coding.h | 14 +-
be/src/util/frame_of_reference_coding.h | 6 +-
be/src/util/interval_tree-inl.h | 2 +-
be/src/util/mysql_row_buffer.cpp | 4 +-
be/src/util/proto_util.h | 34 +-
be/src/util/rle_encoding.h | 50 +--
be/src/util/simd/bits.h | 4 +-
be/src/util/string_parser.hpp | 16 +-
be/src/util/string_util.h | 6 +-
be/src/util/type_traits.h | 4 +-
be/src/util/uid_util.h | 4 +-
be/src/vec/CMakeLists.txt | 6 +-
.../aggregate_function_bitmap.cpp | 2 +-
.../aggregate_function_min_max.cpp | 6 +-
.../aggregate_function_min_max_by.cpp | 10 +-
.../aggregate_function_stddev.cpp | 8 +-
.../aggregate_function_window.cpp | 4 +-
be/src/vec/aggregate_functions/factory_helpers.h | 2 +-
be/src/vec/common/aggregation_common.h | 22 +-
be/src/vec/common/bit_helpers.h | 6 +-
.../vec/common/hash_table/hash_table_key_holder.h | 6 +-
be/src/vec/common/hex.h | 6 +-
be/src/vec/common/memcmp_small.h | 26 +-
be/src/vec/common/nan_utils.h | 8 +-
be/src/vec/common/unaligned.h | 4 +-
be/src/vec/common/volnitsky.h | 7 +-
be/src/vec/core/call_on_type_index.h | 2 +-
be/src/vec/core/field.cpp | 6 +-
be/src/vec/data_types/data_type.h | 26 +-
be/src/vec/data_types/data_type_decimal.h | 20 +-
be/src/vec/data_types/data_type_factory.hpp | 19 ++
be/src/vec/data_types/data_type_struct.cpp | 5 +-
be/src/vec/exec/format/orc/vorc_reader.cpp | 7 +-
.../vec/exec/format/parquet/bool_rle_decoder.cpp | 85 +++++
.../exec/format/parquet/bool_rle_decoder.h} | 45 +--
be/src/vec/exec/format/parquet/decoder.cpp | 44 +++
.../exec/format/parquet/delta_bit_pack_decoder.cpp | 319 +++++++++++++++++++
.../exec/format/parquet/delta_bit_pack_decoder.h | 280 +++++++++++++++++
be/src/vec/exec/scan/new_olap_scan_node.cpp | 7 +-
be/src/vec/exec/scan/pip_scanner_context.h | 62 +++-
be/src/vec/exec/scan/scanner_context.cpp | 105 ++++---
be/src/vec/exec/scan/scanner_context.h | 49 ++-
be/src/vec/exec/scan/vscan_node.cpp | 105 +++++--
be/src/vec/exec/scan/vscan_node.h | 12 +-
be/src/vec/exprs/vslot_ref.cpp | 9 +-
be/src/vec/exprs/vslot_ref.h | 2 -
.../functions/array/function_array_aggregation.cpp | 2 +
.../functions/array/function_array_difference.h | 3 +
.../vec/functions/array/function_array_distinct.h | 9 +
.../vec/functions/array/function_array_element.h | 12 +
be/src/vec/functions/array/function_array_index.h | 24 ++
be/src/vec/functions/array/function_array_join.h | 10 +
.../functions/array/function_array_pushfront.cpp | 113 +++++++
.../functions/array/function_array_register.cpp | 2 +
be/src/vec/functions/array/function_array_remove.h | 19 ++
be/src/vec/functions/array/function_array_set.h | 3 +
be/src/vec/functions/array/function_array_sort.h | 10 +
.../vec/functions/array/function_arrays_overlap.h | 14 +-
be/src/vec/functions/function.cpp | 65 ++++
be/src/vec/functions/function.h | 42 +--
be/src/vec/functions/function_cast.h | 81 ++++-
be/src/vec/functions/function_encryption.cpp | 10 +-
be/src/vec/functions/function_helpers.h | 4 +-
be/src/vec/functions/function_string.h | 4 +-
be/src/vec/functions/function_timestamp.cpp | 2 +-
be/src/vec/functions/functions_logical.cpp | 20 +-
be/src/vec/functions/functions_logical.h | 4 +-
be/src/vec/functions/regexps.h | 8 +-
be/src/vec/io/io_helper.h | 18 +-
be/src/vec/io/var_int.h | 4 +-
be/src/vec/runtime/shared_hash_table_controller.h | 2 +-
be/src/vec/runtime/shared_scanner_controller.h | 69 +++++
be/src/vec/sink/vmysql_result_writer.cpp | 2 +-
be/src/vec/utils/arrow_column_to_doris_column.cpp | 23 +-
be/src/vec/utils/histogram_helpers.hpp | 15 +-
.../utils/arrow_column_to_doris_column_test.cpp | 20 +-
docker/runtime/broker/Dockerfile | 2 +-
docker/runtime/k8s/doris_be.yml | 105 +++++++
docker/runtime/k8s/doris_cn.yml | 106 +++++++
docker/runtime/k8s/doris_follower.yml | 132 ++++++++
docs/en/docs/admin-manual/certificate.md | 57 ++++
.../admin-manual/http-actions/fe/profile-action.md | 2 +-
docs/en/docs/data-table/data-partition.md | 6 +
docs/en/docs/ecosystem/kyuubi.md | 38 +--
docs/en/docs/get-starting/get-starting.md | 20 ++
docs/en/docs/lakehouse/multi-catalog/hive.md | 19 +-
docs/en/docs/lakehouse/multi-catalog/iceberg.md | 6 +-
.../array-functions/array_pushfront.md | 85 +++++
docs/sidebars.json | 2 +
docs/zh-CN/docs/admin-manual/certificate.md | 61 ++++
.../admin-manual/http-actions/fe/profile-action.md | 2 +-
docs/zh-CN/docs/ecosystem/kyuubi.md | 35 +--
docs/zh-CN/docs/faq/install-faq.md | 8 +-
docs/zh-CN/docs/get-starting/get-starting.md | 20 ++
docs/zh-CN/docs/lakehouse/multi-catalog/hive.md | 17 +
docs/zh-CN/docs/lakehouse/multi-catalog/iceberg.md | 6 +-
.../array-functions/array_pushfront.md | 85 +++++
.../main/java/org/apache/doris/catalog/Type.java | 3 +
.../org/apache/doris/analysis/DescriptorTable.java | 12 +-
.../main/java/org/apache/doris/analysis/Expr.java | 19 ++
.../apache/doris/analysis/FunctionCallExpr.java | 39 ++-
.../apache/doris/common/proc/ReplicasProcNode.java | 10 +-
.../apache/doris/common/proc/TabletsProcDir.java | 8 +-
.../common/profile/MultiProfileTreeBuilder.java | 4 -
.../doris/common/profile/ProfileTreeBuilder.java | 6 -
.../apache/doris/common/util/RuntimeProfile.java | 2 +-
.../apache/doris/datasource/InternalCatalog.java | 12 +-
.../doris/datasource/JdbcExternalCatalog.java | 2 +-
.../doris/datasource/hive/HiveMetaStoreCache.java | 4 +-
.../datasource/hive/event/AddPartitionEvent.java | 2 +-
.../datasource/hive/event/AlterPartitionEvent.java | 2 +-
.../datasource/hive/event/AlterTableEvent.java | 2 +-
.../datasource/hive/event/CreateTableEvent.java | 3 +-
.../datasource/hive/event/DropPartitionEvent.java | 2 +-
.../datasource/hive/event/DropTableEvent.java | 2 +-
.../hive/event/GzipJSONMessageDeserializer.java | 172 +++++++++++
.../hive/event/MetastoreEventsProcessor.java | 14 +-
.../apache/doris/mysql/privilege/DbPrivTable.java | 7 +-
.../org/apache/doris/nereids/NereidsPlanner.java | 18 +-
.../org/apache/doris/nereids/StatementContext.java | 8 +-
.../nereids/analyzer/UnboundOneRowRelation.java | 10 +-
.../doris/nereids/analyzer/UnboundRelation.java | 12 +-
.../doris/nereids/analyzer/UnboundTVFRelation.java | 10 +-
.../glue/translator/PhysicalPlanTranslator.java | 28 +-
.../glue/translator/RuntimeFilterTranslator.java | 4 +-
.../doris/nereids/jobs/batch/NereidsRewriter.java | 4 +-
.../doris/nereids/jobs/cascades/ApplyRuleJob.java | 1 +
.../java/org/apache/doris/nereids/memo/Group.java | 8 +-
.../apache/doris/nereids/memo/GroupExpression.java | 20 +-
.../java/org/apache/doris/nereids/memo/Memo.java | 20 +-
.../processor/post/RuntimeFilterContext.java | 12 +-
.../processor/post/RuntimeFilterGenerator.java | 8 +-
.../properties/ChildOutputPropertyDeriver.java | 5 +-
.../org/apache/doris/nereids/rules/RuleType.java | 6 +-
.../rules/exploration/join/InnerJoinLAsscom.java | 11 +-
.../exploration/join/InnerJoinLAsscomProject.java | 13 +-
.../rules/rewrite/logical/AdjustNullable.java | 2 +-
...CrossJoin.java => ConvertInnerOrCrossJoin.java} | 25 +-
...xtractSingleTableExpressionFromDisjunction.java | 149 ++++-----
.../rules/rewrite/logical/PushdownLimit.java | 4 +-
.../doris/nereids/stats/StatsErrorEstimator.java | 3 +
.../doris/nereids/trees/AbstractTreeNode.java | 4 +-
.../doris/nereids/trees/expressions/Alias.java | 2 +-
.../doris/nereids/trees/expressions/ExprId.java | 2 +-
.../nereids/trees/expressions/SlotReference.java | 8 +-
...ionUtil.java => StatementScopeIdGenerator.java} | 11 +-
.../trees/expressions/VirtualSlotReference.java | 2 +-
.../trees/plans/{RelationId.java => ObjectId.java} | 17 +-
.../trees/plans/logical/LogicalAggregate.java | 2 +-
.../nereids/trees/plans/logical/LogicalEsScan.java | 6 +-
.../trees/plans/logical/LogicalFileScan.java | 6 +-
.../nereids/trees/plans/logical/LogicalFilter.java | 53 +---
.../trees/plans/logical/LogicalJdbcScan.java | 6 +-
.../nereids/trees/plans/logical/LogicalJoin.java | 2 +-
.../trees/plans/logical/LogicalOlapScan.java | 14 +-
.../trees/plans/logical/LogicalProject.java | 2 +-
.../trees/plans/logical/LogicalRelation.java | 12 +-
.../trees/plans/logical/LogicalSchemaScan.java | 6 +-
.../nereids/trees/plans/logical/LogicalSort.java | 2 +-
.../trees/plans/logical/LogicalTVFRelation.java | 6 +-
.../nereids/trees/plans/logical/RelationUtil.java | 8 +-
.../trees/plans/physical/PhysicalDistribute.java | 2 +-
.../trees/plans/physical/PhysicalEsScan.java | 6 +-
.../trees/plans/physical/PhysicalFileScan.java | 6 +-
.../trees/plans/physical/PhysicalFilter.java | 2 +-
.../plans/physical/PhysicalHashAggregate.java | 2 +-
.../trees/plans/physical/PhysicalHashJoin.java | 2 +-
.../trees/plans/physical/PhysicalJdbcScan.java | 6 +-
.../trees/plans/physical/PhysicalOlapScan.java | 6 +-
.../trees/plans/physical/PhysicalProject.java | 2 +-
.../trees/plans/physical/PhysicalRelation.java | 10 +-
.../trees/plans/physical/PhysicalSchemaScan.java | 6 +-
.../trees/plans/physical/PhysicalTVFRelation.java | 6 +-
.../trees/plans/physical/PhysicalWindow.java | 2 +-
.../apache/doris/planner/NestedLoopJoinNode.java | 4 +-
.../org/apache/doris/planner/OlapScanNode.java | 45 +++
.../TableFormatType.java => OlapSplitter.java} | 18 +-
.../java/org/apache/doris/planner/SelectNode.java | 5 +-
.../{external/HiveSplit.java => Split.java} | 16 +-
.../TableFormatType.java => Splitter.java} | 18 +-
.../doris/planner/external/FileScanProviderIf.java | 7 -
.../external/{HiveSplit.java => FileSplit.java} | 27 +-
.../doris/planner/external/FileSplitStrategy.java | 2 -
.../doris/planner/external/HiveScanProvider.java | 93 +-----
.../doris/planner/external/HiveSplitter.java | 155 ++++++++++
.../doris/planner/external/IcebergSplitter.java | 154 ++++++++++
.../doris/planner/external/LoadScanProvider.java | 8 -
.../doris/planner/external/QueryScanProvider.java | 149 +++++----
.../doris/planner/external/TVFScanProvider.java | 19 +-
.../apache/doris/planner/external/TVFSplitter.java | 56 ++++
.../doris/planner/external/TableFormatType.java | 1 +
.../external/iceberg/IcebergScanProvider.java | 118 +------
.../planner/external/iceberg/IcebergSplit.java | 4 +-
.../main/java/org/apache/doris/qe/Coordinator.java | 28 +-
.../java/org/apache/doris/qe/QeProcessorImpl.java | 10 -
.../java/org/apache/doris/qe/SessionVariable.java | 14 +
.../java/org/apache/doris/service/FeServer.java | 17 +-
.../java/org/apache/doris/system/FQDNManager.java | 17 +
.../org/apache/doris/system/SystemInfoService.java | 4 -
.../doris/transaction/GlobalTransactionMgr.java | 17 +-
.../event/GzipJSONMessageDeserializerTest.java | 63 ++++
.../datasets/tpch/AnalyzeCheckTestBase.java | 4 +-
.../postprocess/MergeProjectPostProcessTest.java | 4 +-
.../rules/analysis/AnalyzeSubQueryTest.java | 6 +-
.../rules/analysis/AnalyzeWhereSubqueryTest.java | 6 +-
.../rules/analysis/BindSlotReferenceTest.java | 4 +-
.../nereids/rules/analysis/RegisterCTETest.java | 6 +-
.../rules/exploration/join/OuterJoinAssocTest.java | 4 +-
.../rewrite/ExpressionRewriteTestHelper.java | 4 +-
.../expression/rewrite/SimplifyRangeTest.java | 4 +-
.../rules/rewrite/logical/EliminateFilterTest.java | 40 +--
.../logical/EliminateGroupByConstantTest.java | 198 ++++++------
.../rules/rewrite/logical/EliminateLimitTest.java | 45 +--
.../rewrite/logical/EliminateOuterJoinTest.java | 4 +-
.../logical/EliminateUnnecessaryProjectTest.java | 49 ++-
...ctSingleTableExpressionFromDisjunctionTest.java | 4 +-
.../logical/FindHashConditionForJoinTest.java | 32 +-
.../rules/rewrite/logical/MergeFiltersTest.java | 47 ++-
.../rules/rewrite/logical/MergeLimitsTest.java | 46 ++-
.../logical/PhysicalStorageLayerAggregateTest.java | 4 +-
.../logical/PruneOlapScanPartitionTest.java | 74 +++--
.../rewrite/logical/PruneOlapScanTabletTest.java | 24 +-
.../PushdownExpressionsInHashConditionTest.java | 4 +-
.../logical/PushdownJoinOtherConditionTest.java | 170 +++++-----
.../rules/rewrite/logical/PushdownLimitTest.java | 8 +-
.../logical/PushdownProjectThroughLimitTest.java | 4 +-
.../rules/rewrite/logical/SplitLimitTest.java | 25 +-
.../apache/doris/nereids/sqltest/SqlTestBase.java | 4 +-
.../doris/nereids/trees/expressions/ViewTest.java | 4 +-
.../doris/nereids/trees/plans/PlanEqualsTest.java | 2 +-
.../nereids/trees/plans/PlanToStringTest.java | 12 +-
.../doris/nereids/util/LogicalPlanBuilder.java | 7 +
.../org/apache/doris/nereids/util/PlanChecker.java | 7 +
.../apache/doris/nereids/util/PlanConstructor.java | 8 +-
.../apache/doris/nereids/util/ReadLockTest.java | 6 +-
.../org/apache/doris/planner/QueryPlanTest.java | 6 +-
.../doris/planner/SingleNodePlannerTest.java | 298 ------------------
.../apache/doris/utframe/TestWithFeService.java | 4 +-
fe/pom.xml | 2 -
gensrc/script/doris_builtins_functions.py | 31 +-
gensrc/script/gen_builtins_functions.py | 6 +-
gensrc/thrift/PaloInternalService.thrift | 4 +
.../data/correctness_p0/test_current_timestamp.out | 35 ++-
.../test_current_timestamp_streamload.json | 2 +
...ection.out => test_inlineview_with_project.out} | 0
.../hive/test_external_catalog_glue_table.out | 131 ++++++++
.../load_p0/stream_load/case_sensitive_json.json | 2 +
.../data/load_p0/stream_load/test_json_load.out | 5 +
.../test_create_mv_datatype.out | 31 --
.../agg_have_dup_base/agg_have_dup_base.out | 0
.../k1ap2spa/k1ap2spa.out | 0
.../multi_slot_k123p/multi_slot_k123p.out | 0
.../multi_slot_k1a2p2ap3p.out | 0
.../multi_slot_k1a2p2ap3ps.out | 0
.../multi_slot_k1p2ap3p/multi_slot_k1p2ap3p.out | 0
.../multi_slot_k1p2ap3ps/multi_slot_k1p2ap3ps.out | 0
.../multi_slot_multi_mv/multi_slot_multi_mv.out | 0
.../mv_with_view/mv_with_view.out | 0
.../test_dup_group_by_mv_abs.out | 0
.../test_dup_group_by_mv_plus.out | 0
.../test_dup_mv_abs/test_dup_mv_abs.out | 0
.../test_dup_mv_bin/test_dup_mv_bin.out | 0
.../test_dup_mv_bitmap_hash.out | 0
.../test_dup_mv_plus/test_dup_mv_plus.out | 0
.../test_dup_mv_year/test_dup_mv_year.out | 0
.../testIncorrectMVRewriteInSubquery.out} | 11 +-
.../ut/testProjectionMV1/testProjectionMV1.out | 0
.../ut/testProjectionMV2/testProjectionMV2.out | 0
.../ut/testProjectionMV3/testProjectionMV3.out | 0
.../ut/testProjectionMV4/testProjectionMV4.out | 0
.../ut/testQueryOnStar/testQueryOnStar.out} | 5 +-
.../testSelectMVWithTableAlias.out} | 12 +-
.../data/nereids_syntax_p0/grouping_sets.out | 11 +-
.../nereids_syntax_p0/sub_query_correlated.out | 90 +++---
.../sub_query_diff_old_optimize.out | 12 +-
.../aggregate_functions/test_aggregate_collect.out | 64 ----
.../test_aggregate_histogram.out | 8 +-
.../test_array_aggregation_functions.out | 40 +--
.../array_functions/test_array_functions.out | 286 +++++++++++++++++
.../test_array_functions_by_literal.out | 87 ++++++
.../array_functions/test_array_with_scale_type.out | 20 ++
.../test_cast_array_functions_by_literal.out | 3 +
.../cast_function/test_cast_struct.out | 43 +++
regression-test/pipeline/p0/conf/fe.conf | 4 +-
regression-test/pipeline/p1/conf/fe.conf | 4 +-
.../correctness_p0/test_colocate_join.groovy | 2 +-
.../correctness_p0/test_current_timestamp.groovy | 69 ++++-
.../test_inlineview_with_project.groovy | 80 +++++
.../duplicate/storage/test_dup_tab_char.groovy | 2 +-
.../suites/export/test_array_export.groovy | 2 +-
.../suites/export/test_map_export.groovy | 2 +-
.../hive/test_external_catalog_glue_table.groovy | 54 ++++
.../load_p0/stream_load/test_json_load.groovy | 16 +-
.../agg_have_dup_base/agg_have_dup_base.groovy | 12 +-
.../k1ap2spa/k1ap2spa.groovy | 12 +-
.../multi_slot_k123p/multi_slot_k123p.groovy | 12 +-
.../multi_slot_k1a2p2ap3p.groovy | 12 +-
.../multi_slot_k1a2p2ap3ps.groovy | 12 +-
.../multi_slot_k1p2ap3p/multi_slot_k1p2ap3p.groovy | 12 +-
.../multi_slot_k1p2ap3ps.groovy | 12 +-
.../multi_slot_multi_mv/multi_slot_multi_mv.groovy | 12 +-
.../mv_with_view/mv_with_view.groovy | 4 +
.../test_create_mv_complex_type.groovy | 13 -
.../test_dup_group_by_mv_abs.groovy | 12 +-
.../test_dup_group_by_mv_plus.groovy | 12 +-
.../test_dup_mv_abs/test_dup_mv_abs.groovy | 12 +-
.../test_dup_mv_bin/test_dup_mv_bin.groovy | 12 +-
.../test_dup_mv_bitmap_hash.groovy | 13 +-
.../test_dup_mv_plus/test_dup_mv_plus.groovy | 12 +-
.../test_dup_mv_year/test_dup_mv_year.groovy | 13 +-
.../test_mv_useless/test_agg_mv_useless.groovy | 0
.../test_mv_useless/test_dup_mv_useless.groovy | 0
.../test_mv_useless/test_uniq_mv_useless.groovy | 0
.../testIncorrectMVRewriteInSubquery.groovy | 49 +++
.../ut/testProjectionMV1/testProjectionMV1.groovy | 0
.../ut/testProjectionMV2/testProjectionMV2.groovy | 0
.../ut/testProjectionMV3/testProjectionMV3.groovy | 0
.../ut/testProjectionMV4/testProjectionMV4.groovy | 0
.../ut/testQueryOnStar/testQueryOnStar.groovy} | 11 +-
.../testSelectMVWithTableAlias.groovy} | 36 +--
.../nereids_function_p0/gen_function/gen.groovy | 2 +-
.../suites/nereids_syntax_p0/grouping_sets.groovy | 4 +-
.../suites/nereids_syntax_p0/set_operation.groovy | 6 +-
.../nereids_syntax_p0/sub_query_correlated.groovy | 46 +--
.../sub_query_diff_old_optimize.groovy | 4 +-
.../performance_p0/redundant_conjuncts.groovy | 4 +-
.../test_aggregate_collect.groovy | 28 +-
.../test_array_aggregation_functions.groovy | 24 +-
.../array_functions/test_array_functions.groovy | 49 ++-
.../test_array_functions_by_literal.groovy | 31 ++
.../test_array_with_scale_type.groovy | 6 +
.../test_cast_array_functions_by_literal.groovy | 2 +
.../cast_function/test_cast_struct.groovy | 78 +++++
.../conditional_functions/test_query_limit.groovy | 6 +-
.../window_functions/test_window_fn.groovy | 20 +-
.../test_segcompaction_agg_keys.groovy | 2 +-
.../test_segcompaction_dup_keys.groovy | 2 +-
.../test_segcompaction_unique_keys.groovy | 2 +-
.../test_segcompaction_unique_keys_mow.groovy | 2 +-
.../demo/flink/dbsync/DatabaseFullSyncV2.java | 204 ++++++++++++
ui/src/pages/query-profile/index.tsx | 2 +-
390 files changed, 5987 insertions(+), 2966 deletions(-)
create mode 100644 be/src/vec/exec/format/parquet/bool_rle_decoder.cpp
copy be/src/{util/type_traits.h => vec/exec/format/parquet/bool_rle_decoder.h}
(53%)
create mode 100644 be/src/vec/exec/format/parquet/delta_bit_pack_decoder.cpp
create mode 100644 be/src/vec/exec/format/parquet/delta_bit_pack_decoder.h
create mode 100644 be/src/vec/functions/array/function_array_pushfront.cpp
create mode 100644 be/src/vec/runtime/shared_scanner_controller.h
create mode 100644 docker/runtime/k8s/doris_be.yml
create mode 100644 docker/runtime/k8s/doris_cn.yml
create mode 100644 docker/runtime/k8s/doris_follower.yml
create mode 100644 docs/en/docs/admin-manual/certificate.md
create mode 100644
docs/en/docs/sql-manual/sql-functions/array-functions/array_pushfront.md
create mode 100644 docs/zh-CN/docs/admin-manual/certificate.md
create mode 100644
docs/zh-CN/docs/sql-manual/sql-functions/array-functions/array_pushfront.md
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/hive/event/GzipJSONMessageDeserializer.java
rename
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/logical/{InnerToCrossJoin.java
=> ConvertInnerOrCrossJoin.java} (61%)
rename
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/{NamedExpressionUtil.java
=> StatementScopeIdGenerator.java} (80%)
rename
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/{RelationId.java
=> ObjectId.java} (76%)
copy
fe/fe-core/src/main/java/org/apache/doris/planner/{external/TableFormatType.java
=> OlapSplitter.java} (71%)
copy
fe/fe-core/src/main/java/org/apache/doris/planner/{external/HiveSplit.java =>
Split.java} (70%)
copy
fe/fe-core/src/main/java/org/apache/doris/planner/{external/TableFormatType.java
=> Splitter.java} (71%)
rename
fe/fe-core/src/main/java/org/apache/doris/planner/external/{HiveSplit.java =>
FileSplit.java} (64%)
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/planner/external/HiveSplitter.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/planner/external/IcebergSplitter.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/planner/external/TVFSplitter.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/datasource/hive/event/GzipJSONMessageDeserializerTest.java
create mode 100644
regression-test/data/correctness_p0/test_current_timestamp_streamload.json
copy regression-test/data/correctness_p0/{test_join_with_projection.out =>
test_inlineview_with_project.out} (100%)
create mode 100644
regression-test/data/external_table_emr_p2/hive/test_external_catalog_glue_table.out
create mode 100644
regression-test/data/load_p0/stream_load/case_sensitive_json.json
delete mode 100644
regression-test/data/materialized_view_p0/test_create_mv_datatype.out
rename regression-test/data/{materialized_view_p0 =>
mv_p0}/agg_have_dup_base/agg_have_dup_base.out (100%)
copy regression-test/data/{materialized_view_p0 =>
mv_p0}/k1ap2spa/k1ap2spa.out (100%)
rename regression-test/data/{materialized_view_p0 =>
mv_p0}/multi_slot_k123p/multi_slot_k123p.out (100%)
rename regression-test/data/{materialized_view_p0 =>
mv_p0}/multi_slot_k1a2p2ap3p/multi_slot_k1a2p2ap3p.out (100%)
rename regression-test/data/{materialized_view_p0 =>
mv_p0}/multi_slot_k1a2p2ap3ps/multi_slot_k1a2p2ap3ps.out (100%)
rename regression-test/data/{materialized_view_p0 =>
mv_p0}/multi_slot_k1p2ap3p/multi_slot_k1p2ap3p.out (100%)
rename regression-test/data/{materialized_view_p0 =>
mv_p0}/multi_slot_k1p2ap3ps/multi_slot_k1p2ap3ps.out (100%)
rename regression-test/data/{materialized_view_p0 =>
mv_p0}/multi_slot_multi_mv/multi_slot_multi_mv.out (100%)
rename regression-test/data/{materialized_view_p0 =>
mv_p0}/mv_with_view/mv_with_view.out (100%)
rename regression-test/data/{materialized_view_p0 =>
mv_p0}/test_dup_group_by_mv_abs/test_dup_group_by_mv_abs.out (100%)
rename regression-test/data/{materialized_view_p0 =>
mv_p0}/test_dup_group_by_mv_plus/test_dup_group_by_mv_plus.out (100%)
rename regression-test/data/{materialized_view_p0 =>
mv_p0}/test_dup_mv_abs/test_dup_mv_abs.out (100%)
rename regression-test/data/{materialized_view_p0 =>
mv_p0}/test_dup_mv_bin/test_dup_mv_bin.out (100%)
rename regression-test/data/{materialized_view_p0 =>
mv_p0}/test_dup_mv_bitmap_hash/test_dup_mv_bitmap_hash.out (100%)
rename regression-test/data/{materialized_view_p0 =>
mv_p0}/test_dup_mv_plus/test_dup_mv_plus.out (100%)
rename regression-test/data/{materialized_view_p0 =>
mv_p0}/test_dup_mv_year/test_dup_mv_year.out (100%)
copy regression-test/data/{materialized_view_p0/k1ap2spa/k1ap2spa.out =>
mv_p0/ut/testIncorrectMVRewriteInSubquery/testIncorrectMVRewriteInSubquery.out}
(71%)
copy regression-test/data/{materialized_view_p0 =>
mv_p0}/ut/testProjectionMV1/testProjectionMV1.out (100%)
rename regression-test/data/{materialized_view_p0 =>
mv_p0}/ut/testProjectionMV2/testProjectionMV2.out (100%)
rename regression-test/data/{materialized_view_p0 =>
mv_p0}/ut/testProjectionMV3/testProjectionMV3.out (100%)
rename regression-test/data/{materialized_view_p0 =>
mv_p0}/ut/testProjectionMV4/testProjectionMV4.out (100%)
rename
regression-test/data/{materialized_view_p0/ut/testProjectionMV1/testProjectionMV1.out
=> mv_p0/ut/testQueryOnStar/testQueryOnStar.out} (82%)
rename regression-test/data/{materialized_view_p0/k1ap2spa/k1ap2spa.out =>
mv_p0/ut/testSelectMVWithTableAlias/testSelectMVWithTableAlias.out} (71%)
create mode 100644
regression-test/data/query_p0/sql_functions/cast_function/test_cast_struct.out
create mode 100644
regression-test/suites/correctness_p0/test_inlineview_with_project.groovy
create mode 100644
regression-test/suites/external_table_emr_p2/hive/test_external_catalog_glue_table.groovy
rename regression-test/suites/{materialized_view_p0 =>
mv_p0}/agg_have_dup_base/agg_have_dup_base.groovy (83%)
rename regression-test/suites/{materialized_view_p0 =>
mv_p0}/k1ap2spa/k1ap2spa.groovy (78%)
rename regression-test/suites/{materialized_view_p0 =>
mv_p0}/multi_slot_k123p/multi_slot_k123p.groovy (79%)
rename regression-test/suites/{materialized_view_p0 =>
mv_p0}/multi_slot_k1a2p2ap3p/multi_slot_k1a2p2ap3p.groovy (79%)
rename regression-test/suites/{materialized_view_p0 =>
mv_p0}/multi_slot_k1a2p2ap3ps/multi_slot_k1a2p2ap3ps.groovy (82%)
rename regression-test/suites/{materialized_view_p0 =>
mv_p0}/multi_slot_k1p2ap3p/multi_slot_k1p2ap3p.groovy (79%)
rename regression-test/suites/{materialized_view_p0 =>
mv_p0}/multi_slot_k1p2ap3ps/multi_slot_k1p2ap3ps.groovy (78%)
rename regression-test/suites/{materialized_view_p0 =>
mv_p0}/multi_slot_multi_mv/multi_slot_multi_mv.groovy (84%)
rename regression-test/suites/{materialized_view_p0 =>
mv_p0}/mv_with_view/mv_with_view.groovy (97%)
rename regression-test/suites/{materialized_view_p0 =>
mv_p0/test_create_mv_complex_type}/test_create_mv_complex_type.groovy (92%)
rename regression-test/suites/{materialized_view_p0 =>
mv_p0}/test_dup_group_by_mv_abs/test_dup_group_by_mv_abs.groovy (80%)
rename regression-test/suites/{materialized_view_p0 =>
mv_p0}/test_dup_group_by_mv_plus/test_dup_group_by_mv_plus.groovy (80%)
rename regression-test/suites/{materialized_view_p0 =>
mv_p0}/test_dup_mv_abs/test_dup_mv_abs.groovy (85%)
rename regression-test/suites/{materialized_view_p0 =>
mv_p0}/test_dup_mv_bin/test_dup_mv_bin.groovy (86%)
rename regression-test/suites/{materialized_view_p0 =>
mv_p0}/test_dup_mv_bitmap_hash/test_dup_mv_bitmap_hash.groovy (84%)
rename regression-test/suites/{materialized_view_p0 =>
mv_p0}/test_dup_mv_plus/test_dup_mv_plus.groovy (86%)
rename regression-test/suites/{materialized_view_p0 =>
mv_p0}/test_dup_mv_year/test_dup_mv_year.groovy (84%)
rename regression-test/suites/{materialized_view_p0 =>
mv_p0}/test_mv_useless/test_agg_mv_useless.groovy (100%)
rename regression-test/suites/{materialized_view_p0 =>
mv_p0}/test_mv_useless/test_dup_mv_useless.groovy (100%)
rename regression-test/suites/{materialized_view_p0 =>
mv_p0}/test_mv_useless/test_uniq_mv_useless.groovy (100%)
create mode 100644
regression-test/suites/mv_p0/ut/testIncorrectMVRewriteInSubquery/testIncorrectMVRewriteInSubquery.groovy
copy regression-test/suites/{materialized_view_p0 =>
mv_p0}/ut/testProjectionMV1/testProjectionMV1.groovy (100%)
rename regression-test/suites/{materialized_view_p0 =>
mv_p0}/ut/testProjectionMV2/testProjectionMV2.groovy (100%)
rename regression-test/suites/{materialized_view_p0 =>
mv_p0}/ut/testProjectionMV3/testProjectionMV3.groovy (100%)
rename regression-test/suites/{materialized_view_p0 =>
mv_p0}/ut/testProjectionMV4/testProjectionMV4.groovy (100%)
copy
regression-test/suites/{materialized_view_p0/ut/testProjectionMV1/testProjectionMV1.groovy
=> mv_p0/ut/testQueryOnStar/testQueryOnStar.groovy} (83%)
rename
regression-test/suites/{materialized_view_p0/ut/testProjectionMV1/testProjectionMV1.groovy
=> mv_p0/ut/testSelectMVWithTableAlias/testSelectMVWithTableAlias.groovy} (56%)
create mode 100644
regression-test/suites/query_p0/sql_functions/cast_function/test_cast_struct.groovy
create mode 100644
samples/doris-demo/flink-demo-v1.1/src/main/java/org/apache/doris/demo/flink/dbsync/DatabaseFullSyncV2.java
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
