This is an automated email from the ASF dual-hosted git repository.
morningman pushed a change to branch auto-pick-61759-branch-4.1
in repository https://gitbox.apache.org/repos/asf/doris.git
discard 231f6ca06e5 [fix](iceberg) Avoid dict reads on mixed-encoding position
delete files (#61759)
add d160a4afa58 branch-4.1: [refactor](rec_cte)recursive-CTE refactor
(#62039)
add f2777abb7f3 [fix](compaction) Remove upper limit for
vertical_compaction_num_columns_per_group #61992 (#61993)
add aee9b0b60e2 branch-4.1: [chore](cloud) list_snapshot allow query
parameter #61885 (#62012)
add 01120c9e8d0 branch-4.1: [Exec](performance) change colocate execution
parallel num #60677 (#61982)
add ca6191430fe branch-4.1: [fix](build) Fix compilation errors with
implicit conversions and unnecessary virtual keywords (#61836) (#61967)
add cae98934272 branch-4.1: [minor](Prefetch) Revert some useless check
#59601 (#62001)
add b918e156a04 branch-4.1: [opt](memory) lazy-allocate PrefetchBuffer
backing buffer to reduce peak memory (#62005)
add bb722c84b6e branch-4.1: [enhancement](config) Change
segments_key_bounds_truncation_threshold default to 36 #61984 (#62003)
add 5b1e21449d1 branch-4.1: [chore](cloud) process_unknown add warning log
and improve msg #61695 (#62011)
add 860603a0be4 branch-4.1: [fix](snapshot) skip link binlog files if
binlog is not enabled #61949 (#62014)
add b1858e023cb branch-4.1: (fix)[case] fix
test_mow_compaction_and_rowset_not_exist #61947 (#62026)
add 54586843ef5 branch-4.1: [fix](cloud) add get_version and
get_tablet_stats case #61915 (#62028)
add f53d17160de branch-4.1: [fix](scan) Fix missing predicate filter when
Native and JNI readers are mixed in FileScanner #61929 (#62078)
add 64ceb992d67 branch-4.1: [fix](spill) Fix revocable_mem_size to return
0 when partition reaches max repartition depth #61973 (#62073)
add 8003eea5bcb branch-4.1: [enhance](load)only set brokerLoadBatchSize
when enableMemtableOnSinkNode is true (#62089)
add d0906071727 branch-4.1: [fix](csv reader) fix incorrect column parsing
when using enclose for CSV files with UTF-8 BOM (#62093)
add c551f395d87 branch-4.1: [fix](variant) preserve subcolumns after
row-store partial update #62067 (#62076)
add 6d2ba4b8fef branch-4.1: [opt](table-function) Optimize explode with
block fast path #60352 (#62048)
add 79958629c69 branch-4.1: [fix](job) fix StreamingJob loadStatistic
reset to zero after FE checkpoint restart #61997 (#62049)
add b03b432cac7 branch-4.1: [fix](job) fix StreamingInsertJob incorrectly
counting canceled tasks #61894 (#62055)
add 64c700f7821 branch-4.1: [fix](backup) Fix
UnsupportedOperationException in RestoreCommand by copying properties before
modification #61890 (#62072)
add b8e113b5f2a branch-4.1: [fix](streaming-job) Fix PG replication slot
leak when streaming task is cancelled during pause/resume #62010 (#62033)
add edeecbb6fd4 branch-4.1: [minor](fuzzy) make enableQueryCache fuzzy
#61583 (#61964)
add ff2a20abb7b branch-4.1: [fix](cloud) Fix cloud not print ms log #61766
(#62020)
add 45e59b36ddc [fix](iceberg) Avoid dict reads on mixed-encoding position
delete files (#61759)
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (231f6ca06e5)
\
N -- N -- N refs/heads/auto-pick-61759-branch-4.1 (45e59b36ddc)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
be/CMakeLists.txt | 5 +-
be/src/common/config.cpp | 2 +-
be/src/exec/common/ipv6_to_binary.h | 25 +-
.../partitioned_aggregation_source_operator.cpp | 5 +
.../partitioned_hash_join_probe_operator.cpp | 5 +
be/src/exec/operator/table_function_operator.cpp | 233 +++++++-
be/src/exec/operator/table_function_operator.h | 12 +
be/src/exec/pipeline/pipeline_fragment_context.cpp | 15 +-
be/src/exec/pipeline/pipeline_fragment_context.h | 2 +-
be/src/exec/scan/file_scanner.cpp | 5 +-
be/src/exprs/table_function/table_function.h | 14 +
be/src/exprs/table_function/vexplode.cpp | 19 +
be/src/exprs/table_function/vexplode.h | 4 +
be/src/exprs/table_function/vexplode_v2.cpp | 18 +
be/src/exprs/table_function/vexplode_v2.h | 4 +
be/src/format/csv/csv_reader.cpp | 18 +-
be/src/format/csv/csv_reader.h | 3 +
.../file_reader/new_plain_text_line_reader.h | 9 +
be/src/io/fs/buffered_reader.cpp | 16 +-
be/src/io/fs/buffered_reader.h | 2 +-
be/src/storage/segment/segment_writer.cpp | 5 +
be/src/storage/segment/vertical_segment_writer.cpp | 5 +
be/src/storage/snapshot/snapshot_manager.cpp | 2 +-
...artitioned_aggregation_source_operator_test.cpp | 38 ++
.../partitioned_hash_join_probe_operator_test.cpp | 45 ++
.../exec/operator/table_function_operator_test.cpp | 613 ++++++++++++++++++++-
be/test/exec/scan/vfile_scanner_exception_test.cpp | 41 ++
cloud/script/start.sh | 5 +-
cloud/src/main.cpp | 14 +-
cloud/src/meta-service/meta_service_http.cpp | 23 +-
cloud/test/meta_service_http_test.cpp | 2 +-
.../org/apache/doris/analysis/DescriptorTable.java | 5 -
.../apache/doris/catalog/CloudTabletStatMgr.java | 8 +
.../doris/catalog/RecursiveCteTempTable.java | 35 --
.../java/org/apache/doris/catalog/TableIf.java | 6 +-
.../transaction/CloudGlobalTransactionMgr.java | 4 +
.../apache/doris/common/util/PropertyAnalyzer.java | 4 +-
.../insert/streaming/StreamingInsertJob.java | 11 +-
.../apache/doris/load/loadv2/BrokerLoadJob.java | 17 +
.../apache/doris/load/loadv2/LoadLoadingTask.java | 4 +-
.../java/org/apache/doris/nereids/CTEContext.java | 26 +-
.../org/apache/doris/nereids/CascadesContext.java | 37 +-
.../org/apache/doris/nereids/StatementContext.java | 9 +
.../glue/translator/PhysicalPlanTranslator.java | 97 ++--
.../doris/nereids/parser/LogicalPlanBuilder.java | 2 +
.../apache/doris/nereids/pattern/MemoPatterns.java | 24 -
.../processor/post/RuntimeFilterPruner.java | 9 +-
.../processor/pre/PullUpSubqueryAliasToCTE.java | 4 +-
.../properties/ChildOutputPropertyDeriver.java | 15 +-
.../nereids/properties/RequestPropertyDeriver.java | 11 +-
.../org/apache/doris/nereids/rules/RuleSet.java | 14 +-
.../org/apache/doris/nereids/rules/RuleType.java | 7 +-
.../doris/nereids/rules/analysis/AnalyzeCTE.java | 107 ++--
.../doris/nereids/rules/analysis/BindRelation.java | 31 +-
.../nereids/rules/analysis/CollectRelation.java | 12 +-
.../nereids/rules/analysis/SubExprAnalyzer.java | 4 +-
...UnionAnchorToPhysicalRecursiveUnionAnchor.java} | 20 +-
...nProducerToPhysicalRecursiveUnionProducer.java} | 10 +-
...calRecursiveUnionToPhysicalRecursiveUnion.java} | 24 +-
...ableReferenceToPhysicalWorkTableReference.java} | 22 +-
.../nereids/rules/rewrite/AdjustNullable.java | 48 --
.../doris/nereids/rules/rewrite/CTEInline.java | 25 +-
.../doris/nereids/rules/rewrite/ColumnPruning.java | 55 --
.../doris/nereids/stats/StatsCalculator.java | 95 +++-
.../trees/copier/LogicalPlanDeepCopier.java | 23 +-
.../apache/doris/nereids/trees/plans/PlanType.java | 4 +-
.../trees/plans/commands/RestoreCommand.java | 12 +-
.../plans/commands/UpdateMvByPartitionCommand.java | 2 +-
.../commands/info/ModifyTablePropertiesOp.java | 4 +-
.../worker/job/UnassignedJobBuilder.java | 13 -
.../worker/job/UnassignedRecursiveCteScanJob.java | 65 ---
.../job/UnassignedScanBucketOlapTableJob.java | 36 ++
.../nereids/trees/plans/logical/LogicalCTE.java | 26 +-
.../plans/logical/LogicalRecursiveCteScan.java | 73 ---
...ecursiveCte.java => LogicalRecursiveUnion.java} | 210 +++++--
...Child.java => LogicalRecursiveUnionAnchor.java} | 53 +-
...ild.java => LogicalRecursiveUnionProducer.java} | 38 +-
.../trees/plans/logical/LogicalSubQueryAlias.java | 32 +-
.../plans/logical/LogicalWorkTableReference.java | 149 +++++
.../plans/physical/PhysicalRecursiveCteScan.java | 85 ---
...cursiveCte.java => PhysicalRecursiveUnion.java} | 78 +--
...hild.java => PhysicalRecursiveUnionAnchor.java} | 55 +-
...ld.java => PhysicalRecursiveUnionProducer.java} | 40 +-
.../plans/physical/PhysicalWorkTableReference.java | 152 +++++
.../nereids/trees/plans/visitor/PlanVisitor.java | 38 +-
.../trees/plans/visitor/RelationVisitor.java | 12 +-
.../org/apache/doris/nereids/types/DataType.java | 4 +
.../org/apache/doris/nereids/types/StructType.java | 24 +
.../apache/doris/nereids/types/VariantType.java | 21 +
.../apache/doris/planner/RecursiveCteScanNode.java | 70 +--
.../java/org/apache/doris/qe/SessionVariable.java | 8 +-
.../doris/load/loadv2/LoadLoadingTaskTest.java | 57 ++
.../nereids/rules/analysis/AnalyzeCTETest.java | 26 +-
.../doris/nereids/rules/rewrite/CTEInlineTest.java | 6 +-
.../nereids/rules/rewrite/ColumnPruningTest.java | 2 +-
.../trees/plans/commands/RestoreCommandTest.java | 16 +
.../doris/qe/LocalShuffleWithBucketJoinTest.java | 4 +-
.../source/reader/JdbcIncrementalSourceReader.java | 12 +
.../source/reader/mysql/MySqlSourceReader.java | 12 +
...eaming_job_cdc_stream_postgres_pause_resume.out | 2 +-
...reaming_job_cdc_stream_postgres_restart_fe.out} | 9 +-
.../data/load_p0/stream_load/enclose_with_bom.csv | 2 +
.../test_csv_with_enclose_and_escape.out | 2 +
.../data/rec_cte_p0/rec_cte/rec_cte.out | 17 +
.../tablets/test_tablet_stat_syncer.groovy | 150 +++++
.../cloud_p0/version/test_version_syncer.groovy | 112 ++++
...test_mow_compaction_and_rowset_not_exist.groovy | 11 +-
...ob_cdc_stream_postgres_latest_alter_cred.groovy | 4 +-
...ing_job_cdc_stream_postgres_pause_resume.groovy | 27 +-
...aming_job_cdc_stream_postgres_restart_fe.groovy | 288 ++++++++++
.../test_streaming_job_restart_fe.groovy | 3 +-
.../test_csv_with_enclose_and_escape.groovy | 10 +
.../suites/rec_cte_p0/rec_cte/rec_cte.groovy | 79 +++
.../suites/variant_p0/delete_update.groovy | 40 ++
114 files changed, 3195 insertions(+), 1034 deletions(-)
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/catalog/RecursiveCteTempTable.java
rename
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/implementation/{LogicalRecursiveCteToPhysicalRecursiveCte.java
=> LogicalRecursiveUnionAnchorToPhysicalRecursiveUnionAnchor.java} (62%)
rename
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/implementation/{LogicalRecursiveCteRecursiveChildToPhysicalRecursiveCteRecursiveChild.java
=> LogicalRecursiveUnionProducerToPhysicalRecursiveUnionProducer.java} (74%)
rename
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/implementation/{LogicalRecursiveCteScanToPhysicalRecursiveCteScan.java
=> LogicalRecursiveUnionToPhysicalRecursiveUnion.java} (58%)
copy
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/implementation/{LogicalEsScanToPhysicalEsScan.java
=> LogicalWorkTableReferenceToPhysicalWorkTableReference.java} (59%)
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/distribute/worker/job/UnassignedRecursiveCteScanJob.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/logical/LogicalRecursiveCteScan.java
rename
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/logical/{LogicalRecursiveCte.java
=> LogicalRecursiveUnion.java} (50%)
copy
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/logical/{LogicalRecursiveCteRecursiveChild.java
=> LogicalRecursiveUnionAnchor.java} (63%)
rename
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/logical/{LogicalRecursiveCteRecursiveChild.java
=> LogicalRecursiveUnionProducer.java} (70%)
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/logical/LogicalWorkTableReference.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/physical/PhysicalRecursiveCteScan.java
rename
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/physical/{PhysicalRecursiveCte.java
=> PhysicalRecursiveUnion.java} (74%)
copy
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/physical/{PhysicalRecursiveCteRecursiveChild.java
=> PhysicalRecursiveUnionAnchor.java} (64%)
rename
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/physical/{PhysicalRecursiveCteRecursiveChild.java
=> PhysicalRecursiveUnionProducer.java} (70%)
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/physical/PhysicalWorkTableReference.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/load/loadv2/LoadLoadingTaskTest.java
copy
regression-test/data/job_p0/streaming_job/cdc/tvf/{test_streaming_job_cdc_stream_mysql.out
=> test_streaming_job_cdc_stream_postgres_restart_fe.out} (80%)
create mode 100644
regression-test/data/load_p0/stream_load/enclose_with_bom.csv
create mode 100644
regression-test/suites/cloud_p0/tablets/test_tablet_stat_syncer.groovy
create mode 100644
regression-test/suites/cloud_p0/version/test_version_syncer.groovy
create mode 100644
regression-test/suites/job_p0/streaming_job/cdc/tvf/test_streaming_job_cdc_stream_postgres_restart_fe.groovy
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
