This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a change to branch branch-2.1
in repository https://gitbox.apache.org/repos/asf/doris.git
from e21c9dca9cf [fix](mtmv)compatibility metadata without refreshsnapshot
#30735
new 9889683ae32 [Feature](Job)STARTS and AT allow setting
current_timestamp (#30593)
new e5bdc369e2e [runtimefilter](nereids)push down RF into cte producer
(#30568)
new 0f47f7f3892 [Feature](runtime filter) normalize ignore runtime filter
(#30152)
new fd000dbdb3c [chore](ci) try to set 'performance (Doris Performance)'
to required (#30752)
new 286f3970170 [chore](ci) revert "try to set 'performance (Doris
Performance)' to required #30752" (#30755)
new 0e2d2f61698 [Bug](pipeline) fix pipeline task execute without wait
second start rpc (#30659)
new 79a83e9e1a6 [fix](move-memtable) use signed integer when calculating
remain ms (#30750)
new bc8c1875acd [enhencement](config) change default write buffer size
(#30749)
new 5687ca977dd [Bug](java-udf) fix core dump when javaudf input 0 row
block (#30720)
new e413dbec91d [fix](nereids)need substitute agg function using agg
node's output if it's in order by key (#30704)
new ac681e8e8c5 [ehmancement](binlog) Add show proc '/binlog' impl (#30770)
new 4f8730d092e [improvement](jdbc catalog) Optimize connection pool
parameter settings (#30588)
new 24433710c8e [fix](regression) Make catalog recycle bin regression
stable (#30736)
new 151735748bf [test](Nereids): add push_down_distinct_through_join
regression test (#30760)
new 8a0ea4b6515 [enhancement](Nereids): datetime support microsecond
overflow (#30744)
new 0c7c8240ee7 [Fix](ser) serialize bitmap/hll some not fixed length
object maybe over INT_MAX (#30773)
new a3a73162e58 [Fix](Job)Fix One-Time type JOB parameter verification
error (#30779)
new d99bb51d365 [fix](legacy-planner) fixed loss of BetweenPredicate
rewrite on reanalyze in legacy planner (29798) (#30328)
new f0181375006 [doc](jdbc catalog) add connection pool doc (#30641)
new 4f0414d13eb [fix](Nereids) date >= simplify to > by mistake (#30765)
new d33f3be436a [Doc](auto-inc) Add doc for auto increment start value
(#30741)
new 6289f7e6059 [Fix](multi-catalog) Fix truncate_char_or_varchar_column
crash. (#30731)
new 119615dc502 [Fix](parquet-reader) Fix decimal test case out files.
(#30716)
new 5aed3abb8a8 [Fix](Nereids) Fix rewrite by materialized view fail when
join input has agg (#30734)
The 24 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
be/src/common/config.cpp | 7 +-
be/src/common/config.h | 7 +-
be/src/exprs/create_predicate_function.h | 2 +-
be/src/exprs/minmax_predicate.h | 117 +++---
be/src/exprs/runtime_filter.h | 4 +-
be/src/olap/accept_null_predicate.h | 90 ++---
be/src/olap/bitmap_filter_predicate.h | 14 +-
be/src/olap/bloom_filter_predicate.h | 38 +-
be/src/olap/column_predicate.h | 43 ++-
be/src/olap/comparison_predicate.h | 72 ++--
be/src/olap/in_list_predicate.h | 61 +--
be/src/olap/like_column_predicate.cpp | 4 +-
be/src/olap/like_column_predicate.h | 6 +-
be/src/olap/match_predicate.h | 9 +-
be/src/olap/null_predicate.cpp | 4 +-
be/src/olap/null_predicate.h | 6 +-
be/src/olap/rowset/segment_v2/segment_iterator.cpp | 28 +-
be/src/olap/schema_change.cpp | 5 +
be/src/pipeline/pipeline_task.cpp | 25 +-
be/src/pipeline/pipeline_task.h | 1 +
be/src/runtime/descriptors.cpp | 33 +-
be/src/runtime/descriptors.h | 22 +-
be/src/vec/data_types/data_type_bitmap.cpp | 2 +-
be/src/vec/data_types/data_type_hll.cpp | 2 +-
be/src/vec/data_types/data_type_quantilestate.cpp | 2 +-
be/src/vec/exec/jni_connector.cpp | 11 +-
be/src/vec/exec/scan/new_jdbc_scanner.cpp | 11 +-
be/src/vec/exec/scan/vfile_scanner.cpp | 12 +-
be/src/vec/exec/vjdbc_connector.cpp | 13 +-
be/src/vec/exec/vjdbc_connector.h | 11 +-
be/src/vec/exprs/vruntimefilter_wrapper.cpp | 3 +-
be/src/vec/exprs/vruntimefilter_wrapper.h | 8 +-
be/src/vec/sink/writer/vjdbc_table_writer.cpp | 11 +-
be/src/vec/sink/writer/vtablet_writer_v2.cpp | 14 +-
.../olap/bitmap_filter_column_predicate_test.cpp | 4 +-
docs/en/docs/advanced/auto-increment.md | 53 ++-
docs/en/docs/lakehouse/multi-catalog/jdbc.md | 16 +
.../join-optimization/runtime-filter.md | 2 +-
.../Create/CREATE-TABLE.md | 7 +-
docs/zh-CN/docs/advanced/auto-increment.md | 53 ++-
docs/zh-CN/docs/lakehouse/multi-catalog/jdbc.md | 16 +
.../join-optimization/runtime-filter.md | 2 +-
.../Create/CREATE-TABLE.md | 5 +-
.../java/org/apache/doris/jdbc/JdbcDataSource.java | 50 +++
.../apache/doris/jdbc/JdbcDataSourceConfig.java | 75 ++--
.../java/org/apache/doris/jdbc/JdbcExecutor.java | 95 +++--
fe/fe-core/src/main/cup/sql_parser.cup | 19 +-
.../java/org/apache/doris/analysis/Analyzer.java | 10 +
.../org/apache/doris/analysis/CreateJobStmt.java | 15 +-
.../doris/analysis/CreateMaterializedViewStmt.java | 6 +
.../apache/doris/analysis/FunctionCallExpr.java | 4 +-
.../java/org/apache/doris/analysis/SelectStmt.java | 1 +
.../org/apache/doris/binlog/BinlogConfigCache.java | 4 +-
.../org/apache/doris/binlog/BinlogManager.java | 24 ++
.../java/org/apache/doris/binlog/BinlogUtils.java | 4 +
.../java/org/apache/doris/binlog/DBBinlog.java | 64 ++++
.../java/org/apache/doris/binlog/TableBinlog.java | 82 ++++
.../org/apache/doris/catalog/JdbcResource.java | 115 ++++--
.../java/org/apache/doris/catalog/JdbcTable.java | 63 +++-
.../doris/catalog/MaterializedIndexMeta.java | 2 +-
.../java/org/apache/doris/catalog/Resource.java | 2 +
.../java/org/apache/doris/catalog/ResourceMgr.java | 1 +
.../doris/catalog/external/JdbcExternalTable.java | 11 +-
.../{ProcDirInterface.java => BinlogProcDir.java} | 25 +-
.../org/apache/doris/common/proc/ProcService.java | 1 +
.../doris/datasource/jdbc/JdbcExternalCatalog.java | 71 ++--
.../doris/datasource/jdbc/client/JdbcClient.java | 25 +-
.../datasource/jdbc/client/JdbcClientConfig.java | 93 +++--
.../doris/job/base/JobExecutionConfiguration.java | 7 +-
.../nereids/processor/post/PlanPostProcessors.java | 1 +
.../post/RegisterParent.java} | 13 +-
.../processor/post/RuntimeFilterContext.java | 36 --
.../processor/post/RuntimeFilterGenerator.java | 418 +++++++++------------
.../processor/post/RuntimeFilterPruner.java | 47 +++
.../nereids/rules/analysis/FillUpMissingSlots.java | 7 +-
.../mv/AbstractMaterializedViewAggregateRule.java | 28 +-
.../mv/AbstractMaterializedViewJoinRule.java | 25 +-
.../nereids/rules/exploration/mv/StructInfo.java | 89 ++---
.../expression/rules/DateFunctionRewrite.java | 4 +-
.../trees/expressions/literal/DateLiteral.java | 14 +-
.../trees/expressions/literal/DateTimeLiteral.java | 16 +-
.../expressions/literal/DateTimeV2Literal.java | 9 +-
.../doris/nereids/trees/plans/AbstractPlan.java | 15 +
.../trees/plans/physical/AbstractPhysicalPlan.java | 5 +-
.../trees/plans/physical/PhysicalCTEConsumer.java | 18 +-
.../trees/plans/physical/PhysicalDistribute.java | 4 +-
.../trees/plans/physical/PhysicalFilter.java | 4 +-
.../trees/plans/physical/PhysicalHashJoin.java | 28 +-
.../trees/plans/physical/PhysicalOlapScan.java | 11 +-
.../trees/plans/physical/PhysicalProject.java | 12 +-
.../apache/doris/nereids/types/DateTimeV2Type.java | 5 +-
.../doris/nereids/util/DateTimeFormatterUtils.java | 5 +-
.../java/org/apache/doris/nereids/util/Utils.java | 12 +-
.../java/org/apache/doris/qe/SessionVariable.java | 2 +-
.../org/apache/doris/statistics/Statistics.java | 2 +-
.../org/apache/doris/catalog/JdbcResourceTest.java | 146 ++++++-
.../datasource/jdbc/JdbcExternalCatalogTest.java | 45 ++-
.../job/base/JobExecutionConfigurationTest.java | 17 +-
.../rules/analysis/FillUpMissingSlotsTest.java | 23 ++
.../rules/SimplifyComparisonPredicateSqlTest.java | 9 -
.../trees/expressions/literal/DateLiteralTest.java | 10 +-
.../expressions/literal/DateTimeLiteralTest.java | 34 +-
.../nereids/util/DateTimeFormatterUtilsTest.java | 4 +-
fe/pom.xml | 2 +-
gensrc/thrift/Descriptors.thrift | 11 +-
gensrc/thrift/Types.thrift | 12 +-
.../data/javaudf_p0/test_javaudf_float.out | 5 +
.../mv/join/dphyp_outer/outer_join_dphyp.out | 14 +
.../mv/join/left_outer/outer_join.out | 14 +
.../push_down_distinct_through_join.out | 42 +++
.../data/nereids_syntax_p0/test_literal.out | 22 ++
.../nereids_tpcds_shape_sf1000_p0/shape/query1.out | 6 +-
.../shape/query14.out | 28 +-
.../nereids_tpcds_shape_sf1000_p0/shape/query2.out | 4 +-
.../shape/query23.out | 16 +-
.../shape/query30.out | 2 +-
.../shape/query59.out | 8 +-
.../shape/query81.out | 2 +-
.../shape/query95.out | 14 +-
.../constraints/query23.out | 16 +-
.../noStatsRfPrune/query1.out | 4 +-
.../noStatsRfPrune/query14.out | 32 +-
.../noStatsRfPrune/query2.out | 4 +-
.../noStatsRfPrune/query23.out | 12 +-
.../noStatsRfPrune/query30.out | 2 +-
.../noStatsRfPrune/query31.out | 4 +-
.../noStatsRfPrune/query38.out | 8 +-
.../noStatsRfPrune/query5.out | 12 +-
.../noStatsRfPrune/query59.out | 10 +-
.../noStatsRfPrune/query66.out | 4 +-
.../noStatsRfPrune/query76.out | 12 +-
.../noStatsRfPrune/query77.out | 8 +-
.../noStatsRfPrune/query80.out | 8 +-
.../noStatsRfPrune/query81.out | 2 +-
.../noStatsRfPrune/query87.out | 8 +-
.../noStatsRfPrune/query95.out | 14 +-
.../no_stats_shape/query1.out | 2 +-
.../no_stats_shape/query14.out | 30 +-
.../no_stats_shape/query2.out | 4 +-
.../no_stats_shape/query23.out | 12 +-
.../no_stats_shape/query30.out | 2 +-
.../no_stats_shape/query59.out | 4 +-
.../no_stats_shape/query81.out | 2 +-
.../no_stats_shape/query95.out | 12 +-
.../rf_prune/query1.out | 6 +-
.../rf_prune/query14.out | 26 +-
.../rf_prune/query2.out | 4 +-
.../rf_prune/query23.out | 16 +-
.../rf_prune/query30.out | 2 +-
.../rf_prune/query31.out | 4 +-
.../rf_prune/query33.out | 4 +-
.../rf_prune/query38.out | 8 +-
.../rf_prune/query5.out | 12 +-
.../rf_prune/query59.out | 10 +-
.../rf_prune/query66.out | 4 +-
.../rf_prune/query77.out | 8 +-
.../rf_prune/query80.out | 8 +-
.../rf_prune/query81.out | 2 +-
.../rf_prune/query87.out | 8 +-
.../rf_prune/query95.out | 14 +-
.../nereids_tpcds_shape_sf100_p0/shape/query1.out | 6 +-
.../nereids_tpcds_shape_sf100_p0/shape/query14.out | 6 +-
.../nereids_tpcds_shape_sf100_p0/shape/query2.out | 4 +-
.../nereids_tpcds_shape_sf100_p0/shape/query23.out | 16 +-
.../nereids_tpcds_shape_sf100_p0/shape/query30.out | 2 +-
.../nereids_tpcds_shape_sf100_p0/shape/query59.out | 8 +-
.../nereids_tpcds_shape_sf100_p0/shape/query81.out | 2 +-
.../nereids_tpcds_shape_sf100_p0/shape/query95.out | 14 +-
.../nostats_rf_prune/q21.out | 4 +-
.../shape_no_stats/q21.out | 4 +-
.../data/query_p0/having/having_between.out | 31 ++
.../data/variable_p0/set_and_unset_variable.out | 20 +-
.../suites/catalog_recycle_bin_p0/show.groovy | 8 +-
.../suites/javaudf_p0/test_javaudf_float.groovy | 7 +
.../suites/job_p0/test_base_insert_job.groovy | 26 +-
.../mv/join/dphyp_outer/outer_join_dphyp.groovy | 60 +++
.../mv/join/left_outer/outer_join.groovy | 85 ++++-
.../push_down_distinct_through_join.groovy} | 19 +-
.../suites/nereids_syntax_p0/test_literal.groovy | 28 ++
.../nereids_tpcds_shape_sf1000_p0/ddl/shape.tmpl | 2 +-
.../shape/query1.groovy | 2 +-
.../shape/query10.groovy | 2 +-
.../shape/query11.groovy | 2 +-
.../shape/query12.groovy | 2 +-
.../shape/query13.groovy | 2 +-
.../shape/query14.groovy | 2 +-
.../shape/query15.groovy | 2 +-
.../shape/query16.groovy | 2 +-
.../shape/query17.groovy | 2 +-
.../shape/query18.groovy | 2 +-
.../shape/query19.groovy | 2 +-
.../shape/query2.groovy | 2 +-
.../shape/query20.groovy | 2 +-
.../shape/query21.groovy | 2 +-
.../shape/query22.groovy | 2 +-
.../shape/query23.groovy | 2 +-
.../shape/query24.groovy | 2 +-
.../shape/query25.groovy | 2 +-
.../shape/query26.groovy | 2 +-
.../shape/query27.groovy | 2 +-
.../shape/query28.groovy | 2 +-
.../shape/query29.groovy | 2 +-
.../shape/query3.groovy | 2 +-
.../shape/query30.groovy | 2 +-
.../shape/query31.groovy | 2 +-
.../shape/query32.groovy | 2 +-
.../shape/query33.groovy | 2 +-
.../shape/query34.groovy | 2 +-
.../shape/query35.groovy | 2 +-
.../shape/query36.groovy | 2 +-
.../shape/query37.groovy | 2 +-
.../shape/query38.groovy | 2 +-
.../shape/query39.groovy | 2 +-
.../shape/query4.groovy | 2 +-
.../shape/query40.groovy | 2 +-
.../shape/query41.groovy | 2 +-
.../shape/query42.groovy | 2 +-
.../shape/query43.groovy | 2 +-
.../shape/query44.groovy | 2 +-
.../shape/query45.groovy | 2 +-
.../shape/query46.groovy | 2 +-
.../shape/query47.groovy | 2 +-
.../shape/query48.groovy | 2 +-
.../shape/query49.groovy | 2 +-
.../shape/query5.groovy | 2 +-
.../shape/query50.groovy | 2 +-
.../shape/query51.groovy | 2 +-
.../shape/query52.groovy | 2 +-
.../shape/query53.groovy | 2 +-
.../shape/query54.groovy | 2 +-
.../shape/query55.groovy | 2 +-
.../shape/query56.groovy | 2 +-
.../shape/query57.groovy | 2 +-
.../shape/query58.groovy | 2 +-
.../shape/query59.groovy | 2 +-
.../shape/query6.groovy | 2 +-
.../shape/query60.groovy | 2 +-
.../shape/query61.groovy | 2 +-
.../shape/query62.groovy | 2 +-
.../shape/query63.groovy | 2 +-
.../shape/query64.groovy | 2 +-
.../shape/query65.groovy | 2 +-
.../shape/query66.groovy | 2 +-
.../shape/query67.groovy | 2 +-
.../shape/query68.groovy | 2 +-
.../shape/query69.groovy | 2 +-
.../shape/query7.groovy | 2 +-
.../shape/query70.groovy | 2 +-
.../shape/query71.groovy | 2 +-
.../shape/query72.groovy | 2 +-
.../shape/query73.groovy | 2 +-
.../shape/query74.groovy | 2 +-
.../shape/query75.groovy | 2 +-
.../shape/query76.groovy | 2 +-
.../shape/query77.groovy | 2 +-
.../shape/query78.groovy | 2 +-
.../shape/query79.groovy | 2 +-
.../shape/query8.groovy | 2 +-
.../shape/query80.groovy | 2 +-
.../shape/query81.groovy | 2 +-
.../shape/query82.groovy | 2 +-
.../shape/query83.groovy | 2 +-
.../shape/query84.groovy | 2 +-
.../shape/query85.groovy | 2 +-
.../shape/query86.groovy | 2 +-
.../shape/query87.groovy | 2 +-
.../shape/query88.groovy | 2 +-
.../shape/query89.groovy | 2 +-
.../shape/query9.groovy | 2 +-
.../shape/query90.groovy | 2 +-
.../shape/query91.groovy | 2 +-
.../shape/query92.groovy | 2 +-
.../shape/query93.groovy | 2 +-
.../shape/query94.groovy | 2 +-
.../shape/query95.groovy | 2 +-
.../shape/query96.groovy | 2 +-
.../shape/query97.groovy | 2 +-
.../shape/query98.groovy | 2 +-
.../shape/query99.groovy | 2 +-
.../suites/query_p0/having/having_between.groovy | 72 ++++
280 files changed, 2577 insertions(+), 1454 deletions(-)
copy
fe/fe-core/src/main/java/org/apache/doris/common/proc/{ProcDirInterface.java =>
BinlogProcDir.java} (62%)
copy
fe/fe-core/src/main/java/org/apache/doris/nereids/{trees/plans/visitor/DefaultPlanVisitor.java
=> processor/post/RegisterParent.java} (71%)
create mode 100644
regression-test/data/nereids_rules_p0/push_down_distinct_through_join/push_down_distinct_through_join.out
create mode 100644 regression-test/data/nereids_syntax_p0/test_literal.out
create mode 100644 regression-test/data/query_p0/having/having_between.out
copy
regression-test/suites/nereids_rules_p0/{push_down_limit_distinct/push_down_limit_distinct.groovy
=> push_down_distinct_through_join/push_down_distinct_through_join.groovy}
(71%)
create mode 100644 regression-test/suites/query_p0/having/having_between.groovy
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
