This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a change to branch branch-2.1
in repository https://gitbox.apache.org/repos/asf/doris.git
from 60e20a3afea [fix](pipeline_x) Crc32HashPartitioner should use
ShuffleChannelIds (#34147)
new fdf91759b65 [fix](nereids)prevent null pointer access if translate
expression fails (#33990)
new b7b87fbb954 [fix](planner)cast expr should do nothing in
compactForLiteral method (#34047)
new 5adc823b146 [fix](nereids)move ReplaceVariableByLiteral rule to
analyze phase (#33997)
new a34ed4643ad [fix](planner)date_add function should accept date type as
its param (#34035)
new b41a5339d38 [Fix](nereids) fix rule merge_aggregate when has project
(#33892)
new b24ff9953d1 [fix](Nereids) column pruning should prune map in cte
consumer (#34079)
new 91887a285e2 Implement HLL with 128 buckets to support statistics
cache. (#34124)
new acc2b532e7f [Test](hive-writer) Adjust test_hive_write_partitions
regression test to resolve special characters issue with git on windows.
(#34026)
new 0f0c0a266b6 [opt](parquet)Skip page with offset index (#33082)
new 627245f93ae [fix](Nereids) support not in predicate for delete command
(#34153)
The 10 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../parquet/vparquet_column_chunk_reader.cpp | 82 +++---
.../format/parquet/vparquet_column_chunk_reader.h | 11 +-
.../exec/format/parquet/vparquet_column_reader.cpp | 7 +-
.../exec/format/parquet/vparquet_column_reader.h | 22 +-
.../exec/format/parquet/vparquet_group_reader.cpp | 12 +-
.../exec/format/parquet/vparquet_group_reader.h | 1 -
.../exec/format/parquet/vparquet_page_reader.cpp | 15 +-
.../vec/exec/format/parquet/vparquet_page_reader.h | 123 ++++++++-
be/src/vec/exec/format/parquet/vparquet_reader.cpp | 11 +-
be/src/vec/exec/format/parquet/vparquet_reader.h | 3 +-
be/test/vec/exec/parquet/parquet_thrift_test.cpp | 3 +-
.../hive/scripts/create_preinstalled_table.hql | 13 +-
.../varchar_col=varchar_value1/000000_0 | Bin
.../varchar_col=varchar_value1/000000_0 | Bin
.../varchar_col=varchar_value1/000000_0 | Bin
.../main/java/org/apache/doris/common/io/Hll.java | 15 +-
.../java/org/apache/doris/common/io/HllTest.java | 4 +-
.../java/org/apache/doris/analysis/CastExpr.java | 5 +
.../apache/doris/analysis/FunctionCallExpr.java | 16 ++
.../org/apache/doris/nereids/CascadesContext.java | 26 +-
.../org/apache/doris/nereids/StatementContext.java | 41 +--
.../glue/translator/PhysicalPlanTranslator.java | 6 +-
.../doris/nereids/jobs/executor/Analyzer.java | 12 +
.../doris/nereids/jobs/executor/Rewriter.java | 8 +-
.../org/apache/doris/nereids/rules/RuleType.java | 3 +-
.../nereids/rules/analysis/VariableToLiteral.java | 32 +--
.../rules/expression/ExpressionNormalization.java | 2 -
.../expression/rules/FoldConstantRuleOnBE.java | 5 +
.../rules/rewrite/CollectCteConsumerOutput.java | 34 +--
.../rules/rewrite/CollectProjectAboveConsumer.java | 81 ------
.../doris/nereids/rules/rewrite/ColumnPruning.java | 31 ++-
.../nereids/rules/rewrite/MergeAggregate.java | 23 +-
.../nereids/rules/rewrite/RewriteCteChildren.java | 17 +-
.../trees/plans/commands/DeleteFromCommand.java | 2 +
.../trees/plans/logical/LogicalCTEConsumer.java | 22 +-
.../org/apache/doris/statistics/util/Hll128.java | 214 +++++++++++++++
.../apache/doris/statistics/util/Hll128Test.java | 204 +++++++++++++++
gensrc/script/doris_builtins_functions.py | 2 +
.../hive/test_hive_parquet_skip_page.out | 289 +++++++++++++++++++++
.../hive/write/test_hive_write_partitions.out | 105 ++++----
.../data/nereids_hint_tpcds_p0/shape/query1.out | 3 +-
.../data/nereids_hint_tpcds_p0/shape/query24.out | 3 +-
.../data/nereids_p0/hint/multi_leading.out | 6 +-
.../merge_aggregate/merge_aggregate.out | 51 ++++
.../nereids_tpcds_shape_sf1000_p0/shape/query1.out | 3 +-
.../shape/query23.out | 12 +-
.../shape/query24.out | 3 +-
.../shape/query30.out | 3 +-
.../shape/query31.out | 48 ++--
.../shape/query39.out | 10 +-
.../shape/query47.out | 5 +-
.../shape/query57.out | 5 +-
.../shape/query59.out | 19 +-
.../shape/query70.out | 1 +
.../shape/query81.out | 3 +-
.../shape/query95.out | 6 +-
.../constraints/query23.out | 12 +-
.../noStatsRfPrune/query1.out | 3 +-
.../noStatsRfPrune/query23.out | 12 +-
.../noStatsRfPrune/query24.out | 3 +-
.../noStatsRfPrune/query30.out | 3 +-
.../noStatsRfPrune/query31.out | 40 +--
.../noStatsRfPrune/query39.out | 10 +-
.../noStatsRfPrune/query47.out | 5 +-
.../noStatsRfPrune/query57.out | 5 +-
.../noStatsRfPrune/query59.out | 19 +-
.../noStatsRfPrune/query70.out | 1 +
.../noStatsRfPrune/query81.out | 3 +-
.../noStatsRfPrune/query95.out | 6 +-
.../no_stats_shape/query1.out | 3 +-
.../no_stats_shape/query23.out | 12 +-
.../no_stats_shape/query24.out | 3 +-
.../no_stats_shape/query30.out | 3 +-
.../no_stats_shape/query31.out | 40 +--
.../no_stats_shape/query39.out | 10 +-
.../no_stats_shape/query47.out | 5 +-
.../no_stats_shape/query57.out | 5 +-
.../no_stats_shape/query59.out | 19 +-
.../no_stats_shape/query81.out | 3 +-
.../no_stats_shape/query95.out | 6 +-
.../rf_prune/query1.out | 3 +-
.../rf_prune/query23.out | 12 +-
.../rf_prune/query24.out | 3 +-
.../rf_prune/query30.out | 3 +-
.../rf_prune/query31.out | 48 ++--
.../rf_prune/query39.out | 10 +-
.../rf_prune/query47.out | 5 +-
.../rf_prune/query57.out | 5 +-
.../rf_prune/query59.out | 19 +-
.../rf_prune/query70.out | 1 +
.../rf_prune/query81.out | 3 +-
.../rf_prune/query95.out | 6 +-
.../nereids_tpcds_shape_sf100_p0/shape/query1.out | 3 +-
.../nereids_tpcds_shape_sf100_p0/shape/query23.out | 12 +-
.../nereids_tpcds_shape_sf100_p0/shape/query24.out | 3 +-
.../nereids_tpcds_shape_sf100_p0/shape/query30.out | 3 +-
.../nereids_tpcds_shape_sf100_p0/shape/query31.out | 48 ++--
.../nereids_tpcds_shape_sf100_p0/shape/query39.out | 10 +-
.../nereids_tpcds_shape_sf100_p0/shape/query47.out | 5 +-
.../nereids_tpcds_shape_sf100_p0/shape/query57.out | 5 +-
.../nereids_tpcds_shape_sf100_p0/shape/query59.out | 19 +-
.../nereids_tpcds_shape_sf100_p0/shape/query81.out | 3 +-
.../nereids_tpcds_shape_sf100_p0/shape/query95.out | 6 +-
.../correctness/test_date_function_const.groovy | 5 +-
.../suites/correctness_p0/test_cast_decimal.groovy | 11 +
.../hive/test_hive_parquet_skip_page.groovy | 131 ++++++++++
.../hive/write/test_hive_write_partitions.groovy | 2 +-
.../nereids_p0/cte/test_cte_column_pruning.groovy | 143 ++++++++++
.../suites/nereids_p0/test_user_var.groovy | 35 +++
.../merge_aggregate/merge_aggregate.groovy | 80 ++++++
110 files changed, 1879 insertions(+), 677 deletions(-)
rename
docker/thirdparties/docker-compose/hive/scripts/preinstalled_data/parquet_table/all_partition_types2_parquet_snappy_src/decimal_col=-123456.789012/string_col=string_value/{binary_col=62
69 6e 61 72 79 5f 76 61 6c 75 65/date_col=2024-03-21/timestamp_col=2024-03-21
12%3A00%3A00/char_col=char_value1 =>
date_col=2024-03-21/char_col=char_value1}/varchar_col=varchar_value1/000000_0
(100%)
rename
docker/thirdparties/docker-compose/hive/scripts/preinstalled_data/parquet_table/all_partition_types2_parquet_snappy_src/decimal_col=-123456.789012/string_col=string_value/{binary_col=62
69 6e 61 72 79 5f 76 61 6c 75 65/date_col=2024-03-22/timestamp_col=2024-03-22
12%3A00%3A00/char_col=char_value1 =>
date_col=2024-03-22/char_col=char_value1}/varchar_col=varchar_value1/000000_0
(100%)
rename
docker/thirdparties/docker-compose/hive/scripts/preinstalled_data/parquet_table/all_partition_types2_parquet_snappy_src/decimal_col=123456.789012/string_col=string_value/{binary_col=62
69 6e 61 72 79 5f 76 61 6c 75 65/date_col=2024-03-20/timestamp_col=2024-03-20
12%3A00%3A00/char_col=char_value1 =>
date_col=2024-03-20/char_col=char_value1}/varchar_col=varchar_value1/000000_0
(100%)
copy regression-test/suites/correctness_p0/test_cast_decimal.groovy =>
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/analysis/VariableToLiteral.java
(51%)
copy regression-test/suites/correctness_p0/test_cast_decimal.groovy =>
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/CollectCteConsumerOutput.java
(50%)
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/CollectProjectAboveConsumer.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/statistics/util/Hll128.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/statistics/util/Hll128Test.java
create mode 100644
regression-test/data/external_table_p0/hive/test_hive_parquet_skip_page.out
create mode 100644
regression-test/suites/external_table_p0/hive/test_hive_parquet_skip_page.groovy
create mode 100644
regression-test/suites/nereids_p0/cte/test_cte_column_pruning.groovy
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
