This is an automated email from the ASF dual-hosted git repository.
kxiao pushed a change to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/doris.git
from f91bdb7c87 [Chore](status) avoid empty error msg on status (#21454)
(#21712)
new b714c72e59 [fix](test) fix two case bug #21124
new 664a6b167c [Improvement](statistics, multi catalog)Support iceberg
table stats collection (#21481)
new 459f295704 [Enhancement](inverted index) reset global instance for
InvertedIndexSearcherCache when destroy (#21601)
new 984b1e9fb2 [refactor](udf) refactor java-udf execute method by using
for loop (#21388)
new 4cd9c20cd8 [Fix](pipeline) fix ExchangeSinkBuffer request id memory
alloc problem (#21647)
new f050426a86 [opt](Nereids): Join use List<Plan> as children (#21608)
new 8cd31f6042 [improvement](statistics)Support external table show table
stats, modify column stats and drop stats (#21624)
new a7e8b28287 [improvement](ssb) run-ssb-queries.sh and
run-ssb-flat-queries.sh add analyze database with sync and calculate total time
#21653
new 64ee35ea81 [Enhancement](Compaction) Caculate all committed rowsets
delete bitmaps when do comapction (#20907)
new 721d688764 [enchancement](stats) Inject partition statistics #21543
new 0e883e9d2b [Profile](runtimefilter) fix merge time of runtime filter
(#21654)
new 772411271f [feature](jdbc) Support jdbc catalog to read json types
(#21341)
new 22582d61b5 fix: toCalendar should use Calendar.MONTH instead MONDAY
(#21665)
new 91351014bf [enhancement](flush) make writer write fail status visible
(#21530)
new 4a50c354ee [fix](create table) modify varchar default length 1 to
65533 (#21302)
new 2470f31236 [test](spill) disable fuzzy spill variables for now (#21677)
new 6b21fb1813 fix merge conflict for
91351014bfc2a7ec7619b7281c5e7772bdfad0a2 [enhancement](flush) make writer
write fail status visible (#21530)
new dc5a9e7653 [fix](runtime_filter) runtime_profile was not initialized
in multi_cast_data_stream_source (#21690)
new bae501f958 [improvement](timezone) support mixed uppper-lower case of
timezone names (#21572)
new faef406fe9 [feature](partial update)support insert new rows in
non-strict mode partial update with nullable unmentioned columns (#21623)
new 0faf3dab71 [fix](regression)update some case with timediff (#21697)
new 1407c97ea9 [fix](sparkdpp) Change spark dpp default version to
1.2-SNAPSHOT (#21698)
new 1c7e7ccaab [improvement](column) Support for the default value of
current_timestamp in microsecond (#21487)
new fa27dedb89 [fix](jdbc scan) `1=1` does not translate to `TRUE` (#21688)
new ca9a9de288 [enhancement](mysql) Add have_query_cache variable to be
compatible with old mysql client (#21701)
new 2323761faf [regression] add bitmap filter p1 regression case (#21591)
new 9457dc6b48 [Improve](hash-fun)improve nested hash with range #21699
new 84f53bea36 [Bug](dynamic table) set all CreateTableStmt from cup
parser dynamic table flag false (#21706)
new 79a5663a7d [fix](config) set FE header size limit to 1MB from 10k
(#21719)
new d8f6199d99 [enhancement](nereids) Sync stats across FE cluster after
analyze #21482
The 30 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
be/src/exec/tablet_info.cpp | 5 +
be/src/exec/tablet_info.h | 3 +-
be/src/exprs/bloom_filter_func.h | 1 +
be/src/olap/compaction.cpp | 32 +-
be/src/olap/delta_writer.cpp | 1 +
be/src/olap/rowset/beta_rowset_writer.cpp | 3 +-
.../olap/rowset/segment_v2/inverted_index_cache.h | 12 +
be/src/olap/rowset/segment_v2/segment_writer.cpp | 71 +-
be/src/olap/rowset/segment_v2/segment_writer.h | 3 +-
be/src/olap/tablet.cpp | 7 +-
be/src/olap/tablet.h | 2 +-
be/src/olap/tablet_meta.cpp | 7 +-
be/src/olap/tablet_meta.h | 7 +
be/src/olap/tablet_schema.cpp | 8 +-
be/src/olap/tablet_schema.h | 12 +-
be/src/olap/txn_manager.cpp | 22 +
be/src/olap/txn_manager.h | 30 +-
be/src/pipeline/exec/exchange_sink_buffer.cpp | 6 +-
.../exec/multi_cast_data_stream_source.cpp | 3 +-
be/src/runtime/exec_env_init.cpp | 3 +
be/src/runtime/runtime_filter_mgr.cpp | 7 +-
be/src/runtime/runtime_filter_mgr.h | 3 +-
be/src/runtime/types.h | 2 +
be/src/util/timezone_utils.cpp | 39 +-
be/src/util/timezone_utils.h | 5 +
be/src/vec/columns/column.h | 8 +-
be/src/vec/columns/column_array.cpp | 77 +-
be/src/vec/columns/column_array.h | 6 +-
be/src/vec/columns/column_const.h | 8 +-
be/src/vec/columns/column_decimal.cpp | 50 +-
be/src/vec/columns/column_decimal.h | 14 +-
be/src/vec/columns/column_map.cpp | 80 +-
be/src/vec/columns/column_map.h | 6 +-
be/src/vec/columns/column_nullable.cpp | 34 +-
be/src/vec/columns/column_nullable.h | 6 +-
be/src/vec/columns/column_string.h | 42 +-
be/src/vec/columns/column_struct.cpp | 10 +-
be/src/vec/columns/column_struct.h | 6 +-
be/src/vec/columns/column_vector.h | 42 +-
be/src/vec/exec/runtime_filter_consumer.cpp | 9 +
be/src/vec/exec/runtime_filter_consumer.h | 2 +
be/src/vec/exec/scan/vscan_node.cpp | 7 +-
be/src/vec/exec/vjdbc_connector.cpp | 76 +
be/src/vec/exec/vjdbc_connector.h | 7 +
be/src/vec/functions/function_java_udf.cpp | 302 +--
be/src/vec/functions/function_java_udf.h | 4 +
be/test/olap/tablet_meta_test.cpp | 2 +-
be/test/vec/columns/column_hash_func_test.cpp | 79 +
be/test/vec/function/function_time_test.cpp | 3 +
.../utils/arrow_column_to_doris_column_test.cpp | 1 +
.../docker-compose/clickhouse/clickhouse.yaml.tpl | 2 +-
.../clickhouse/init/03-create-table.sql | 11 +-
.../docker-compose/clickhouse/init/04-insert.sql | 9 +-
.../postgresql/init/02-create-table.sql | 12 +
.../docker-compose/postgresql/init/04-insert.sql | 40 +
docs/en/docs/admin-manual/config/fe-config.md | 37 +-
docs/en/docs/advanced/variables.md | 4 +
.../import/import-scenes/load-strict-mode.md | 50 +-
docs/en/docs/lakehouse/multi-catalog/jdbc.md | 5 +-
docs/en/docs/query-acceleration/statistics.md | 3 +-
.../Create/CREATE-TABLE.md | 2 +-
.../Load/STREAM-LOAD.md | 2 +-
docs/zh-CN/docs/admin-manual/config/fe-config.md | 22 +-
docs/zh-CN/docs/advanced/variables.md | 4 +
.../import/import-scenes/load-strict-mode.md | 49 +-
docs/zh-CN/docs/lakehouse/multi-catalog/jdbc.md | 5 +-
docs/zh-CN/docs/query-acceleration/statistics.md | 3 +-
.../Create/CREATE-TABLE.md | 2 +-
.../Load/STREAM-LOAD.md | 2 +-
.../main/java/org/apache/doris/udf/UdfConvert.java | 1760 ++++++++++++++++
.../java/org/apache/doris/udf/UdfExecutor.java | 406 ++++
.../java/org/apache/doris/jdbc/JdbcExecutor.java | 17 +
.../main/java/org/apache/doris/common/Config.java | 15 +-
fe/fe-core/src/main/cup/sql_parser.cup | 10 +-
.../doris/analysis/AlterColumnStatsStmt.java | 56 +-
.../java/org/apache/doris/analysis/ColumnDef.java | 79 +-
.../apache/doris/analysis/DefaultValueExprDef.java | 49 +-
.../org/apache/doris/analysis/DropStatsStmt.java | 17 +
.../apache/doris/analysis/NativeInsertStmt.java | 2 +-
.../apache/doris/analysis/ShowTableStatsStmt.java | 3 -
.../doris/catalog/external/HMSExternalTable.java | 25 +-
.../catalog/external/IcebergExternalTable.java | 10 +
.../apache/doris/datasource/InternalCatalog.java | 8 +-
.../apache/doris/external/hive/util/HiveUtil.java | 2 +-
.../doris/external/jdbc/JdbcMySQLClient.java | 3 +-
.../doris/external/jdbc/JdbcPostgreSQLClient.java | 4 +-
.../doris/load/loadv2/LoadingTaskPlanner.java | 2 +-
.../hypergraph/receiver/PlanReceiver.java | 2 +-
.../doris/nereids/pattern/GroupMatching.java | 2 -
.../nereids/properties/LogicalProperties.java | 4 -
.../nereids/rules/analysis/BindExpression.java | 4 +-
.../exploration/join/InnerJoinLAsscomProject.java | 5 +-
.../join/InnerJoinLeftAssociateProject.java | 5 +-
.../join/InnerJoinRightAssociateProject.java | 5 +-
.../exploration/join/JoinExchangeBothProject.java | 3 +-
.../exploration/join/JoinExchangeLeftProject.java | 3 +-
.../exploration/join/JoinExchangeRightProject.java | 3 +-
.../join/LogicalJoinSemiJoinTransposeProject.java | 5 +-
.../exploration/join/OuterJoinAssocProject.java | 4 +-
.../exploration/join/OuterJoinLAsscomProject.java | 5 +-
.../join/PushdownProjectThroughInnerJoin.java | 4 +-
.../join/PushdownProjectThroughSemiJoin.java | 2 +-
.../join/SemiJoinSemiJoinTransposeProject.java | 4 +-
.../rules/expression/ExpressionRewrite.java | 2 +-
.../nereids/rules/rewrite/ExistsApplyToJoin.java | 4 +-
.../rules/rewrite/ExtractFilterFromCrossJoin.java | 2 +-
.../rules/rewrite/FindHashConditionForJoin.java | 2 +-
.../doris/nereids/rules/rewrite/InApplyToJoin.java | 4 +-
.../rules/rewrite/PushFilterInsideJoin.java | 2 +-
.../nereids/rules/rewrite/ScalarApplyToJoin.java | 3 +-
.../doris/nereids/trees/plans/AbstractPlan.java | 30 +-
.../nereids/trees/plans/commands/Command.java | 14 -
.../plans/commands/InsertIntoTableCommand.java | 2 +-
.../trees/plans/logical/AbstractLogicalPlan.java | 8 +-
.../nereids/trees/plans/logical/LogicalBinary.java | 5 +
.../nereids/trees/plans/logical/LogicalJoin.java | 45 +-
.../nereids/trees/plans/logical/LogicalUnary.java | 4 -
.../doris/nereids/types/coercion/DateLikeType.java | 2 +-
.../org/apache/doris/planner/JdbcScanNode.java | 3 +
.../org/apache/doris/planner/OlapTableSink.java | 6 +-
.../apache/doris/planner/StreamLoadPlanner.java | 4 +-
.../java/org/apache/doris/qe/SessionVariable.java | 10 +
.../apache/doris/service/FrontendServiceImpl.java | 12 +
.../apache/doris/statistics/AnalysisManager.java | 3 +
.../apache/doris/statistics/ColumnStatistic.java | 49 +-
.../statistics/ColumnStatisticsCacheLoader.java | 32 +-
.../apache/doris/statistics/HMSAnalysisTask.java | 258 ++-
.../org/apache/doris/statistics/Histogram.java | 5 +
.../apache/doris/statistics/HiveAnalysisTask.java | 370 ----
.../doris/statistics/IcebergAnalysisTask.java | 121 --
.../apache/doris/statistics/OlapAnalysisTask.java | 2 +-
.../apache/doris/statistics/StatisticsCache.java | 98 +-
.../doris/statistics/StatisticsCacheKey.java | 7 +
.../doris/statistics/StatisticsRepository.java | 64 +-
.../doris/statistics/util/StatisticsUtil.java | 55 +-
.../org/apache/doris/catalog/CreateTableTest.java | 14 +
.../doris/nereids/util/HyperGraphBuilder.java | 2 +-
.../doris/nereids/util/LogicalPlanBuilder.java | 2 +-
.../apache/doris/planner/OlapTableSinkTest.java | 8 +-
.../doris/planner/TableFunctionPlanTest.java | 4 +-
.../org/apache/doris/statistics/CacheTest.java | 3 +
gensrc/proto/descriptors.proto | 1 +
gensrc/thrift/Descriptors.thrift | 1 +
gensrc/thrift/FrontendService.thrift | 7 +
.../data/correctness_p0/test_current_timestamp.out | 9 +
.../data/ddl_p0/test_createtable_strlen.out | 7 +
.../hive/test_hive_statistic.out | 57 +
.../test_clickhouse_jdbc_catalog.out | Bin 2583 -> 2710 bytes
.../jdbc_catalog_p0/test_mysql_jdbc_catalog.out | 73 +-
.../jdbc_catalog_p0/test_oracle_jdbc_catalog.out | 11 +
.../data/jdbc_catalog_p0/test_pg_jdbc_catalog.out | 2121 ++++++++++----------
.../test_sqlserver_jdbc_catalog.out | 9 +
.../datetime_functions/test_date_function.out | 15 +
.../data/query_p1/test_bitmap_filter_p1.out | 241 +++
.../partial_update/test_partial_update_upsert.out | 12 +
.../unique_with_mow_p0/partial_update/upsert.csv | 3 +
.../correctness_p0/test_char_implicit_cast.groovy | 2 +-
.../correctness_p0/test_current_timestamp.groovy | 89 +-
.../suites/ddl_p0/test_createtable_strlen.groovy | 46 +-
.../hive/test_hive_statistic.groovy | 62 +
.../suites/index_p0/test_bitmap_index.groovy | 6 +-
.../inverted_index_p0/test_bitmap_index.groovy | 6 +-
.../inverted_index_p0/test_inverted_index.groovy | 6 +-
.../test_clickhouse_jdbc_catalog.groovy | 3 +
.../jdbc_catalog_p0/test_mysql_jdbc_catalog.groovy | 3 +
.../test_oracle_jdbc_catalog.groovy | 3 +
.../jdbc_catalog_p0/test_pg_jdbc_catalog.groovy | 5 +
.../test_sqlserver_jdbc_catalog.groovy | 3 +
.../mv_p0/multi_slot_k123p/multi_slot_k123p.groovy | 1 +
.../datetime_functions/test_date_function.groovy | 6 +
.../sql_functions/math_functions/test_round.groovy | 1 +
.../suites/query_p0/test_char_implicit_cast.groovy | 2 +-
.../suites/query_p1/test_bitmap_filter_p1.groovy | 307 +++
.../suites/statistics/analyze_stats.groovy | 4 +
.../test_partial_update_upsert.groovy | 103 +
tools/ssb-tools/bin/run-ssb-flat-queries.sh | 6 +
tools/ssb-tools/bin/run-ssb-queries.sh | 6 +
177 files changed, 6257 insertions(+), 2208 deletions(-)
create mode 100644
fe/be-java-extensions/java-udf/src/main/java/org/apache/doris/udf/UdfConvert.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/statistics/HiveAnalysisTask.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/statistics/IcebergAnalysisTask.java
create mode 100644 regression-test/data/ddl_p0/test_createtable_strlen.out
create mode 100644
regression-test/data/external_table_emr_p2/hive/test_hive_statistic.out
create mode 100644 regression-test/data/query_p1/test_bitmap_filter_p1.out
create mode 100644
regression-test/data/unique_with_mow_p0/partial_update/test_partial_update_upsert.out
create mode 100644
regression-test/data/unique_with_mow_p0/partial_update/upsert.csv
copy be/src/util/timezone_utils.h =>
regression-test/suites/ddl_p0/test_createtable_strlen.groovy (61%)
create mode 100644
regression-test/suites/external_table_emr_p2/hive/test_hive_statistic.groovy
create mode 100644 regression-test/suites/query_p1/test_bitmap_filter_p1.groovy
create mode 100644
regression-test/suites/unique_with_mow_p0/partial_update/test_partial_update_upsert.groovy
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
