This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a change to branch branch-4.2
in repository https://gitbox.apache.org/repos/asf/doris.git
from 6bff765c894 branch-4.2: [fix](mtmv) Check agg/window partition keys
only for tracked partition outputs #65393 (#68295)
new 666eac09109 [branch-4.1](pick) forward profile level to master- #68382
(#68402)
new f402b287c52 [refactor](arrow) Make format convertors explicit and
schema-owned (#68381)
new eb33a237aa6 branch-4.1: [fix](streaming-job) Stabilize lag and TVF
pause-resume tests #68168 (#68400)
new b2c3b9725a6 branch-4.1: [fix](paimon) Invalidate SDK cache on metadata
refresh (#68238)
new 6c05f7a8c23 branch-4.1: [fix](function) Round Decimal values with zero
decimal places #68010 (#68412)
new 091b5b503e0 branch-4.1: [fix](function) Fix decimal rounding overflow
at scale boundaries #68251 (#68415)
new 3020271a76a branch-4.1: [fix](function) Handle odd-byte IPv6
truncation #68109 (#68414)
new 3b26ef36a24 branch-4.1: [fix](function) Split UTF-8 strings correctly
with empty regexp #68020 (#68417)
new 3430c436573 branch-4.1: [fix](function) Preserve NUL bytes in
count-by-enum keys #67984 (#68419)
new 01abfa9a798 branch-4.1: [fix](function) Handle zero-address IPv4 CIDR
correctly #68081 (#68420)
new 5dfbdc58c72 branch-4.1: [fix](function) Preserve NULL values in
parse_data_size #67910 (#68413)
new 82d75b7bb9d branch-4.1: [fix](function) Consider microseconds in week
floor and ceil #67961 (#68416)
new 12cbfd669f7 branch-4.1: [fix](search) Validate ngram search gram
number #67749 (#68418)
new 0ed97584e62 branch-4.1: [fix](be) Fix BIGINT conv overflow during base
conversion #67872 (#68424)
new 25710db991b branch-4.1: [fix](agg) Align complex aggregate null
ordering #67439 (#68422)
new 5ff72ef62ab branch-4.1: [fix](function) Fix MAKE_SET constant NULL
handling #68274 (#68411)
new 855c600cf88 branch-4.1: [fix](be) Preserve NULL propagation in date
floor functions #67951 (#68423)
new 2c201be9f2f branch-4.1: [fix](function) Handle constant NULL arrays in
array pushback #68380 (#68442)
new d87029ea3e3 branch-4.1: [fix](be) Keep all samples when merging
percentile states #68318 (#68475)
new d7a283efbcd branch-4.1: [improvement](be) Optimize pow squares with
exact integer results #68258 (#68443)
new 309fcab1c1c branch-4.1: [fix](cloud) Restore warm-up destination locks
after FE recovery #67867 (#68105)
new 9741804f507 branch-4.1: [fix](cloud) fix
get_prepare_txn_by_coordinator method #67761 (#68059)
new 08fb90314da branch-4.1: [fix](cloud) Prevent FDB CloudUT ASAN
getentropy crash #67316 (#68451)
new 65bec97ea2b branch-4.1: [fix](bitmap) fix SIGSEGV when reading
zero-length serialized BITMAP/HLL/QuantileState cell #67473 (#68497)
new 6ee1e252f05 branch-4.1: [fix](be) Fix nested array_map lambda captures
#64563 (#68471)
The 25 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
be/src/core/column/column_complex.h | 3 +-
.../core/data_type_serde/data_type_array_serde.cpp | 71 +
.../core/data_type_serde/data_type_array_serde.h | 10 +
.../core/data_type_serde/data_type_map_serde.cpp | 103 ++
be/src/core/data_type_serde/data_type_map_serde.h | 10 +
.../data_type_serde/data_type_nullable_serde.cpp | 23 +
.../data_type_serde/data_type_nullable_serde.h | 10 +
be/src/core/data_type_serde/data_type_serde.h | 18 +
.../data_type_serde/data_type_string_serde.cpp | 47 +
.../core/data_type_serde/data_type_string_serde.h | 5 +
.../data_type_serde/data_type_struct_serde.cpp | 71 +
.../core/data_type_serde/data_type_struct_serde.h | 10 +
.../data_type_serde/data_type_variant_v2_serde.cpp | 396 +++---
.../data_type_serde/data_type_variant_v2_serde.h | 10 +
be/src/core/value/ip_address_cidr.h | 6 +-
be/src/exec/common/format_ip.h | 11 +-
.../exec/operator/memory_scratch_sink_operator.cpp | 9 +-
.../writer/iceberg/viceberg_delete_file_writer.cpp | 6 +-
.../writer/iceberg/viceberg_partition_writer.cpp | 8 +-
.../writer/paimon/jni_paimon_write_backend.cpp | 57 +-
.../sink/writer/paimon/jni_paimon_write_backend.h | 11 +-
be/src/exec/sink/writer/vfile_result_writer.cpp | 4 +-
be/src/exec/sink/writer/vhive_partition_writer.cpp | 4 +-
.../aggregate/aggregate_function_count_by_enum.h | 3 +-
.../exprs/aggregate/aggregate_function_min_max.h | 10 +-
.../aggregate/aggregate_function_python_udaf.cpp | 25 +-
.../function/array/function_array_pushback.cpp | 7 +-
.../function/function_datetime_floor_ceil.cpp | 26 +-
be/src/exprs/function/function_ip.h | 8 +-
be/src/exprs/function/function_python_udf.cpp | 24 +-
be/src/exprs/function/function_split_by_regexp.cpp | 4 +-
be/src/exprs/function/function_string.cpp | 55 +-
be/src/exprs/function/function_string_format.h | 2 +-
be/src/exprs/function/function_string_misc.cpp | 3 +-
be/src/exprs/function/math.cpp | 23 +
be/src/exprs/function/round.h | 248 +---
.../lambda_function/lambda_execution_context.h | 116 ++
be/src/exprs/lambda_function/lambda_function.h | 2 +-
.../exprs/lambda_function/varray_map_function.cpp | 246 ++--
.../exprs/lambda_function/varray_sort_function.cpp | 179 ++-
be/src/exprs/math_functions.cpp | 55 +-
.../exprs/table_function/python_udtf_function.cpp | 14 +-
be/src/exprs/vcolumn_ref.h | 60 +-
be/src/exprs/vexpr_context.cpp | 12 +
be/src/exprs/vexpr_context.h | 39 +-
be/src/exprs/vlambda_function_call_expr.h | 2 +-
be/src/exprs/vlambda_function_expr.h | 14 +-
be/src/exprs/vslot_ref.h | 1 +
be/src/format/arrow/arrow_block_convertor.cpp | 329 +++--
be/src/format/arrow/arrow_block_convertor.h | 137 +-
be/src/format/arrow/arrow_row_batch.cpp | 5 +-
.../parquet/parquet_arrow_block_convertor.cpp | 55 +
.../format/parquet/parquet_arrow_block_convertor.h | 46 +
.../format/table/hive/hive_arrow_block_convertor.h | 26 +-
.../iceberg/iceberg_arrow_block_convertor.cpp | 57 +
.../table/iceberg/iceberg_arrow_block_convertor.h | 51 +
.../table/paimon/paimon_arrow_block_convertor.cpp | 58 +
.../table/paimon/paimon_arrow_block_convertor.h | 47 +
.../vfile_format_transformer_factory.cpp | 4 +-
.../src/format/transformer/vhive_parquet_writer.h | 29 +-
.../format/transformer/viceberg_parquet_writer.cpp | 133 ++
.../format/transformer/viceberg_parquet_writer.h | 49 +
.../format/transformer/vjni_format_transformer.h | 2 +-
be/src/format/transformer/vorc_transformer.h | 2 +-
...parquet_transformer.cpp => vparquet_writer.cpp} | 196 +--
.../{vparquet_transformer.h => vparquet_writer.h} | 42 +-
.../arrow_flight/arrow_flight_batch_reader.cpp | 8 +-
be/src/util/percentile_util.h | 36 +-
be/test/core/column/column_complex_test.cpp | 146 ++
.../core/data_type/common_data_type_serder_test.h | 4 +-
.../data_type_serde/data_type_serde_arrow_test.cpp | 628 ++++++++-
.../data_type_variant_v2_serde_output_test.cpp | 80 +-
be/test/exprs/aggregate/vec_count_by_enum_test.cpp | 48 +
be/test/exprs/function/function_ip_test.cpp | 35 +-
be/test/exprs/function/function_math_test.cpp | 216 ++-
be/test/exprs/function/function_round_test.cpp | 88 ++
be/test/exprs/function/function_string_test.cpp | 51 +
be/test/exprs/function/function_time_test.cpp | 15 +
.../lambda_function/array_map_function_test.cpp | 1409 ++++++++++++++++++++
be/test/exprs/math_functions_test.cpp | 40 +
be/test/exprs/vcolumn_ref_test.cpp | 148 ++
.../format/arrow/arrow_block_convertor_test.cpp | 291 ++++
...ansformer_test.cpp => vparquet_writer_test.cpp} | 108 +-
be/test/util/percentile_util_test.cpp | 84 ++
cloud/src/common/config.h | 4 +
cloud/src/meta-service/meta_service_txn.cpp | 180 ++-
cloud/src/meta-store/mem_txn_kv.cpp | 2 -
cloud/test/CMakeLists.txt | 37 +
cloud/test/fdb_getentropy_compat.c | 113 ++
cloud/test/meta_service_test.cpp | 448 +++++++
.../apache/doris/analysis/LambdaFunctionExpr.java | 1 +
.../org/apache/doris/catalog/RefreshManager.java | 178 ++-
.../apache/doris/cloud/CacheHotspotManager.java | 14 +
.../org/apache/doris/cloud/catalog/CloudEnv.java | 1 +
.../apache/doris/datasource/ExternalCatalog.java | 47 +-
.../apache/doris/datasource/ExternalDatabase.java | 45 +-
.../doris/datasource/ExternalMetaCacheMgr.java | 18 +-
.../doris/datasource/hive/HiveMetadataOps.java | 6 +-
.../datasource/iceberg/IcebergMetadataOps.java | 18 +-
.../maxcompute/MaxComputeMetadataOps.java | 6 +-
.../datasource/metacache/ExternalMetaCache.java | 23 +
.../metacache/paimon/PaimonTableLoader.java | 35 +-
.../datasource/operations/ExternalMetadataOps.java | 28 +-
.../datasource/paimon/PaimonExternalCatalog.java | 278 +++-
.../datasource/paimon/PaimonExternalMetaCache.java | 71 +-
.../doris/datasource/paimon/PaimonMetadataOps.java | 116 +-
.../insert/streaming/StreamingInsertTask.java | 2 +
.../offset/jdbc/JdbcTvfSourceOffsetProvider.java | 5 +
.../expressions/functions/scalar/NgramSearch.java | 24 +-
.../java/org/apache/doris/qe/SessionVariable.java | 2 +-
.../CdcStreamTableValuedFunction.java | 1 +
.../doris/analysis/LambdaFunctionExprTest.java | 42 +
.../doris/catalog/RefreshManagerReplayTest.java | 97 ++
.../cloud/CacheHotspotManagerRecoveryTest.java | 272 ++++
.../lance/LanceCatalogLifecycleTest.java | 8 +
.../paimon/PaimonExternalMetaCacheTest.java | 1127 +++++++++++++++-
.../datasource/paimon/PaimonMetadataOpsTest.java | 23 +
.../streaming/StreamingInsertTaskAuditTest.java | 13 +
.../org/apache/doris/cdcclient/common/Env.java | 14 +-
.../cdcclient/service/PipelineCoordinator.java | 70 +-
.../org/apache/doris/cdcclient/common/EnvTest.java | 88 ++
gensrc/thrift/Exprs.thrift | 4 +
.../data/function_p0/test_pow_square.out | 79 ++
.../pythonudf_p0/test_pythonudf_base_data_type.out | 20 +-
.../pythonudf_p0/test_pythonudf_data_types.out | 2 +-
.../aggregate/percentile_bucketed_agg_merge.out | 23 +
.../test_aggregate_all_functions2.out | 14 +-
.../test_array_functions_by_literal.out | 36 +
.../array_functions/test_nested_array_map.out | 13 +
.../test_week_floor_ceil_microsecond.out | 8 +
.../math_functions/test_format_round.out | 9 +-
.../test_parse_data_size_nullable.out | 24 +
.../string_functions/test_split_by_regexp.out | 6 +
.../suites/function_p0/test_pow_square.groovy | 114 ++
.../cdc/test_streaming_mysql_job_lag.groovy | 25 +-
.../cdc/test_streaming_postgres_job_lag.groovy | 34 +-
.../test_python_arrow_convertor_timezone.groovy | 88 ++
.../pythonudf_p0/test_pythonudf_data_types.groovy | 1 +
.../aggregate/percentile_bucketed_agg_merge.groovy | 93 ++
.../test_aggregate_all_functions2.groovy | 1 +
.../test_array_functions_by_literal.groovy | 2 +
.../array_functions/test_nested_array_map.groovy | 90 ++
.../test_week_floor_ceil_microsecond.groovy | 55 +
.../math_functions/test_format_round.groovy | 9 +-
.../test_parse_data_size_nullable.groovy | 50 +
.../string_functions/test_split_by_regexp.groovy | 2 +
.../string_functions/test_string_function.groovy | 8 +
147 files changed, 9631 insertions(+), 1420 deletions(-)
create mode 100644 be/src/exprs/lambda_function/lambda_execution_context.h
create mode 100644 be/src/format/parquet/parquet_arrow_block_convertor.cpp
create mode 100644 be/src/format/parquet/parquet_arrow_block_convertor.h
copy
fs_brokers/cdc_client/src/test/java/org/apache/doris/cdcclient/common/EnvTest.java
=> be/src/format/table/hive/hive_arrow_block_convertor.h (55%)
create mode 100644
be/src/format/table/iceberg/iceberg_arrow_block_convertor.cpp
create mode 100644 be/src/format/table/iceberg/iceberg_arrow_block_convertor.h
create mode 100644 be/src/format/table/paimon/paimon_arrow_block_convertor.cpp
create mode 100644 be/src/format/table/paimon/paimon_arrow_block_convertor.h
copy
fs_brokers/cdc_client/src/test/java/org/apache/doris/cdcclient/common/EnvTest.java
=> be/src/format/transformer/vhive_parquet_writer.h (53%)
create mode 100644 be/src/format/transformer/viceberg_parquet_writer.cpp
create mode 100644 be/src/format/transformer/viceberg_parquet_writer.h
rename be/src/format/transformer/{vparquet_transformer.cpp =>
vparquet_writer.cpp} (54%)
rename be/src/format/transformer/{vparquet_transformer.h => vparquet_writer.h}
(72%)
create mode 100644 be/test/core/column/column_complex_test.cpp
create mode 100644 be/test/exprs/lambda_function/array_map_function_test.cpp
create mode 100644 be/test/exprs/vcolumn_ref_test.cpp
create mode 100644 be/test/format/arrow/arrow_block_convertor_test.cpp
rename be/test/format/transformer/{vparquet_transformer_test.cpp =>
vparquet_writer_test.cpp} (61%)
create mode 100644 cloud/test/fdb_getentropy_compat.c
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/analysis/LambdaFunctionExprTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/catalog/RefreshManagerReplayTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/cloud/CacheHotspotManagerRecoveryTest.java
create mode 100644 regression-test/data/function_p0/test_pow_square.out
create mode 100644
regression-test/data/query_p0/aggregate/percentile_bucketed_agg_merge.out
create mode 100644
regression-test/data/query_p0/sql_functions/array_functions/test_nested_array_map.out
create mode 100644
regression-test/data/query_p0/sql_functions/datetime_functions/test_week_floor_ceil_microsecond.out
create mode 100644
regression-test/data/query_p0/sql_functions/string_functions/test_parse_data_size_nullable.out
create mode 100644 regression-test/suites/function_p0/test_pow_square.groovy
create mode 100644
regression-test/suites/pythonudf_p0/test_python_arrow_convertor_timezone.groovy
create mode 100644
regression-test/suites/query_p0/aggregate/percentile_bucketed_agg_merge.groovy
create mode 100644
regression-test/suites/query_p0/sql_functions/array_functions/test_nested_array_map.groovy
create mode 100644
regression-test/suites/query_p0/sql_functions/datetime_functions/test_week_floor_ceil_microsecond.groovy
create mode 100644
regression-test/suites/query_p0/sql_functions/string_functions/test_parse_data_size_nullable.groovy
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
