This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a change to branch auto-pick-70181-branch-4.2
in repository https://gitbox.apache.org/repos/asf/doris.git
discard e6c2e80979dc [fix](fe) Fix publish recovery and task cleanup after
backend outages (#70181)
add 1ed9383f3eef branch-4.2: [fix](be) Add fixed per-task ANN build thread
budgets #70454 (#70543)
add 51e384b919e2 branch-4.2: [fix](be) Fix data race on
Segment::_footer_pb #67720 (#70638)
add 71fc28ca3491 branch-4.2: [improvement](variant) Assert Variant sparse
byte order in debug builds (#70590)
add e4b99b7f8eb2 [branch-4.2](point query) Fix short-circuit point query
returning the default for a column added by light schema change (#70608)
add 71cb7eb9b8c3 branch-4.2: [feature](variant) Add
variant_escape_dots_in_keys to keep dotted keys distinct (#70643)
add f8862153287d [fix](be) Restore file-level cache metadata loading
during startup (#69116) (#70677)
add 1380984ddb4a [fix](fe) Handle BACKUP timeout with immutable properties
(#69424) (#70682)
add 3a84339911e5 branch-4.2: [fix](cloud) Split oversized delete bitmaps
across metadata transactions #70478 (#70644)
add abea8643c94e branch-4.2: [fix](nereids) Honor cast mode for string
literal arithmetic #70518 (#70617)
add 7095636156e2 branch-4.2: [fix](nereids) Merge existing filters when
adding predicates (70409) (#70620)
add 95ea83f8de41 branch-4.2: [fix](be) Fix data race on
Segment::_footer_pb #67720 (#70642)
add 48b17e2b3eba [fix](be) Remove heartbeat tokens from logs and errors
(#69532) (#70692)
add a43275467ad9 branch-4.2: [fix](be) Preserve DATE type in min/max
runtime filters (#70333)
add d0ec1f6b0ff7 branch-4.2: [fix](be) Isolate runtime filter wrappers
shared with local consumers (#70623)
add 1fb0f9ea8a86 [fix](fe) Fix publish recovery and task cleanup after
backend outages (#70181)
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (e6c2e80979dc)
\
N -- N -- N refs/heads/auto-pick-70181-branch-4.2 (1fb0f9ea8a86)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
be/src/agent/heartbeat_server.cpp | 10 +-
be/src/cloud/cloud_internal_service.cpp | 4 +
be/src/cloud/pb_convert.cpp | 4 +
be/src/common/config.cpp | 9 +
be/src/common/config.h | 5 +
be/src/core/column/column_variant.cpp | 8 +
be/src/core/column/variant_column_utils.cpp | 9 +
be/src/core/column/variant_column_utils.h | 5 +
be/src/exec/common/variant_util.cpp | 101 ++-
be/src/exec/common/variant_util.h | 17 +-
be/src/exec/rowid_fetcher.cpp | 14 +
be/src/exec/runtime_filter/runtime_filter_merger.h | 36 +-
.../runtime_filter/runtime_filter_producer.cpp | 8 +-
.../exec/runtime_filter/runtime_filter_wrapper.cpp | 47 +-
.../exec/runtime_filter/runtime_filter_wrapper.h | 6 +-
be/src/exec/scan/olap_scanner.cpp | 2 +
be/src/exprs/bloom_filter_func.h | 34 +
be/src/exprs/filter_base.h | 1 +
be/src/exprs/minmax_predicate.h | 11 +
be/src/io/cache/block_file_cache.cpp | 21 +-
be/src/io/cache/cache_block_meta_store.cpp | 54 +-
be/src/io/cache/cache_block_meta_store.h | 4 +
be/src/io/cache/file_cache_storage.h | 7 +-
be/src/io/cache/fs_file_cache_storage.cpp | 41 +-
be/src/io/cache/fs_file_cache_storage.h | 5 +-
be/src/io/cache/mem_file_cache_storage.cpp | 6 +-
be/src/io/cache/mem_file_cache_storage.h | 5 +-
be/src/service/internal_service.cpp | 4 +-
be/src/service/point_query_executor.cpp | 48 +-
be/src/storage/cache/page_cache.h | 5 +-
be/src/storage/index/ann/faiss_ann_index.cpp | 42 +-
.../index/inverted/reader/predicate_collector.cpp | 3 +-
be/src/storage/schema.cpp | 107 +++
be/src/storage/schema.h | 6 +
be/src/storage/segment/segment.cpp | 31 +-
be/src/storage/segment/segment.h | 6 +-
be/src/storage/segment/segment_iterator.cpp | 2 +-
.../variant/binary_column_extract_iterator.cpp | 2 +
.../segment/variant/hierarchical_data_iterator.cpp | 4 +-
.../segment/variant/hierarchical_data_iterator.h | 2 +-
.../variant/sparse_column_merge_iterator.cpp | 10 +
.../segment/variant/v2/variant_assembler.cpp | 33 +-
.../storage/segment/variant/v2/variant_assembler.h | 5 +-
.../segment/variant/v2/variant_column_writer.cpp | 3 +
.../segment/variant/v2/variant_shredder.cpp | 11 +-
.../storage/segment/variant/v2/variant_shredder.h | 1 +
.../segment/variant/variant_column_reader.cpp | 187 +++-
.../segment/variant/variant_column_reader.h | 12 +-
.../segment/variant/variant_column_writer_impl.cpp | 36 +-
.../variant_doc_snpashot_compact_iterator.h | 38 +-
.../segment/variant/variant_writer_helpers.cpp | 24 +-
be/src/storage/tablet/tablet_meta.cpp | 1 +
be/src/storage/tablet/tablet_schema.cpp | 53 +-
be/src/storage/tablet/tablet_schema.h | 6 +-
be/src/util/json/path_in_data.cpp | 37 +
be/src/util/json/path_in_data.h | 6 +-
be/test/agent/heartbeat_server_token_test.cpp | 90 ++
be/test/core/column/column_variant_test.cpp | 23 +
be/test/core/jsonb/path_in_data_test.cpp | 16 +
be/test/exec/common/schema_util_test.cpp | 44 +-
.../runtime_filter/runtime_filter_merger_test.cpp | 242 +++++
.../runtime_filter_producer_test.cpp | 519 ++++++++++-
.../runtime_filter/runtime_filter_wrapper_test.cpp | 471 ++++++++++
.../io/cache/block_file_cache_test_meta_store.cpp | 181 +++-
be/test/io/cache/cache_block_meta_store_test.cpp | 37 +-
.../cache/cached_remote_file_reader_peer_test.cpp | 84 +-
.../storage/index/ann/faiss_vector_index_test.cpp | 198 +++++
.../inverted/reader/collection_statistics_test.cpp | 63 ++
.../inverted/snii/snii_variant_routing_test.cpp | 5 +-
be/test/storage/pb_convert_test.cpp | 17 +
be/test/storage/schema_test.cpp | 161 ++++
.../binary_column_extract_iterator_test.cpp | 85 ++
.../storage/segment/segment_footer_cache_test.cpp | 283 +++++-
.../segment/variant_assembler_legacy_test.cpp | 24 +
be/test/storage/tablet/tablet_schema_test.cpp | 120 +++
.../variant/variant_column_writer_reader_test.cpp | 857 +++++++++++++++++-
be/test/storage/variant/variant_util_test.cpp | 31 +
cloud/src/meta-service/meta_service.cpp | 411 +++++----
cloud/test/CMakeLists.txt | 1 +
cloud/test/meta_service_large_bitmap_test.cpp | 544 ++++++++++++
cloud/test/schema_kv_test.cpp | 47 +
.../main/java/org/apache/doris/common/Config.java | 7 +
.../org/apache/doris/alter/CloudRollupJobV2.java | 2 +-
.../apache/doris/alter/CloudSchemaChangeJobV2.java | 3 +-
.../doris/alter/MaterializedViewHandler.java | 11 +
.../java/org/apache/doris/alter/RollupJobV2.java | 3 +-
.../apache/doris/alter/SchemaChangeHandler.java | 9 +-
.../org/apache/doris/alter/SchemaChangeJobV2.java | 2 +-
.../java/org/apache/doris/backup/RestoreJob.java | 20 +-
.../main/java/org/apache/doris/catalog/Env.java | 40 +-
.../java/org/apache/doris/catalog/OlapTable.java | 35 +-
.../org/apache/doris/catalog/TableProperty.java | 15 +
.../apache/doris/cloud/backup/CloudRestoreJob.java | 3 +-
.../cloud/datasource/CloudInternalCatalog.java | 5 +-
.../apache/doris/common/util/PropertyAnalyzer.java | 175 ++++
.../apache/doris/datasource/ExternalCatalog.java | 2 +
.../apache/doris/datasource/InternalCatalog.java | 26 +-
.../org/apache/doris/master/ReportHandler.java | 2 +-
.../java/org/apache/doris/mtmv/MTMVPlanUtil.java | 17 +-
.../glue/translator/ExpressionTranslator.java | 2 +-
.../glue/translator/PlanTranslatorContext.java | 10 +-
.../doris/nereids/parser/LogicalPlanBuilder.java | 4 +
.../rules/rewrite/RewriteSearchToSlots.java | 11 +-
.../trees/plans/commands/BackupCommand.java | 11 +-
.../trees/plans/commands/CreateTableCommand.java | 14 +
.../plans/commands/info/AlterMTMVPropertyInfo.java | 2 +
.../trees/plans/commands/info/CreateTableInfo.java | 11 +
.../commands/info/ModifyTablePropertiesOp.java | 1 +
.../apache/doris/nereids/types/VariantType.java | 7 +
.../org/apache/doris/nereids/util/PlanUtils.java | 24 +
.../doris/nereids/util/TypeCoercionUtils.java | 15 +-
.../org/apache/doris/planner/OlapTableSink.java | 2 +
.../org/apache/doris/task/CreateReplicaTask.java | 5 +-
.../apache/doris/analysis/MVColumnItemTest.java | 22 +-
.../org/apache/doris/backup/RestoreJobTest.java | 66 ++
.../org/apache/doris/catalog/OlapTableTest.java | 33 +
.../doris/catalog/VariantEscapeTableTest.java | 433 +++++++++
.../cloud/datasource/CloudInternalCatalogTest.java | 26 +-
.../apache/doris/common/PropertyAnalyzerTest.java | 172 ++++
.../ExternalCatalogVariantPolicyTest.java | 43 +
.../org/apache/doris/mtmv/MTMVPlanUtilTest.java | 37 +
.../glue/translator/ExpressionTranslatorTest.java | 80 ++
.../nereids/rules/rewrite/InferAggNotNullTest.java | 92 ++
.../rules/rewrite/RewriteSearchToSlotsTest.java | 23 +
.../trees/plans/commands/BackupCommandTest.java | 15 +
.../plans/commands/CreateTableCommandTest.java | 81 ++
.../apache/doris/nereids/util/PlanUtilsTest.java | 71 ++
.../doris/nereids/util/TypeCoercionUtilsTest.java | 43 +
.../java/org/apache/doris/task/AgentTaskTest.java | 21 +-
gensrc/proto/olap_file.proto | 4 +
gensrc/proto/segment_v2.proto | 2 +
gensrc/thrift/AgentService.thrift | 1 +
.../datev1/test_datev1_minmax_runtime_filter.out | 8 +
.../nereids_p0/test_string_literal_arithmetic.out | 37 +
.../test_point_query_light_schema_change.out | 28 +
.../dotted_key_escape_added_column_rewrite.out | 31 +
.../data/variant_p0/dotted_key_escape_async_mv.out | 32 +
.../variant_p0/dotted_key_escape_common_type.out | 38 +
.../dotted_key_escape_compaction_failure.out | 9 +
.../dotted_key_escape_complex_rewrite.out | 22 +
.../dotted_key_escape_condition_cache.out | 13 +
.../variant_p0/dotted_key_escape_ctas_template.out | 19 +
.../dotted_key_escape_derived_format.out | 29 +
.../variant_p0/dotted_key_escape_doc_threshold.out | 46 +
.../dotted_key_escape_document_examples.out | 49 ++
.../variant_p0/dotted_key_escape_index_profile.out | 19 +
.../variant_p0/dotted_key_escape_index_rewrite.out | 59 ++
.../data/variant_p0/dotted_key_escape_ingest.json | 2 +
.../variant_p0/dotted_key_escape_layout_switch.out | 155 ++++
.../variant_p0/dotted_key_escape_mow_rewrite.out | 9 +
.../data/variant_p0/dotted_key_escape_rewrite.out | 287 ++++++
.../data/variant_p0/dotted_key_escape_score.out | 13 +
.../variant_p0/dotted_key_escape_semantics.out | 99 +++
.../dotted_key_escape_sparse_promotion.out | 974 +++++++++++++++++++++
.../variant_p0/dotted_key_escape_stream_load.out | 10 +
.../dotted_key_escape_table_property.out | 16 +
.../data/variant_p0/dotted_key_legacy_rewrite.out | 92 ++
.../variant_p0/dotted_key_legacy_semantics.out | 48 +
.../sparse_ancestor_promotion_extended.out | 24 +-
.../test_datev1_minmax_runtime_filter.groovy | 82 ++
.../test_string_literal_arithmetic.groovy | 68 ++
.../suites/nereids_syntax_p0/cast.groovy | 12 +-
.../test_point_query_light_schema_change.groovy | 99 +++
.../suites/variant_p0/column_name.groovy | 2 +-
.../dotted_key_escape_added_column_rewrite.groovy | 58 ++
.../variant_p0/dotted_key_escape_async_mv.groovy | 107 +++
.../dotted_key_escape_common_type.groovy | 83 ++
.../variant_p0/dotted_key_escape_compaction.groovy | 141 +++
.../dotted_key_escape_compaction_failure.groovy | 99 +++
.../dotted_key_escape_complex_rewrite.groovy | 50 ++
.../dotted_key_escape_condition_cache.groovy | 66 ++
.../dotted_key_escape_cross_encoding.groovy | 84 ++
.../dotted_key_escape_ctas_template.groovy | 64 ++
.../dotted_key_escape_derived_format.groovy | 79 ++
.../dotted_key_escape_doc_threshold.groovy | 50 ++
.../dotted_key_escape_document_examples.groovy | 75 ++
.../dotted_key_escape_element_at_mv.groovy | 92 ++
.../dotted_key_escape_index_profile.groovy | 59 ++
.../dotted_key_escape_index_rewrite.groovy | 58 ++
.../dotted_key_escape_layout_switch.groovy | 87 ++
.../dotted_key_escape_mow_rewrite.groovy | 54 ++
.../variant_p0/dotted_key_escape_rewrite.groovy | 167 ++++
.../dotted_key_escape_schema_change.groovy | 93 ++
.../variant_p0/dotted_key_escape_score.groovy | 46 +
.../variant_p0/dotted_key_escape_semantics.groovy | 165 ++++
.../dotted_key_escape_sparse_promotion.groovy | 266 ++++++
.../dotted_key_escape_stream_load.groovy | 57 ++
.../dotted_key_escape_table_property.groovy | 89 ++
.../dotted_key_escape_v1_execution.groovy | 129 +++
.../variant_p0/dotted_key_legacy_rewrite.groovy | 162 ++++
.../variant_p0/dotted_key_legacy_semantics.groovy | 112 +++
.../suites/variant_p0/predefine/load.groovy | 2 +-
.../predefine/predefined_typed_to_sparse.groovy | 2 +-
.../predefined_typed_to_sparse_1shard.groovy | 2 +-
.../test_all_prdefine_type_to_sparse.groovy | 2 +-
.../predefine/test_multi_index_nonCurrent.groovy | 6 +-
.../test_types_with_indexes_profile.groovy | 2 +-
.../test_variant_predefine_comment_escape.groovy | 6 +-
.../suites/variant_p0/rqg/fix_rqg1.groovy | 2 +-
regression-test/suites/variant_p0/rqg/load.groovy | 2 +-
regression-test/suites/variant_p0/rqg/rqg1.groovy | 2 +-
regression-test/suites/variant_p0/rqg/rqg2.groovy | 2 +-
regression-test/suites/variant_p0/rqg/rqg3.groovy | 2 +-
regression-test/suites/variant_p0/rqg/rqg4.groovy | 2 +-
regression-test/suites/variant_p0/rqg/rqg5.groovy | 2 +-
regression-test/suites/variant_p0/rqg/rqg6.groovy | 2 +-
regression-test/suites/variant_p0/rqg/rqg7.groovy | 2 +-
.../test_double_write_when_schema_change.groovy | 2 +-
.../variant_p0/sparse_ancestor_promotion.groovy | 22 +-
.../sparse_ancestor_promotion_extended.groovy | 30 +-
.../suites/variant_p0/test_types_in_variant.groovy | 2 +-
.../test_variant_search_subcolumn_snii.groovy | 16 +-
.../variant_p0/test_variant_sub_path_order.groovy | 12 +-
.../test_array_contains_with_inverted_index.groovy | 2 +-
.../with_index/test_variant_snii_compaction.groovy | 10 +-
.../variant_p0/with_index/var_index_snii.groovy | 16 +-
216 files changed, 12122 insertions(+), 567 deletions(-)
create mode 100644 be/test/agent/heartbeat_server_token_test.cpp
create mode 100644 be/test/storage/schema_test.cpp
create mode 100644 cloud/test/meta_service_large_bitmap_test.cpp
copy regression-test/suites/variant_p0/rqg/load.groovy =>
fe/fe-core/src/test/java/org/apache/doris/analysis/MVColumnItemTest.java (55%)
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/catalog/VariantEscapeTableTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/datasource/ExternalCatalogVariantPolicyTest.java
create mode 100644
regression-test/data/datatype_p0/datev1/test_datev1_minmax_runtime_filter.out
create mode 100644
regression-test/data/nereids_p0/test_string_literal_arithmetic.out
create mode 100644
regression-test/data/point_query_p0/test_point_query_light_schema_change.out
create mode 100644
regression-test/data/variant_p0/dotted_key_escape_added_column_rewrite.out
create mode 100644
regression-test/data/variant_p0/dotted_key_escape_async_mv.out
create mode 100644
regression-test/data/variant_p0/dotted_key_escape_common_type.out
create mode 100644
regression-test/data/variant_p0/dotted_key_escape_compaction_failure.out
create mode 100644
regression-test/data/variant_p0/dotted_key_escape_complex_rewrite.out
create mode 100644
regression-test/data/variant_p0/dotted_key_escape_condition_cache.out
create mode 100644
regression-test/data/variant_p0/dotted_key_escape_ctas_template.out
create mode 100644
regression-test/data/variant_p0/dotted_key_escape_derived_format.out
create mode 100644
regression-test/data/variant_p0/dotted_key_escape_doc_threshold.out
create mode 100644
regression-test/data/variant_p0/dotted_key_escape_document_examples.out
create mode 100644
regression-test/data/variant_p0/dotted_key_escape_index_profile.out
create mode 100644
regression-test/data/variant_p0/dotted_key_escape_index_rewrite.out
create mode 100644
regression-test/data/variant_p0/dotted_key_escape_ingest.json
create mode 100644
regression-test/data/variant_p0/dotted_key_escape_layout_switch.out
create mode 100644
regression-test/data/variant_p0/dotted_key_escape_mow_rewrite.out
create mode 100644
regression-test/data/variant_p0/dotted_key_escape_rewrite.out
create mode 100644 regression-test/data/variant_p0/dotted_key_escape_score.out
create mode 100644
regression-test/data/variant_p0/dotted_key_escape_semantics.out
create mode 100644
regression-test/data/variant_p0/dotted_key_escape_sparse_promotion.out
create mode 100644
regression-test/data/variant_p0/dotted_key_escape_stream_load.out
create mode 100644
regression-test/data/variant_p0/dotted_key_escape_table_property.out
create mode 100644
regression-test/data/variant_p0/dotted_key_legacy_rewrite.out
create mode 100644
regression-test/data/variant_p0/dotted_key_legacy_semantics.out
create mode 100644
regression-test/suites/datatype_p0/datev1/test_datev1_minmax_runtime_filter.groovy
create mode 100644
regression-test/suites/nereids_p0/test_string_literal_arithmetic.groovy
create mode 100644
regression-test/suites/point_query_p0/test_point_query_light_schema_change.groovy
create mode 100644
regression-test/suites/variant_p0/dotted_key_escape_added_column_rewrite.groovy
create mode 100644
regression-test/suites/variant_p0/dotted_key_escape_async_mv.groovy
create mode 100644
regression-test/suites/variant_p0/dotted_key_escape_common_type.groovy
create mode 100644
regression-test/suites/variant_p0/dotted_key_escape_compaction.groovy
create mode 100644
regression-test/suites/variant_p0/dotted_key_escape_compaction_failure.groovy
create mode 100644
regression-test/suites/variant_p0/dotted_key_escape_complex_rewrite.groovy
create mode 100644
regression-test/suites/variant_p0/dotted_key_escape_condition_cache.groovy
create mode 100644
regression-test/suites/variant_p0/dotted_key_escape_cross_encoding.groovy
create mode 100644
regression-test/suites/variant_p0/dotted_key_escape_ctas_template.groovy
create mode 100644
regression-test/suites/variant_p0/dotted_key_escape_derived_format.groovy
create mode 100644
regression-test/suites/variant_p0/dotted_key_escape_doc_threshold.groovy
create mode 100644
regression-test/suites/variant_p0/dotted_key_escape_document_examples.groovy
create mode 100644
regression-test/suites/variant_p0/dotted_key_escape_element_at_mv.groovy
create mode 100644
regression-test/suites/variant_p0/dotted_key_escape_index_profile.groovy
create mode 100644
regression-test/suites/variant_p0/dotted_key_escape_index_rewrite.groovy
create mode 100644
regression-test/suites/variant_p0/dotted_key_escape_layout_switch.groovy
create mode 100644
regression-test/suites/variant_p0/dotted_key_escape_mow_rewrite.groovy
create mode 100644
regression-test/suites/variant_p0/dotted_key_escape_rewrite.groovy
create mode 100644
regression-test/suites/variant_p0/dotted_key_escape_schema_change.groovy
create mode 100644
regression-test/suites/variant_p0/dotted_key_escape_score.groovy
create mode 100644
regression-test/suites/variant_p0/dotted_key_escape_semantics.groovy
create mode 100644
regression-test/suites/variant_p0/dotted_key_escape_sparse_promotion.groovy
create mode 100644
regression-test/suites/variant_p0/dotted_key_escape_stream_load.groovy
create mode 100644
regression-test/suites/variant_p0/dotted_key_escape_table_property.groovy
create mode 100644
regression-test/suites/variant_p0/dotted_key_escape_v1_execution.groovy
create mode 100644
regression-test/suites/variant_p0/dotted_key_legacy_rewrite.groovy
create mode 100644
regression-test/suites/variant_p0/dotted_key_legacy_semantics.groovy
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]