This is an automated email from the ASF dual-hosted git repository.
jacktengg pushed a change to branch auto-pick-67965-branch-4.2
in repository https://gitbox.apache.org/repos/asf/doris.git
omit 9a475512ef3 [fix](be) Correct rounding and bounds in floating-point
decimal casts (#67965)
add 0a24ed86fb8 branch-4.2: [chore](docker)Support rootless Docker for
third-party environments #70457 (#70580)
add a9dfbb7292a branch-4.2: [fix](be) Reject legacy Variant blocks before
deserialization (#70473)
add 64293ab085c branch-4.2: [feature](inverted-index) Add Japanese
(Kuromoji) morphological analyzer #64667 (#70555)
add d83fc096278 branch-4.2: [refactor](variant) Make Variant metadata key
registration allocation-safe #70487 (#70551)
add 55f56ceb008 branch-4.2: [fix](show command) some show command should
not be forward to master #69677 (#70326)
add fde749055bd branch-4.2: [improvement](geo) Decode constant shapes once
in st_contains and other spatial relation functions #70302 (#70629)
add 97c99035a75 branch-4.2: [fix](cast) Declare Variant casts to
non-Variant targets nullable (#70359)
add 1ed9383f3ee branch-4.2: [fix](be) Add fixed per-task ANN build thread
budgets #70454 (#70543)
add 51e384b919e branch-4.2: [fix](be) Fix data race on Segment::_footer_pb
#67720 (#70638)
add 71fc28ca349 branch-4.2: [improvement](variant) Assert Variant sparse
byte order in debug builds (#70590)
add e4b99b7f8eb [branch-4.2](point query) Fix short-circuit point query
returning the default for a column added by light schema change (#70608)
add 71cb7eb9b8c branch-4.2: [feature](variant) Add
variant_escape_dots_in_keys to keep dotted keys distinct (#70643)
add f8862153287 [fix](be) Restore file-level cache metadata loading during
startup (#69116) (#70677)
add 1380984ddb4 [fix](fe) Handle BACKUP timeout with immutable properties
(#69424) (#70682)
add 3a84339911e branch-4.2: [fix](cloud) Split oversized delete bitmaps
across metadata transactions #70478 (#70644)
add abea8643c94 branch-4.2: [fix](nereids) Honor cast mode for string
literal arithmetic #70518 (#70617)
add 7095636156e branch-4.2: [fix](nereids) Merge existing filters when
adding predicates (70409) (#70620)
add 95ea83f8de4 branch-4.2: [fix](be) Fix data race on Segment::_footer_pb
#67720 (#70642)
add 48b17e2b3eb [fix](be) Remove heartbeat tokens from logs and errors
(#69532) (#70692)
add a43275467ad branch-4.2: [fix](be) Preserve DATE type in min/max
runtime filters (#70333)
add d0ec1f6b0ff branch-4.2: [fix](be) Isolate runtime filter wrappers
shared with local consumers (#70623)
add 10bf65d9442 branch-4.2: [fix](be) Eliminate undefined
float-to-wide-integer conversion (#67463) (#70535)
add 71d452d628c branch-4.2: [fix](be) Preserve shared columns in distinct
streaming pass-through (#68880) (#70533)
add 8dc51ede59e branch-4.2: [fix](be) Guard RuntimeState access with task
context lifetime #70691 (#70702)
add 858c1424f1d branch-4.2: [fix](cloud) Retire unused cloud management
RPCs #70494 (#70640)
add 07fc08ab9ec [improvement](file-cache) adapt external cache reads to
I/O latency (branch-4.2) (#69768)
add 9f52e0e7f28 [improvement](fe) Reuse Hudi table schema across native
scan ranges (#70557)
add 64c4dae48dd branch-4.2: [fix](be) Wait before cancelling paused
queries under process memory pressure (#70432)
add 6774f7a556d branch-4.2: [fix](cloud) Preserve VARIANT escape policy
presence in protobuf conversions (#70755)
add 620dc96b37f [fix](be) Correct rounding and bounds in floating-point
decimal casts (#67965)
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (9a475512ef3)
\
N -- N -- N refs/heads/auto-pick-67965-branch-4.2 (620dc96b37f)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.clang-format-ignore | 1 +
.gitignore | 3 +
.licenserc.yaml | 1 +
LICENSE.txt | 8 +
NOTICE.txt | 3 +
be/CMakeLists.txt | 69 +
be/dict/kuromoji/README.md | 60 +
be/src/agent/heartbeat_server.cpp | 10 +-
be/src/cloud/cloud_internal_service.cpp | 4 +
be/src/cloud/pb_convert.cpp | 16 +
be/src/common/config.cpp | 16 +
be/src/common/config.h | 9 +
be/src/core/column/column_variant.cpp | 8 +
be/src/core/column/variant_column_utils.cpp | 9 +
be/src/core/column/variant_column_utils.h | 5 +
be/src/core/data_type/data_type_factory.cpp | 3 +-
be/src/core/data_type/data_type_variant.cpp | 15 +-
be/src/core/data_type/data_type_variant.h | 4 +-
be/src/core/types.h | 1 -
.../core/value/variant/variant_batch_builder.cpp | 118 +-
be/src/core/wide_integer_impl.h | 170 +-
be/src/exec/common/variant_util.cpp | 101 +-
be/src/exec/common/variant_util.h | 17 +-
.../distinct_streaming_aggregation_operator.cpp | 15 +-
be/src/exec/operator/scan_operator.cpp | 9 +
be/src/exec/pipeline/pipeline_task.cpp | 9 +-
be/src/exec/rowid_fetcher.cpp | 14 +
be/src/exec/runtime_filter/runtime_filter_merger.h | 36 +-
.../runtime_filter/runtime_filter_producer.cpp | 8 +-
.../exec/runtime_filter/runtime_filter_wrapper.cpp | 47 +-
.../exec/runtime_filter/runtime_filter_wrapper.h | 6 +-
be/src/exec/scan/file_scan_io_context.h | 4 +
be/src/exec/scan/olap_scanner.cpp | 2 +
be/src/exec/sink/writer/async_result_writer.cpp | 6 +-
be/src/exprs/bloom_filter_func.h | 34 +
be/src/exprs/filter_base.h | 1 +
be/src/exprs/function/function_tokenize.cpp | 2 +-
be/src/exprs/function/geo/functions_geo.cpp | 61 +-
be/src/exprs/function/geo/functions_geo.h | 31 +-
be/src/exprs/function/geo/geo_types.cpp | 55 +-
be/src/exprs/function/geo/geo_types.h | 6 +
be/src/exprs/minmax_predicate.h | 11 +
be/src/io/cache/block_file_cache.cpp | 41 +-
be/src/io/cache/block_file_cache.h | 8 +
be/src/io/cache/block_file_cache_profile.cpp | 48 +
be/src/io/cache/block_file_cache_profile.h | 12 +
be/src/io/cache/cache_block_meta_store.cpp | 54 +-
be/src/io/cache/cache_block_meta_store.h | 4 +
be/src/io/cache/cached_remote_file_reader.cpp | 350 +++-
be/src/io/cache/cached_remote_file_reader.h | 27 +-
be/src/io/cache/file_block.cpp | 1 +
be/src/io/cache/file_cache_common.h | 12 +
be/src/io/cache/file_cache_io_adaptor.cpp | 140 ++
be/src/io/cache/file_cache_io_adaptor.h | 82 +
be/src/io/cache/file_cache_storage.h | 7 +-
be/src/io/cache/fs_file_cache_storage.cpp | 41 +-
be/src/io/cache/fs_file_cache_storage.h | 5 +-
be/src/io/cache/mem_file_cache_storage.cpp | 6 +-
be/src/io/cache/mem_file_cache_storage.h | 5 +-
be/src/io/io_common.h | 26 +
.../workload_group/workload_group_manager.cpp | 433 +++--
.../workload_group/workload_group_manager.h | 31 +-
.../runtime/workload_management/task_controller.h | 15 +
be/src/service/internal_service.cpp | 4 +-
be/src/service/point_query_executor.cpp | 48 +-
be/src/storage/cache/page_cache.h | 5 +-
be/src/storage/index/ann/faiss_ann_index.cpp | 42 +-
.../index/inverted/analysis/analyzer/analyzer.cpp | 27 +-
.../analysis/analyzer/kuromoji/KuromojiAnalyzer.h | 75 +
.../analysis/analyzer/kuromoji/KuromojiMode.h | 48 +
.../analyzer/kuromoji/KuromojiTokenizer.cpp | 204 ++
.../analysis/analyzer/kuromoji/KuromojiTokenizer.h | 53 +
.../analysis/analyzer/kuromoji/dict/darts.h | 1979 ++++++++++++++++++++
.../analyzer/kuromoji/dict/kuromoji_dict_format.h | 129 ++
.../analyzer/kuromoji/dict/kuromoji_dictionary.cpp | 326 ++++
.../analyzer/kuromoji/dict/kuromoji_dictionary.h | 142 ++
.../kuromoji/dict/kuromoji_dictionary_builder.cpp | 263 +++
.../kuromoji/dict/kuromoji_dictionary_builder.h | 75 +
.../kuromoji/dict/kuromoji_ipadic_parser.cpp | 324 ++++
.../kuromoji/dict/kuromoji_ipadic_parser.h | 50 +
.../analyzer/kuromoji/kuromoji_normalize.h | 68 +
.../analyzer/kuromoji/kuromoji_viterbi.cpp | 322 ++++
.../analysis/analyzer/kuromoji/kuromoji_viterbi.h | 64 +
.../inverted/analysis/inverted_index_parser.cpp | 17 +-
.../inverted/analysis/inverted_index_parser.h | 7 +-
.../index/inverted/reader/predicate_collector.cpp | 3 +-
be/src/storage/schema.cpp | 107 ++
be/src/storage/schema.h | 6 +
be/src/storage/segment/segment.cpp | 31 +-
be/src/storage/segment/segment.h | 6 +-
be/src/storage/segment/segment_iterator.cpp | 2 +-
.../variant/binary_column_extract_iterator.cpp | 2 +
.../segment/variant/hierarchical_data_iterator.cpp | 4 +-
.../segment/variant/hierarchical_data_iterator.h | 2 +-
.../variant/sparse_column_merge_iterator.cpp | 10 +
.../segment/variant/v2/variant_assembler.cpp | 33 +-
.../storage/segment/variant/v2/variant_assembler.h | 5 +-
.../segment/variant/v2/variant_column_writer.cpp | 3 +
.../segment/variant/v2/variant_path_builder.cpp | 20 +-
.../segment/variant/v2/variant_shredder.cpp | 11 +-
.../storage/segment/variant/v2/variant_shredder.h | 1 +
.../segment/variant/variant_column_reader.cpp | 187 +-
.../segment/variant/variant_column_reader.h | 12 +-
.../segment/variant/variant_column_writer_impl.cpp | 36 +-
.../variant_doc_snpashot_compact_iterator.h | 38 +-
.../segment/variant/variant_writer_helpers.cpp | 24 +-
be/src/storage/tablet/tablet_meta.cpp | 1 +
be/src/storage/tablet/tablet_schema.cpp | 53 +-
be/src/storage/tablet/tablet_schema.h | 6 +-
be/src/tools/kuromoji_build_dict.cpp | 203 ++
be/src/util/json/path_in_data.cpp | 37 +
be/src/util/json/path_in_data.h | 6 +-
be/src/util/mem_info.h | 7 +
be/test/CMakeLists.txt | 6 +-
be/test/agent/heartbeat_server_token_test.cpp | 90 +
.../core/block/block_variant_wire_format_test.cpp | 100 +
be/test/core/column/column_variant_test.cpp | 23 +
be/test/core/jsonb/path_in_data_test.cpp | 16 +
be/test/core/wide_integer_test.cpp | 40 +
be/test/exec/common/schema_util_test.cpp | 44 +-
...istinct_streaming_aggregation_operator_test.cpp | 68 +
be/test/exec/pipeline/pipeline_task_test.cpp | 22 +
.../runtime_filter/runtime_filter_merger_test.cpp | 242 +++
.../runtime_filter_producer_test.cpp | 519 ++++-
.../runtime_filter/runtime_filter_wrapper_test.cpp | 471 +++++
be/test/exec/scan/file_scanner_v2_test.cpp | 26 +
be/test/exprs/function/function_geo_test.cpp | 146 ++
be/test/exprs/function/geo/geo_types_test.cpp | 41 +
.../block_file_cache_profile_reporter_test.cpp | 48 +
.../io/cache/block_file_cache_test_meta_store.cpp | 181 +-
be/test/io/cache/cache_block_meta_store_test.cpp | 37 +-
.../cache/cached_remote_file_reader_peer_test.cpp | 84 +-
.../io/cache/cached_remote_file_reader_test.cpp | 871 ++++++++-
be/test/io/cache/file_cache_io_adaptor_test.cpp | 435 +++++
.../workload_group/workload_group_manager_test.cpp | 844 ++++++++-
.../storage/index/ann/faiss_vector_index_test.cpp | 198 ++
.../analyzer/kuromoji/darts_smoke_test.cpp | 70 +
.../kuromoji/kuromoji_dict_format_test.cpp | 48 +
.../kuromoji/kuromoji_dictionary_builder_test.cpp | 79 +
.../analyzer/kuromoji/kuromoji_dictionary_test.cpp | 163 ++
.../kuromoji/kuromoji_ipadic_parser_test.cpp | 109 ++
.../analyzer/kuromoji/kuromoji_normalize_test.cpp | 65 +
.../analyzer/kuromoji/kuromoji_real_dict_test.cpp | 178 ++
.../analyzer/kuromoji/kuromoji_viterbi_test.cpp | 272 +++
.../analysis/analyzer/kuromoji_analyzer_test.cpp | 66 +
.../analysis/analyzer/kuromoji_tokenizer_test.cpp} | 31 +-
.../analysis/kuromoji_parser_type_test.cpp} | 34 +-
.../inverted/reader/collection_statistics_test.cpp | 63 +
.../inverted/snii/snii_variant_routing_test.cpp | 5 +-
be/test/storage/pb_convert_test.cpp | 38 +
be/test/storage/schema_test.cpp | 161 ++
.../binary_column_extract_iterator_test.cpp | 85 +
.../storage/segment/segment_footer_cache_test.cpp | 283 ++-
.../segment/variant_assembler_legacy_test.cpp | 24 +
be/test/storage/tablet/tablet_schema_test.cpp | 120 ++
.../variant/variant_column_writer_reader_test.cpp | 888 ++++++++-
be/test/storage/variant/variant_util_test.cpp | 31 +
.../test/testutil/gperftools_stubs.cpp | 26 +-
be/test/testutil/mock/mock_query_task_controller.h | 55 +
.../util/variant/variant_batch_builder_test.cpp | 258 +++
build.sh | 15 +
cloud/src/common/bvars.cpp | 7 -
cloud/src/common/bvars.h | 5 -
cloud/src/common/http_helper.cpp | 14 +-
cloud/src/common/http_helper.h | 2 -
cloud/src/meta-service/meta_service.cpp | 411 ++--
cloud/src/meta-service/meta_service.h | 8 -
cloud/src/meta-service/meta_service_resource.cpp | 90 -
cloud/src/recycler/recycler_service.cpp | 21 +-
cloud/src/recycler/recycler_service.h | 10 +-
cloud/test/CMakeLists.txt | 1 +
cloud/test/meta_service_http_test.cpp | 30 +-
cloud/test/meta_service_large_bitmap_test.cpp | 544 ++++++
cloud/test/meta_service_test.cpp | 68 +-
cloud/test/rpc_kv_bvar_test.cpp | 43 -
cloud/test/schema_kv_test.cpp | 47 +
dist/LICENSE-dist.txt | 2 +
dist/licenses/LICENSE-darts-clone.txt | 11 +
dist/licenses/LICENSE-ipadic.txt | 70 +
docker/thirdparties/README.md | 71 +
docker/thirdparties/docker-command.sh | 92 +
.../thirdparties/docker-compose/db2/db2.yaml.tpl | 5 +-
.../scripts/init-db2.sh} | 23 +-
.../docker-compose/elasticsearch/es.yaml.tpl | 2 +
.../elasticsearch/scripts/es_init.sh | 3 +
docker/thirdparties/docker-compose/hive/README.md | 4 +-
.../thirdparties/docker-compose/hive/README_ZH.md | 4 +-
.../hive/scripts/snapshot-hive-baseline.sh | 31 +-
.../thirdparties/docker-compose/hudi/hudi.env.tpl | 2 +-
.../thirdparties/docker-compose/hudi/hudi.yaml.tpl | 24 +-
.../iceberg-rest/docker-compose.yaml.tpl | 10 +-
.../iceberg-rest/hdfs-rootless.yaml.tpl | 44 +
.../docker-compose/kafka/kafka.yaml.tpl | 5 +-
.../docker-compose/kerberos/kerberos.yaml.tpl | 6 +
.../minio/minio-RELEASE.2024-11-07.yaml.tpl | 1 -
.../docker-compose/mysql/mysql-5.7.yaml.tpl | 5 +-
.../docker-compose/oracle/oracle-11.yaml.tpl | 9 +-
.../docker-compose/polaris/docker-compose.yaml.tpl | 5 +
.../docker-compose/polaris/init-catalog.sh | 2 +
.../docker-compose/polaris/polaris_settings.env | 8 +-
.../postgresql/postgresql-14.yaml.tpl | 5 +-
.../docker-compose/ranger/ranger_settings.env | 6 +-
.../docker-compose/sqlserver/sqlserver.yaml.tpl | 20 +-
docker/thirdparties/docker-health.sh | 4 +-
docker/thirdparties/run-thirdparties-docker.sh | 219 ++-
docker/thirdparties/test/docker-command-test.sh | 138 ++
docker/thirdparties/test/launcher-stop-test.sh | 79 +
.../test/snapshot-hive-baseline-test.sh | 99 +
.../main/java/org/apache/doris/common/Config.java | 7 +
.../org/apache/doris/alter/CloudRollupJobV2.java | 2 +-
.../apache/doris/alter/CloudSchemaChangeJobV2.java | 3 +-
.../doris/alter/MaterializedViewHandler.java | 11 +
.../java/org/apache/doris/alter/RollupJobV2.java | 3 +-
.../apache/doris/alter/SchemaChangeHandler.java | 9 +-
.../org/apache/doris/alter/SchemaChangeJobV2.java | 2 +-
.../apache/doris/analysis/InvertedIndexUtil.java | 17 +-
.../java/org/apache/doris/backup/RestoreJob.java | 20 +-
.../main/java/org/apache/doris/catalog/Env.java | 40 +-
.../java/org/apache/doris/catalog/OlapTable.java | 35 +-
.../org/apache/doris/catalog/TableProperty.java | 15 +
.../apache/doris/cloud/backup/CloudRestoreJob.java | 3 +-
.../cloud/datasource/CloudInternalCatalog.java | 5 +-
.../apache/doris/common/util/PropertyAnalyzer.java | 175 ++
.../apache/doris/datasource/ExternalCatalog.java | 2 +
.../apache/doris/datasource/InternalCatalog.java | 26 +-
.../doris/datasource/hudi/source/HudiScanNode.java | 21 +-
.../org/apache/doris/indexpolicy/IndexPolicy.java | 2 +-
.../org/apache/doris/master/ReportHandler.java | 2 +-
.../java/org/apache/doris/mtmv/MTMVPlanUtil.java | 17 +-
.../glue/translator/ExpressionTranslator.java | 2 +-
.../glue/translator/PlanTranslatorContext.java | 10 +-
.../doris/nereids/parser/LogicalPlanBuilder.java | 4 +
.../rules/rewrite/RewriteSearchToSlots.java | 11 +-
.../doris/nereids/trees/expressions/Cast.java | 12 +-
.../trees/plans/commands/BackupCommand.java | 11 +-
.../trees/plans/commands/CreateTableCommand.java | 14 +
.../trees/plans/commands/ShowProcCommand.java | 10 +
.../plans/commands/ShowWorkloadGroupsCommand.java | 7 +-
.../plans/commands/info/AlterMTMVPropertyInfo.java | 2 +
.../trees/plans/commands/info/CreateTableInfo.java | 11 +
.../commands/info/ModifyTablePropertiesOp.java | 1 +
.../apache/doris/nereids/types/VariantType.java | 7 +
.../org/apache/doris/nereids/util/PlanUtils.java | 24 +
.../doris/nereids/util/TypeCoercionUtils.java | 15 +-
.../org/apache/doris/planner/OlapTableSink.java | 2 +
.../java/org/apache/doris/qe/SessionVariable.java | 7 +
.../org/apache/doris/task/CreateReplicaTask.java | 5 +-
.../InvertedIndexKuromojiValidationTest.java | 50 +
.../analysis/InvertedIndexPropertiesTest.java | 35 +
.../apache/doris/analysis/MVColumnItemTest.java | 22 +-
.../org/apache/doris/backup/RestoreJobTest.java | 66 +
.../org/apache/doris/catalog/OlapTableTest.java | 33 +
.../doris/catalog/VariantEscapeTableTest.java | 433 +++++
.../cloud/datasource/CloudInternalCatalogTest.java | 26 +-
.../apache/doris/common/PropertyAnalyzerTest.java | 172 ++
.../ExternalCatalogVariantPolicyTest.java | 43 +
.../datasource/hudi/source/HudiScanNodeTest.java | 89 +
.../org/apache/doris/mtmv/MTMVPlanUtilTest.java | 37 +
.../glue/translator/ExpressionTranslatorTest.java | 80 +
.../nereids/rules/rewrite/InferAggNotNullTest.java | 92 +
.../rules/rewrite/RewriteSearchToSlotsTest.java | 23 +
.../doris/nereids/trees/expressions/CastTest.java | 12 +-
.../trees/plans/commands/BackupCommandTest.java | 15 +
.../plans/commands/CreateTableCommandTest.java | 81 +
.../apache/doris/nereids/util/PlanUtilsTest.java | 71 +
.../doris/nereids/util/TypeCoercionUtilsTest.java | 43 +
.../qe/SessionVariableFileCacheIoAdaptorTest.java | 85 +
.../java/org/apache/doris/task/AgentTaskTest.java | 21 +-
gensrc/proto/cloud.proto | 13 -
gensrc/proto/olap_file.proto | 4 +
gensrc/proto/segment_v2.proto | 2 +
gensrc/thrift/AgentService.thrift | 1 +
gensrc/thrift/PaloInternalService.thrift | 3 +
.../datev1/test_datev1_minmax_runtime_filter.out | 8 +
.../nereids_p0/test_string_literal_arithmetic.out | 37 +
.../test_point_query_light_schema_change.out | 28 +
.../distinct_streaming_repeated_keys.out | 13 +
.../dotted_key_escape_added_column_rewrite.out | 31 +
.../data/variant_p0/dotted_key_escape_async_mv.out | 32 +
.../variant_p0/dotted_key_escape_common_type.out | 38 +
.../dotted_key_escape_compaction_failure.out | 9 +
.../dotted_key_escape_complex_rewrite.out | 22 +
.../dotted_key_escape_condition_cache.out | 13 +
.../variant_p0/dotted_key_escape_ctas_template.out | 19 +
.../dotted_key_escape_derived_format.out | 29 +
.../variant_p0/dotted_key_escape_doc_threshold.out | 46 +
.../dotted_key_escape_document_examples.out | 49 +
.../variant_p0/dotted_key_escape_index_profile.out | 19 +
.../variant_p0/dotted_key_escape_index_rewrite.out | 59 +
.../data/variant_p0/dotted_key_escape_ingest.json | 2 +
.../variant_p0/dotted_key_escape_layout_switch.out | 155 ++
.../variant_p0/dotted_key_escape_mow_rewrite.out | 9 +
.../data/variant_p0/dotted_key_escape_rewrite.out | 287 +++
.../data/variant_p0/dotted_key_escape_score.out | 13 +
.../variant_p0/dotted_key_escape_semantics.out | 99 +
.../dotted_key_escape_sparse_promotion.out | 974 ++++++++++
.../variant_p0/dotted_key_escape_stream_load.out | 10 +
.../dotted_key_escape_table_property.out | 16 +
.../data/variant_p0/dotted_key_legacy_rewrite.out | 92 +
.../variant_p0/dotted_key_legacy_semantics.out | 48 +
.../predefine/test_all_prdefine_type_to_sparse.out | 1058 +++++------
.../sparse_ancestor_promotion_extended.out | 24 +-
.../test_variant_cast_isnull_pushdown.out | 17 +
.../test_datev1_minmax_runtime_filter.groovy | 82 +
.../analyzer/test_japanese_analyzer.groovy | 104 +
.../inverted_index_p0/test_properties.groovy | 2 +-
.../test_string_literal_arithmetic.groovy | 68 +
.../suites/nereids_syntax_p0/cast.groovy | 12 +-
.../test_point_query_light_schema_change.groovy | 99 +
.../distinct_streaming_repeated_keys.groovy | 146 ++
.../adjust_virtual_slot_nullable.groovy | 4 +-
.../suites/variant_p0/column_name.groovy | 2 +-
.../dotted_key_escape_added_column_rewrite.groovy | 58 +
.../variant_p0/dotted_key_escape_async_mv.groovy | 107 ++
.../dotted_key_escape_common_type.groovy | 83 +
.../variant_p0/dotted_key_escape_compaction.groovy | 141 ++
.../dotted_key_escape_compaction_failure.groovy | 99 +
.../dotted_key_escape_complex_rewrite.groovy | 50 +
.../dotted_key_escape_condition_cache.groovy | 66 +
.../dotted_key_escape_cross_encoding.groovy | 84 +
.../dotted_key_escape_ctas_template.groovy | 64 +
.../dotted_key_escape_derived_format.groovy | 79 +
.../dotted_key_escape_doc_threshold.groovy | 50 +
.../dotted_key_escape_document_examples.groovy | 75 +
.../dotted_key_escape_element_at_mv.groovy | 92 +
.../dotted_key_escape_index_profile.groovy | 59 +
.../dotted_key_escape_index_rewrite.groovy | 58 +
.../dotted_key_escape_layout_switch.groovy | 87 +
.../dotted_key_escape_mow_rewrite.groovy | 54 +
.../variant_p0/dotted_key_escape_rewrite.groovy | 167 ++
.../dotted_key_escape_schema_change.groovy | 93 +
.../variant_p0/dotted_key_escape_score.groovy | 46 +
.../variant_p0/dotted_key_escape_semantics.groovy | 165 ++
.../dotted_key_escape_sparse_promotion.groovy | 266 +++
.../dotted_key_escape_stream_load.groovy | 57 +
.../dotted_key_escape_table_property.groovy | 89 +
.../dotted_key_escape_v1_execution.groovy | 129 ++
.../variant_p0/dotted_key_legacy_rewrite.groovy | 162 ++
.../variant_p0/dotted_key_legacy_semantics.groovy | 112 ++
.../suites/variant_p0/predefine/load.groovy | 2 +-
.../predefine/predefined_typed_to_sparse.groovy | 2 +-
.../predefined_typed_to_sparse_1shard.groovy | 2 +-
.../test_all_prdefine_type_to_sparse.groovy | 2 +-
.../predefine/test_multi_index_nonCurrent.groovy | 6 +-
.../test_types_with_indexes_profile.groovy | 2 +-
...ant_field_pattern_invalid_inverted_index.groovy | 2 +-
.../test_variant_predefine_comment_escape.groovy | 6 +-
.../suites/variant_p0/rqg/fix_rqg1.groovy | 2 +-
regression-test/suites/variant_p0/rqg/load.groovy | 2 +-
regression-test/suites/variant_p0/rqg/rqg1.groovy | 2 +-
regression-test/suites/variant_p0/rqg/rqg2.groovy | 2 +-
regression-test/suites/variant_p0/rqg/rqg3.groovy | 2 +-
regression-test/suites/variant_p0/rqg/rqg4.groovy | 2 +-
regression-test/suites/variant_p0/rqg/rqg5.groovy | 2 +-
regression-test/suites/variant_p0/rqg/rqg6.groovy | 2 +-
regression-test/suites/variant_p0/rqg/rqg7.groovy | 2 +-
.../test_double_write_when_schema_change.groovy | 2 +-
.../variant_p0/sparse_ancestor_promotion.groovy | 22 +-
.../sparse_ancestor_promotion_extended.groovy | 30 +-
.../suites/variant_p0/test_types_in_variant.groovy | 2 +-
.../test_variant_cast_isnull_pushdown.groovy | 42 +
.../test_variant_search_subcolumn_snii.groovy | 16 +-
.../variant_p0/test_variant_sub_path_order.groovy | 12 +-
.../test_array_contains_with_inverted_index.groovy | 2 +-
.../with_index/test_variant_snii_compaction.groovy | 10 +-
.../variant_p0/with_index/var_index_snii.groovy | 16 +-
thirdparty/build-thirdparty.sh | 14 +
thirdparty/vars.sh | 7 +
368 files changed, 24249 insertions(+), 1972 deletions(-)
create mode 100644 be/dict/kuromoji/README.md
create mode 100644 be/src/io/cache/file_cache_io_adaptor.cpp
create mode 100644 be/src/io/cache/file_cache_io_adaptor.h
create mode 100644
be/src/storage/index/inverted/analysis/analyzer/kuromoji/KuromojiAnalyzer.h
create mode 100644
be/src/storage/index/inverted/analysis/analyzer/kuromoji/KuromojiMode.h
create mode 100644
be/src/storage/index/inverted/analysis/analyzer/kuromoji/KuromojiTokenizer.cpp
create mode 100644
be/src/storage/index/inverted/analysis/analyzer/kuromoji/KuromojiTokenizer.h
create mode 100644
be/src/storage/index/inverted/analysis/analyzer/kuromoji/dict/darts.h
create mode 100644
be/src/storage/index/inverted/analysis/analyzer/kuromoji/dict/kuromoji_dict_format.h
create mode 100644
be/src/storage/index/inverted/analysis/analyzer/kuromoji/dict/kuromoji_dictionary.cpp
create mode 100644
be/src/storage/index/inverted/analysis/analyzer/kuromoji/dict/kuromoji_dictionary.h
create mode 100644
be/src/storage/index/inverted/analysis/analyzer/kuromoji/dict/kuromoji_dictionary_builder.cpp
create mode 100644
be/src/storage/index/inverted/analysis/analyzer/kuromoji/dict/kuromoji_dictionary_builder.h
create mode 100644
be/src/storage/index/inverted/analysis/analyzer/kuromoji/dict/kuromoji_ipadic_parser.cpp
create mode 100644
be/src/storage/index/inverted/analysis/analyzer/kuromoji/dict/kuromoji_ipadic_parser.h
create mode 100644
be/src/storage/index/inverted/analysis/analyzer/kuromoji/kuromoji_normalize.h
create mode 100644
be/src/storage/index/inverted/analysis/analyzer/kuromoji/kuromoji_viterbi.cpp
create mode 100644
be/src/storage/index/inverted/analysis/analyzer/kuromoji/kuromoji_viterbi.h
create mode 100644 be/src/tools/kuromoji_build_dict.cpp
create mode 100644 be/test/agent/heartbeat_server_token_test.cpp
create mode 100644 be/test/core/block/block_variant_wire_format_test.cpp
create mode 100644 be/test/io/cache/file_cache_io_adaptor_test.cpp
create mode 100644
be/test/storage/index/inverted/analysis/analyzer/kuromoji/darts_smoke_test.cpp
create mode 100644
be/test/storage/index/inverted/analysis/analyzer/kuromoji/kuromoji_dict_format_test.cpp
create mode 100644
be/test/storage/index/inverted/analysis/analyzer/kuromoji/kuromoji_dictionary_builder_test.cpp
create mode 100644
be/test/storage/index/inverted/analysis/analyzer/kuromoji/kuromoji_dictionary_test.cpp
create mode 100644
be/test/storage/index/inverted/analysis/analyzer/kuromoji/kuromoji_ipadic_parser_test.cpp
create mode 100644
be/test/storage/index/inverted/analysis/analyzer/kuromoji/kuromoji_normalize_test.cpp
create mode 100644
be/test/storage/index/inverted/analysis/analyzer/kuromoji/kuromoji_real_dict_test.cpp
create mode 100644
be/test/storage/index/inverted/analysis/analyzer/kuromoji/kuromoji_viterbi_test.cpp
create mode 100644
be/test/storage/index/inverted/analysis/analyzer/kuromoji_analyzer_test.cpp
copy be/{src/exprs/filter_base.h =>
test/storage/index/inverted/analysis/analyzer/kuromoji_tokenizer_test.cpp} (56%)
copy be/{src/exec/scan/file_scan_io_context.h =>
test/storage/index/inverted/analysis/kuromoji_parser_type_test.cpp} (51%)
create mode 100644 be/test/storage/schema_test.cpp
copy regression-test/suites/variant_p0/rqg/load.groovy =>
be/test/testutil/gperftools_stubs.cpp (66%)
create mode 100644 cloud/test/meta_service_large_bitmap_test.cpp
create mode 100644 dist/licenses/LICENSE-darts-clone.txt
create mode 100644 dist/licenses/LICENSE-ipadic.txt
create mode 100644 docker/thirdparties/README.md
create mode 100755 docker/thirdparties/docker-command.sh
copy docker/thirdparties/docker-compose/{ranger/ranger_settings.env =>
db2/scripts/init-db2.sh} (52%)
mode change 100644 => 100755
create mode 100644
docker/thirdparties/docker-compose/iceberg-rest/hdfs-rootless.yaml.tpl
create mode 100755 docker/thirdparties/test/docker-command-test.sh
create mode 100644 docker/thirdparties/test/launcher-stop-test.sh
create mode 100644 docker/thirdparties/test/snapshot-hive-baseline-test.sh
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/analysis/InvertedIndexKuromojiValidationTest.java
copy regression-test/suites/variant_p0/rqg/load.groovy =>
fe/fe-core/src/test/java/org/apache/doris/analysis/MVColumnItemTest.java (55%)
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/catalog/VariantEscapeTableTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/datasource/ExternalCatalogVariantPolicyTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/qe/SessionVariableFileCacheIoAdaptorTest.java
create mode 100644
regression-test/data/datatype_p0/datev1/test_datev1_minmax_runtime_filter.out
create mode 100644
regression-test/data/nereids_p0/test_string_literal_arithmetic.out
create mode 100644
regression-test/data/point_query_p0/test_point_query_light_schema_change.out
create mode 100644
regression-test/data/query_p0/set_operations/distinct_streaming_repeated_keys.out
create mode 100644
regression-test/data/variant_p0/dotted_key_escape_added_column_rewrite.out
create mode 100644
regression-test/data/variant_p0/dotted_key_escape_async_mv.out
create mode 100644
regression-test/data/variant_p0/dotted_key_escape_common_type.out
create mode 100644
regression-test/data/variant_p0/dotted_key_escape_compaction_failure.out
create mode 100644
regression-test/data/variant_p0/dotted_key_escape_complex_rewrite.out
create mode 100644
regression-test/data/variant_p0/dotted_key_escape_condition_cache.out
create mode 100644
regression-test/data/variant_p0/dotted_key_escape_ctas_template.out
create mode 100644
regression-test/data/variant_p0/dotted_key_escape_derived_format.out
create mode 100644
regression-test/data/variant_p0/dotted_key_escape_doc_threshold.out
create mode 100644
regression-test/data/variant_p0/dotted_key_escape_document_examples.out
create mode 100644
regression-test/data/variant_p0/dotted_key_escape_index_profile.out
create mode 100644
regression-test/data/variant_p0/dotted_key_escape_index_rewrite.out
create mode 100644
regression-test/data/variant_p0/dotted_key_escape_ingest.json
create mode 100644
regression-test/data/variant_p0/dotted_key_escape_layout_switch.out
create mode 100644
regression-test/data/variant_p0/dotted_key_escape_mow_rewrite.out
create mode 100644
regression-test/data/variant_p0/dotted_key_escape_rewrite.out
create mode 100644 regression-test/data/variant_p0/dotted_key_escape_score.out
create mode 100644
regression-test/data/variant_p0/dotted_key_escape_semantics.out
create mode 100644
regression-test/data/variant_p0/dotted_key_escape_sparse_promotion.out
create mode 100644
regression-test/data/variant_p0/dotted_key_escape_stream_load.out
create mode 100644
regression-test/data/variant_p0/dotted_key_escape_table_property.out
create mode 100644
regression-test/data/variant_p0/dotted_key_legacy_rewrite.out
create mode 100644
regression-test/data/variant_p0/dotted_key_legacy_semantics.out
create mode 100644
regression-test/data/variant_p0/test_variant_cast_isnull_pushdown.out
create mode 100644
regression-test/suites/datatype_p0/datev1/test_datev1_minmax_runtime_filter.groovy
create mode 100644
regression-test/suites/inverted_index_p0/analyzer/test_japanese_analyzer.groovy
create mode 100644
regression-test/suites/nereids_p0/test_string_literal_arithmetic.groovy
create mode 100644
regression-test/suites/point_query_p0/test_point_query_light_schema_change.groovy
create mode 100644
regression-test/suites/query_p0/set_operations/distinct_streaming_repeated_keys.groovy
create mode 100644
regression-test/suites/variant_p0/dotted_key_escape_added_column_rewrite.groovy
create mode 100644
regression-test/suites/variant_p0/dotted_key_escape_async_mv.groovy
create mode 100644
regression-test/suites/variant_p0/dotted_key_escape_common_type.groovy
create mode 100644
regression-test/suites/variant_p0/dotted_key_escape_compaction.groovy
create mode 100644
regression-test/suites/variant_p0/dotted_key_escape_compaction_failure.groovy
create mode 100644
regression-test/suites/variant_p0/dotted_key_escape_complex_rewrite.groovy
create mode 100644
regression-test/suites/variant_p0/dotted_key_escape_condition_cache.groovy
create mode 100644
regression-test/suites/variant_p0/dotted_key_escape_cross_encoding.groovy
create mode 100644
regression-test/suites/variant_p0/dotted_key_escape_ctas_template.groovy
create mode 100644
regression-test/suites/variant_p0/dotted_key_escape_derived_format.groovy
create mode 100644
regression-test/suites/variant_p0/dotted_key_escape_doc_threshold.groovy
create mode 100644
regression-test/suites/variant_p0/dotted_key_escape_document_examples.groovy
create mode 100644
regression-test/suites/variant_p0/dotted_key_escape_element_at_mv.groovy
create mode 100644
regression-test/suites/variant_p0/dotted_key_escape_index_profile.groovy
create mode 100644
regression-test/suites/variant_p0/dotted_key_escape_index_rewrite.groovy
create mode 100644
regression-test/suites/variant_p0/dotted_key_escape_layout_switch.groovy
create mode 100644
regression-test/suites/variant_p0/dotted_key_escape_mow_rewrite.groovy
create mode 100644
regression-test/suites/variant_p0/dotted_key_escape_rewrite.groovy
create mode 100644
regression-test/suites/variant_p0/dotted_key_escape_schema_change.groovy
create mode 100644
regression-test/suites/variant_p0/dotted_key_escape_score.groovy
create mode 100644
regression-test/suites/variant_p0/dotted_key_escape_semantics.groovy
create mode 100644
regression-test/suites/variant_p0/dotted_key_escape_sparse_promotion.groovy
create mode 100644
regression-test/suites/variant_p0/dotted_key_escape_stream_load.groovy
create mode 100644
regression-test/suites/variant_p0/dotted_key_escape_table_property.groovy
create mode 100644
regression-test/suites/variant_p0/dotted_key_escape_v1_execution.groovy
create mode 100644
regression-test/suites/variant_p0/dotted_key_legacy_rewrite.groovy
create mode 100644
regression-test/suites/variant_p0/dotted_key_legacy_semantics.groovy
create mode 100644
regression-test/suites/variant_p0/test_variant_cast_isnull_pushdown.groovy
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]