xylaaaaa opened a new pull request, #70243: URL: https://github.com/apache/doris/pull/70243
### Background The BE service registers a file cache factory in `ExecEnv` at startup. The BE unit-test runner reads the server configuration without performing that startup step. Both writer construction and tablet cleanup can reach the file cache. ### What problem does this PR solve? Issue Number: None Related PR: #68797 (enabled file caching by default), #69605 (initial S3 fixture fix), #69670 (overlapping S3 client teardown) PR Type: Test Problem Summary: S3, Hive report-writer, HDFS and packed-file-manager unit tests can crash when run alone or after a fixture leaves an empty cache factory. A downloader or cloud-tablet test can mask the crash by leaving caching disabled. With caching enabled but no populated factory, dropping a tablet can also crash in rowset cleanup. Set the file-cache default to false in the test runner’s copy of `be.conf`. Writer fixtures that exercise caching install a populated cache, explicitly enable it, and restore both the factory and the flag afterward. ### Details - **Failure:** Isolated writers dereference a missing cache factory during S3/HDFS writer construction or a packed-file cache write. With an empty factory, the same lookup takes a hash modulo zero. In a shuffled run, the S3 buffer-boundary test also removed the suite's mock callbacks, which made later tests send requests to S3. - **Cause:** The writer fixtures relied on process-wide state set by other tests. The boundary test used its own in-memory client but also cleared all registered callbacks and left a changed buffer size behind. - **Tablet cleanup:** The service default enables file caching, but the unit-test executable does not initialize a cache. Once the writer and downloader fixtures correctly restore the enable flag, `TestDeltaWriterClusterKey.vec_sequence_col` can call `Rowset::clear_cache()` with an empty factory while dropping the tablet. The test runner now writes its default to the copy of `be.conf` used by the test binary. - **Stability evidence:** The 76 focused tests pass in the default order and three shuffled runs (seeds 7301, 7302 and 7303). The HDFS write test also passes alone and immediately after a block-cache fixture that leaves an empty factory. The scope tests exercise a missing factory, an empty factory, reuse of a populated factory, and restoration with either initial value of the enable flag. ### Testing - Reproduced three ASAN segmentation faults with the fixtures from `315c2e0033`. `S3FileWriterTest.multi_part_io_error` and `VHivePartitionWriterReportLifecycleTest.PeriodicReportDefersMetadataAndFinalReportTransfersUploadIdentity` fail during writer construction. `PackedFileManagerTest.ProcessUploadingFilesSetsFailedWhenMetaUpdateFails` fails during the cache write. All three pass in separate processes with this change. The Azure Hive case and three additional packed-upload cases also pass in separate processes (7/7). - In [BE UT build 1093099](http://43.132.222.7:8111/viewLog.html?buildId=1093099&buildTypeId=Doris_DorisBeUt_BeUt), `HdfsFileSystemTest.Write` aborted with ASAN FPE in `FileCacheFactory::get_by_path`. Reproduced the failure locally both alone and following `BlockFileCacheTest.file_cache_profile_specialized_write_cache_counters`. Both runs pass after giving the HDFS writer a scoped cache. The test still requests cache writes and verifies the writer installed its cache builder. - In [BE UT build 1093619](http://43.132.222.7:8111/viewLog.html?buildId=1093619&buildTypeId=Doris_DorisBeUt_BeUt), `HdfsFileSystemTest.Write` passed. The run later aborted in `TestDeltaWriterClusterKey.vec_sequence_col` while dropping a tablet: `TabletManager::drop_tablet` → `Rowset::clear_cache` → `FileCacheFactory::get_by_path` (ASAN FPE). On macOS the old runner reproduced the crash at the same cache lookup when the block-cache test ran immediately before the DeltaWriter case. With the new runner the same pair passes, and the DeltaWriter case also passes alone under ASAN. All 85 tests in a combined run also passed, covering the DeltaWriter, MemTable, Segment, cache and writer fixtures, including the original HDFS failure. Checked that the generated config disables caching even when the source config explicitly enabled it and had no final newline. - Ran the 76 tests below on macOS under ASAN. All 76 passed in the default order and in each iteration of `GTEST_SHUFFLE=1 GTEST_REPEAT=3 GTEST_RANDOM_SEED=7301`, and both processes exited with status 0. ```sh ./run-be-ut.sh --run --filter='ScopedFileCacheTest.*:S3FileWriterTest.*:VHivePartitionWriterReportLifecycleTest.*:PackedFileManagerTest.*:FileCacheBlockDownloaderTest.*:CloudTabletSyncMetaTest.*:HdfsFileSystemTest.*:BlockFileCacheTest.file_cache_profile_specialized_write_cache_counters' ``` The macOS full suite stopped before the HDFS test on two failures that also reproduce in isolation: `ColumnVariantV2Test.TypedComparisonBoundariesMatchEncodedWithoutMaterializing` and `LocalExchangerTest.TestShuffleExchangerWrongMap`. Both passed on Linux in build 1093099. The local Linux x86_64 build was stopped before running tests. The [full Linux BE UT, build 1094205](http://43.132.222.7:8111/viewLog.html?buildId=1094205&buildTypeId=Doris_DorisBeUt_BeUt&guest=1), passed on `5d9cf757c6`. GoogleTest reported 16,200 passed, 68 skipped and zero failures. The HDFS write and DeltaWriter tablet-cleanup cases both passed in the full run. The [P0 regression](http://43.132.222.7:8111/viewLog.html?buildId=1094289&buildTypeId=Doris_DorisRegression_P0Regression&guest=1) and [Cloud P0 regression](http://43.132.222.7:8111/viewLog.html?buildId=1094317&buildTypeId=Doris_DorisRegression_CloudP0&guest=1) also passed on the same commit. TeamCity reported one muted case in P0 and two in Cloud P0, including the STS role-configuration case in both jobs. Clang-format 16, clang-tidy on the changed lines, and the BE build-hygiene checks passed. ### Notes for Reviewers The scope reuses an existing populated factory. Otherwise it creates a 16 MiB memory cache. The S3 and packed-file fixtures drain pending uploads before releasing the cache. The S3 fixture also releases its filesystem before AWS SDK shutdown. ### Release note None ### Check List (For Author) - Test: BE unit tests under ASAN. - Behavior changed: No - Does this need documentation: No - Pre-submission status: Focused ASAN tests passed locally, and the full Linux BE UT passed on this PR head. ### AI Disclosure - Tool: OpenAI Codex (GPT-6) - Generated: Implementation, unit tests, and PR description. - Human review: Pending author review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
