matanper opened a new issue, #68894: URL: https://github.com/apache/doris/issues/68894
### Search before asking - [x] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Version 4.1.4 (cloud mode); master is the same. ### What's Wrong? File cache block writes go through `LocalFileWriter`. On finalize it calls `sync_file_range(fd, 0, 0, SYNC_FILE_RANGE_WRITE)`, which only starts writeback, and never drops the pages afterwards (no `posix_fadvise(POSIX_FADV_DONTNEED)`). The written cache blocks therefore stay in the page cache, dirty until writeback finishes and clean afterwards, and are charged to the BE's cgroup. In a container with a memory limit, sustained write-through to the file cache grows dirty page cache to 8–20 GB. The cgroup then sits at `memory.max`, and allocating threads, including S3 upload threads, enter direct reclaim. Loads stall even though the process's own memory is well below the limit. The data was just written to the cache's local disk, so keeping it in the page cache is rarely useful: reads go through the cache's own block files and the remote fallback. ### What You Expected? File cache writes should not fill the cgroup's page cache with blocks that were just written. ### How to Reproduce? Run the BE in a container (cgroup v2) with a memory limit and `enable_file_cache_write_from_s3_file_writer=true`, under sustained ingestion. Watch `memory.stat` (`file_dirty`, `file`) and `memory.pressure` while the BE's own allocations stay flat. ### Anything Else? What fixed it for us: an option on `FileWriterOptions` (on for file cache writes, behind a mutable BE config) that on close calls `sync_file_range(WAIT_BEFORE | WRITE | WAIT_AFTER)` and then `posix_fadvise(DONTNEED)`, plus bvars for dropped bytes and wait time. Dirty pages fell from 8–20 GB to under 0.4 GB under the same load. ### Are you willing to submit PR? - [x] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
