mrhhsg opened a new pull request, #69781:
URL: https://github.com/apache/doris/pull/69781
### What problem does this PR solve?
Issue Number: None
Related PR: None
Problem Summary: `SpillFileWriter` mixed common spill-part serialization,
footer writing, rotation, and lifecycle management with S3-only upload queue
admission, cancellation checks, timing, and remote accounting. This made the
common writer depend on object-storage behavior and spread remote branches
across its write and close paths.
This change introduces `SpillS3FileWriter` for remote spill files while
keeping the common part lifecycle and local writing in `SpillFileWriter`.
Narrow virtual hooks preserve S3 upload admission, remote I/O metrics, and
failed-part reconciliation. The derived destructor closes the writer before
base destruction so implicit close still dispatches the S3 hooks. The writer
API and spill file format remain unchanged.
### Release note
None
### Check List (For Author)
- Test: Unit Test / Regression test
- `BUILD_TYPE=ASAN ./build.sh --be` passed.
- `./run-be-ut.sh --run --filter='SpillFileS3Test.*:SpillFileTest.*'`
passed (84 tests).
- Existing `spill_p0/window_function_spill` local regression passed (1
suite, 0 failures).
- BE formatting, build-hygiene, and changed-line clang-tidy checks
passed; FE checkstyle passed as part of the local regression build.
- Behavior changed: No; this is an internal responsibility split that
preserves the existing local and S3 spill behavior.
- Does this need documentation: No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
