gavinchou commented on PR #68032: URL: https://github.com/apache/doris/pull/68032#issuecomment-6251154113
Please add consistent file-level lifecycle logging in the spill layer (`SpillFileManager` / `SpillFile` / `SpillFileWriter`). The existing S3 upload and recursive-delete INFO logs are useful, but they do not provide a consistent logical `SpillFile` lifecycle. Spill-layer creation and successful GC currently have no corresponding lifecycle logs, while normal local filesystem operations are only logged at VLOG_DEBUG. Please log: - Logical file creation, write/close result, and deletion result, including eventual deletion success after a retry. Distinguish allocation from a successfully completed write. - Query ID, local/remote storage type, path (identifiable bucket/vault for remote files), relevant byte/part counts, and status/error, so creation and cleanup can be correlated. - Successful lifecycle events at INFO and failures at WARNING; rate-limit repeated retry failures while retaining the correlation fields. Keep this at file granularity, not per Block or S3 UploadPart. This can stay within the spill layer without changing the generic S3 writer. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
