pan3793 opened a new pull request, #57676: URL: https://github.com/apache/spark/pull/57676
### What changes were proposed in this pull request? This PR extends History Server on-demand event-log loading to V1 event logs. It adds `spark.history.fs.eventLog.v1.onDemandLoadEnabled`, which is independent from the existing rolling event-log configuration. When enabled, the History Server probes V1 completed, compressed, and in-progress event-log names after a requested application is absent from the listing. ### Why are the changes needed? V1 event logs remain supported, but on-demand loading currently only supports rolling event logs. A requested V1 application can therefore return Not Found until periodic scanning discovers its log. ### Does this PR introduce _any_ user-facing change? Yes. History Server can load V1 event logs on demand. The new `spark.history.fs.eventLog.v1.onDemandLoadEnabled` configuration defaults to `true`. ### How was this patch tested? Added tests for: - enabled and disabled V1 on-demand loading - uncompressed and LZF-compressed logs - in-progress logs and cleaner safety - attempt IDs - multiple log directories - scan-disabled directories Ran the focused `FsHistoryProviderSuite` and `SingleEventLogFileWriterSuite` tests. Also ran `SparkConfigBindingPolicySuite`. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: gpt-5.6-terra -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
