viirya commented on PR #56890: URL: https://github.com/apache/spark/pull/56890#issuecomment-4847369290
Small correction to my note above on the tradeoff: when the appender fails, the error is surfaced via log4j's own `StatusLogger` (which defaults to `System.err`), **not** written into the driver log file itself — I was imprecise saying "still logged." So a disk-full/permission failure on the local driver-log file doesn't go silent, but you see it on stderr rather than in the driver log. Two caveats on that: `DefaultErrorHandler` throttles repeats (~5-minute interval), so a persistent write failure won't produce one stderr line per event — most are suppressed. And the authoritative signal for the DFS-sync path is unaffected anyway, since `DfsAsyncWriter` reports its own failures through Spark's `logError`. None of this changes the conclusion; just making the tradeoff precise. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
