dongjoon-hyun commented on PR #51604: URL: https://github.com/apache/spark/pull/51604#issuecomment-3103429856
Thank you for review, @LuciferYang and @mridulm . This is a continuation of Apache Spark 4.0.0 [SPARK-45869 Revisit and Improve Spark Standalone Cluster](https://issues.apache.org/jira/browse/SPARK-45869) to support Apache Spark as a subsystem. 1. No, this PR intentionally doesn't update the list UI. The list UI will update periodically in the existing manner to sync with the storage. New log entry will be there with the uploaded correct information. For the typo(?) entry, it will be removed because it's the same situation when a user deletes log files from the storage arbitrarily. > What is the behavior if the application does not exist ? (typo in user query for example) Will the listing now have an invalid entry ? 2. Yes, you are right. Initially, I also had a single file event log logic in this PR, but I removed it in the current PR to simplify this PR and to focus rolling event logs only. Legacy log format is out of scope in this PR. As you may noticed at SPARK-45771, I have been using rolling event logs always since Nov 2, 2023 in order to guarantee to get partial logs. > We should not assume it will be RollingEventLogFilesWriter, users dont need to be running with the default enabled, right ? - https://github.com/apache/spark/pull/43638 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
