HeartSaVioR commented on issue #23260: [SPARK-26311][YARN] New feature: custom log URL for stdout/stderr URL: https://github.com/apache/spark/pull/23260#issuecomment-455373732 @squito > The one thing I am wondering about is if we should change it so that when the SHS is looking at an inprogress file, it does not do any replacement. I'm sorry but could you elaborate? I'm honestly not getting familiar with UI/SHS codebase, so I can be missing here. Btw, I'm seeing my guess below may be wrong. > Now we just require end users to modify the configuration and restart SHS (unless there's a cache mechanism to avoid re-reading events when restarting SHS.) I'm taking a look at FsHistoryProvider again, and realize it may not replay events all the time (since there's a logic which loads KVStore from disk) which is against my guess... (@squito did you mean to point out this?) Looks like current approach is the simplest way to replace log url without invasion on codebase which are both used for UI and SHS, but I may need to find alternative... Maybe the class what I should extend and override is `AppStatusStore` instead of `AppStatusListener`?
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
