vanzin commented on issue #23260: [SPARK-26311][YARN] New feature: custom log URL for stdout/stderr URL: https://github.com/apache/spark/pull/23260#issuecomment-451018488 A general solution requires two things: - the application to save some information in the event log so that the SHS can use it - the SHS to use that information to build the log URLs when building an application's UI later The mechanism to do that doesn't need to be RM-specific. The data (i.e. the information saved in the event logs, the log URL template itself) can be, but the mechanism doesn't need to. If you think about the code you have, you already defined a set of parameters for the log URL. All you need is, instead of parsing the log URL on the application side, do that on the SHS side, based on data written by the app. You can even only implement that for YARN, but you'd have a generic mechanism that later can also be implemented for other RMs without having to change the SHS.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
