HeartSaVioR commented on issue #23260: [SPARK-26311][YARN] New feature: custom log URL for stdout/stderr URL: https://github.com/apache/spark/pull/23260#issuecomment-455032458 Updated with new approach: - Only SHS provides custom executor log URLs (via instrumenting events a bit while reading) - The approach is open to all resource managers (any RM can provide executor attributes and let SHS leverages these as variables for patterns) - For now, only YARN is able to provide this functionality, since other RMs even don't seem to provide log URLs I have to spend time to run this in YARN cluster as well (will update the result here), but I guess the new test cases show how it works and whether it works. @vanzin @squito @tgravescs Could you take a second look? Thanks in advance.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
