tgravescs commented on issue #23260: [SPARK-26311][CORE] New feature: apply custom log URL pattern for executor log URLs in SHS URL: https://github.com/apache/spark/pull/23260#issuecomment-456827963 Sorry I haven't had time to look through all the code so this might be a separate jira, but one thing I thought of here is it would be really nice not to have specifically stderr/stdout. users can specify any log4j.properties and some tools like oozie by default end up using hadoop log4j rather then spark log4j, so files aren't necessarily the same. Also users can put in other logs files so it would be nice to have links to those from the UI. It seems simpler if we just had a link to the directory and it read the files within there. Other things in Hadoop do it this way, but I'm not sure if that works well for other resource managers, any thoughts on that? As long as this doesn't prevent the above I can file a separate jira for it.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
