vanzin commented on issue #23260: [SPARK-26311][YARN] New feature: custom log URL for stdout/stderr URL: https://github.com/apache/spark/pull/23260#issuecomment-454519656 > If we are happy with pre-defining parameters by ourselves and let end users just be flexible with constructing log URL, things would be fairly simpler. Yes, this is what I mean. By "not pre-defined" I mean Spark as a whole doesn't define the parameters; each resource manager backend can define a different set of parameters that are saved to the logs. It would be nice if some of them at least overlapped (e.g. app id, app attempt), but I can see that some RMs may need or provide different parameters that may not be available elsewhere (e.g. cluster ID).
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
