pan3793 commented on PR #39160: URL: https://github.com/apache/spark/pull/39160#issuecomment-1367164854
@dongjoon-hyun The design does not force to use POD_NAME nor SVC_NAME as criteria to access driver/executor logs, it totally depends on how the external log service aggregates logs. To allow identifying Spark driver logs by service name, we just need to expose it as an attribute in `KubernetesClusterSchedulerBackend#getDriverAttributes`, then it can be used in the log URL pattern. So, here we need to expose the common attributes so that users can use them as criteria to fetch logs from the external log service. As I said in the proposal, I think the following attributes are generic > My proposed generic attributes are > - `APP_ID` > - `KUBENETES_POD_NAME` > - `KUBENETES_NAMESPACE` we can expose `DRIVER_SVC_NAME` as well but also keep `KUBENETES_POD_NAME`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
