swamirishi commented on PR #38377: URL: https://github.com/apache/spark/pull/38377#issuecomment-1301028102
> Makes sense ... why not simply `val dfsLogFile = new Path(rootDir, appId + DRIVER_LOG_FILE_SUFFIX)` instead btw ? If we do necessarily need the fully qualified path, we can use `fileSystem.resolvePath(new Path ...)` instead. I am trying to see if I am missing anything here ... Currently the logs are written based on the current working directory of the job when the path is relative. So the output path would go to the home directory of the user in the default filesystem instead of the CWD which would be the output log directory configured in the job. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
