DanielLeens commented on issue #9890: URL: https://github.com/apache/seatunnel/issues/9890#issuecomment-4899080781
Sorry this thread sat too long. I checked the current starter path, and your suspicion is reasonable. In cluster mode, `FileUtils.getConfigPath(...)` strips the config path down to the file name, and `SparkTaskExecuteCommand.execute()` checks that path before Spark-side resolution happens. In the current branch, this still does not go through a `SparkFiles`-style lookup, so this does look like a real bug in the Spark Standalone cluster-mode config loading path rather than just an environment problem on your side. That also matches the symptom you reported: - client mode works because the local path is used directly - cluster mode fails early because SeaTunnel looks for the bare file name before the distributed file location is resolved - the problem is specifically around how the config file is located, not around the job content itself For now, the practical workaround is to use client mode for Spark Standalone deployments. I would not recommend relying on the driver working directory layout as a long-term workaround, because that path is not stable and should not be part of the user contract. So we should keep this open as a bug. Since the issue did not opt into a PR, I also added `help wanted` so other contributors can pick it up. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
