beliefer commented on issue #23841: [SPARK-26936][SQL] Fix bug of insert overwrite local dir can not create temporary path in local staging directory URL: https://github.com/apache/spark/pull/23841#issuecomment-476472025 According to the error: `FileFormatWriter` will create `SingleDirectoryWriteTask` and call the method `execute`. The `SingleDirectoryWriteTask` runs on one of nodes in yarn cluster. The original code use a temp path start with `file:/ `, but this path is not a valid path in this node. the temp path only valid in driver node.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
