Github user avulanov commented on the issue:
https://github.com/apache/spark/pull/13868
I used `makeQualifiedPath` from `SessionCatalog` to construct paths. It
uses hadoop's `Path` and `FileSystem`. Each attempt of path construction should
use this approach in order to be consistent with Spark API. There are 36
occurrences of `"file:"` or `"file://"` or `"file:/"` in Spark Scala code as
String constants. It is either checking if the path is a local file or
construction of a path for a local file. Approaches differ. For example:
https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/SparkContext.scala#L1717
(uses java.net.URI)
https://github.com/apache/spark/blob/master/yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala#L611
(uses java.File)
The same constant is frequently used in Spark Python.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
