Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/15382#discussion_r83183636 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -757,7 +758,10 @@ private[sql] class SQLConf extends Serializable with CatalystConf with Logging { def variableSubstituteDepth: Int = getConf(VARIABLE_SUBSTITUTE_DEPTH) - def warehousePath: String = new Path(getConf(WAREHOUSE_PATH)).toString + def warehousePath: String = { + val path = new Path(getConf(WAREHOUSE_PATH)) + FileSystem.get(path.toUri, new Configuration()).makeQualified(path).toString --- End diff -- Yeah I agree that's why this behaves this way, and it's how similar paths/URIs would be treated in a lot of places in Spark. Therefore I'm wondering if that's the right thing to do, locally, to accept the same behavior and quirks in parsing this path (and then consider changing that globally later if needed). If I'm right then there is at least a syntax available for hdfs: and file: URIs, and one that works for Linux and Windows, including paths with spaces. Not all possible strings work as expected, maybe, but the more common ones seem to. The net change at this point is just to use `Utils.resolveURI` to resolve this thing like other places in Spark, and then default to "spark-warehouse" which will now go back to defaulting to a local working dir path as intended. The rest is just test/doc/example updates. What's your opinion on this current state, compared to the problem being solved here?
--- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org