Repository: spark Updated Branches: refs/heads/master 28d41d627 -> cc275f4b7
[SQL] [SPARK-3236] Reading Parquet tables from Metastore mangles location Currently we do `relation.hiveQlTable.getDataLocation.getPath`, which returns the path-part of the URI (e.g., "s3n://my-bucket/my-path" => "/my-path"). We should do `relation.hiveQlTable.getDataLocation.toString` instead, as a URI's toString returns a faithful representation of the full URI, which can later be passed into a Hadoop Path. Author: Aaron Davidson <aa...@databricks.com> Closes #2150 from aarondav/parquet-location and squashes the following commits: 459f72c [Aaron Davidson] [SQL] [SPARK-3236] Reading Parquet tables from Metastore mangles location Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/cc275f4b Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/cc275f4b Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/cc275f4b Branch: refs/heads/master Commit: cc275f4b7910f6d0ad266a43bac2fdae58e9739e Parents: 28d41d6 Author: Aaron Davidson <aa...@databricks.com> Authored: Wed Aug 27 15:05:47 2014 -0700 Committer: Michael Armbrust <mich...@databricks.com> Committed: Wed Aug 27 15:05:47 2014 -0700 ---------------------------------------------------------------------- .../src/main/scala/org/apache/spark/sql/hive/HiveStrategies.scala | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/cc275f4b/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveStrategies.scala ---------------------------------------------------------------------- diff --git a/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveStrategies.scala b/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveStrategies.scala index 10fa831..47e24f0 100644 --- a/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveStrategies.scala +++ b/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveStrategies.scala @@ -135,7 +135,7 @@ private[hive] trait HiveStrategies { .fakeOutput(projectList.map(_.toAttribute)):: Nil } else { hiveContext - .parquetFile(relation.hiveQlTable.getDataLocation.getPath) + .parquetFile(relation.hiveQlTable.getDataLocation.toString) .lowerCase .where(unresolvedOtherPredicates) .select(unresolvedProjection:_*) --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
