GitHub user scwf opened a pull request:
https://github.com/apache/spark/pull/3413
[SPARK-4552][SQL] query for empty parquet table in spark sql hive get
IllegalArgumentException
create table test_parquet(key int, value string) stored as parquet;
select * from test_parquet;
get error as follow
java.lang.IllegalArgumentException: Could not find Parquet metadata at path
file:/user/hive/warehouse/test_parquet
at
org.apache.spark.sql.parquet.ParquetTypesConverter$$anonfun$readMetaData$4.apply(ParquetTypes.scala:459)
at
org.apache.spark.sql.parquet.ParquetTypesConverter$$anonfun$readMetaData$4.apply(ParquetTypes.scala:459)
at scala.Option.getOrElse(Option.scala:120)
at
org.apache.spark.sql.parquet.ParquetTypesConverter$.readMetaData(ParquetTypes.sc
link to #3352
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/scwf/spark parquet-empty
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/3413.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3413
----
commit c38201dc9cabac67fa6f99ebc0d5bd2e6e5b16fa
Author: scwf <[email protected]>
Date: 2014-11-22T08:53:19Z
fix parquet empty issue
commit be3b00656957fea7dcd9e3e0f264e67430d3b39f
Author: scwf <[email protected]>
Date: 2014-11-22T08:53:26Z
Merge branch 'master' of https://github.com/apache/spark into parquet-1
commit 1e9cdfc9af06731aa4b18287e32c7ec82ac8aed1
Author: scwf <[email protected]>
Date: 2014-11-22T09:11:45Z
minor fix
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
