Shivam Dalmia created PARQUET-1080:
--------------------------------------
Summary: Empty Parquet Files created as a result of spark jobs
fail when read
Key: PARQUET-1080
URL: https://issues.apache.org/jira/browse/PARQUET-1080
Project: Parquet
Issue Type: Bug
Components: parquet-format
Reporter: Shivam Dalmia
Priority: Minor
I have faced an issue intermittently with certain spark jobs writing parquet
files which apparently succeed but the written .parquet directory in HDFS is an
empty directory (with no _SUCCESS and _metadata parts, even). Surprisingly, no
errors are thrown from spark dataframe writer.
However, when attempting to read this written file, spark throws the error:
{{Unable to infer schema for Parquet. It must be specified manually}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
