[
https://issues.apache.org/jira/browse/CRUNCH-370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Micah Whitacre updated CRUNCH-370:
----------------------------------
Fix Version/s: 0.8.3
0.10.0
> Update Parquet dependency in Crunch pom
> ---------------------------------------
>
> Key: CRUNCH-370
> URL: https://issues.apache.org/jira/browse/CRUNCH-370
> Project: Crunch
> Issue Type: Improvement
> Components: IO
> Affects Versions: 0.9.0, 0.8.2
> Reporter: Anandsagar Kothapalli
> Assignee: Micah Whitacre
> Fix For: 0.10.0, 0.8.3
>
> Attachments: CRUNCH-370.patch
>
>
> Currently crunch is supporting avro to parquet conversion using
> AvroParquetFileTarget, AvroParquetFileSource classes. When I used these
> classes to convert avro to parquet files, I got the following exception in
> some cases: "org.apache.crunch.CrunchRuntimeException:
> parquet.io.ParquetEncodingException: empty fields are illegal, the field
> should be ommited completely instead"
> After further debugging I found out that this issue is related to
> AvroWriteSupport class in parquet, which was fixed as part of milestone 1.2.3
> https://github.com/Parquet/parquet-mr/issues/162. Latest parquet version is
> 1.3.2.
> But crunch is still using parquet 1.2.0
> https://github.com/apache/crunch/blob/master/pom.xml#L77
> As part of this improvement, parquet dependency version in crunch will be
> updated if not to latest then at least to 1.2.3
--
This message was sent by Atlassian JIRA
(v6.2#6252)
