[
https://issues.apache.org/jira/browse/PARQUET-364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14706959#comment-14706959
]
Cheng Lian commented on PARQUET-364:
------------------------------------
I tested the Thrift case with Thrift 0.9.2 because I can't get Thrift 0.7.0
compiled on Mac OS X 10.10 because of lacking proper C++ header files. I assume
that this doesn't change the essence of this issue.
(BTW, any plan to upgrade to Thrift 0.9.2?)
> Parque-avro cannot decode Avro/Thrift array of primitive array (e.g.
> array<array<int>>)
> ---------------------------------------------------------------------------------------
>
> Key: PARQUET-364
> URL: https://issues.apache.org/jira/browse/PARQUET-364
> Project: Parquet
> Issue Type: Bug
> Components: parquet-mr
> Affects Versions: 1.5.0, 1.6.0, 1.7.0, 1.8.0
> Reporter: Cheng Lian
>
> The problematic Avro and Thrift schemas are:
> {noformat}
> record AvroArrayOfArray {
> array<array<int>> int_arrays_column;
> }
> {noformat}
> and
> {noformat}
> struct ThriftListOfList {
> 1: list<list<i32>> intArraysColumn;
> }
> {noformat}
> They are converted to the following Parquet schemas by parquet-avro 1.7.0 and
> parquet-thrift 1.7.0 respectively:
> {noformat}
> message AvroArrayOfArray {
> required group int_arrays_column (LIST) {
> repeated group array (LIST) {
> repeated int32 array;
> }
> }
> }
> {noformat}
> and
> {noformat}
> message ParquetSchema {
> required group intListsColumn (LIST) {
> repeated group intListsColumn_tuple (LIST) {
> repeated int32 intListsColumn_tuple_tuple;
> }
> }
> }
> {noformat}
> {{AvroIndexedRecordConverter}} cannot decode such records correctly. The
> reason is that the 2nd level repeated group {{array}} doesn't pass
> {{AvroIndexedRecordConverter.isElementType()}} check. We should check for
> field name "array" and field name suffix "_thrift" in {{isElementType()}} to
> fix this issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
