[
https://issues.apache.org/jira/browse/PARQUET-2039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17335226#comment-17335226
]
Gabor Szadovszky commented on PARQUET-2039:
-------------------------------------------
I am a bit confused about the version numbers. Are you comparing 1.11.1 and
1.10.0 or 1.11.1 and 1.11.0? Anyway, PARQUET-1879 seems to be related. It is
around the same area and got in to 1.11.1.
You are invoking {{setRequestedProjection}}. Could you please list the exact
avro schema you are using for the projection?
[~maccamlc], since your modification potentially caused this regression(?) you
may want to check this out.
> AvroReadSupport.setRequestedProjection in 1.11.1+ not backwards compatible
> with MAPS
> ------------------------------------------------------------------------------------
>
> Key: PARQUET-2039
> URL: https://issues.apache.org/jira/browse/PARQUET-2039
> Project: Parquet
> Issue Type: Bug
> Components: parquet-avro, parquet-mr
> Affects Versions: 1.11.0, 1.11.1
> Reporter: Bil Bingham
> Priority: Minor
> Attachments: PQMapTest.java
>
>
>
> using AvroReadSupport.setRequestedProjection in 1.11.1 reading a 1.10.0
> generated parquet file sets MAP<STRING,STRING> fields to null (and vice versa
> 1.10.0 reader against a 1.11.1 generated file)
> - Not using a projected schema works, the map fields are converted correctly.
> In my case. Parquet file is generated by hive
> {code:java}
> CREATE TABLE parquetmaptest (
> a string,
> b MAP<string,string>
> )
> STORED AS PARQUET
> tblproperties(
> "parquet.compression"="SNAPPY"
> );
> insert into parquetmaptest select "a",map("k1","v1","k2","v2");{code}
> Using parquet-avro 1.11.1 (and appropriate dependencies) result in field "b"
> being null.
> {code:java}
> data:null
> row:{"a": "a", "b": null}{code}
> Using parquet-avro 1.11.0 (and appropriate dependencies) result in field "b"
> being the right map value.
> {code:java}
> data:{k1=v1, k2=v2}
> row:{"a": "a", "b": {"k1": "v1", "k2": "v2"}}{code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
