[
https://issues.apache.org/jira/browse/PARQUET-402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15051312#comment-15051312
]
Ryan Blue commented on PARQUET-402:
-----------------------------------
What was the type of my_map before you used the explicit type? It seems to me
that the type should be something we can work with. I wonder if there is still
a bug here we should address.
Also, could you open an issue for the second problem you found, that you now
have to use the full path to ParquetStorer? That's probably something we can
fix easily and (I'm guessing) happened when we moved to org.apache packages.
> Apache Pig cannot store Map data type into Parquet format
> ---------------------------------------------------------
>
> Key: PARQUET-402
> URL: https://issues.apache.org/jira/browse/PARQUET-402
> Project: Parquet
> Issue Type: Bug
> Components: parquet-pig
> Affects Versions: 1.6.0, 1.8.1
> Reporter: Jerry Ylilammi
>
> Trying to store simple map with two entries gives me following exception:
> {code}table_with_map_data: {my_map: map[]}
> 2015-12-10 11:58:54,478 [main] INFO
> org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is
> deprecated. Instead, use fs.defaultFS
> 2015-12-10 11:58:54,498 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR
> 2999: Unexpected internal error. Invalid map Schema, schema should contain
> exactly one field: my_map: map{code}
> For example taking any input and doing this gives me the exception:
> {code}table_with_map_data = FOREACH random_data GENERATE TOMAP('123',
> 'hello', '456', 'world') as (my_map);
> DESCRIBE table_with_map_data;
> STORE table_with_map_data INTO '...' USING ParquetStorer();{code}
> I'm using latest version of Pig: Apache Pig version 0.15.0 (r1682971)
> compiled Jun 01 2015, 11:44:35
> and Parquet: parquet-pig-bundle-1.6.0.jar
> EDIT: I noticed Parquet 1.8.1 is out. I switched to it and were forced to
> update the pig script to use full path with ParquetStorer. However this gives
> me same error as 1.6.0.
> {code}STORE table_with_map_data INTO
> '/Users/jerry/tmp/parquet/output/parquet' USING
> org.apache.parquet.pig.ParquetStorer();{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
