[ https://issues.apache.org/jira/browse/PARQUET-402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15051521#comment-15051521 ]
Ryan Blue commented on PARQUET-402: ----------------------------------- Is there anything we can do about it? Maybe we should at least throw an exception when the map type passed to Parquet can't be converted to a valid Parquet schema because the KV types are missing. > Apache Pig cannot store Map data type into Parquet format > --------------------------------------------------------- > > Key: PARQUET-402 > URL: https://issues.apache.org/jira/browse/PARQUET-402 > Project: Parquet > Issue Type: Bug > Components: parquet-pig > Affects Versions: 1.6.0, 1.8.1 > Reporter: Jerry Ylilammi > > Trying to store simple map with two entries gives me following exception: > {code}table_with_map_data: {my_map: map[]} > 2015-12-10 11:58:54,478 [main] INFO > org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is > deprecated. Instead, use fs.defaultFS > 2015-12-10 11:58:54,498 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR > 2999: Unexpected internal error. Invalid map Schema, schema should contain > exactly one field: my_map: map{code} > For example taking any input and doing this gives me the exception: > {code}table_with_map_data = FOREACH random_data GENERATE TOMAP('123', > 'hello', '456', 'world') as (my_map); > DESCRIBE table_with_map_data; > STORE table_with_map_data INTO '...' USING ParquetStorer();{code} > I'm using latest version of Pig: Apache Pig version 0.15.0 (r1682971) > compiled Jun 01 2015, 11:44:35 > and Parquet: parquet-pig-bundle-1.6.0.jar > EDIT: I noticed Parquet 1.8.1 is out. I switched to it and were forced to > update the pig script to use full path with ParquetStorer. However this gives > me same error as 1.6.0. > {code}STORE table_with_map_data INTO > '/Users/jerry/tmp/parquet/output/parquet' USING > org.apache.parquet.pig.ParquetStorer();{code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)