hehuiyuan edited a comment on pull request #15712:
URL: https://github.com/apache/flink/pull/15712#issuecomment-834083104
> > > Hi @hehuiyuan , I think hive-2.0.0 doesn't support writing empty maps
into parquet table: related to HIVE-13632.
> > > Could you elaborate how you encountered this issue? E.g. how are the
parquet files generated?
> >
> >
> > Empty map data is writed by other way e.g. hive client or other
application write parquet file...
> > Then read hive by flink
>
> I managed to generate some parquet file with empty maps with higher
version hive. And when I read this file with hive-2.0.0, it returns `NULL` for
the empty maps. Could you verify what hive-2.0.0 returns for the empty map in
your case?
>
> We can consider adding such a parquet file for the test. I think it's
better than generating the file using filesystem connector.
Yes, I'm already using the filesystem for test case.
But
```
Caused by: java.lang.UnsupportedOperationException: Unsupported type:
MAP<STRING, STRING>
May 06 08:39:07 at
org.apache.flink.formats.parquet.utils.ParquetSchemaConverter.convertToParquetType(ParquetSchemaConverter.java:615)
May 06 08:39:07 at
org.apache.flink.formats.parquet.utils.ParquetSchemaConverter.convertToParquetType(ParquetSchemaConverter.java:553)
May 06 08:39:07 at
org.apache.flink.formats.parquet.utils.ParquetSchemaConverter.convertToParquetMessageType(ParquetSchemaConverter.java:547)
May 06 08:39:07 at
org.apache.flink.formats.parquet.row.ParquetRowDataBuilder$ParquetWriteSupport.<init>(ParquetRowDataBuilder.java:72)
May 06 08:39:07 at
org.apache.flink.formats.parquet.row.ParquetRowDataBuilder$ParquetWriteSupport.<init>(ParquetRowDataBuilder.java:70)
May 06 08:39:07 at
org.apache.flink.formats.parquet.row.ParquetRowDataBuilder.getWriteSupport(ParquetRowDataBuilder.java:67)
May 06 08:39:07 at
org.apache.parquet.hadoop.ParquetWriter$Builder.build(ParquetWriter.java:563)
May 06 08:39:07 at
org.apache.flink.formats.parquet.row.ParquetRowDataBuilder$FlinkParquetBuilder.createWriter(ParquetRowDataBuilder.java:135)
May 06 08:39:07 at
org.apache.flink.formats.parquet.ParquetWriterFactory.create(ParquetWriterFactory.java:56)
May 06 08:39:07 at
org.apache.flink.table.filesystem.FileSystemTableSink$3.open(FileSystemTableSink.java:426)
May 06 08:39:07 at
org.apache.flink.table.filesystem.PartitionWriter$Context.createNewOutputFormat(PartitionWriter.java:61)
May 06 08:39:07 at
org.apache.flink.table.filesystem.SingleDirectoryWriter.createFormat(SingleDirectoryWriter.java:57)
May 06 08:39:07 at
org.apache.flink.table.filesystem.SingleDirectoryWriter.write(SingleDirectoryWriter.java:67)
May 06 08:39:07 at
org.apache.flink.table.filesystem.FileSystemOutputFormat.writeRecord(FileSystemOutputFormat.java:129)
May 06 08:39:07 ... 20 more
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
