hehuiyuan edited a comment on pull request #15712:
URL: https://github.com/apache/flink/pull/15712#issuecomment-834081851


   > > > Hi @hehuiyuan , I think hive-2.0.0 doesn't support writing empty maps 
into parquet table: related to HIVE-13632.
   > > > Could you elaborate how you encountered this issue? E.g. how are the 
parquet files generated?
   > > 
   > > 
   > > Empty map data is writed by other way e.g. hive client or other 
application write parquet file...
   > > Then read hive by flink
   > 
   > I managed to generate some parquet file with empty maps with higher 
version hive. And when I read this file with hive-2.0.0, it returns `NULL` for 
the empty maps. Could you verify what hive-2.0.0 returns for the empty map in 
your case?
   > 
   > We can consider adding such a parquet file for the test. I think it's 
better than generating the file using filesystem connector.
   
![image](https://user-images.githubusercontent.com/18002496/117402090-53916900-af38-11eb-81bc-c6165fd4c2f3.png)
   
   ```
   Map<?, ?> map = mapInspector.getMap(data);
   Map<Object, Object> result = new HashMap<>(map.size());
   ```
   mapArray.length == 0 then Null is returned here,then throw NPE when  call 
map.size. 
   
   Local Hive table :
   ```
   CREATE EXTERNAL TABLE `search_rank_feature_label_integration2`(
     `pvid` string COMMENT '',
     `id_list_feature` map<string,string> COMMENT '',
     `ts` bigint COMMENT '')
   PARTITIONED BY (
     `dt` string)
   ROW FORMAT SERDE
     'org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe'
   STORED AS INPUTFORMAT
     'org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat'
   OUTPUTFORMAT
     'org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat'
   ```
   Local Hive table data
   
![image](https://user-images.githubusercontent.com/18002496/117402849-bf280600-af39-11eb-981d-d886823682c4.png)
   
   ```
   hehuiyuan    {}      222     2021-03-26
   ```
   read this local hive table then NPE 
   
![image](https://user-images.githubusercontent.com/18002496/117402724-7ec88800-af39-11eb-97e3-261f497c3593.png)
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[email protected]


Reply via email to