[GitHub] [hudi] Zouxxyy commented on pull request #6930: [HUDI-4716] Avoid parquet-hadoop-bundle in hudi-hadoop-mr

Mon, 05 Jun 2023 09:22:44 -0700 


Zouxxyy commented on PR #6930:
URL: https://github.com/apache/hudi/pull/6930#issuecomment-1577091968

   @xushiyan use `parquet-hadoop` 1.8.1 (hive2) with `parquet-avro` 1.10.1 have 
one problem:
   
   ```java
       // get base schema
       ParquetMetadata fileFooter = ParquetFileReader.readFooter(conf, path, 
ParquetMetadataConverter.NO_FILTER);
       MessageType messageType = fileFooter.getFileMetaData().getSchema();
   ```
   
   the Schema get from footer is wrong:
   
   
![image](https://github.com/apache/hudi/assets/37108074/5bbfa13e-1d50-442c-b14a-5ff63ec59be6)
   
   **It will not fetch the information in `parquet.avro.schema`**
   
   <img width="1434" alt="image" 
src="https://github.com/apache/hudi/assets/37108074/03574475-bde6-4efa-9359-ff8f5f4f581b";>
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to