Gatsby-Lee commented on issue #6232: URL: https://github.com/apache/hudi/issues/6232#issuecomment-1424657330
hmm interesting. I am on Glue 3 + Hudi 0.10.1. While I did bulk_insert from Hudi Table A to Hudi Table B to change recordKey, I encountered an issue related Timestamp. Hudi Table A - timestamp datatype in Table Schema in AWS Glue Catalog - Parquet meta shows `OPTIONAL INT64 L:TIMESTAMP(MICROS,true) R:0 D:1` Hudi Table B - timestamp datatype in Table Schema in AWS Glue Catalog - Parquet meta shows `OPTIONAL INT64 L:TIMESTAMP(MILLIS,true) R:0 D:1` - The data is queryable and looks working and valid. HOWEVER, when the new record is coming in with MICROS format into the Parquet's meta `TIMESTAMP(MILLIS)`, things become super weird. The fix for me is `hoodie.parquet.outputtimestamptype: TIMESTAMP_MICROS` I can see the precision as well like `2023-02-09 05:01:44.626` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
