okalinin opened a new pull request #1428: DRILL-6670: align Parquet TIMESTAMP_MICROS logical type handling with earlier versions URL: https://github.com/apache/drill/pull/1428 ## DRILL-6670: align Parquet TIMESTAMP_MICROS logical type handling with earlier versions ### Background Parquet dependencies were upgraded with DRILL-6353 which changed behavior in how Parquet handles `TIMESTAMP_MICROS` logical type. Previously, calling `SchemaElement.getConverted_type()` was returning null and logical type was ignored. With updated Parquet version above call returns `TIMESTAMP_MICROS` which triggers query exception. Issue impacts both ParquetRecordReader and DrillParquetReader. This change aims at restoring original functionality - handling `TIMESTAMP_MICROS` as `INT64` with no logical type in both Parquet readers. It doesn't seem to make sense to do more since `TIMESTAMP_MICROS` is deprecated logical type as per Parquet [current documentation](https://github.com/apache/parquet-format/blob/master/LogicalTypes.md). ### Change description - Added `TIMESTAMP_MICROS` logical type in both Parquet readers in order to handle it as regular `INT64` - Modified `ParquetSimpleTestFileGenerator` to include `TIMESTAMP_MICROS` columns in Parquet test files and updated existing test data files - Modified respective tests with minor regrouping - Fixed `TestDrillParquetReader` to make sure correct reader is actually used to handle test queries - Improved debug lines to avoid using ambiguous 'new/old Parquet reader' terminology ### Level of testing - Sample file and query provided by issue reporter - Unit tests
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
