vvysotskyi opened a new pull request #2026: DRILL-7330: Implement metadata usage for all format plugins URL: https://github.com/apache/drill/pull/2026 # [DRILL-7330](https://issues.apache.org/jira/browse/DRILL-7330): Implement metadata usage for all format plugins ## Description This pull request adds support of all format plugins for Metastore. It contains two parts: - Making changes to collect metadata for easy group scan - Allowing usage Metastore metadata for easy group scan during querying data To reach the first goal was added AnalyzeFileInfoProviderImpl to handle correctly EasyGroupScan tables in MetastoreAnalyzeTableHandler and specify correct table type. Accommodated EVF scan framework to incorporate Metastore-specific additional implicit columns and handle the case of empty files when metadata should be returned. To reach the second goal, EasyGroupScan was updated to extend AbstractGroupScanWithMetadata. MetastoreFileTableMetadataProvider based on existing MetastoreParquetTableMetadataProvider was introduced to provide metastore metadata for file-based tables. Accommodated ParquetPushDownFilter rule to work with easy format plugins. Refactored some code and fixed additional issues. ## Documentation After this PR is merged, Drill Web Site docs should be updated to reflect this feature. ## Testing Added tests to verify the correctness of collected metadata and ensure that new optimizations work correctly.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
