RussellSpitzer commented on pull request #2953: URL: https://github.com/apache/iceberg/pull/2953#issuecomment-896934206
> > This in turn requires fixing the alternate PruneColumn methods for Avro > > and Parquet to respect that they will now have selected field ID's for non > > primitive nodes. > > Do you know if anything needs to be done for ORC @RussellSpitzer? I'm helping out with ORC more going forward and if you're aware of anything that needs to be updated there, if you don't have time to update it, if you make an issue I'll see if I can grab it (or at the least we'll have the issue to track). > > If you're not sure then possibly you can add a unit test for ORC as well? If it fails, open an issue (or I will) and then we can follow up on it after. =) I don't think so, we don't have any custom ORC projection code as far as I can tell. It just uses the output of a TypeUtil.selectNot() https://github.com/apache/iceberg/blob/68091037944ff7e9de91e7b619f313a8e98c1adc/spark/src/main/java/org/apache/iceberg/spark/source/RowDataReader.java#L158-L159 against the ORC schema to determine what to read and then uses that directly here : https://github.com/apache/iceberg/blob/970e8aac60081c7d286ab0f7e12e428b3309dd34/orc/src/main/java/org/apache/iceberg/orc/ORC.java#L315-L319 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
