LuciferYang opened a new pull request #26106: [SPARK-29454][SQL]Reduce one unsafeProjection call when read parquet file URL: https://github.com/apache/spark/pull/26106 ### What changes were proposed in this pull request? ParquetGroupConverter call unsafeProjection function to covert SpecificInternalRow to UnsafeRow every times when read Parquet data file use ParquetRecordReader, then ParquetFileFormat will call unsafeProjection function to covert this UnsafeRow to another UnsafeRow again when partitionSchema is not empty , and on the other hand we PartitionReaderWithPartitionValues always do this convert process when use DataSourceV2. I think the first time convert in ParquetGroupConverter is redundant and ParquetRecordReader return a SpecificInternalRow is enough. ### How was this patch tested? Existing test case is enough.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
