Ngone51 commented on issue #28195: [SPARK-31425][SQL][CORE] UnsafeKVExternalSorter should also respect UnsafeAlignedOffset URL: https://github.com/apache/spark/pull/28195#issuecomment-613280460 I fixed `VariableLengthRowBasedKeyValueBatch` as well. To be clarify, it won't cause data correctness issue as `UnsafeKVExternalSorter` does. Unlike `UnsafeKVExternalSorter` may reading records from `BytesToBytesMap`, `VariableLengthRowBasedKeyValueBatch` writes and reads records by itself. Thus, similar to #22053 and [your comment](https://github.com/apache/spark/pull/22053#issuecomment-411975239) there, it's more likely an improvement for the support of SPARC platform. BTW, I believe we don't need to fix `FixedLengthRowBasedKeyValueBatch` as it doesn't need to write any length. cc @kiszk
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
