pitrou commented on issue #50548: URL: https://github.com/apache/arrow/issues/50548#issuecomment-5022654241
For the record, the min-max check was added by @emkornfield in https://github.com/apache/arrow/pull/8177/changes#diff-05802abb678d7f865de06497651d9ca057bf0b44ec37ceeae2a5d720ffe98e28R160-R168 I think we have 3-4 options here: 1) determine that the check is unnecessary and remove it entirely 2) determine that the check is necessary and: a) decide it's cheap enough and not worth optimizing b) consider it too expensive, optimize the non-AVX2 version to be better autovectorizable (or hand-optimize it using xsimd) c) consider it too expensive, fold the min-max checking into the decoding itself (which would be a win for large RLE runs) However, the approach of only checking the first and last values does not make sense, IMHO :-) cc @AntoinePrv for opinions -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
