pitrou commented on PR #50483: URL: https://github.com/apache/arrow/pull/50483#issuecomment-5036452927
To avoid the regression, we could arrange to run the bulk filter only if the entire block does not contain a NUL byte (which can be checked extremely quickly using `memcmp`). NUL bytes being exceptional in CSV files, common cases would not see a slowdown. (of course, this would be only for SSE4.2, not NEON) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
