pitrou commented on pull request #11101: URL: https://github.com/apache/arrow/pull/11101#issuecomment-914181655
Takeaway: both filter types are quite beneficial with long CSV values, but incur a slowdown if all CSV columns have short values. One approach would be to only enable those filters when the _row length / num columns_ ratio is above a given constant, e.g. 10. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
