pitrou commented on code in PR #50483:
URL: https://github.com/apache/arrow/pull/50483#discussion_r3629517074
##########
cpp/src/arrow/csv/parser.cc:
##########
@@ -533,12 +534,13 @@ class BlockParserImpl {
}
if (batch_.num_rows_ > start_num_rows && batch_.num_cols_ > 0) {
- // Use bulk filter only if average value length is >= 10 bytes,
- // as the bulk filter has a fixed cost that isn't compensated
- // when values are too short.
+ // Use bulk filter only if average value length is >= 10 bytes
+ // (its fixed cost isn't compensated for short values), and the block
+ // has no embedded NUL bytes (see block_has_nul_).
Review Comment:
I think Copilot is right that `csv/chunker.cc` must be fixed as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
