HannaWeissberg commented on code in PR #50483:
URL: https://github.com/apache/arrow/pull/50483#discussion_r3629621386
##########
cpp/src/arrow/csv/parser_test.cc:
##########
@@ -936,5 +936,56 @@ TEST(BlockParser, RowNumberAppendedToError) {
}
}
+TEST(BlockParser, EmbeddedNulBytesDisableBulkFilter) {
+ // Regression test for GH-50481: the bulk filter's implicit-length SIMD
+ // compare can miss a delimiter sharing a word with an embedded NUL byte.
+ // The fix disables the bulk filter for any block containing a NUL, so
+ // every cell here carries one. num_cols is explicit so the bulk filter
+ // isn't delayed by the single-row column-count-inference parse path.
+ constexpr int32_t num_cols = 64;
Review Comment:
The test is for checking if there's an embedded NUL byte
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
