efegokdemir opened a new pull request, #11709: URL: https://github.com/apache/arrow-rs/pull/11709
# Which issue does this PR close? - Closes #11579 # Rationale for this change CSV fields are concatenated before UTF-8 validation, so bytes separated by a delimiter or row terminator can form a valid code point together even when the individual fields are invalid. The reader could then construct invalid `&str` values through unchecked field slicing. # What changes are included in this PR? The decoder now checks field offsets are UTF-8 character boundaries and returns the existing line/field error if a boundary splits a code point. Regression tests cover delimiters, row terminators, error locations, and both `Utf8` and `Utf8View`. # Are these changes tested? The regression failed before the fix and passes now. `cargo test -p arrow-csv` passed (95 unit tests, 14 doctests); formatting, Clippy and typos checks passed. Clippy reports an existing unknown `clippy::assert_is_empty` lint from the workspace config. AI assistance was used to analyse the issue and generate the implementation and regression tests. # Are there any significant user-facing changes? Malformed CSV that could previously produce invalid string fields now returns a `CsvError`. There is no API change. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
