rootvector2 opened a new pull request, #618: URL: https://github.com/apache/commons-csv/pull/618
`nextToken` evaluates the side-effecting `isDelimiter(c)` twice on the same character when `ignoreSurroundingSpaces` is on: once in the leading-whitespace skip loop and again in the start-of-token check. `isDelimiter` consumes the trailing characters of a multi-character delimiter as it matches, so for a delimiter whose first character is whitespace the loop eats the delimiter and the second call then peeks past it, mismatches, and the empty field at that boundary is dropped. With delimiter `" |"` and `ignoreSurroundingSpaces` enabled, `a | |b` parsed to two fields (`a`, `" b"`) instead of three (`a`, empty, `b`), and ` |a` to one field (`" a"`) instead of two (empty, `a`). Fixed by evaluating `isDelimiter` once in the whitespace loop and reusing that result for the start-of-token decision. Found while auditing the multi-character delimiter look-ahead against `ignoreSurroundingSpaces`. - [x] Read the [contribution guidelines](CONTRIBUTING.md) for this project. - [ ] Read the [ASF Generative Tooling Guidance](https://www.apache.org/legal/generative-tooling.html) if you use Artificial Intelligence (AI). - [ ] I used AI to create any part of, or all of, this pull request. Which AI tool was used to create this pull request, and to what extent did it contribute? - [x] Run a successful build using the default [Maven](https://maven.apache.org/) goal with `mvn`; that's `mvn` on the command line by itself. - [x] Write unit tests that match behavioral changes, where the tests fail if the changes to the runtime are not applied. This may not always be possible, but it is a best practice. - [x] Write a pull request description that is detailed enough to understand what the pull request does, how, and why. - [x] Each commit in the pull request should have a meaningful subject line and body. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
