rootvector2 opened a new pull request, #622: URL: https://github.com/apache/commons-csv/pull/622
`getEncodedCharLength` re-encodes each character with the configured `CharsetEncoder` to count its bytes, but the convenience `encode` call re-emits a byte-order mark on every invocation for charsets like `UTF-16`. Parsing a UTF-16 stream with `setTrackBytes(true)`, for example an Excel Unicode export, counts every code unit as 4 bytes instead of 2, so `CSVRecord.getBytePosition()` grows at twice the true rate. Found by comparing reported byte offsets against the encoded prefix length across charsets. Measure the encoder`s per-call BOM once and subtract it from each character`s length. The count already lives in `getEncodedCharLength`, so keeping the adjustment there covers both the single-char and array read paths, and it stays a no-op for charsets that emit no BOM (`UTF-8`, `UTF-16LE`/`BE`, single-byte). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
