Hi, On Tue, 1 Sept 2026 at 07:41, Greg Burd <[email protected]> wrote: > > For anyone following along, the mechanism: commit e0a3a3fd53 added > CopyReadLineTextSIMDHelper, which refills the input buffer once fewer > than sizeof(Vector8) bytes remain: > > if (copy_buf_len - input_buf_ptr < sizeof(Vector8)) /* reads ahead */ > > whereas scalar CopyReadLineText only refills when actually empty: > > if (input_buf_ptr >= copy_buf_len || need_data) > > That difference is the issue. CopyConvertBuf() deliberately tolerates an > invalid byte sequence after the \. end-of-copy marker (see the comment > near its convertedlen == 0 block), deferring the error. But > CopyLoadInputBuf() raises it the moment input_reached_error is set. The > SIMD read-ahead reaches that raise before the scalar loop has consumed > \., so a file ending in \.\n<valid bytes><invalid byte> errors out where > the scalar path would have stopped cleanly at \.
Thank you for great explanation! On Mon, 31 Aug 2026 at 20:48, Nazir Bilal Yavuz <[email protected]> wrote: > > That seems like the easiest fix without complicating the code. I will > review this code in more detail tomorrow, but I think adding the > reproducer above as a test case makes sense to me. I re-checked the patch and the fix LGTM. My only comment is adding the reproducer as a test case. -- Regards, Nazir Bilal Yavuz Microsoft
