jpfeuffer commented on issue #43604: URL: https://github.com/apache/arrow/issues/43604#issuecomment-2286200684
> > By the way, I just saw that even if I do not limit the chunks (and the process succeeds without segfault), the written file is always corrupt statrting from a few thousand lines in, with lines being mixed together etc. > > That's not something I see with my reproducer above, I think (with mixed together, you mean that there are newline markers missing?) Mmh I am not sure if it was 'just' missing newlines. It felt like it was even more garbled as if the concurrent streams are not flushed at the right time. It was about after 1500 rows. I think if you do a few chunks and then tail the file you should see it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
