nevi-me edited a comment on pull request #7297: URL: https://github.com/apache/arrow/pull/7297#issuecomment-647578097
@andygrove thanks. I pushed some of what I worked on over the weekend. The main problem seems to be that we don't read all record batches from the Arrow files. As a result we end up getting "no more record batches" errors. I haven't yet been able to figure out why we don't read all batches, the block sizes indicate the correct batch counts, but the reader just fails to read them all 😞 ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
