hcrosse commented on PR #1571: URL: https://github.com/apache/arrow-go/pull/1571#issuecomment-6925530401
@AynSiddiqui Thanks for taking this. On this branch, my tests show the page after the limit is rejected, but later `Close` calls don't all return the error: - Plain column: the column, row group and file `Close` calls return nil, and the footer claims 32,770 rows while the reader returns 32,768. - `pqarrow`'s `FileWriter.Write` closes the file when a write fails, so its plain columns end up like this even when the caller checks the error. - Dictionary column in a serial row group: the column's `Close` returns the error, but the row group and file close cleanly and the column reads back no values. - Dictionary column in a buffered row group: `Close` returns the error, but 4 MiB stays allocated. I think every later `Close` should return the error, so no footer is written. A test through the column writer would catch this, and I'm happy to share mine. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
