hcrosse commented on PR #1571:
URL: https://github.com/apache/arrow-go/pull/1571#issuecomment-6925530401

   @AynSiddiqui Thanks for taking this. On this branch, my tests show the page 
after the limit is rejected, but later `Close` calls don't all return the error:
   
   - Plain column: the column, row group and file `Close` calls return nil, and 
the footer claims 32,770 rows while the reader returns 32,768.
   - `pqarrow`'s `FileWriter.Write` closes the file when a write fails, so its 
plain columns end up like this even when the caller checks the error.
   - Dictionary column in a serial row group: the column's `Close` returns the 
error, but the row group and file close cleanly and the column reads back no 
values.
   - Dictionary column in a buffered row group: `Close` returns the error, but 
4 MiB stays allocated.
   
   I think every later `Close` should return the error, so no footer is 
written. A test through the column writer would catch this, and I'm happy to 
share mine.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to