AynSiddiqui opened a new pull request, #1621:
URL: https://github.com/apache/arrow-go/pull/1621

   ### Rationale for this change
   
   Follow-up to #1597, covering the optional point @zeroshade raised in his 
review. #1597 keeps page-write errors on the column writer. Other errors from 
`FlushCurrentPage`, such as `FlushValues`, the page size check or 
`getPageStatistics`, were returned without being kept, so a later column 
`Close` could finish the chunk with missing data.
   
   ### What changes are included in this PR?
   
   `FlushCurrentPage` now records its error with `recordError`, as #1597 does 
for page writes. That covers flushes triggered by `WriteBatch`, 
`WriteBatchSpaced`, the bitmap, dictionary-index and Arrow write paths, 
explicit `FlushCurrentPage` calls and `FlushBufferedDataPages`. I did it in 
`FlushCurrentPage` rather than in each recover handler so one change covers the 
template, generated and hand-written writers, and unrelated panics from bad 
input are not kept. Happy to move it if you prefer the handlers. `recordError` 
keeps the first error, so column `Close` returns the first saved error on every 
call.
   
   ### Are these changes tested?
   
   Two tests make the first `FlushValues` fail (once, through `WriteBatch` and 
through an explicit `FlushCurrentPage`). Both check column and file `Close` and 
that zero bytes are appended after the failure. The batch test also checks 
repeated closes, row group `Close` and that no readable footer is written. Both 
fail on main and pass with this change.
   
   ```sh
   go test -short ./parquet/...
   go test -race ./parquet/file ./parquet/pqarrow
   golangci-lint run ./parquet/file/...
   ```
   
   `go generate ./parquet/file` leaves the diff unchanged.
   
   ### Are there any user-facing changes?
   
   After an error returned by `FlushCurrentPage`, column `Close` returns the 
saved error and the file writer does not write the footer. Row group `Close` 
may still report an earlier validation error first.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to