wombatu-kun opened a new pull request, #8923:
URL: https://github.com/apache/paimon/pull/8923
### Purpose
`SingleFileWriter` and `FormatTableSingleFileWriter` open the output stream
in their constructor, so the file exists on the filesystem before the format
writer is built. The constructor sets `deleteFileUponAbort = true` up front,
with the comment `// true first to clean file in exception`, but the cleanup
only ran from `catch (IOException e)`, and only when `out != null`. Two
problems follow.
A non-IOException failure leaves an orphan file that no snapshot references,
plus an unclosed stream. This is reachable: `file.compression` is not
validated, so an unknown value reaches
`CompressionKind.valueOf(compression.toUpperCase())` in `OrcWriterFactory` and
throws `IllegalArgumentException`, and building a Parquet or ORC writer throws
`UnsupportedOperationException("Unsupported type: ...")` for a type the format
does not support.
The cleanup also called the overridable `abort()` from the base constructor.
`RowDataFileWriter` overrides it and iterates `auxiliaryFileWriters`, a field
assigned after `super(...)`, so the cleanup threw `NullPointerException` out of
the constructor, replaced the real cause, and never reached `super.abort()`,
leaving the file behind anyway. That covers every append, blob and vector data
file, and it happens on the existing `IOException` path as well.
Both constructors now track whether opening completed and clean up from
`finally`. `SingleFileWriter` does so through a private, non-overridable
method, so no subclass state is touched while the super constructor is still
running. The guard is `out != null || writer != null`: when
`newOutputStream(path, false)` refuses to overwrite an existing file, nothing
was created and that file is not ours to delete.
Out of scope: a throw inside `SupportsDirectWrite.create` can still leave a
file behind (Vortex creates it eagerly). That interface states the format has
full control over its file IO, so the base class does not delete it.
### Tests
New `SingleFileWriterTest` (8 cases) and `FormatTableSingleFileWriterTest`
(3 cases). 6 of the 11 fail on master:
- runtime exception while opening, for both `asyncWrite` values: the
original exception surfaces and no file survives
- a subclass whose `abort()` touches state assigned after `super(...)`: the
real failure is not replaced by `NullPointerException`, checked on both the
runtime-exception and the `IOException` path
- failure after only the format writer was created: the writer is closed and
the file deleted
- regression guards: the `IOException` path still wraps into
`UncheckedIOException`, an already existing file survives a failed open with
its content intact, and a successful open leaves the stream open until `close()`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
