JingsongLi commented on code in PR #9317:
URL: https://github.com/apache/paimon/pull/9317#discussion_r3819967073
##########
paimon-python/pypaimon/write/writer/dedicated_format_writer.py:
##########
@@ -335,11 +351,7 @@ def abort(self):
blob_writer.abort()
if self.vector_writer is not None:
self.vector_writer.abort()
- committed_non_blob_files = [
- file_meta for file_meta in self.committed_files
- if not DataFileMeta.is_blob_file(file_meta.file_name)
- ]
- self._delete_committed_files(committed_non_blob_files)
+ self._delete_committed_files(self.committed_files)
Review Comment:
Please preserve `BlobConsumer`-owned blob files here. After row-count
rolling, `_close_current_writers` transfers blob metadata to
`self.committed_files` and clears the child writer metadata, so this
unconditional deletion removes blob files even when a consumer has already
received descriptors pointing to them. This differs from the Java
implementation, where `BlobFormatWriter.deleteFileUponAbort()` returns `false`
when a consumer is configured. With `target-file-row-num=3`, writing 7 rows and
then aborting leaves 6 of the 7 emitted descriptors unreadable. Please retain
the per-file delete-on-abort policy when transferring ownership and add a
rolling + consumer + abort regression test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
