Stephen0421 commented on code in PR #10570:
URL: https://github.com/apache/paimon/pull/10570#discussion_r4376344143
##########
paimon-python/pypaimon/execution/python/write/file_store_write.py:
##########
@@ -220,13 +223,10 @@ def max_seq_number():
default = -1 if self.table.is_primary_key_table else 1
return self._seq_number_stats(partition).get(bucket, default)
- # Dedicated Blob files are an append-table layout. PK tables require
- # managed packs and references attached to their key-value Parquet
files.
- if self._has_blob_columns():
- if self.table.is_primary_key_table:
- raise NotImplementedError(
- 'Primary-key Blob writes require the native writer; '
- 'enable write.native.enabled and write Arrow batches')
+ # Dedicated Blob files are an append-table layout. Primary-key tables
+ # store descriptors in the key-value file and list packs in a .blobref
+ # sidecar written by KeyValueDataWriter.
+ if self._has_blob_columns() and not self.table.is_primary_key_table:
Review Comment:
Thanks, confirmed. The primary-key route skipped the inline reference check
that `DedicatedFormatWriter` performs, so a raw `b"not-a-reference"` value in a
`blob-descriptor-field` / `blob-view-field` could be committed and only fail on
read.
Fixed:
- Moved the descriptor/view validation from
`DedicatedFormatWriter._validate_inline_stored_fields_input` into a shared
`validate_inline_blob_fields` helper in `pypaimon/write/row_utils.py`. The
checks and error messages are unchanged; `DedicatedFormatWriter` now calls the
helper.
- `KeyValueDataWriter.write` validates every incoming batch before it is
buffered, so malformed inline values are rejected at `write_arrow` time and
nothing is persisted. `PostponeDataWriter` goes through the same `write` path.
- Added
`ManagedBlobPrimaryKeyWriteTest.test_inline_blob_fields_reject_raw_payloads`.
It covers both `blob-descriptor-field` and `blob-view-field` on a PK table with
the native writer disabled. Writing `b"not-a-reference"` now raises a
`ValueError` naming the expected `BlobDescriptor` / `BlobViewStruct`. The
following `prepare_commit()` returns no files, no snapshot is created, and no
data file is left in the table directory. Both cases fail without the writer
change.
The existing append-table inline validation tests still pass.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
