JingsongLi commented on code in PR #10570:
URL: https://github.com/apache/paimon/pull/10570#discussion_r4375018913
##########
paimon-python/pypaimon/execution/python/write/file_store_write.py:
##########
@@ -220,13 +223,10 @@ def max_seq_number():
default = -1 if self.table.is_primary_key_table else 1
return self._seq_number_stats(partition).get(bucket, default)
- # Dedicated Blob files are an append-table layout. PK tables require
- # managed packs and references attached to their key-value Parquet
files.
- if self._has_blob_columns():
- if self.table.is_primary_key_table:
- raise NotImplementedError(
- 'Primary-key Blob writes require the native writer; '
- 'enable write.native.enabled and write Arrow batches')
+ # Dedicated Blob files are an append-table layout. Primary-key tables
+ # store descriptors in the key-value file and list packs in a .blobref
+ # sidecar written by KeyValueDataWriter.
+ if self._has_blob_columns() and not self.table.is_primary_key_table:
Review Comment:
[P2] Preserve inline BLOB validation when selecting the PK writer
The new primary-key route bypasses
`DedicatedFormatWriter._validate_inline_stored_fields_input`, and
`KeyValueDataWriter` does not perform the equivalent check. For a fresh PK
table with `blob-descriptor-field=payload` or `blob-view-field=payload` and
`write.native.enabled=false`, I wrote an Arrow row containing
`b"not-a-reference"`: both `prepare_commit()` and commit succeeded, but the
normal public read then failed with `Expected BlobDescriptor bytes, got raw
bytes` / `Expected BlobViewStruct bytes`. These fields promise inline
serialized references, and Java rejects raw inline values. Please reuse the
descriptor/view validation in the shared or PK writer before
buffering/persisting, so malformed input cannot commit unreadable data.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
