JingsongLi commented on code in PR #10655:
URL: https://github.com/apache/paimon/pull/10655#discussion_r4374710343


##########
paimon-python/pypaimon/execution/python/write/file_store_write.py:
##########
@@ -127,27 +131,32 @@ def write_row(
         require_columns(values_by_name, column_names, 'write_row')
         writer = self.data_writers.get(key)
         data = None
-        if writer is None and self._has_blob_columns():
-            # Dedicated writers consume live payloads themselves. Inline fields
-            # instead encode to Arrow before the writer captures its layout.
-            descriptor_fields = self.options.blob_descriptor_fields()
-            view_fields = self.options.blob_view_fields()
+        if self._has_blob_columns():
+            descriptor_fields = 
CoreOptions.blob_descriptor_fields(self.options)
+            view_fields = CoreOptions.blob_view_fields(self.options)
             inline_columns = [name for name in column_names if name in 
descriptor_fields or name in view_fields]
             if inline_columns:
                 values_by_name = dict(values_by_name)
                 for name in inline_columns:
                     values_by_name[name] = inline_blob_value(
                         values_by_name[name], name in descriptor_fields, name 
in view_fields)

Review Comment:
   [P2] Normalize inline BLOB rows before postpone bucket planning
   
   This normalization is too late for 
`table.new_postpone_fixed_bucket_write_builder().new_write()`. On a fresh PK 
table with `bucket=-2`, `postpone.batch-write-fixed-bucket=true`, 
`blob-descriptor-field=picture`, and `write.native.enabled=false`, writing a 
`GenericRow` containing `Blob.from_descriptor_bytes(...)` to a new partition 
raises `ValueError: Blob values cannot be converted to Arrow without 
materializing the stream`. `PythonPostponeFixedBucketWrite.write_row` first 
calls `row_values_to_arrow_table` at `postpone_batch_table_write.py:132` to 
estimate the unknown partition's size, so it never reaches this inline encoding.
   
   I reproduced this with Python 3.13 / PyArrow 22.0.0. Serialized descriptor 
bytes through Arrow commit and read correctly on the same table; after that 
partition exists, the same Blob-object row succeeds. A fresh ordinary 
fixed-bucket PK table also succeeds. Please share the inline-reference 
normalization with the postpone sizing/buffering path before its Arrow 
conversion, and cover the first write into a new partition for descriptor/view 
fields.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to