zhoulii opened a new issue, #9412: URL: https://github.com/apache/paimon/issues/9412
### Search before asking - [x] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar. ### Paimon version 2.0 ### Compute Engine spark3.2 ### Minimal reproduce step CREATE TABLE t (id INT, name STRING) TBLPROPERTIES ( 'row-tracking.enabled' = 'true', 'data-evolution.enabled' = 'true' ); -- Keep the historical rows in one normal anchor. INSERT INTO t SELECT /*+ REPARTITION(1) */ * FROM VALUES (1, 'name1'), (2, 'name2'), (3, 'name3') AS v(id, name); -- The historical anchor has no physical baseline for this column. ALTER TABLE t ADD COLUMN picture BINARY COMMENT '__BLOB_FIELD'; -- Create a snapshot using the new schema to avoid the unrelated -- no-new-schema-snapshot planning issue. INSERT INTO t VALUES (4, 'name4', CAST(NULL AS BINARY)); CREATE TABLE s (id INT, picture BINARY); INSERT INTO s VALUES (1, X'4E4557'); MERGE INTO t USING s ON t.id = s.id WHEN MATCHED THEN UPDATE SET t.picture = s.picture; SELECT id, picture FROM t WHERE id <= 3 ORDER BY id; This reproduces with both Spark V1 and V2 write paths. ### What doesn't meet your expectations? The query should return the updated BLOB for row 1 and NULL for the unchanged historical rows: 1, X'4E4557' 2, NULL 3, NULL Instead, it fails while converting an internal placeholder to Spark BINARY: java.lang.UnsupportedOperationException: Should never call this method for placeholder blob. at org.apache.paimon.data.BlobPlaceholder.toData at org.apache.paimon.spark.data.Spark3InternalRowWithBlob.getBinary The MERGE writes a full-range BLOB file for the historical anchor. The updated row contains the new value, while the unchanged rows contain placeholders. Because the file has one sequence group and fully covers the logical range, DataEvolutionSplitRead uses the sequentialReadFiles fast path. This bypasses BlobFallbackRecordReader, so unresolved placeholders are returned directly to Spark. Full physical row-id coverage does not imply that the file is placeholder-free. ### Anything else? Do not bypass placeholder resolution for BlobFileBunch. Remove the sequentialReadOptimize() fast path in DataEvolutionSplitRead and route all BLOB bunches through BlobFallbackRecordReader. BlobFallbackRecordReader already provides the required semantics: - Return the latest non-placeholder value. - Fall back to older sequence groups when necessary. - Return NULL when no physical baseline exists. - Handle BLOB, ARRAY<BLOB>, and MAP<X, BLOB> consistently. ### Are you willing to submit a PR? - [x] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
