wwj6591812 commented on code in PR #8219:
URL: https://github.com/apache/paimon/pull/8219#discussion_r3417638753
##########
paimon-core/src/main/java/org/apache/paimon/append/ExternalStorageBlobWriter.java:
##########
@@ -201,12 +205,14 @@ private static ExternalStorageBlobFieldWriter
createFieldWriter(
FileSource fileSource,
boolean asyncFileWrite,
boolean statsDenseStore,
- long targetFileSize) {
+ long targetFileSize,
+ boolean writeNullOnMissingFile) {
int fieldIndex = writeSchema.getFieldIndex(fieldName);
ExternalStorageBlobFieldWriter fieldWriter = new
ExternalStorageBlobFieldWriter(fieldIndex);
BlobFileFormat blobFileFormat = new BlobFileFormat();
blobFileFormat.setWriteConsumer(fieldWriter);
+ blobFileFormat.setWriteNullOnMissingFile(writeNullOnMissingFile);
Review Comment:
@JingsongLi
Thanks for catching this — you're absolutely right. When the
external-storage copy hit the 404 fallback, BlobConsumer.accept(field, null)
made writeAndReplace return null, and FallbackMappingRow treated that as "no
override", so the inline descriptor field still kept the original HTTP URL
instead of NULL.
I've fixed this by copying the input row first and applying explicit field
outcomes afterward: missing-file fallback now sets an explicit NULL on the
descriptor field, while a successful copy still replaces it with the new
external-storage descriptor. I also added
testExternalStorageMissingHttpBlobWritesNull, which writes an external-storage
descriptor pointing to a missing HTTP URL and verifies the stored field is NULL
on readback.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
