kumarpritam863 commented on code in PR #18739:
URL: https://github.com/apache/iceberg/pull/18739#discussion_r4520093247
##########
flink/v2.3/flink/src/main/java/org/apache/iceberg/flink/sink/dynamic/DataConverter.java:
##########
@@ -134,6 +139,15 @@ static DataConverter nullable(DataConverter converter) {
return value -> value == null ? null : converter.convert(value);
}
+ static DataConverter nonNullable(DataConverter converter, String fieldName) {
+ return value -> {
+ if (value == null) {
+ throw new IllegalArgumentException("Null value for non-nullable field:
" + fieldName);
Review Comment:
You're right: for upsert, BaseDeltaTaskWriter only calls deleteKey for
DELETE and ignores UPDATE_BEFORE. A tombstone with a null non-key column failed
the job with this change, whereas main deletes the row. The required-column
check now only applies to INSERT and UPDATE_AFTER rows. Tests:
TestRowDataConverter#nullValueForRequiredFieldAllowedForRowsThatOnlyDelete,
TestDynamicRecordProcessor#doesNotDivertRowThatOnlyDeletes and
TestDynamicTableUpdateOperator#upsertDeleteWithoutRequiredColumnsIsNotDiverted.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
