rdblue commented on code in PR #4519:
URL: https://github.com/apache/iceberg/pull/4519#discussion_r870920405
##########
flink/v1.12/flink/src/main/java/org/apache/iceberg/flink/sink/FlinkSink.java:
##########
@@ -375,8 +375,22 @@ private SingleOutputStreamOperator<WriteResult>
appendWriter(DataStream<RowData>
boolean upsertMode = upsert ||
PropertyUtil.propertyAsBoolean(table.properties(),
UPSERT_ENABLED, UPSERT_ENABLED_DEFAULT);
- // Validate the equality fields and partition fields if we enable the
upsert mode.
+ // `upsert` mode should not be used in Flink 1.12 due to correctness
issues.
+ // As part of a patch release, apachee-iceberg-flink-runtime_1.12:0.13.2,
+ // it has been decided the best course of action would be to log a
warning
+ // asking people to upgrade, as Flink 1.12 is deprecated from both
upstream
+ // Flink as well as Iceberg as of Iceberg 0.14.0. But we allow the
configuration
+ // given that it's a patch release and the change would be otherwise
very breaking.
+ //
+ // See in https://github.com/apache/iceberg/pull/4364 for more
information.
if (upsertMode) {
+ String deprecationNotice =
+ "This job is running in upsert mode. Upsert mode should not be
used with Flink 1.12 due to correctness " +
+ "issues. Please upgrade to Flink 1.13+ if upsert mode is truly
needed. If upsert mode is not needed, " +
+ "set the table property `write.upsert.enabled` to 'false' and
don't use `upsert(true)` while " +
Review Comment:
I think including the part about "if truly needed" isn't necessary. I think
people only use upsert if it is needed. It would simplify the message to remove
that part of it.
Also, can we be more specific about the problem? For example, "Upsert mode
should not be used with Flink 1.12 because it will write incorrect delete file
metadata, which could prevent deletes from being correctly applied. Upgrading
to Flink 1.13+ is recommended. To safely use Flink 1.12, set manifest metrics
to counts only."
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
