[
https://issues.apache.org/jira/browse/FLINK-41511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiabao Sun updated FLINK-41511:
-------------------------------
Description:
Flink 2.3.0 (FLIP-558, FLINK-38926) tightened the planner: a changelog query
whose upsert key differs from the sink's primary key is now rejected unless the
sink DDL declares an ON CONFLICT clause.
{{MongoPartitionedTableSinkITCase.testSinkIntoPartitionedTableWithMutableShardKey}}
fails on Flink 2.3.0 with:
{noformat}
org.apache.flink.table.api.ValidationException: The query has an upsert key
that differs from the primary key of the sink table
'default_catalog.default_database.mongo_sink'. Primary key: [a], upsert key:
none. This can lead to non-deterministic results when multiple records with
different upsert keys map to the same primary key. Please specify an ON
CONFLICT clause to define how conflicts should be handled: ON CONFLICT DO
DEDUPLICATE (update to the latest record, state intensive, since we need to
keep the entire history), or ON CONFLICT DO ERROR (fail on conflict), or ON
CONFLICT DO NOTHING (keep first record).
{noformat}
was:
Flink 2.3.0 (FLIP-558, FLINK-38926) tightened the planner: a changelog query
whose upsert key differs from the sink's primary key is now rejected unless the
sink DDL declares an ON CONFLICT clause.
{{MongoPartitionedTableSinkITCase.testSinkIntoPartitionedTableWithMutableShardKey}}
- the shared base test inherited by every database module's sink ITCase -
fails on Flink 2.3.0 with:
{noformat}
org.apache.flink.table.api.ValidationException: The query has an upsert key
that differs from the primary key of the sink table
'default_catalog.default_database.mongo_sink'. Primary key: [a], upsert key:
none. This can lead to non-deterministic results when multiple records with
different upsert keys map to the same primary key. Please specify an ON
CONFLICT clause to define how conflicts should be handled: ON CONFLICT DO
DEDUPLICATE (update to the latest record, state intensive, since we need to
keep the entire history), or ON CONFLICT DO ERROR (fail on conflict), or ON
CONFLICT DO NOTHING (keep first record).
{noformat}
> MongoDB table sink incompatible with Flink 2.3.0 ON CONFLICT / upsert-key
> validation
> ------------------------------------------------------------------------------------
>
> Key: FLINK-41511
> URL: https://issues.apache.org/jira/browse/FLINK-41511
> Project: Flink
> Issue Type: Bug
> Components: Connectors / MongoDB
> Reporter: Jiabao Sun
> Assignee: Jiabao Sun
> Priority: Major
>
> Flink 2.3.0 (FLIP-558, FLINK-38926) tightened the planner: a changelog query
> whose upsert key differs from the sink's primary key is now rejected unless
> the sink DDL declares an ON CONFLICT clause.
> {{MongoPartitionedTableSinkITCase.testSinkIntoPartitionedTableWithMutableShardKey}}
> fails on Flink 2.3.0 with:
> {noformat}
> org.apache.flink.table.api.ValidationException: The query has an upsert key
> that differs from the primary key of the sink table
> 'default_catalog.default_database.mongo_sink'. Primary key: [a], upsert key:
> none. This can lead to non-deterministic results when multiple records with
> different upsert keys map to the same primary key. Please specify an ON
> CONFLICT clause to define how conflicts should be handled: ON CONFLICT DO
> DEDUPLICATE (update to the latest record, state intensive, since we need to
> keep the entire history), or ON CONFLICT DO ERROR (fail on conflict), or ON
> CONFLICT DO NOTHING (keep first record).
> {noformat}
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)