[ 
https://issues.apache.org/jira/browse/FLINK-41511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jiabao Sun updated FLINK-41511:
-------------------------------
    Description: 
Flink 2.3.0 (FLIP-558, FLINK-38926) tightened the planner: a changelog query 
whose upsert key differs from the sink's primary key is now rejected unless the 
sink DDL declares an ON CONFLICT clause.

{{MongoPartitionedTableSinkITCase.testSinkIntoPartitionedTableWithMutableShardKey}}
  fails on Flink 2.3.0 with:
{noformat}
org.apache.flink.table.api.ValidationException: The query has an upsert key 
that differs from the primary key of the sink table 
'default_catalog.default_database.mongo_sink'. Primary key: [a], upsert key: 
none. This can lead to non-deterministic results when multiple records with 
different upsert keys map to the same primary key. Please specify an ON 
CONFLICT clause to define how conflicts should be handled: ON CONFLICT DO 
DEDUPLICATE (update to the latest record, state intensive, since we need to 
keep the entire history), or ON CONFLICT DO ERROR (fail on conflict), or ON 
CONFLICT DO NOTHING (keep first record).
{noformat}

  was:
Flink 2.3.0 (FLIP-558, FLINK-38926) tightened the planner: a changelog query 
whose upsert key differs from the sink's primary key is now rejected unless the 
sink DDL declares an ON CONFLICT clause.

{{MongoPartitionedTableSinkITCase.testSinkIntoPartitionedTableWithMutableShardKey}}
 - the shared base test inherited by every database module's sink ITCase - 
fails on Flink 2.3.0 with:
{noformat}
org.apache.flink.table.api.ValidationException: The query has an upsert key 
that differs from the primary key of the sink table 
'default_catalog.default_database.mongo_sink'. Primary key: [a], upsert key: 
none. This can lead to non-deterministic results when multiple records with 
different upsert keys map to the same primary key. Please specify an ON 
CONFLICT clause to define how conflicts should be handled: ON CONFLICT DO 
DEDUPLICATE (update to the latest record, state intensive, since we need to 
keep the entire history), or ON CONFLICT DO ERROR (fail on conflict), or ON 
CONFLICT DO NOTHING (keep first record).
{noformat}


> MongoDB table sink incompatible with Flink 2.3.0 ON CONFLICT / upsert-key 
> validation
> ------------------------------------------------------------------------------------
>
>                 Key: FLINK-41511
>                 URL: https://issues.apache.org/jira/browse/FLINK-41511
>             Project: Flink
>          Issue Type: Bug
>          Components: Connectors / MongoDB
>            Reporter: Jiabao Sun
>            Assignee: Jiabao Sun
>            Priority: Major
>
> Flink 2.3.0 (FLIP-558, FLINK-38926) tightened the planner: a changelog query 
> whose upsert key differs from the sink's primary key is now rejected unless 
> the sink DDL declares an ON CONFLICT clause.
> {{MongoPartitionedTableSinkITCase.testSinkIntoPartitionedTableWithMutableShardKey}}
>   fails on Flink 2.3.0 with:
> {noformat}
> org.apache.flink.table.api.ValidationException: The query has an upsert key 
> that differs from the primary key of the sink table 
> 'default_catalog.default_database.mongo_sink'. Primary key: [a], upsert key: 
> none. This can lead to non-deterministic results when multiple records with 
> different upsert keys map to the same primary key. Please specify an ON 
> CONFLICT clause to define how conflicts should be handled: ON CONFLICT DO 
> DEDUPLICATE (update to the latest record, state intensive, since we need to 
> keep the entire history), or ON CONFLICT DO ERROR (fail on conflict), or ON 
> CONFLICT DO NOTHING (keep first record).
> {noformat}



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to