[
https://issues.apache.org/jira/browse/KAFKA-15182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yash Mayya updated KAFKA-15182:
-------------------------------
Description:
See discussion
[here|https://github.com/apache/kafka/pull/13945#discussion_r1260946148]
TLDR: When users attempt to externally modify source connector offsets via the
{{PATCH /offsets}} endpoint (introduced in
[KIP-875|https://cwiki.apache.org/confluence/display/KAFKA/KIP-875%3A+First-class+offsets+support+in+Kafka+Connect]),
type mismatches can occur between offsets passed to
{{SourceConnector::alterOffsets}} and the offsets that are retrieved by
connectors / tasks via an instance of {{OffsetStorageReader}} after the offsets
have been modified. In order to prevent this type mismatch that could lead to
subtle bugs in connectors, we could serialize + deserialize the offsets using
the worker's internal JSON converter before invoking
{{{}SourceConnector::alterOffsets{}}}.
was:
See discussion
[here|https://github.com/apache/kafka/pull/13945#discussion_r1260946148]
TLDR: When users attempt to externally modify source connector offsets via the
{{PATCH /offsets}} endpoint (introduced in
[KIP-875|https://cwiki.apache.org/confluence/display/KAFKA/KIP-875%3A+First-class+offsets+support+in+Kafka+Connect]),
type mismatches can occur between offsets passed to
{{SourceConnector::alterOffsets}} and the offsets that are retrieved by
connectors / tasks via an instance of {{OffsetStorageReader }}after the offsets
have been modified. In order to prevent this type mismatch that could lead to
subtle bugs in connectors, we could serialize + deserialize the offsets using
the worker's internal JSON converter before invoking
{{{}SourceConnector::alterOffsets{}}}.
> Normalize offsets before invoking SourceConnector::alterOffsets
> ---------------------------------------------------------------
>
> Key: KAFKA-15182
> URL: https://issues.apache.org/jira/browse/KAFKA-15182
> Project: Kafka
> Issue Type: Improvement
> Components: KafkaConnect
> Affects Versions: 3.6.0
> Reporter: Yash Mayya
> Assignee: Yash Mayya
> Priority: Major
> Fix For: 3.6.0
>
>
> See discussion
> [here|https://github.com/apache/kafka/pull/13945#discussion_r1260946148]
>
> TLDR: When users attempt to externally modify source connector offsets via
> the {{PATCH /offsets}} endpoint (introduced in
> [KIP-875|https://cwiki.apache.org/confluence/display/KAFKA/KIP-875%3A+First-class+offsets+support+in+Kafka+Connect]),
> type mismatches can occur between offsets passed to
> {{SourceConnector::alterOffsets}} and the offsets that are retrieved by
> connectors / tasks via an instance of {{OffsetStorageReader}} after the
> offsets have been modified. In order to prevent this type mismatch that could
> lead to subtle bugs in connectors, we could serialize + deserialize the
> offsets using the worker's internal JSON converter before invoking
> {{{}SourceConnector::alterOffsets{}}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
