[
https://issues.apache.org/jira/browse/IGNITE-18992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ilya Shishkov updated IGNITE-18992:
-----------------------------------
Description:
In order to read data with guaranteed order, metadata topic must have only one
partition. To achieve this we should:
* Write binary types and marshaller mappings only to a single partition in
{{IgniteToKafkaCdcStreamer#onMappings}} and {{#onTypes}}.
* Print warnings into logs, when topic has more than one partition.
* Topic size expected to have fixed size, su we can use assign instead of
subscribe, but {{KafkaConsumer}} in {{KafkaToIgniteMetadataUpdater}} must be
assigned to all partitions.
was:
In order to read data with guaranteed order, metadata topic must have only one
partition. To achieve this we should:
* Write binary types and marshaller mappings only to a single partition in
{{IgniteToKafkaCdcStreamer#onMappings}} and {{#onTypes}}.
* Print warnings into logs, when topic has more than one partition.
* Because topic size is expected we can use assign instead of subscribe, but
{{KafkaConsumer}} in {{KafkaToIgniteMetadataUpdater}} must be assigned to all
partitions.
> CDC: Use single partition topic for metadata updates
> -----------------------------------------------------
>
> Key: IGNITE-18992
> URL: https://issues.apache.org/jira/browse/IGNITE-18992
> Project: Ignite
> Issue Type: Task
> Components: extensions
> Reporter: Ilya Shishkov
> Priority: Major
> Labels: IEP-59, ise
>
> In order to read data with guaranteed order, metadata topic must have only
> one partition. To achieve this we should:
> * Write binary types and marshaller mappings only to a single partition in
> {{IgniteToKafkaCdcStreamer#onMappings}} and {{#onTypes}}.
> * Print warnings into logs, when topic has more than one partition.
> * Topic size expected to have fixed size, su we can use assign instead of
> subscribe, but {{KafkaConsumer}} in {{KafkaToIgniteMetadataUpdater}} must be
> assigned to all partitions.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
