[
https://issues.apache.org/jira/browse/IGNITE-2730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15214730#comment-15214730
]
Denis Magda commented on IGNITE-2730:
-------------------------------------
Roman,
Before getting deeper into the details please clarify on the following.
1. In the current design several events of the same type can be added to an
{{IgniteQueue}} of a particular {{IgniteSourceTask}}. The reason is because
remote listeners are triggered on both primary and backup nodes.
If you are ok to implement a gurantee when an even can be missed then in the
body of the remote filter you can check whether it's called on a primary node
and if so the filter will return true and add an event into the queue.
Moreover, I don't see a big reason to have a distributed queue here. We can use
a local listener where the event will be sent in case the remote filter returns
true and from the local listener body the event can be sent directly to Kafka
or can be put into a local plain Java queue that will be polled by some thread.
In case you want to support a stronger guarantee like at least once then there
is no need to filter out events on backup nodes in the remote filter but the
user of Kafka should be ready to have duplicated events coming from Ignite.
2. Why did you decide to use JdkMarshaller? It's not clear from the code.
> Ignite Events Source Streaming to Kafka
> ---------------------------------------
>
> Key: IGNITE-2730
> URL: https://issues.apache.org/jira/browse/IGNITE-2730
> Project: Ignite
> Issue Type: New Feature
> Components: streaming
> Reporter: Roman Shtykh
> Assignee: Roman Shtykh
> Labels: community
>
> Streaming specified Ignite events
> (https://apacheignite.readme.io/docs/events) to Kafka via Kafka Connect.
> It has to be added to org.apache.ignite.stream.kafka.connect package.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
