[
https://issues.apache.org/jira/browse/IGNITE-22530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ilya Shishkov updated IGNITE-22530:
-----------------------------------
Description:
*Motivation:*
Currently, we can specify cache names _only explicitly as list_ in
configuration files. But, if we create new cache, it will not be processed by
CDC, until cluster and CDC applications are not restarted with updated
configuration.
For example, if we need all caches to be replicated, it seems that most
convenient way to do so, is to specify some kind of asterisk or regex filter.
*Proposed solution:*
# Add regex filters to all CDC classes with configured cache names: CdcConsumer
implementations, ConflictResolver implementations and KafkaToIgnite streamers.
# Current way to specify caches list must remain available in order to preserve
compatibility.
# Optional exclude filter also should be implemented.
# Multiple include and exclude filters should be implemented.
# It should be compatible with realtime CDC.
was:
We need metrics for received events count and last received event timestamp for
the {{KafkaToIgniteStreamer}}, because there can be situations, when events
sent by {{IgniteToKafkaStreamer}} are lost in Kafka due to Kafka failures or
topic's retention configuration.
In such situations we should be able to compare to sent messages count on
source cluster with received messages count on destination one. Timestamp of
last received event would be also useful.
> CDC: Add regex filters for cache names
> --------------------------------------
>
> Key: IGNITE-22530
> URL: https://issues.apache.org/jira/browse/IGNITE-22530
> Project: Ignite
> Issue Type: New Feature
> Components: extensions
> Reporter: Ilya Shishkov
> Priority: Major
> Labels: IEP-59, ise
>
> *Motivation:*
> Currently, we can specify cache names _only explicitly as list_ in
> configuration files. But, if we create new cache, it will not be processed by
> CDC, until cluster and CDC applications are not restarted with updated
> configuration.
> For example, if we need all caches to be replicated, it seems that most
> convenient way to do so, is to specify some kind of asterisk or regex filter.
> *Proposed solution:*
> # Add regex filters to all CDC classes with configured cache names:
> CdcConsumer implementations, ConflictResolver implementations and
> KafkaToIgnite streamers.
> # Current way to specify caches list must remain available in order to
> preserve compatibility.
> # Optional exclude filter also should be implemented.
> # Multiple include and exclude filters should be implemented.
> # It should be compatible with realtime CDC.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
