Hello, Alexey.
Sorry, for the long answer.
> - The interface exposes WALRecord which is a private API
Not it's fixed.
CDC consumer should use public API to get notifications about a data change.
This API can be found in IEP [1] and PR [2]
```
@IgniteExperimental
public interface
Alexey,
>> If a CDC agent is restarted, it will have to start from scratch
>> If a CDC reader does not keep up with the WAL write rate (e.g. there
is a short-term write burst and WAL archive is small), the Ignite node
will
delete WAL segments while the consumer is still reading it.
I think
Hello Nikolay,
Thanks for the suggestion, it definitely may be a good feature, however, I
do not see any significant value that it currently adds to the already
existing WAL Iterator. I think the following issues should be addressed,
otherwise, no regular user will be able to use the CDC
Hi,
I definitely agree with Pavel and Evgenii ideas and comments.
>From my point of view the proposal is not about Apache Ignite
features. Described functionality could be implemented outside of the
Apache Ignite project. Perhaps, Debezium connector or WAL-G module are
the best candidates for
Hi,
>On the segment archiving, utility iterates it using existing WALIterator
>Wait and respond to some specific events or data changes.
It seems like this solution will have an unpredictable delay for
synchronization for handling events.
Why can't we just implement a Debezium connector for
Hell. Pavel.
Thanks for the feedback.
> But what do you think to integrate these ideas into WAL-G project?
Looked into WAL-G description and look like it’s use-case are more restricted
then CDC itself.
Self-definition - "WAL-G is an archival restoration tool for Postgres(beta for
MySQL,
This tool is also can be used to store snapshots in an external warehouse.
ср, 14 окт. 2020 г. в 14:57, Pavel Kovalenko :
> Hi Nikolay,
>
> The idea is good. But what do you think to integrate these ideas into
> WAL-G project?
> https://github.com/wal-g/wal-g
> It's a well-known tool that is
Hi Nikolay,
The idea is good. But what do you think to integrate these ideas into WAL-G
project?
https://github.com/wal-g/wal-g
It's a well-known tool that is already used to stream WAL for PostgreSQL,
MySQL, and MongoDB.
The advantages are integration with S3, GCP, Azure out of the box,
Hello, Igniters.
I want to start a discussion of the new feature [1]
CDC - capture data change. The feature allows the consumer to receive online
notifications about data record changes.
It can be used in the following scenarios:
* Export data into some warehouse, full-text search, or
