Re: CDC Tools

[Johnny Miller]

Dor - that looks very useful. Looking forward to trying the CDC Kafka
connector!

On Thu, 28 May 2020 at 02:53, Dor Laor  wrote:

> If it's helpful, IMO, the approach Cassandra needs to take isn't
> by tracking the individual node commit log and putting the burden
> on the client. At Scylla, we had the 'opportunity' to be a late comer
> and see what approach Cassadnra took and what DynamoDB streams
> took.
>
> We've implemented CDC as a regular CQL table [1].
> Not only it's super easy to consume, it's also consistent and
> you can choose to read the older values.
>
> I recommend Cassandra should pick up our design, a small
> contribution back. We're implementing an OSS kafka CDCk connector
> too.
>
> Dor
>
> [1]
> https://www.scylladb.com/2020/03/31/observing-data-changes-with-change-data-capture-cdc/
>
> On Wed, May 27, 2020 at 5:41 PM Erick Ramirez 
> wrote:
>
>> I have looked at DataStax CDC but I think it works only for DSE !
>>>
>>
>> Yes, thanks for the correction.  I just got confirmation myself -- the
>> Kafka-Cassandra connector works with OSS C* but the CDC connector relies on
>> a DSE feature that's not yet available in OSS C*. Cheers!
>>
>

-- 



--

The information contained in this electronic message and any 
attachments to this message are intended for the exclusive use of the 
addressee(s) and may contain proprietary, confidential or privileged 
information. If you are not the intended recipient, you should not 
disseminate, distribute or copy this e-mail. Please notify the sender 
immediately and destroy all copies of this message and any attachments. 
WARNING: Computer viruses can be transmitted via email. The recipient 
should check this email and any attachments for the presence of viruses. 
The company accepts no liability for any damage caused by any virus 
transmitted by this email. www.digitalis.io 

Re: CDC Tools

[Dor Laor]

If it's helpful, IMO, the approach Cassandra needs to take isn't
by tracking the individual node commit log and putting the burden
on the client. At Scylla, we had the 'opportunity' to be a late comer
and see what approach Cassadnra took and what DynamoDB streams
took.

We've implemented CDC as a regular CQL table [1].
Not only it's super easy to consume, it's also consistent and
you can choose to read the older values.

I recommend Cassandra should pick up our design, a small
contribution back. We're implementing an OSS kafka CDCk connector
too.

Dor

[1]
https://www.scylladb.com/2020/03/31/observing-data-changes-with-change-data-capture-cdc/

On Wed, May 27, 2020 at 5:41 PM Erick Ramirez 
wrote:

> I have looked at DataStax CDC but I think it works only for DSE !
>>
>
> Yes, thanks for the correction.  I just got confirmation myself -- the
> Kafka-Cassandra connector works with OSS C* but the CDC connector relies on
> a DSE feature that's not yet available in OSS C*. Cheers!
>

Re: CDC Tools

[Erick Ramirez]

>
> I have looked at DataStax CDC but I think it works only for DSE !
>

Yes, thanks for the correction.  I just got confirmation myself -- the
Kafka-Cassandra connector works with OSS C* but the CDC connector relies on
a DSE feature that's not yet available in OSS C*. Cheers!

Re: CDC Tools

[Ahmed Eljami]

Hi Erick,
I have looked at DataStax CDC but I think it works only for DSE !
Maybe it will be open-sourced like other drivers :)

Cheers.

Le mer. 27 mai 2020 à 04:42, Erick Ramirez  a
écrit :

> I think I recall seeing you post the same question on ASF Slack yesterday.
>
> I haven't used Debezium so I can't comment on it. But are you looking for
> something to consume the CDC logs? If so, you might be interested in the
> DataStax CDC Kafka connector that offers bi-directional data movement.
>
> For details, check out this blog post from Christopher Bradford [1] and
> this Kafka connector resources page [2]. Cheers!
>
> [1]
> https://www.datastax.com/blog/2019/09/datastax-cdc-connector-apache-kafkatm-enables-globally-replicated-microservices
> [2] https://www.datastax.com/dev/datastax-apache-kafka-connector
>
>>

-- 
Cordialement;

Ahmed ELJAMI

Re: CDC Tools

[Erick Ramirez]

I think I recall seeing you post the same question on ASF Slack yesterday.

I haven't used Debezium so I can't comment on it. But are you looking for
something to consume the CDC logs? If so, you might be interested in the
DataStax CDC Kafka connector that offers bi-directional data movement.

For details, check out this blog post from Christopher Bradford [1] and
this Kafka connector resources page [2]. Cheers!

[1]
https://www.datastax.com/blog/2019/09/datastax-cdc-connector-apache-kafkatm-enables-globally-replicated-microservices
[2] https://www.datastax.com/dev/datastax-apache-kafka-connector

>

Re: CDC Tools

[guo Maxwell]

I have found some project that support parse Commitlog (CDC), such as :
https://github.com/rustyrazorblade/commitlog-viz  this seems wrote by Jon
Haddad , but is  not work yet. and  commitlog extract tool :
https://github.com/carloscm/cassandra-commitlog-extract but it also do some
output to other software. the Debezium I have not yet saw the feature .


Ahmed Eljami  于2020年5月26日周二 下午8:59写道：

> Hi guys,
> I'm looking for a tool that helps me to parse CommitLog (CDC).
>
> I found Debezium https://debezium.io/documentation/reference/1.2/ and I
> want to know  if someone has used it or if you could advise me other
> solutions?
>
> Cheers.
>


-- 
you are the apple of my eye !