Re: Multi DC replication between different Cassandra versions
> > If you're upgrading the whole cluster, I'd recommend going ahead and > upgrading all the way to 3.11.6 if possible. In my experience it's been > noticeably faster, more reliable, and easier to manage compared to 3.0.x. > Thanks, Elliott. That's really good to know.
Re: Multi DC replication between different Cassandra versions
If you're upgrading the whole cluster, I'd recommend going ahead and upgrading all the way to 3.11.6 if possible. In my experience it's been noticeably faster, more reliable, and easier to manage compared to 3.0.x. On Thu, Apr 16, 2020 at 6:37 PM Ashika Umagiliya wrote: > Thank you for the clarifications, > > If this is not recommended, our last resort is to upgrade the entire > cluster. > > About Kafka Connect, we sound following Source Connectors which can be > used to Ingest data from C* to Kafka . > > https://debezium.io/documentation/reference/connectors/cassandra.html > https://docs.lenses.io/2.0/connectors/source/cassandra-cdc.html > https://docs.lenses.io/2.0/connectors/source/cassandra.html > > https://www.datastax.com/press-release/datastax-announces-change-data-capture-cdc-connector-apache-kafka > > > > > On Thu, Apr 16, 2020 at 9:42 PM Durity, Sean R < > sean_r_dur...@homedepot.com> wrote: > >> I agree – do not aim for a mixed version as normal. Mixed versions are >> fine during an upgrade process, but the goal is to complete the upgrade as >> soon as possible. >> >> >> >> As for other parts of your plan, the Kafka Connector is a “sink-only,” >> which means that it can only insert into Cassandra. It doesn’t go the other >> way. >> >> >> >> I usually suggest that if the data is needed in two (or more) places, >> that the application write to a queue. Then, let the queue feed all the >> downstream destinations. >> >> >> >> >> >> Sean Durity – Staff Systems Engineer, Cassandra >> >> >> >> *From:* Christopher Bradford >> *Sent:* Thursday, April 16, 2020 1:13 AM >> *To:* user@cassandra.apache.org >> *Subject:* [EXTERNAL] Re: Multi DC replication between different >> Cassandra versions >> >> >> >> It’s worth noting there can be issues with streaming between different >> versions of C*. Note this excerpt from >> >> https://thelastpickle.com/blog/2019/02/26/data-center-switch.html >> [thelastpickle.com] >> <https://urldefense.com/v3/__https:/thelastpickle.com/blog/2019/02/26/data-center-switch.html__;!!M-nmYVHPHQ!cPTqPpkLv1YvwMse4ApY-5I_2FOybuD6fqw3uE0fYHJnJAmMKgcpysJMQInae06wWpM26pA$> >> >> >> >> >> Note that with an upgrade it’s important to keep in mind that *streaming >> in a cluster running mixed versions of Casandra is not recommended* >> >> >> >> Emphasis mine. With the approach you’re suggesting streaming would be >> involved both during bootstrap and repair. Would it be possible to upgrade >> to a more recent release prior to pursuing this course of action? >> >> >> >> On Thu, Apr 16, 2020 at 1:02 AM Erick Ramirez >> wrote: >> >> I don't mean any disrespect but let me offer you a friendly advice -- >> don't do it to yourself. I think you would have a very hard time finding >> someone who would recommend implementing a solution that involves mixed >> versions. If you run into issues, it would be hell trying to unscramble >> that egg. >> >> >> >> On top of that, Cassandra 3.0.9 is an ancient version released 4 years >> ago (September 2016). There are several pages of fixes deployed since then. >> So in the nicest possible way, what you're planning to do is not a good >> idea. I personally wouldn't do it. Cheers! >> >> -- >> >> >> Christopher Bradford >> >> >> >> -- >> >> The information in this Internet Email is confidential and may be legally >> privileged. It is intended solely for the addressee. Access to this Email >> by anyone else is unauthorized. If you are not the intended recipient, any >> disclosure, copying, distribution or any action taken or omitted to be >> taken in reliance on it, is prohibited and may be unlawful. When addressed >> to our clients any opinions or advice contained in this Email are subject >> to the terms and conditions expressed in any applicable governing The Home >> Depot terms of business or client engagement letter. The Home Depot >> disclaims all responsibility and liability for the accuracy and content of >> this attachment and for any damages or losses arising from any >> inaccuracies, errors, viruses, e.g., worms, trojan horses, etc., or other >> items of a destructive nature, which may be contained in this attachment >> and shall not be liable for direct, indirect, consequential or special >> damages in connection with this e-mail message or its attachment. >> >
Re: Multi DC replication between different Cassandra versions
Thank you for the clarifications, If this is not recommended, our last resort is to upgrade the entire cluster. About Kafka Connect, we sound following Source Connectors which can be used to Ingest data from C* to Kafka . https://debezium.io/documentation/reference/connectors/cassandra.html https://docs.lenses.io/2.0/connectors/source/cassandra-cdc.html https://docs.lenses.io/2.0/connectors/source/cassandra.html https://www.datastax.com/press-release/datastax-announces-change-data-capture-cdc-connector-apache-kafka On Thu, Apr 16, 2020 at 9:42 PM Durity, Sean R wrote: > I agree – do not aim for a mixed version as normal. Mixed versions are > fine during an upgrade process, but the goal is to complete the upgrade as > soon as possible. > > > > As for other parts of your plan, the Kafka Connector is a “sink-only,” > which means that it can only insert into Cassandra. It doesn’t go the other > way. > > > > I usually suggest that if the data is needed in two (or more) places, that > the application write to a queue. Then, let the queue feed all the > downstream destinations. > > > > > > Sean Durity – Staff Systems Engineer, Cassandra > > > > *From:* Christopher Bradford > *Sent:* Thursday, April 16, 2020 1:13 AM > *To:* user@cassandra.apache.org > *Subject:* [EXTERNAL] Re: Multi DC replication between different > Cassandra versions > > > > It’s worth noting there can be issues with streaming between different > versions of C*. Note this excerpt from > > https://thelastpickle.com/blog/2019/02/26/data-center-switch.html > [thelastpickle.com] > <https://urldefense.com/v3/__https:/thelastpickle.com/blog/2019/02/26/data-center-switch.html__;!!M-nmYVHPHQ!cPTqPpkLv1YvwMse4ApY-5I_2FOybuD6fqw3uE0fYHJnJAmMKgcpysJMQInae06wWpM26pA$> > > > > > Note that with an upgrade it’s important to keep in mind that *streaming > in a cluster running mixed versions of Casandra is not recommended* > > > > Emphasis mine. With the approach you’re suggesting streaming would be > involved both during bootstrap and repair. Would it be possible to upgrade > to a more recent release prior to pursuing this course of action? > > > > On Thu, Apr 16, 2020 at 1:02 AM Erick Ramirez > wrote: > > I don't mean any disrespect but let me offer you a friendly advice -- > don't do it to yourself. I think you would have a very hard time finding > someone who would recommend implementing a solution that involves mixed > versions. If you run into issues, it would be hell trying to unscramble > that egg. > > > > On top of that, Cassandra 3.0.9 is an ancient version released 4 years ago > (September 2016). There are several pages of fixes deployed since then. So > in the nicest possible way, what you're planning to do is not a good idea. > I personally wouldn't do it. Cheers! > > -- > > > Christopher Bradford > > > > -- > > The information in this Internet Email is confidential and may be legally > privileged. It is intended solely for the addressee. Access to this Email > by anyone else is unauthorized. If you are not the intended recipient, any > disclosure, copying, distribution or any action taken or omitted to be > taken in reliance on it, is prohibited and may be unlawful. When addressed > to our clients any opinions or advice contained in this Email are subject > to the terms and conditions expressed in any applicable governing The Home > Depot terms of business or client engagement letter. The Home Depot > disclaims all responsibility and liability for the accuracy and content of > this attachment and for any damages or losses arising from any > inaccuracies, errors, viruses, e.g., worms, trojan horses, etc., or other > items of a destructive nature, which may be contained in this attachment > and shall not be liable for direct, indirect, consequential or special > damages in connection with this e-mail message or its attachment. >
RE: Multi DC replication between different Cassandra versions
I agree – do not aim for a mixed version as normal. Mixed versions are fine during an upgrade process, but the goal is to complete the upgrade as soon as possible. As for other parts of your plan, the Kafka Connector is a “sink-only,” which means that it can only insert into Cassandra. It doesn’t go the other way. I usually suggest that if the data is needed in two (or more) places, that the application write to a queue. Then, let the queue feed all the downstream destinations. Sean Durity – Staff Systems Engineer, Cassandra From: Christopher Bradford Sent: Thursday, April 16, 2020 1:13 AM To: user@cassandra.apache.org Subject: [EXTERNAL] Re: Multi DC replication between different Cassandra versions It’s worth noting there can be issues with streaming between different versions of C*. Note this excerpt from https://thelastpickle.com/blog/2019/02/26/data-center-switch.html [thelastpickle.com]<https://urldefense.com/v3/__https:/thelastpickle.com/blog/2019/02/26/data-center-switch.html__;!!M-nmYVHPHQ!cPTqPpkLv1YvwMse4ApY-5I_2FOybuD6fqw3uE0fYHJnJAmMKgcpysJMQInae06wWpM26pA$> Note that with an upgrade it’s important to keep in mind that streaming in a cluster running mixed versions of Casandra is not recommended Emphasis mine. With the approach you’re suggesting streaming would be involved both during bootstrap and repair. Would it be possible to upgrade to a more recent release prior to pursuing this course of action? On Thu, Apr 16, 2020 at 1:02 AM Erick Ramirez mailto:erick.rami...@datastax.com>> wrote: I don't mean any disrespect but let me offer you a friendly advice -- don't do it to yourself. I think you would have a very hard time finding someone who would recommend implementing a solution that involves mixed versions. If you run into issues, it would be hell trying to unscramble that egg. On top of that, Cassandra 3.0.9 is an ancient version released 4 years ago (September 2016). There are several pages of fixes deployed since then. So in the nicest possible way, what you're planning to do is not a good idea. I personally wouldn't do it. Cheers! -- Christopher Bradford The information in this Internet Email is confidential and may be legally privileged. It is intended solely for the addressee. Access to this Email by anyone else is unauthorized. If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited and may be unlawful. When addressed to our clients any opinions or advice contained in this Email are subject to the terms and conditions expressed in any applicable governing The Home Depot terms of business or client engagement letter. The Home Depot disclaims all responsibility and liability for the accuracy and content of this attachment and for any damages or losses arising from any inaccuracies, errors, viruses, e.g., worms, trojan horses, etc., or other items of a destructive nature, which may be contained in this attachment and shall not be liable for direct, indirect, consequential or special damages in connection with this e-mail message or its attachment.
Re: Multi DC replication between different Cassandra versions
It’s worth noting there can be issues with streaming between different versions of C*. Note this excerpt from https://thelastpickle.com/blog/2019/02/26/data-center-switch.html Note that with an upgrade it’s important to keep in mind that *streaming in a cluster running mixed versions of Casandra is not recommended* Emphasis mine. With the approach you’re suggesting streaming would be involved both during bootstrap and repair. Would it be possible to upgrade to a more recent release prior to pursuing this course of action? On Thu, Apr 16, 2020 at 1:02 AM Erick Ramirez wrote: > I don't mean any disrespect but let me offer you a friendly advice -- > don't do it to yourself. I think you would have a very hard time finding > someone who would recommend implementing a solution that involves mixed > versions. If you run into issues, it would be hell trying to unscramble > that egg. > > On top of that, Cassandra 3.0.9 is an ancient version released 4 years ago > (September 2016). There are several pages of fixes deployed since then. So > in the nicest possible way, what you're planning to do is not a good idea. > I personally wouldn't do it. Cheers! > -- Christopher Bradford
Re: Multi DC replication between different Cassandra versions
I don't mean any disrespect but let me offer you a friendly advice -- don't do it to yourself. I think you would have a very hard time finding someone who would recommend implementing a solution that involves mixed versions. If you run into issues, it would be hell trying to unscramble that egg. On top of that, Cassandra 3.0.9 is an ancient version released 4 years ago (September 2016). There are several pages of fixes deployed since then. So in the nicest possible way, what you're planning to do is not a good idea. I personally wouldn't do it. Cheers!
Multi DC replication between different Cassandra versions
Greetings, We have an existing Cassandra cluster (3.0.9) running on production. Now ,we want to create data pipelines to ingest data from Cassandra and persist in hadoop. We are thinking of using CDC feature (available from Cassandra 3.8) along with Kafka Connect. We are thinking of creating a new read-only DC which will replicate data from the Production DC.This new DC will be running the latest Cassandra version (3.8+) with CDC enabled. Shown in this diagram : https://i.stack.imgur.com/hkGBh.png My questions: 1. For replication to work, do we need both dc's running same version of Cassandra? Can't we achieve this without upgrading the DC used by the service? 2. Is it possible to enable CDC feature only in the new read-only DC?