Re: Compaction resume after node restart and upgrade

[Nitan Kainth]

Thanks Jeff.

On Mon, Jun 17, 2019 at 8:08 PM Jeff Jirsa  wrote:

> Yes  - the incomplete sstable will be deleted during startup (in 3.0 and
> newer there’s a transaction log of each compaction in progress - that gets
> cleaned during the startup process)
>
> On Jun 17, 2019, at 6:02 PM, Nitan Kainth  wrote:
>
> The interim/temp sstable will delete by itself?
>
>
> Regards,
>
> Nitan
>
> Cell: 510 449 9629
>
> On Jun 17, 2019, at 7:15 PM, Jeff Jirsa  wrote:
>
>
>
> On Jun 17, 2019, at 4:22 PM, Nitan Kainth  wrote:
>
> Hi,
>
> C* version 3.0.16.
> Two questions:
> 1. If major compaction is in progress and we restart the node, will
> compaction start from beginning or will it resume from where the node was
> shut down?
>
>
> You’ll have to restart it manually , and it’ll restart from the beginning
>
>
> 2. If we upgrade binaries to C* 4.0 while compaction is running, will it
> resume the compaction or start over or no compaction would kick in?
>
>
> Normal compactions will start over on restart but major compaction would
> need to be run again manually
>
>

Re: Compaction resume after node restart and upgrade

[Jeff Jirsa]

Yes  - the incomplete sstable will be deleted during startup (in 3.0 and newer 
there’s a transaction log of each compaction in progress - that gets cleaned 
during the startup process)

> On Jun 17, 2019, at 6:02 PM, Nitan Kainth  wrote:
> 
> The interim/temp sstable will delete by itself?
> 
> 
> Regards,
> Nitan
> Cell: 510 449 9629
> 
>> On Jun 17, 2019, at 7:15 PM, Jeff Jirsa  wrote:
>> 
>> 
>> 
>>> On Jun 17, 2019, at 4:22 PM, Nitan Kainth  wrote:
>>> 
>>> Hi,
>>> 
>>> C* version 3.0.16.
>>> Two questions:
>>> 1. If major compaction is in progress and we restart the node, will 
>>> compaction start from beginning or will it resume from where the node was 
>>> shut down?
>> 
>> You’ll have to restart it manually , and it’ll restart from the beginning
>> 
>> 
>>> 2. If we upgrade binaries to C* 4.0 while compaction is running, will it 
>>> resume the compaction or start over or no compaction would kick in?
>>> 
>> 
>> Normal compactions will start over on restart but major compaction would 
>> need to be run again manually 

Re: Compaction resume after node restart and upgrade

[Nitan Kainth]

The interim/temp sstable will delete by itself?


Regards,
Nitan
Cell: 510 449 9629

> On Jun 17, 2019, at 7:15 PM, Jeff Jirsa  wrote:
> 
> 
> 
>> On Jun 17, 2019, at 4:22 PM, Nitan Kainth  wrote:
>> 
>> Hi,
>> 
>> C* version 3.0.16.
>> Two questions:
>> 1. If major compaction is in progress and we restart the node, will 
>> compaction start from beginning or will it resume from where the node was 
>> shut down?
> 
> You’ll have to restart it manually , and it’ll restart from the beginning
> 
> 
>> 2. If we upgrade binaries to C* 4.0 while compaction is running, will it 
>> resume the compaction or start over or no compaction would kick in?
>> 
> 
> Normal compactions will start over on restart but major compaction would need 
> to be run again manually 

Re: Compaction resume after node restart and upgrade

[Nitan Kainth]

Thanks Jeff!


Regards,
Nitan
Cell: 510 449 9629

> On Jun 17, 2019, at 7:15 PM, Jeff Jirsa  wrote:
> 
> 
> 
>> On Jun 17, 2019, at 4:22 PM, Nitan Kainth  wrote:
>> 
>> Hi,
>> 
>> C* version 3.0.16.
>> Two questions:
>> 1. If major compaction is in progress and we restart the node, will 
>> compaction start from beginning or will it resume from where the node was 
>> shut down?
> 
> You’ll have to restart it manually , and it’ll restart from the beginning
> 
> 
>> 2. If we upgrade binaries to C* 4.0 while compaction is running, will it 
>> resume the compaction or start over or no compaction would kick in?
>> 
> 
> Normal compactions will start over on restart but major compaction would need 
> to be run again manually 

Re: Compaction resume after node restart and upgrade

[Jeff Jirsa]

> On Jun 17, 2019, at 4:22 PM, Nitan Kainth  wrote:
> 
> Hi,
> 
> C* version 3.0.16.
> Two questions:
> 1. If major compaction is in progress and we restart the node, will 
> compaction start from beginning or will it resume from where the node was 
> shut down?

You’ll have to restart it manually , and it’ll restart from the beginning


> 2. If we upgrade binaries to C* 4.0 while compaction is running, will it 
> resume the compaction or start over or no compaction would kick in?
> 

Normal compactions will start over on restart but major compaction would need 
to be run again manually 

Compaction resume after node restart and upgrade

[Nitan Kainth]

Hi,

C* version 3.0.16.
Two questions:
1. If major compaction is in progress and we restart the node, will
compaction start from beginning or will it resume from where the node was
shut down?
2. If we upgrade binaries to C* 4.0 while compaction is running, will it
resume the compaction or start over or no compaction would kick in?

RE: [EXTERNAL] Re: Cassandra migration from 1.25 to 3.x

[Durity, Sean R]

The advice so far is exactly correct for an in-place kind of upgrade. The blog 
post you mentioned is different. They decided to jump versions in Cassandra by 
standing up a new cluster and using a dual-write/dual-read process for their 
app. They also wrote code to read and interpret sstables in order to migrate 
existing data. Getting that right with compaction running, data consistency, 
etc. it not easy. That is what Cassandra does, of course. They had to reverse 
engineer that process.

I would not personally take that path as it seems a more difficult way to go -- 
for the DBA/admin. It is a nice path for the development team, though. They 
only had to look at their reads and writes (already encapsulated in a DAO) for 
the dual clusters. In a multi-upgrade scenario, drivers and statements probably 
have to get upgraded at several steps along the way (including a move from 
Thrift to CQL, probably). More app testing is required each upgrade. So, the 
decision has to be based on which resources you have and trust (app dev and 
testing + Cassandra upgrades or data migration and testing). Once you have 
automated/semi-automated Cassandra upgrades in place, that is an easier path, 
but that company obviously hadn't invested there.

Sean Durity

-Original Message-
From: Michael Shuler  On Behalf Of Michael Shuler
Sent: Monday, June 17, 2019 8:26 AM
To: user@cassandra.apache.org
Subject: [EXTERNAL] Re: Cassandra migration from 1.25 to 3.x

First and foremost, read NEWS.txt from your current version to the version you 
wish to upgrade to. There are too may details that you many need to be aware 
of. For instance, in the 2.0.0 Upgrading notes:

https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_cassandra_blob_cassandra-2D3.11_NEWS.txt-23L1169-2DL1178=DwIDaQ=MtgQEAMQGqekjTjiAhkudQ=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ=CPWy2XBaDFHzVDapNO4E7kLIFFfbkRd8KqftSrypjSU=D3Y18E9gxewpushCMETjHt9cS8lKvLMrhUdhPriF4Dk=

I assume you meant 1.2.5, so you're first step is to upgrade to at least
1.2.9 (I would suggest using latest 1.2.x, which is 1.2.19). Then you can to to 
2.0.x and up.

Practicing on a scratch cluster is valuable experience. Reading the upgrade 
notes in NEWS.txt is a must.

--
Kind regards,
Michael

On 6/17/19 3:34 AM, Anurag Sharma wrote:
> Thanks Alex,
>
> I came across some interesting and efficient ways of upgrading from
> 1.x to 3.x as described in the blog here
>  eezilkha_database-2Dmigration-2Dat-2Dscale-2Dae85c14c3621=DwIDaQ=M
> tgQEAMQGqekjTjiAhkudQ=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ=
> CPWy2XBaDFHzVDapNO4E7kLIFFfbkRd8KqftSrypjSU=7ekpEXHT1Qm_xL9l6_1Kty32
> fDDerlB_PgO1-4K1-VQ= > and others. Was curious if someone has
> open-sourced their custom utility.  :D
>
> Regards
> Anurag
>
> On Mon, Jun 17, 2019 at 1:27 PM Oleksandr Shulgin
> mailto:oleksandr.shul...@zalando.de>> wrote:
>
> On Mon, Jun 17, 2019 at 9:30 AM Anurag Sharma
> mailto:anurag.rp.sha...@gmail.com>> wrote:
>
>
> We are upgrading Cassandra from 1.25 to 3.X. Just curious if
> there is any recommended open source utility for the same.
>
>
> Hi,
>
> The "recommended  open source utility" is the Apache Cassandra
> itself. ;-)
>
> Given the huge difference between the major versions, though, you
> will need a decent amount of planning and preparation to
> successfully complete such a migration.  Most likely you will want
> to do it in small steps, first upgrading to the latest minor version
> in the 1.x series, then making a jump to 2.x, then to 3.0, and only
> then to 3.x if you really mean to.  On each upgrade step, be sure to
> examine the release notes carefully to understand if there is any
> impact for your cluster and/or client applications.  Do have a test
> system with preferably identical setup and configuration and execute
> the upgrade steps there first to verify your expectations.
>
> Good luck!
> --
> Alex
>


-
To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
For additional commands, e-mail: user-h...@cassandra.apache.org




The information in this Internet Email is confidential and may be legally 
privileged. It is intended solely for the addressee. Access to this Email by 
anyone else is unauthorized. If you are not the intended recipient, any 
disclosure, copying, distribution or any action taken or omitted to be taken in 
reliance on it, is prohibited and may be unlawful. When addressed to our 
clients any opinions or advice contained in this Email are subject to the terms 
and conditions expressed in any applicable governing The Home Depot terms of 
business or client engagement letter. The Home Depot disclaims all 
responsibility and liability for the accuracy and content of this attachment 
and for any 

Re: Cassandra migration from 1.25 to 3.x

[Michael Shuler]

First and foremost, read NEWS.txt from your current version to the 
version you wish to upgrade to. There are too may details that you many 
need to be aware of. For instance, in the 2.0.0 Upgrading notes:


https://github.com/apache/cassandra/blob/cassandra-3.11/NEWS.txt#L1169-L1178

I assume you meant 1.2.5, so you're first step is to upgrade to at least 
1.2.9 (I would suggest using latest 1.2.x, which is 1.2.19). Then you 
can to to 2.0.x and up.


Practicing on a scratch cluster is valuable experience. Reading the 
upgrade notes in NEWS.txt is a must.


--
Kind regards,
Michael

On 6/17/19 3:34 AM, Anurag Sharma wrote:

Thanks Alex,

I came across some interesting and efficient ways of upgrading from 1.x 
to 3.x as described in the blog here 
 and 
others. Was curious if someone has open-sourced their custom utility.  :D


Regards
Anurag

On Mon, Jun 17, 2019 at 1:27 PM Oleksandr Shulgin 
mailto:oleksandr.shul...@zalando.de>> wrote:


On Mon, Jun 17, 2019 at 9:30 AM Anurag Sharma
mailto:anurag.rp.sha...@gmail.com>> wrote:


We are upgrading Cassandra from 1.25 to 3.X. Just curious if
there is any recommended open source utility for the same.


Hi,

The "recommended  open source utility" is the Apache Cassandra
itself. ;-)

Given the huge difference between the major versions, though, you
will need a decent amount of planning and preparation to
successfully complete such a migration.  Most likely you will want
to do it in small steps, first upgrading to the latest minor version
in the 1.x series, then making a jump to 2.x, then to 3.0, and only
then to 3.x if you really mean to.  On each upgrade step, be sure to
examine the release notes carefully to understand if there is any
impact for your cluster and/or client applications.  Do have a test
system with preferably identical setup and configuration and execute
the upgrade steps there first to verify your expectations.

Good luck!
--
Alex




-
To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
For additional commands, e-mail: user-h...@cassandra.apache.org

Re: Cassandra migration from 1.25 to 3.x

[Anurag Sharma]

Thanks Alex,

I came across some interesting and efficient ways of upgrading from 1.x to
3.x as described in the blog here
 and
others. Was curious if someone has open-sourced their custom utility.  :D

Regards
Anurag

On Mon, Jun 17, 2019 at 1:27 PM Oleksandr Shulgin <
oleksandr.shul...@zalando.de> wrote:

> On Mon, Jun 17, 2019 at 9:30 AM Anurag Sharma 
> wrote:
>
>>
>> We are upgrading Cassandra from 1.25 to 3.X. Just curious if there is any
>> recommended open source utility for the same.
>>
>
> Hi,
>
> The "recommended  open source utility" is the Apache Cassandra itself. ;-)
>
> Given the huge difference between the major versions, though, you will
> need a decent amount of planning and preparation to successfully complete
> such a migration.  Most likely you will want to do it in small steps, first
> upgrading to the latest minor version in the 1.x series, then making a jump
> to 2.x, then to 3.0, and only then to 3.x if you really mean to.  On each
> upgrade step, be sure to examine the release notes carefully to understand
> if there is any impact for your cluster and/or client applications.  Do
> have a test system with preferably identical setup and configuration and
> execute the upgrade steps there first to verify your expectations.
>
> Good luck!
> --
> Alex
>

Re: Cassandra migration from 1.25 to 3.x

[Oleksandr Shulgin]

On Mon, Jun 17, 2019 at 9:30 AM Anurag Sharma 
wrote:

>
> We are upgrading Cassandra from 1.25 to 3.X. Just curious if there is any
> recommended open source utility for the same.
>

Hi,

The "recommended  open source utility" is the Apache Cassandra itself. ;-)

Given the huge difference between the major versions, though, you will need
a decent amount of planning and preparation to successfully complete such a
migration.  Most likely you will want to do it in small steps, first
upgrading to the latest minor version in the 1.x series, then making a jump
to 2.x, then to 3.0, and only then to 3.x if you really mean to.  On each
upgrade step, be sure to examine the release notes carefully to understand
if there is any impact for your cluster and/or client applications.  Do
have a test system with preferably identical setup and configuration and
execute the upgrade steps there first to verify your expectations.

Good luck!
--
Alex

Cassandra migration from 1.25 to 3.x

[Anurag Sharma]

Hi,

We are upgrading Cassandra from 1.25 to 3.X. Just curious if there is any
recommended open source utility for the same.

Regards
Anurag

Re: How can I check cassandra cluster has a real working function of high availability?

[Oleksandr Shulgin]

On Sat, Jun 15, 2019 at 4:31 PM Nimbus Lin  wrote:

> Dear cassandra's pioneers:
> I am a 5 years' newbie,  it is until now that I have time to use
> cassandra. but I cann't check cassandra's high availabily when I stop a
> seed node or none seed DN as CGE or Greenplum.
> Would someone can tell me how to check the cassandra's high
> availability? even I change the consistency level from one to local_one,
> the cqlsh's select is always return an error of NoHostAvailable.
>
>  By the way, would you like to answer me other two questions:
> 2nd question: although cassandrfa's consistency is a per-operation
> setting, isn't there a whole all operations' consistency setting method?
> 3rd question: how can I can cassandra cluster's running variables as
> mysql's show global variables? such as hidden variable of  auto_bootstrap?
>

Hi,

For the purpose of serving client requests, all nodes are equal -- seed or
not.  So it shouldn't matter which node you are stopping (or making it
unavailable for the rest of the cluster using other means).

In order to test it with cqlsh you should ensure that the replication
factors of the keyspace you're testing with is sufficient.  Given the
NoHostAvailable exception that you are experiencing at consistency level
ONE (or LOCAL_ONE), I can guess that you are testing with a keyspace with
replication factor 1 and the node which is unavailable happen to be
responsible for the particular partition.

For your second question: it depends on a client (or "client driver") you
are using.  In cqlsh you can set consistency level that will be applied for
all subsequent queries using the "CONSISTENCY ..." command.  I think that
the Java driver does have an option to set the default consistency level,
as well as has an option to set consistency level per query.  Most likely
this is also true for Python and other drivers.

And for the third question: I'm not aware of a CQL or nodetool command that
would fulfill the need.  Most likely it is possible to learn (and update)
most of the configuration parameters using JMX, e.g. with JConsole:
https://docs.datastax.com/en/archived/cassandra/3.0/cassandra/operations/opsMonitoring.html#opsMonitoringJconsole

Cheers,
--
Alex