Re: where does c* store the schema?

[Jinhua Luo]

Yes, I know it must be in system schema.

But how c* replicates the user defined schema to all nodes? If it
applies the same RWN model to them, then what's the R and W?
And when a failed node comes back to the cluster, how to recover the
schema updates it may miss during the outage?

2018-04-16 17:01 GMT+08:00 DuyHai Doan :
> There is a system_schema keyspace to store all the schema information
>
> https://docs.datastax.com/en/cql/3.3/cql/cql_using/useQuerySystem.html#useQuerySystem__table_bhg_1bw_4v
>
> On Mon, Apr 16, 2018 at 10:48 AM, Jinhua Luo  wrote:
>>
>> Hi All,
>>
>> Does c* use predefined keyspace/tables to store the user defined schema?
>> If so, what's the RWN of those meta schema? And what's the procedure
>> to update them?
>>
>> -
>> To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
>> For additional commands, e-mail: user-h...@cassandra.apache.org
>>
>

-
To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
For additional commands, e-mail: user-h...@cassandra.apache.org

Re: about the tombstone and hinted handoff

[kurt greaves]

I don't think that's true/maybe that comment is misleading. Tombstones
AFAIK will be propagated by hints, and the hint system doesn't do anything
to check if a particular row has been tombstoned. To the node receiving the
hints it just looks like it's receiving a bunch of writes, it doesn't know
they are hints.

On 12 April 2018 at 13:51, Jinhua Luo  wrote:

> Hi All,
>
> In the doc:
> https://docs.datastax.com/en/cassandra/3.0/cassandra/dml/dml
> AboutDeletes.html
>
> It said "When an unresponsive node recovers, Cassandra uses hinted
> handoff to replay the database mutationsthe node missed while it was
> down. Cassandra does not replay a mutation for a tombstoned record
> during its grace period.".
>
> The tombstone here is on the recovered node or coordinator?
> The tombstone is a special write record, so it must have writetime.
> We could compare the writetime between the version in the hint and the
> version of the tombstone, which is enough to make choice, so why we
> need to wait for gc_grace_seconds here?
>
> -
> To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: user-h...@cassandra.apache.org
>
>

Re: Shifting data to DCOS

[Faraz Mateen]

Thanks for the response guys.

Let me try setting token ranges manually and move the data again to correct
nodes. Will update with the outcome soon.


On Tue, Apr 17, 2018 at 5:42 AM, kurt greaves  wrote:

> Sorry for the delay.
>
>> Is the problem related to token ranges? How can I find out token range
>> for each node?
>> What can I do to further debug and root cause this?
>
> Very likely. See below.
>
> My previous cluster has 3 nodes but replication factor is 2. I am not
>> exactly sure how I would handle the tokens. Can you explain that a bit?
>
> The new cluster will have to have the same token ring as the old if you
> are copying from node to node. Basically you should get the set of tokens
> for each node (from nodetool ring) and when you spin up your 3 new nodes,
> set initial_tokens in the yaml to be the comma-separated list of tokens for 
> *exactly
> one* node from the previous cluster. When restoring the SSTables you need
> to make sure you take the SSTables from the original node and place it on
> the new node that has the *same* list of tokens. If you don't do this it
> won't be a replica for all the data in those SSTables and consequently
> you'll lose data (or it simply won't be available).
> ​
>



-- 
Faraz Mateen

Re: Shifting data to DCOS

[kurt greaves]

Sorry for the delay.

> Is the problem related to token ranges? How can I find out token range for
> each node?
> What can I do to further debug and root cause this?

Very likely. See below.

My previous cluster has 3 nodes but replication factor is 2. I am not
> exactly sure how I would handle the tokens. Can you explain that a bit?

The new cluster will have to have the same token ring as the old if you are
copying from node to node. Basically you should get the set of tokens for
each node (from nodetool ring) and when you spin up your 3 new nodes, set
initial_tokens in the yaml to be the comma-separated list of tokens
for *exactly
one* node from the previous cluster. When restoring the SSTables you need
to make sure you take the SSTables from the original node and place it on
the new node that has the *same* list of tokens. If you don't do this it
won't be a replica for all the data in those SSTables and consequently
you'll lose data (or it simply won't be available).
​

Re: Time serial column family design

[Nate McCall]

>
>
> Select * from test where vin =“ZD41578123DSAFWE12313” and create_date in
> (20180416, 20180415, 20180414, 20180413, 20180412….);
>
> But this cause the cql query is very long,and I don’t know whether there
> is limitation for the length of the cql.
>
> Please give me some advice,thanks in advance.
>

Using the SELECT ... IN syntax  means that:
- the driver will not be able to route the queries to the nodes which have
the partition
- a single coordinator must scatter-gather the query and results

Break this up into a series of single statements using the executeAsync
method and gather the results via something like Futures in Guava or
similar.

Re: Cassandra downgrade version

[Lerh Chuan Low]

You should just be able to install 3.1.0 again if you need to as they are
in the 3.X line. To be really safe you can also take a snapshot and backup
your existing SSTables first..and always remember to test before upgrading
in Production :)

On 17 April 2018 at 07:48, Abdul Patel  wrote:

> Hi All,
>
> I am.planning to upgrade my cassandra cluster from 3.1.0 to 3.11.2 . Just
> in case if somethings goes back then do we have any rollback or downgrade
> option in cassandra  to older/ previous version?
>
> Thanks
>
>

Cassandra downgrade version

[Abdul Patel]

Hi All,

I am.planning to upgrade my cassandra cluster from 3.1.0 to 3.11.2 . Just
in case if somethings goes back then do we have any rollback or downgrade
option in cassandra  to older/ previous version?

Thanks

Re: copy from one table to another

[Kyrylo Lebediev]

Thanks,  Ali.
I just need to copy a large table in production without actual copying by using 
hardlinks. After this both tables should be used independently (RW). Is this a 
supported way or not?

Regards,
Kyrill

From: Ali Hubail 
Sent: Monday, April 16, 2018 6:51:51 PM
To: user@cassandra.apache.org
Subject: Re: copy from one table to another

If you want to copy a portion of the data to another table, you can also use 
sstable cql writer. It is more of an advanced feature and can be tricky, but 
doable.
once you write the new sstables, you can then use the sstableloader to stream 
the new data into the new table.
check this out:
https://www.datastax.com/dev/blog/using-the-cassandra-bulk-loader-updated

I have recently used this to clean up 500 GB worth of sstable data in order to 
purge tombstones that were mistakenly generated by the client.
obviously this is not as fast as hardlinks + refresh, but it's much faster and 
more efficient than using cql to copy data accross the tables.
take advantage of CQLSSTableWriter.builder.sorted() if you can, and utilize 
writetime if you have to.

Ali Hubail

Confidentiality warning: This message and any attachments are intended only for 
the persons to whom this message is addressed, are confidential, and may be 
privileged. If you are not the intended recipient, you are hereby notified that 
any review, retransmission, conversion to hard copy, copying, modification, 
circulation or other use of this message and any attachments is strictly 
prohibited. If you receive this message in error, please notify the sender 
immediately by return email, and delete this message and any attachments from 
your system. Petrolink International Limited its subsidiaries, holding 
companies and affiliates disclaims all responsibility from and accepts no 
liability whatsoever for the consequences of any unauthorized person acting, or 
refraining from acting, on any information contained in this message. For 
security purposes, staff training, to assist in resolving complaints and to 
improve our customer service, email communications may be monitored and 
telephone calls may be recorded.


Kyrylo Lebediev 

04/16/2018 10:37 AM
Please respond to
user@cassandra.apache.org




To
"user@cassandra.apache.org" ,
cc

Subject
Re: copy from one table to another







Any issues if we:

1) create an new empty table with the same structure as the old one
2) create hardlinks ("ln without -s"): 
.../-/--* ---> 
.../-/--*
3) run nodetool refresh -- newkeyspacename newtable

and then query/modify both tables independently/simultaneously?

In theory, as SSTables are immutable, this should work, but could there be some 
hidden issues?

Regards,
Kyrill



From: Dmitry Saprykin 
Sent: Sunday, April 8, 2018 7:33:03 PM
To: user@cassandra.apache.org
Subject: Re: copy from one table to another

You can copy hardlinks to ALL SSTables from old to new table and then delete 
part of data you do not need in a new one.

On Sun, Apr 8, 2018 at 10:20 AM, Nitan Kainth 
mailto:nitankai...@gmail.com>> wrote:
If it for testing and you don’t need any specific data, just copy a set of 
sstables with all files of that sequence and move to target tables directory 
and rename it.

Restart target node or run nodetool refresh

Sent from my iPhone

On Apr 8, 2018, at 4:15 AM, onmstester onmstester 
mailto:onmstes...@zoho.com>> wrote:

Is there any way to copy some part of a table to another table in cassandra? A 
large amount of data should be copied so i don't want to fetch data to client 
and stream it back to cassandra using cql.

Sent using Zoho Mail

Re: copy from one table to another

[Ali Hubail]

If you want to copy a portion of the data to another table, you can also 
use sstable cql writer. It is more of an advanced feature and can be 
tricky, but doable.
once you write the new sstables, you can then use the sstableloader to 
stream the new data into the new table.
check this out:
https://www.datastax.com/dev/blog/using-the-cassandra-bulk-loader-updated

I have recently used this to clean up 500 GB worth of sstable data in 
order to purge tombstones that were mistakenly generated by the client.
obviously this is not as fast as hardlinks + refresh, but it's much faster 
and more efficient than using cql to copy data accross the tables.
take advantage of CQLSSTableWriter.builder.sorted() if you can, and 
utilize writetime if you have to.

Ali Hubail

Confidentiality warning: This message and any attachments are intended 
only for the persons to whom this message is addressed, are confidential, 
and may be privileged. If you are not the intended recipient, you are 
hereby notified that any review, retransmission, conversion to hard copy, 
copying, modification, circulation or other use of this message and any 
attachments is strictly prohibited. If you receive this message in error, 
please notify the sender immediately by return email, and delete this 
message and any attachments from your system. Petrolink International 
Limited its subsidiaries, holding companies and affiliates disclaims all 
responsibility from and accepts no liability whatsoever for the 
consequences of any unauthorized person acting, or refraining from acting, 
on any information contained in this message. For security purposes, staff 
training, to assist in resolving complaints and to improve our customer 
service, email communications may be monitored and telephone calls may be 
recorded.



Kyrylo Lebediev  
04/16/2018 10:37 AM
Please respond to
user@cassandra.apache.org


To
"user@cassandra.apache.org" , 
cc

Subject
Re: copy from one table to another






Any issues if we:

1) create an new empty table with the same structure as the old one 
2) create hardlinks ("ln without -s"): 
.../-/--* ---> 
.../-/--* 
3) run nodetool refresh -- newkeyspacename newtable

and then query/modify both tables independently/simultaneously?

In theory, as SSTables are immutable, this should work, but could there be 
some hidden issues? 

Regards, 
Kyrill

From: Dmitry Saprykin 
Sent: Sunday, April 8, 2018 7:33:03 PM
To: user@cassandra.apache.org
Subject: Re: copy from one table to another 
 
You can copy hardlinks to ALL SSTables from old to new table and then 
delete part of data you do not need in a new one.

On Sun, Apr 8, 2018 at 10:20 AM, Nitan Kainth  
wrote:
If it for testing and you don’t need any specific data, just copy a set of 
sstables with all files of that sequence and move to target tables 
directory and rename it. 

Restart target node or run nodetool refresh 

Sent from my iPhone

On Apr 8, 2018, at 4:15 AM, onmstester onmstester  
wrote:

Is there any way to copy some part of a table to another table in 
cassandra? A large amount of data should be copied so i don't want to 
fetch data to client and stream it back to cassandra using cql.

Sent using Zoho Mail

Re: copy from one table to another

[Kyrylo Lebediev]

Any issues if we:


1) create an new empty table with the same structure as the old one

2) create hardlinks ("ln without -s"): 
.../-/--* ---> 
.../-/--*

3) run nodetool refresh -- newkeyspacename newtable


and then query/modify both tables independently/simultaneously?


In theory, as SSTables are immutable, this should work, but could there be some 
hidden issues?


Regards,

Kyrill


From: Dmitry Saprykin 
Sent: Sunday, April 8, 2018 7:33:03 PM
To: user@cassandra.apache.org
Subject: Re: copy from one table to another

You can copy hardlinks to ALL SSTables from old to new table and then delete 
part of data you do not need in a new one.

On Sun, Apr 8, 2018 at 10:20 AM, Nitan Kainth 
mailto:nitankai...@gmail.com>> wrote:
If it for testing and you don’t need any specific data, just copy a set of 
sstables with all files of that sequence and move to target tables directory 
and rename it.

Restart target node or run nodetool refresh

Sent from my iPhone

On Apr 8, 2018, at 4:15 AM, onmstester onmstester 
mailto:onmstes...@zoho.com>> wrote:

Is there any way to copy some part of a table to another table in cassandra? A 
large amount of data should be copied so i don't want to fetch data to client 
and stream it back to cassandra using cql.


Sent using Zoho Mail

Re: where does c* store the schema?

[DuyHai Doan]

There is a system_schema keyspace to store all the schema information

https://docs.datastax.com/en/cql/3.3/cql/cql_using/useQuerySystem.html#useQuerySystem__table_bhg_1bw_4v

On Mon, Apr 16, 2018 at 10:48 AM, Jinhua Luo  wrote:

> Hi All,
>
> Does c* use predefined keyspace/tables to store the user defined schema?
> If so, what's the RWN of those meta schema? And what's the procedure
> to update them?
>
> -
> To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: user-h...@cassandra.apache.org
>
>

where does c* store the schema?

[Jinhua Luo]

Hi All,

Does c* use predefined keyspace/tables to store the user defined schema?
If so, what's the RWN of those meta schema? And what's the procedure
to update them?

-
To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
For additional commands, e-mail: user-h...@cassandra.apache.org

RE: Mailing list server IPs

[Jacques-Henri Berthemet]

Hi Nate,

Thanks for the help, I passed the info to our IT.

Regards,
--
Jacques-Henri Berthemet

From: Nate McCall [mailto:n...@thelastpickle.com]
Sent: Monday, April 16, 2018 12:27 AM
To: Cassandra Users 
Subject: Re: Mailing list server IPs

Hi Jacques,
Thanks for bringing this up. I took a quick look through the INFRA project and 
saw a couple of resolved issues that might help:
https://issues.apache.org/jira/browse/INFRA-6584?jql=project%20%3D%20INFRA%20AND%20text%20~%20%22mail%20server%20whitelist%22

If those don't do it for you, please open a new issue with INFRA.


On Sat, Apr 14, 2018 at 1:19 AM, Jacques-Henri Berthemet 
mailto:jacques-henri.berthe...@genesys.com>>
 wrote:
I checked with IT and I missed an email on the period where I got the last 
bounce. It’s not a very big deal but I’d like to have it fixed if possible.

Gmail servers are very picky on SMTP traffic and reject a lot of things.

--
Jacques-Henri Berthemet

From: Nicolas Guyomar 
[mailto:nicolas.guyo...@gmail.com]
Sent: Friday, April 13, 2018 3:15 PM
To: user@cassandra.apache.org
Subject: Re: Mailing list server IPs

Hi,

I receive similar messages from time to time, and I'm using Gmail ;)  I believe 
I never missed a mail on the ML and that you can safely ignore this message

On 13 April 2018 at 15:06, Jacques-Henri Berthemet 
mailto:jacques-henri.berthe...@genesys.com>>
 wrote:
Hi,

I’m getting bounce messages from the ML from time to time, see attached 
example. Our IT told me that they need to whitelist all IPs used by Cassandra 
ML server. Is there a way to get those IPs?

Sorry if it’s not really related to Cassandra itself but I didn’t find anything 
in 
http://untroubled.org/ezmlm/ezman/ezman5.html
 commands.

Regards,
--
Jacques-Henri Berthemet


-- Forwarded message --
From: "user-h...@cassandra.apache.org" 
mailto:user-h...@cassandra.apache.org>>
To: Jacques-Henri Berthemet 
mailto:jacques-henri.berthe...@genesys.com>>
Cc:
Bcc:
Date: Fri, 6 Apr 2018 20:47:22 +
Subject: Warning from 
user@cassandra.apache.org
Hi! This is the ezmlm program. I'm managing the
user@cassandra.apache.org mailing list.


Messages to you from the user mailing list seem to
have been bouncing. I've attached a copy of the first bounce
message I received.

If this message bounces too, I will send you a probe. If the probe bounces,
I will remove your address from the user mailing list,
without further notice.


I've kept a list of which messages from the user mailing list have
bounced from your address.

Copies of these messages may be in the archive.
To retrieve a set of messages 123-145 (a maximum of 100 per request),
send a short message to:
   
mailto:user-get.123_...@cassandra.apache.org>>

To receive a subject and author list for the last 100 or so messages,
send a short message to:
   mailto:user-in...@cassandra.apache.org>>

Here are the message numbers:

   60535
   60536
   60548

--- Enclosed is a copy of the bounce message I received.

Return-Path: <>
Received: (qmail 8848 invoked for bounce); 27 Mar 2018 14:22:11 -
Date: 27 Mar 2018 14:22:11 -
From: mailer-dae...@apache.org
To: 
user-return-605...@cassandra.apache.org
Subject: failure notice




-
To unsubscribe, e-mail: 
user-unsubscr...@cassandra.apache.org
For additional commands, e-mail: 
user-h...@cassandra.apache.org




--
-
Nate McCall
Wellington, NZ
@zznate

CTO
Apache Cassandra Consulting
http://www.thelastpickle.com