Re: Cassandra 2.x Stability

[Shalom Sagges]

Thanks a lot Kai!


Shalom Sagges
DBA
T: +972-74-700-4035
 
 We Create Meaningful Connections



On Thu, Dec 1, 2016 at 5:18 PM, Kai Wang  wrote:

> Just based on a few observations on this list. Not one week goes by
> without people asking which release is the most stable on 3.x line. Folks
> at instaclustr also provide their own 3.x fork for stability issues. etc
>
> We developers already have enough to think about. I really don't feel like
> spending time researching which release of C* I should choose. So for me,
> 2.2.x is the choice in production.
>
> That being said, I have nothing against 3.x. I do like its new storage
> engine. If I start a brand new project today with zero previous C*
> experience, I probably would choose 3.0.10 as my starting point. However if
> I were to upgrade to 3.x, I would have to test it thoroughly in a dev
> environment with real production load and monitor it very closely on
> performance, compaction, repair, bootstrap, replacing etc. Data is simply
> too important to take chances with.
>
>
> On Thu, Dec 1, 2016 at 9:38 AM, Shalom Sagges 
> wrote:
>
>> Hey Kai,
>>
>> Thanks for the info. Can you please elaborate on the reasons you'd pick
>> 2.2.6 over 3.0?
>>
>>
>> Shalom Sagges
>> DBA
>> T: +972-74-700-4035 <+972%2074-700-4035>
>>  
>>  We Create Meaningful Connections
>>
>> 
>>
>>
>> On Thu, Dec 1, 2016 at 2:26 PM, Kai Wang  wrote:
>>
>>> I have been running 2.2.6 in production. As of today I would still pick
>>> it over 3.x for production.
>>>
>>> On Nov 30, 2016 5:42 AM, "Shalom Sagges"  wrote:
>>>
 Hi Everyone,

 I'm about to upgrade our 2.0.14 version to a newer 2.x version.
 At first I thought of upgrading to 2.2.8, but I'm not sure how stable
 it is, as I understand the 2.2 version was supposed to be a sort of beta
 version for 3.0 feature-wise, whereas 3.0 upgrade will mainly handle the
 storage modifications (please correct me if I'm wrong).

 So my question is, if I need a 2.x version (can't upgrade to 3 due to
 client considerations), which one should I choose, 2.1.x or 2.2.x? (I'm
 don't require any new features available in 2.2).

 Thanks!

 Shalom Sagges
 DBA
 T: +972-74-700-4035 <+972%2074-700-4035>
 
   We
 Create Meaningful Connections

 


 This message may contain confidential and/or privileged information.
 If you are not the addressee or authorized to receive this on behalf of
 the addressee you must not use, copy, disclose or take action based on this
 message or any information herein.
 If you have received this message in error, please advise the sender
 immediately by reply email and delete this message. Thank you.

>>>
>>
>> This message may contain confidential and/or privileged information.
>> If you are not the addressee or authorized to receive this on behalf of
>> the addressee you must not use, copy, disclose or take action based on this
>> message or any information herein.
>> If you have received this message in error, please advise the sender
>> immediately by reply email and delete this message. Thank you.
>>
>
>

-- 
This message may contain confidential and/or privileged information. 
If you are not the addressee or authorized to receive this on behalf of the 
addressee you must not use, copy, disclose or take action based on this 
message or any information herein. 
If you have received this message in error, please advise the sender 
immediately by reply email and delete this message. Thank you.

Re: Why does `now()` produce different times within the same query?

[Edward Capriolo]

On Thu, Dec 1, 2016 at 4:06 AM, Sylvain Lebresne 
wrote:

> One can of course always open a JIRA, but I'm going to strongly disagree
> with a
> change here (outside of a documentation one that is).
>
> The now() function is a timeuuid generator, and it thus generates a unique
> timeuuid on every call, as specified by the timeuuid spec. I'll note that
> document lists it under "Timeuuid functions", and has sentences like
> "the value returned by now() is guaranteed to be unique", so while I'm
> sure the
> documentation can be further clarified, I think it's pretty clear it's not
> the
> now() of SQL, and getting unique values on every call shouldn't be *that*
> surprising.
>
> Also, now() was primarily meant for use on timeuuid clustering columns for
> a
> time-series like table, something like:
>   CREATE TABLE ts (
> k int,
> t timeuuid,
> v text,
> PRIMARY KEY (k, t)
>   )
> and if you use it multiple times in a batch, this would look something
> like:
>   BEGIN BATCH
> INSERT INTO ts (k, t, v) VALUES (0, now(), 'foo');
> INSERT INTO ts (k, t, v) VALUES (0, now(), 'bar');
>   APPLY BATCH
> and you definitively want that to insert 2 "events", not just one.
>
> This is also why changing the behavior of this method *would* be a breaking
> change.
>
> Another reason this work the way it is is that functions in CQL are just
> that,
> functions. Each execution is unique and they have no notion of being
> executed in
> the same statement/batch/whatever. I actually think this is sensible,
> assuming
> one stops being obsessed with what other databases that aren't Apache
> Cassandra
> do.
>
> I will note that Ben seems to suggest keeping the return of now() unique
> across
> call while keeping the time component equals, thus varying the rest of the
> uuid
> bytes. However:
>  - I'm starting to wonder what this would buy us. Why would someone be
> super
>confused by the time changing across calls (in a single
> statement/batch), but
>be totally not confused by the actual full return to not be equal? And
> how is
>that actually useful: you're having different result anyway and you're
>letting the server pick the timestamp in the first place, so you're
> probably
>not caring about milliseconds precision of that timestamp in the first
> place.
>  - This would basically be a violation of the timeuuid spec
>  - This would be a big pain in the code and make of now() a special case
> among functions. I'm unconvinced special cases are making things easier
> in general.
>
> So I'm all for improving the documentation if this confuses users due to
> expectations (mistakenly) carried from prior experiences, and please
> feel free to open a JIRA for that. I'm a lot less in agreement that there
> is
> something wrong with the way the function behave in principle.
>
> > I can see why this issue has been largely ignored and hasn't had a
> chance for
> > the behaviour to be formally defined
>
> Don't make too much assumptions. The behavior is perfectly well defined:
> now()
> is a "normal" function and is evaluated whenever it's called according to
> the
> timeuuid spec (or as close to it as we can make it).
>
> On Thu, Dec 1, 2016 at 7:25 AM, Benjamin Roth 
> wrote:
>
>> Great comment. +1
>>
>> Am 01.12.2016 06:29 schrieb "Ben Bromhead" :
>>
>>> tl;dr +1 yup raise a jira to discuss how now() should behave in a single
>>> statement (and possible extend to batch statements).
>>>
>>> The values of now should be the same if you assume that now() works like
>>> it does in relational databases such as postgres or mysql, however at the
>>> moment it instead works like sysdate() in mysql. Given that CQL is supposed
>>> to be SQL like, I think the assumption around the behaviour of now() was a
>>> fair one to make.
>>>
>>> I definitely agree that raising a jira ticket would be a great place to
>>> discuss what the behaviour of now() should be for Cassandra. Personally I
>>> would be in favour of seeing the deterministic component (the actual time
>>> part) being the same across multiple calls in the one statement or multiple
>>> statements in a batch.
>>>
>>> Cassandra documentation does not make any claims as to how now() works
>>> within a single statement and reading the code it shows the intent is to
>>> work like sysdate() from MySQL rather than now(). One of the identified
>>> dangers of making cql similar to sql is that, while yes it aids adoption,
>>> users will find that SQL like things don't behave as expected. Of course as
>>> a user, one shouldn't have to read the source code to determine correct
>>> behaviour.
>>>
>>> Given that a timeuuid is made up of deterministic and (pseudo)
>>> non-deterministic components I can see why this issue has been largely
>>> ignored and hasn't had a chance for the behaviour to be formally defined
>>> (you would expect now to return the same time in the one statement despite

Re: Cassandra 2.x Stability

[Shalom Sagges]

Hey Kai,

Thanks for the info. Can you please elaborate on the reasons you'd pick
2.2.6 over 3.0?


Shalom Sagges
DBA
T: +972-74-700-4035
 
 We Create Meaningful Connections



On Thu, Dec 1, 2016 at 2:26 PM, Kai Wang  wrote:

> I have been running 2.2.6 in production. As of today I would still pick it
> over 3.x for production.
>
> On Nov 30, 2016 5:42 AM, "Shalom Sagges"  wrote:
>
>> Hi Everyone,
>>
>> I'm about to upgrade our 2.0.14 version to a newer 2.x version.
>> At first I thought of upgrading to 2.2.8, but I'm not sure how stable it
>> is, as I understand the 2.2 version was supposed to be a sort of beta
>> version for 3.0 feature-wise, whereas 3.0 upgrade will mainly handle the
>> storage modifications (please correct me if I'm wrong).
>>
>> So my question is, if I need a 2.x version (can't upgrade to 3 due to
>> client considerations), which one should I choose, 2.1.x or 2.2.x? (I'm
>> don't require any new features available in 2.2).
>>
>> Thanks!
>>
>> Shalom Sagges
>> DBA
>> T: +972-74-700-4035 <+972%2074-700-4035>
>>  
>>  We Create Meaningful Connections
>>
>> 
>>
>>
>> This message may contain confidential and/or privileged information.
>> If you are not the addressee or authorized to receive this on behalf of
>> the addressee you must not use, copy, disclose or take action based on this
>> message or any information herein.
>> If you have received this message in error, please advise the sender
>> immediately by reply email and delete this message. Thank you.
>>
>

-- 
This message may contain confidential and/or privileged information. 
If you are not the addressee or authorized to receive this on behalf of the 
addressee you must not use, copy, disclose or take action based on this 
message or any information herein. 
If you have received this message in error, please advise the sender 
immediately by reply email and delete this message. Thank you.

Re: Why does `now()` produce different times within the same query?

[Bruce Heath]

Get Outlook for Android


From: Edward Capriolo 
Sent: Thursday, December 1, 2016 10:44:10 AM
To: user@cassandra.apache.org
Subject: Re: Why does `now()` produce different times within the same query?



On Thu, Dec 1, 2016 at 4:06 AM, Sylvain Lebresne 
> wrote:
One can of course always open a JIRA, but I'm going to strongly disagree with a
change here (outside of a documentation one that is).

The now() function is a timeuuid generator, and it thus generates a unique
timeuuid on every call, as specified by the timeuuid spec. I'll note that
document lists it under "Timeuuid functions", and has sentences like
"the value returned by now() is guaranteed to be unique", so while I'm sure the
documentation can be further clarified, I think it's pretty clear it's not the
now() of SQL, and getting unique values on every call shouldn't be *that*
surprising.

Also, now() was primarily meant for use on timeuuid clustering columns for a
time-series like table, something like:
  CREATE TABLE ts (
k int,
t timeuuid,
v text,
PRIMARY KEY (k, t)
  )
and if you use it multiple times in a batch, this would look something like:
  BEGIN BATCH
INSERT INTO ts (k, t, v) VALUES (0, now(), 'foo');
INSERT INTO ts (k, t, v) VALUES (0, now(), 'bar');
  APPLY BATCH
and you definitively want that to insert 2 "events", not just one.

This is also why changing the behavior of this method *would* be a breaking
change.

Another reason this work the way it is is that functions in CQL are just that,
functions. Each execution is unique and they have no notion of being executed in
the same statement/batch/whatever. I actually think this is sensible, assuming
one stops being obsessed with what other databases that aren't Apache Cassandra
do.

I will note that Ben seems to suggest keeping the return of now() unique across
call while keeping the time component equals, thus varying the rest of the uuid
bytes. However:
 - I'm starting to wonder what this would buy us. Why would someone be super
   confused by the time changing across calls (in a single statement/batch), but
   be totally not confused by the actual full return to not be equal? And how is
   that actually useful: you're having different result anyway and you're
   letting the server pick the timestamp in the first place, so you're probably
   not caring about milliseconds precision of that timestamp in the first place.
 - This would basically be a violation of the timeuuid spec
 - This would be a big pain in the code and make of now() a special case
among functions. I'm unconvinced special cases are making things easier
in general.

So I'm all for improving the documentation if this confuses users due to
expectations (mistakenly) carried from prior experiences, and please
feel free to open a JIRA for that. I'm a lot less in agreement that there is
something wrong with the way the function behave in principle.

> I can see why this issue has been largely ignored and hasn't had a chance for
> the behaviour to be formally defined

Don't make too much assumptions. The behavior is perfectly well defined: now()
is a "normal" function and is evaluated whenever it's called according to the
timeuuid spec (or as close to it as we can make it).

On Thu, Dec 1, 2016 at 7:25 AM, Benjamin Roth 
> wrote:

Great comment. +1

Am 01.12.2016 06:29 schrieb "Ben Bromhead" 
>:
tl;dr +1 yup raise a jira to discuss how now() should behave in a single 
statement (and possible extend to batch statements).

The values of now should be the same if you assume that now() works like it 
does in relational databases such as postgres or mysql, however at the moment 
it instead works like sysdate() in mysql. Given that CQL is supposed to be SQL 
like, I think the assumption around the behaviour of now() was a fair one to 
make.

I definitely agree that raising a jira ticket would be a great place to discuss 
what the behaviour of now() should be for Cassandra. Personally I would be in 
favour of seeing the deterministic component (the actual time part) being the 
same across multiple calls in the one statement or multiple statements in a 
batch.

Cassandra documentation does not make any claims as to how now() works within a 
single statement and reading the code it shows the intent is to work like 
sysdate() from MySQL rather than now(). One of the identified dangers of making 
cql similar to sql is that, while yes it aids adoption, users will find that 
SQL like things don't behave as expected. Of course as a user, one shouldn't 
have to read the source code to determine correct behaviour.

Given that a timeuuid is made up of deterministic and (pseudo) 
non-deterministic components I can see why this issue has been largely ignored 
and hasn't had a 

Re: Cassandra 2.x Stability

[Benjamin Roth]

A little experience report on MVs:

We use them in production (3.10-trunk) and they work really well on normal
read/write operations but streaming operations (bootstrap, repair, rebuild,
decommision) can kill your cluster and/or your nerves.
We will stay with MVs as we need them and want them.
I rolled out a patch on MV streaming on our production cluster a few hours
ago as we had problems with bootstrapping new nodes.

Before:
- Error log was completely flooded with WTEs
- Bootstrap either failed due to exceptions or wasn't even close to finish
after 24h - it just did not work

After
- Bootstrap finished without a single error log after less than 5:30h

I started to roll out that patch to the whole cluster to see how repairs
are affected. Will keep you updated.

There is no dedicated JIRA issue assigned as it addresses multiple tickets
like CASSANDRA-12905 + CASSANDRA-12888


2016-12-01 16:21 GMT+01:00 Jonathan Haddad :

> I agree with everything you just said, Kai.  I'd start a new project with
> 3.0.10.  I'd stay away from MVs though.
>
> On Thu, Dec 1, 2016 at 10:19 AM Kai Wang  wrote:
>
>> Just based on a few observations on this list. Not one week goes by
>> without people asking which release is the most stable on 3.x line. Folks
>> at instaclustr also provide their own 3.x fork for stability issues. etc
>>
>> We developers already have enough to think about. I really don't feel
>> like spending time researching which release of C* I should choose. So for
>> me, 2.2.x is the choice in production.
>>
>> That being said, I have nothing against 3.x. I do like its new storage
>> engine. If I start a brand new project today with zero previous C*
>> experience, I probably would choose 3.0.10 as my starting point. However if
>> I were to upgrade to 3.x, I would have to test it thoroughly in a dev
>> environment with real production load and monitor it very closely on
>> performance, compaction, repair, bootstrap, replacing etc. Data is simply
>> too important to take chances with.
>>
>>
>> On Thu, Dec 1, 2016 at 9:38 AM, Shalom Sagges 
>> wrote:
>>
>> Hey Kai,
>>
>> Thanks for the info. Can you please elaborate on the reasons you'd pick
>> 2.2.6 over 3.0?
>>
>>
>> Shalom Sagges
>> DBA
>> T: +972-74-700-4035 <+972%2074-700-4035>
>>  
>>  We Create Meaningful Connections
>>
>> 
>>
>>
>> On Thu, Dec 1, 2016 at 2:26 PM, Kai Wang  wrote:
>>
>> I have been running 2.2.6 in production. As of today I would still pick
>> it over 3.x for production.
>>
>> On Nov 30, 2016 5:42 AM, "Shalom Sagges"  wrote:
>>
>> Hi Everyone,
>>
>> I'm about to upgrade our 2.0.14 version to a newer 2.x version.
>> At first I thought of upgrading to 2.2.8, but I'm not sure how stable it
>> is, as I understand the 2.2 version was supposed to be a sort of beta
>> version for 3.0 feature-wise, whereas 3.0 upgrade will mainly handle the
>> storage modifications (please correct me if I'm wrong).
>>
>> So my question is, if I need a 2.x version (can't upgrade to 3 due to
>> client considerations), which one should I choose, 2.1.x or 2.2.x? (I'm
>> don't require any new features available in 2.2).
>>
>> Thanks!
>>
>> Shalom Sagges
>> DBA
>> T: +972-74-700-4035 <+972%2074-700-4035>
>>  
>>  We Create Meaningful Connections
>>
>> 
>>
>>
>> This message may contain confidential and/or privileged information.
>> If you are not the addressee or authorized to receive this on behalf of
>> the addressee you must not use, copy, disclose or take action based on this
>> message or any information herein.
>> If you have received this message in error, please advise the sender
>> immediately by reply email and delete this message. Thank you.
>>
>>
>>
>> This message may contain confidential and/or privileged information.
>> If you are not the addressee or authorized to receive this on behalf of
>> the addressee you must not use, copy, disclose or take action based on this
>> message or any information herein.
>> If you have received this message in error, please advise the sender
>> immediately by reply email and delete this message. Thank you.
>>
>>
>>


-- 
Benjamin Roth
Prokurist

Jaumo GmbH · www.jaumo.com
Wehrstraße 46 · 73035 Göppingen · Germany
Phone +49 7161 304880-6 · Fax +49 7161 304880-1
AG Ulm · HRB 731058 · Managing Director: Jens Kammerer

Re: Cassandra 2.x Stability

[Kai Wang]

Just based on a few observations on this list. Not one week goes by without
people asking which release is the most stable on 3.x line. Folks at
instaclustr also provide their own 3.x fork for stability issues. etc

We developers already have enough to think about. I really don't feel like
spending time researching which release of C* I should choose. So for me,
2.2.x is the choice in production.

That being said, I have nothing against 3.x. I do like its new storage
engine. If I start a brand new project today with zero previous C*
experience, I probably would choose 3.0.10 as my starting point. However if
I were to upgrade to 3.x, I would have to test it thoroughly in a dev
environment with real production load and monitor it very closely on
performance, compaction, repair, bootstrap, replacing etc. Data is simply
too important to take chances with.


On Thu, Dec 1, 2016 at 9:38 AM, Shalom Sagges 
wrote:

> Hey Kai,
>
> Thanks for the info. Can you please elaborate on the reasons you'd pick
> 2.2.6 over 3.0?
>
>
> Shalom Sagges
> DBA
> T: +972-74-700-4035 <+972%2074-700-4035>
>  
>  We Create Meaningful Connections
>
> 
>
>
> On Thu, Dec 1, 2016 at 2:26 PM, Kai Wang  wrote:
>
>> I have been running 2.2.6 in production. As of today I would still pick
>> it over 3.x for production.
>>
>> On Nov 30, 2016 5:42 AM, "Shalom Sagges"  wrote:
>>
>>> Hi Everyone,
>>>
>>> I'm about to upgrade our 2.0.14 version to a newer 2.x version.
>>> At first I thought of upgrading to 2.2.8, but I'm not sure how stable it
>>> is, as I understand the 2.2 version was supposed to be a sort of beta
>>> version for 3.0 feature-wise, whereas 3.0 upgrade will mainly handle the
>>> storage modifications (please correct me if I'm wrong).
>>>
>>> So my question is, if I need a 2.x version (can't upgrade to 3 due to
>>> client considerations), which one should I choose, 2.1.x or 2.2.x? (I'm
>>> don't require any new features available in 2.2).
>>>
>>> Thanks!
>>>
>>> Shalom Sagges
>>> DBA
>>> T: +972-74-700-4035 <+972%2074-700-4035>
>>>  
>>>  We Create Meaningful Connections
>>>
>>> 
>>>
>>>
>>> This message may contain confidential and/or privileged information.
>>> If you are not the addressee or authorized to receive this on behalf of
>>> the addressee you must not use, copy, disclose or take action based on this
>>> message or any information herein.
>>> If you have received this message in error, please advise the sender
>>> immediately by reply email and delete this message. Thank you.
>>>
>>
>
> This message may contain confidential and/or privileged information.
> If you are not the addressee or authorized to receive this on behalf of
> the addressee you must not use, copy, disclose or take action based on this
> message or any information herein.
> If you have received this message in error, please advise the sender
> immediately by reply email and delete this message. Thank you.
>

Re: Cassandra 2.x Stability

[Jonathan Haddad]

I agree with everything you just said, Kai.  I'd start a new project with
3.0.10.  I'd stay away from MVs though.

On Thu, Dec 1, 2016 at 10:19 AM Kai Wang  wrote:

> Just based on a few observations on this list. Not one week goes by
> without people asking which release is the most stable on 3.x line. Folks
> at instaclustr also provide their own 3.x fork for stability issues. etc
>
> We developers already have enough to think about. I really don't feel like
> spending time researching which release of C* I should choose. So for me,
> 2.2.x is the choice in production.
>
> That being said, I have nothing against 3.x. I do like its new storage
> engine. If I start a brand new project today with zero previous C*
> experience, I probably would choose 3.0.10 as my starting point. However if
> I were to upgrade to 3.x, I would have to test it thoroughly in a dev
> environment with real production load and monitor it very closely on
> performance, compaction, repair, bootstrap, replacing etc. Data is simply
> too important to take chances with.
>
>
> On Thu, Dec 1, 2016 at 9:38 AM, Shalom Sagges 
> wrote:
>
> Hey Kai,
>
> Thanks for the info. Can you please elaborate on the reasons you'd pick
> 2.2.6 over 3.0?
>
>
> Shalom Sagges
> DBA
> T: +972-74-700-4035 <+972%2074-700-4035>
>  
>  We Create Meaningful Connections
>
> 
>
>
> On Thu, Dec 1, 2016 at 2:26 PM, Kai Wang  wrote:
>
> I have been running 2.2.6 in production. As of today I would still pick it
> over 3.x for production.
>
> On Nov 30, 2016 5:42 AM, "Shalom Sagges"  wrote:
>
> Hi Everyone,
>
> I'm about to upgrade our 2.0.14 version to a newer 2.x version.
> At first I thought of upgrading to 2.2.8, but I'm not sure how stable it
> is, as I understand the 2.2 version was supposed to be a sort of beta
> version for 3.0 feature-wise, whereas 3.0 upgrade will mainly handle the
> storage modifications (please correct me if I'm wrong).
>
> So my question is, if I need a 2.x version (can't upgrade to 3 due to
> client considerations), which one should I choose, 2.1.x or 2.2.x? (I'm
> don't require any new features available in 2.2).
>
> Thanks!
>
> Shalom Sagges
> DBA
> T: +972-74-700-4035 <+972%2074-700-4035>
>  
>  We Create Meaningful Connections
>
> 
>
>
> This message may contain confidential and/or privileged information.
> If you are not the addressee or authorized to receive this on behalf of
> the addressee you must not use, copy, disclose or take action based on this
> message or any information herein.
> If you have received this message in error, please advise the sender
> immediately by reply email and delete this message. Thank you.
>
>
>
> This message may contain confidential and/or privileged information.
> If you are not the addressee or authorized to receive this on behalf of
> the addressee you must not use, copy, disclose or take action based on this
> message or any information herein.
> If you have received this message in error, please advise the sender
> immediately by reply email and delete this message. Thank you.
>
>
>

Re: Cassandra: maximum size of collection list type

[Selvam Raman]

I am getting the below error when i am having huge list( greater than
3lakh).

"Cassandra timeout during write query at consistency LOCAL_ONE (1 replica
were required but only 0 acknowledged the write)"

2016-12-01 16:20 GMT+00:00 Selvam Raman :

> Hi,
>
> What is the maximum size which can be stored into collection list(in a row
> ) in cassandra.
>
> --
> Selvam Raman
> "லஞ்சம் தவிர்த்து நெஞ்சம் நிமிர்த்து"
>



-- 
Selvam Raman
"லஞ்சம் தவிர்த்து நெஞ்சம் நிமிர்த்து"

Re: Cassandra 2.x Stability

[Kai Wang]

Ben, I just read through those two tickets. It's scarier than I thought.
Thank you for all the investigations and comments.

On Thu, Dec 1, 2016 at 10:31 AM, Benjamin Roth 
wrote:

> A little experience report on MVs:
>
> We use them in production (3.10-trunk) and they work really well on normal
> read/write operations but streaming operations (bootstrap, repair, rebuild,
> decommision) can kill your cluster and/or your nerves.
> We will stay with MVs as we need them and want them.
> I rolled out a patch on MV streaming on our production cluster a few hours
> ago as we had problems with bootstrapping new nodes.
>
> Before:
> - Error log was completely flooded with WTEs
> - Bootstrap either failed due to exceptions or wasn't even close to finish
> after 24h - it just did not work
>
> After
> - Bootstrap finished without a single error log after less than 5:30h
>
> I started to roll out that patch to the whole cluster to see how repairs
> are affected. Will keep you updated.
>
> There is no dedicated JIRA issue assigned as it addresses multiple tickets
> like CASSANDRA-12905 + CASSANDRA-12888
>
>
> 2016-12-01 16:21 GMT+01:00 Jonathan Haddad :
>
>> I agree with everything you just said, Kai.  I'd start a new project with
>> 3.0.10.  I'd stay away from MVs though.
>>
>> On Thu, Dec 1, 2016 at 10:19 AM Kai Wang  wrote:
>>
>>> Just based on a few observations on this list. Not one week goes by
>>> without people asking which release is the most stable on 3.x line. Folks
>>> at instaclustr also provide their own 3.x fork for stability issues. etc
>>>
>>> We developers already have enough to think about. I really don't feel
>>> like spending time researching which release of C* I should choose. So for
>>> me, 2.2.x is the choice in production.
>>>
>>> That being said, I have nothing against 3.x. I do like its new storage
>>> engine. If I start a brand new project today with zero previous C*
>>> experience, I probably would choose 3.0.10 as my starting point. However if
>>> I were to upgrade to 3.x, I would have to test it thoroughly in a dev
>>> environment with real production load and monitor it very closely on
>>> performance, compaction, repair, bootstrap, replacing etc. Data is simply
>>> too important to take chances with.
>>>
>>>
>>> On Thu, Dec 1, 2016 at 9:38 AM, Shalom Sagges 
>>> wrote:
>>>
>>> Hey Kai,
>>>
>>> Thanks for the info. Can you please elaborate on the reasons you'd pick
>>> 2.2.6 over 3.0?
>>>
>>>
>>> Shalom Sagges
>>> DBA
>>> T: +972-74-700-4035 <+972%2074-700-4035>
>>>  
>>>  We Create Meaningful Connections
>>>
>>> 
>>>
>>>
>>> On Thu, Dec 1, 2016 at 2:26 PM, Kai Wang  wrote:
>>>
>>> I have been running 2.2.6 in production. As of today I would still pick
>>> it over 3.x for production.
>>>
>>> On Nov 30, 2016 5:42 AM, "Shalom Sagges"  wrote:
>>>
>>> Hi Everyone,
>>>
>>> I'm about to upgrade our 2.0.14 version to a newer 2.x version.
>>> At first I thought of upgrading to 2.2.8, but I'm not sure how stable it
>>> is, as I understand the 2.2 version was supposed to be a sort of beta
>>> version for 3.0 feature-wise, whereas 3.0 upgrade will mainly handle the
>>> storage modifications (please correct me if I'm wrong).
>>>
>>> So my question is, if I need a 2.x version (can't upgrade to 3 due to
>>> client considerations), which one should I choose, 2.1.x or 2.2.x? (I'm
>>> don't require any new features available in 2.2).
>>>
>>> Thanks!
>>>
>>> Shalom Sagges
>>> DBA
>>> T: +972-74-700-4035 <+972%2074-700-4035>
>>>  
>>>  We Create Meaningful Connections
>>>
>>> 
>>>
>>>
>>> This message may contain confidential and/or privileged information.
>>> If you are not the addressee or authorized to receive this on behalf of
>>> the addressee you must not use, copy, disclose or take action based on this
>>> message or any information herein.
>>> If you have received this message in error, please advise the sender
>>> immediately by reply email and delete this message. Thank you.
>>>
>>>
>>>
>>> This message may contain confidential and/or privileged information.
>>> If you are not the addressee or authorized to receive this on behalf of
>>> the addressee you must not use, copy, disclose or take action based on this
>>> message or any information herein.
>>> If you have received this message in error, please advise the sender
>>> immediately by reply email and delete this message. Thank you.
>>>
>>>
>>>
>
>
> --
> Benjamin Roth
> Prokurist
>
> Jaumo GmbH 

Re: Why does `now()` produce different times within the same query?

[Sylvain Lebresne]

On Thu, Dec 1, 2016 at 4:44 PM, Edward Capriolo 
wrote:

>
> I am not sure you saw my reply on thread but I believe everyone's needs
> can be met I will copy that here:
>

I saw it, but the real problem that was raised initially was not that of
UDF and of allowing both behavior. It's a matter of people being confused
by the behavior of a non-UDF function, now(), and suggesting it should be
changed.

The Hive idea is interesting I guess, and we can switch to discussing that,
but it's a different problem really and I'm not a fond of derailing
threads. I will just note though that if we're not talking about a
confusion issue but rather how to get a timeuuid to be fixed within a
statement, then there is much much more trivial solution: generate it
client side. The `now()` function is a small convenience but there is
nothing you cannot do without it client side, and that actually basically
stands for almost any use of (non aggregate) function in Cassandra
currently.


>
>
> "Food for thought: Hive's UDFs introduced an annotation
> @UDFType(deterministic = false)
>
> http://dmtolpeko.com/2014/10/15/invoking-stateful-udf-at-map
> -and-reduce-side-in-hive/
>
> The effect is the query planner can see when such a UDF is in use and
> determine the value once at the start of a very long query."
>
> Essentially hive had a similar if not identical problem, during a long
> running distributed process like map/reduce some users wanted the semantics
> of:
>
> 1) Each call should have a new timestamps
>
> While other users wanted the semantics of:
>
> 2) Each call should generate the same timestamp
>
> The solution implemented was to add an annotation to udf such that the
> query planner would pick up the annotation and act accordingly.
>
> (Here is a related issue https://issues.apache.org/jira/browse/HIVE-1986
>
> As a result you can essentially implement two UDFS
>
> @UDFType(deterministic = false)
> public class UDFNow
>
> and for the other people
>
> @UDFType(deterministic = true)
> public class UDFNowOnce extends UDFNow
>
> Both user cases are met in a sensible way.
>

Re: Why does `now()` produce different times within the same query?

[Jonathan Haddad]

+1 to everything Sylvan said.
On Thu, Dec 1, 2016 at 11:09 AM Sylvain Lebresne 
wrote:

> On Thu, Dec 1, 2016 at 4:44 PM, Edward Capriolo 
> wrote:
>
>
> I am not sure you saw my reply on thread but I believe everyone's needs
> can be met I will copy that here:
>
>
> I saw it, but the real problem that was raised initially was not that of
> UDF and of allowing both behavior. It's a matter of people being confused
> by the behavior of a non-UDF function, now(), and suggesting it should be
> changed.
>
> The Hive idea is interesting I guess, and we can switch to discussing
> that, but it's a different problem really and I'm not a fond of derailing
> threads. I will just note though that if we're not talking about a
> confusion issue but rather how to get a timeuuid to be fixed within a
> statement, then there is much much more trivial solution: generate it
> client side. The `now()` function is a small convenience but there is
> nothing you cannot do without it client side, and that actually basically
> stands for almost any use of (non aggregate) function in Cassandra
> currently.
>
>
>
>
> "Food for thought: Hive's UDFs introduced an annotation  
> @UDFType(deterministic
> = false)
>
>
> http://dmtolpeko.com/2014/10/15/invoking-stateful-udf-at-map-and-reduce-side-in-hive/
>
> The effect is the query planner can see when such a UDF is in use and
> determine the value once at the start of a very long query."
>
> Essentially hive had a similar if not identical problem, during a long
> running distributed process like map/reduce some users wanted the semantics
> of:
>
> 1) Each call should have a new timestamps
>
> While other users wanted the semantics of:
>
> 2) Each call should generate the same timestamp
>
> The solution implemented was to add an annotation to udf such that the
> query planner would pick up the annotation and act accordingly.
>
> (Here is a related issue https://issues.apache.org/jira/browse/HIVE-1986
>
> As a result you can essentially implement two UDFS
>
> @UDFType(deterministic = false)
> public class UDFNow
>
> and for the other people
>
> @UDFType(deterministic = true)
> public class UDFNowOnce extends UDFNow
>
> Both user cases are met in a sensible way.
>
>

Re: Why does `now()` produce different times within the same query?

[Cody Yancey]

On Thu, Dec 1, 2016 at 11:09 AM Sylvain Lebresne 
wrote:

> there is much much more trivial solution: generate it client side. The
> `now()` function is a small convenience but there is nothing you cannot do
> without it client side
>

Please see my post above as to why this is a bad idea for inserts based on
request time where knowing the time the request was made is actually
important.

Cody

>

Cassandra: maximum size of collection list type

[Selvam Raman]

Hi,

What is the maximum size which can be stored into collection list(in a row
) in cassandra.

-- 
Selvam Raman
"லஞ்சம் தவிர்த்து நெஞ்சம் நிமிர்த்து"

Re: Why does `now()` produce different times within the same query?

[Edward Capriolo]

On Thu, Dec 1, 2016 at 11:09 AM, Sylvain Lebresne 
wrote:

> On Thu, Dec 1, 2016 at 4:44 PM, Edward Capriolo 
> wrote:
>
>>
>> I am not sure you saw my reply on thread but I believe everyone's needs
>> can be met I will copy that here:
>>
>
> I saw it, but the real problem that was raised initially was not that of
> UDF and of allowing both behavior. It's a matter of people being confused
> by the behavior of a non-UDF function, now(), and suggesting it should be
> changed.
>
> The Hive idea is interesting I guess, and we can switch to discussing
> that, but it's a different problem really and I'm not a fond of derailing
> threads. I will just note though that if we're not talking about a
> confusion issue but rather how to get a timeuuid to be fixed within a
> statement, then there is much much more trivial solution: generate it
> client side. The `now()` function is a small convenience but there is
> nothing you cannot do without it client side, and that actually basically
> stands for almost any use of (non aggregate) function in Cassandra
> currently.
>
>
>>
>>
>> "Food for thought: Hive's UDFs introduced an annotation
>> @UDFType(deterministic = false)
>>
>> http://dmtolpeko.com/2014/10/15/invoking-stateful-udf-at-map
>> -and-reduce-side-in-hive/
>>
>> The effect is the query planner can see when such a UDF is in use and
>> determine the value once at the start of a very long query."
>>
>> Essentially hive had a similar if not identical problem, during a long
>> running distributed process like map/reduce some users wanted the semantics
>> of:
>>
>> 1) Each call should have a new timestamps
>>
>> While other users wanted the semantics of:
>>
>> 2) Each call should generate the same timestamp
>>
>> The solution implemented was to add an annotation to udf such that the
>> query planner would pick up the annotation and act accordingly.
>>
>> (Here is a related issue https://issues.apache.org/jira/browse/HIVE-1986
>>
>> As a result you can essentially implement two UDFS
>>
>> @UDFType(deterministic = false)
>> public class UDFNow
>>
>> and for the other people
>>
>> @UDFType(deterministic = true)
>> public class UDFNowOnce extends UDFNow
>>
>> Both user cases are met in a sensible way.
>>
>
>
I agree that changing the semantics of something already in existence is a
bad idea. What is there "now" no pun on works should stay working as is.

I will also point out that presto addresses this issue with specific
functions:

https://prestodb.io/docs/current/functions/datetime.html

localtime -> time

Returns the current time as of the start of the query.
localtimestamp -> timestamp

Returns the current timestamp as of the start of the query.

Re: Cassandra: maximum size of collection list type

[Benjamin Roth]

You can read it in the docs but i think it was 2^16 aka 64k

Am 01.12.2016 18:00 schrieb "Selvam Raman" :

> Hi,
>
> What is the maximum size which can be stored into collection list(in a row
> ) in cassandra.
>
> --
> Selvam Raman
> "லஞ்சம் தவிர்த்து நெஞ்சம் நிமிர்த்து"
>

Re: Cassandra: maximum size of collection list type

[Jeff Jirsa]

A 64k element list will not be performant. Even 10% of that will be painful for 
many users. 

 

 

 

 

From: Vladimir Yudovin 
Reply-To: "user@cassandra.apache.org" 
Date: Thursday, December 1, 2016 at 10:34 AM
To: user 
Subject: Re: Cassandra: maximum size of collection list type

 

As doc says:

 The maximum size of an item in a collection is 64K.
Keep collections small to prevent delays during querying because Cassandra 
reads a collection in its entirety. The collection is not paged internally.
As discussed earlier, collections are designed to store only a small amount of 
data.
Never insert more than 64K items in a collection. 
If you insert more than 64K items into a collection, only 64K of them will be 
queryable, resulting in data loss.
 

Best regards, Vladimir Yudovin, 

Winguzone - Cloud Cassandra Hosting

 

 

 On Thu, 01 Dec 2016 12:17:23 -0500 Benjamin Roth  
wrote 

 

You can read it in the docs but i think it was 2^16 aka 64k

 

Am 01.12.2016 18:00 schrieb "Selvam Raman" :

Hi,

 

What is the maximum size which can be stored into collection list(in a row ) in 
cassandra.

 

-- 

Selvam Raman

"லஞ்சம் தவிர்த்து நெஞ்சம் நிமிர்த்து"

 



smime.p7s
Description: S/MIME cryptographic signature

Re: Cassandra: maximum size of collection list type

[DuyHai Doan]

My usual rule-of-thumb recommendation for most use-cases is not to exceed
1000 elements. The reasoning behind this is if you have more than 1000
elements, maybe using extra clustering columns to handle them is cleaner
and put less pressure on the heap since all the collection data are fetched
at once for each access

On Thu, Dec 1, 2016 at 7:38 PM, Jeff Jirsa 
wrote:

> A 64k element list will not be performant. Even 10% of that will be
> painful for many users.
>
>
>
>
>
>
>
>
>
> *From: *Vladimir Yudovin 
> *Reply-To: *"user@cassandra.apache.org" 
> *Date: *Thursday, December 1, 2016 at 10:34 AM
> *To: *user 
> *Subject: *Re: Cassandra: maximum size of collection list type
>
>
>
> As doc says:
>
>-  The maximum size of an item in a collection is 64K.
>
>
>- Keep collections small to prevent delays during querying because
>Cassandra reads a collection in its entirety. The collection is not paged
>internally.
>As discussed earlier, collections are designed to store only a small
>amount of data.
>
>
>- Never insert more than 64K items in a collection.
>If you insert more than 64K items into a collection, only 64K of them
>will be queryable, resulting in data loss.
>
>
>
> Best regards, Vladimir Yudovin,
>
> *Winguzone
> 
> - Cloud Cassandra Hosting*
>
>
>
>
>
>  On Thu, 01 Dec 2016 12:17:23 -0500 *Benjamin Roth
> >* wrote 
>
>
>
> You can read it in the docs but i think it was 2^16 aka 64k
>
>
>
> Am 01.12.2016 18:00 schrieb "Selvam Raman" :
>
> Hi,
>
>
>
> What is the maximum size which can be stored into collection list(in a row
> ) in cassandra.
>
>
>
> --
>
> Selvam Raman
>
> "லஞ்சம் தவிர்த்து நெஞ்சம் நிமிர்த்து"
>
>
>

Re: Cassandra: maximum size of collection list type

[Vladimir Yudovin]

As doc says:

 The maximum size of an item in a collection is 64K.


Keep collections small to prevent delays during querying because Cassandra 
reads a collection in its entirety. The collection is not paged internally.
As discussed earlier, collections are designed to store only a small amount of 
data.


Never insert more than 64K items in a collection. 
If you insert more than 64K items into a collection, only 64K of them will be 
queryable, resulting in data loss.




Best regards, Vladimir Yudovin, 

Winguzone - Cloud Cassandra Hosting






 On Thu, 01 Dec 2016 12:17:23 -0500 Benjamin Roth 
benjamin.r...@jaumo.com wrote 




You can read it in the docs but i think it was 2^16 aka 64k



Am 01.12.2016 18:00 schrieb "Selvam Raman" sel...@gmail.com:

Hi,



What is the maximum size which can be stored into collection list(in a row ) in 
cassandra.



-- 

Selvam Raman

"லஞ்சம் தவிர்த்து நெஞ்சம் நிமிர்த்து"

Re: Why does `now()` produce different times within the same query?

[Ben Bromhead]

>
>
>
> I will note that Ben seems to suggest keeping the return of now() unique
> across
> call while keeping the time component equals, thus varying the rest of the
> uuid
> bytes. However:
>  - I'm starting to wonder what this would buy us. Why would someone be
> super
>confused by the time changing across calls (in a single
> statement/batch), but
>be totally not confused by the actual full return to not be equal?
>
Given that a common way of interacting with timeuuids is with toTimestamp I
can see the confusion and assumptions around behaviour.

And how is
>that actually useful: you're having different result anyway and you're
>letting the server pick the timestamp in the first place, so you're
> probably
>not caring about milliseconds precision of that timestamp in the first
> place.
>
If you want consistency of timestamps within your query as OP did I can see
how this is useful. Postgres claims this is a "feature".

 - This would basically be a violation of the timeuuid spec
>

Not quite... Type 1 uuids let you swap out the low 47 bits of the node
component with other randomly generated bits (
https://www.ietf.org/rfc/rfc4122.txt)

 - This would be a big pain in the code and make of now() a special case
> among functions. I'm unconvinced special cases are making things easier
> in general.
>

On reflection, I have to agree here, now() has been around for ever and
this is the first anecdote I've seen of someone getting caught out.

However with my user advocate hat on I think it would be worth
investigating further beyond a documentation update if others found it a
sticking point in Cassandra adoption.


> So I'm all for improving the documentation if this confuses users due to
> expectations (mistakenly) carried from prior experiences, and please
> feel free to open a JIRA for that. I'm a lot less in agreement that there
> is
> something wrong with the way the function behave in principle.
>


> > I can see why this issue has been largely ignored and hasn't had a
> chance for
> > the behaviour to be formally defined
>
> Don't make too much assumptions. The behavior is perfectly well defined:
> now()
> is a "normal" function and is evaluated whenever it's called according to
> the
> timeuuid spec (or as close to it as we can make it).
>
Maybe formally defined is the wrong term... Formally documented?

>
> On Thu, Dec 1, 2016 at 7:25 AM, Benjamin Roth 
> wrote:
>
> Great comment. +1
>
> Am 01.12.2016 06:29 schrieb "Ben Bromhead" :
>
> tl;dr +1 yup raise a jira to discuss how now() should behave in a single
> statement (and possible extend to batch statements).
>
> The values of now should be the same if you assume that now() works like
> it does in relational databases such as postgres or mysql, however at the
> moment it instead works like sysdate() in mysql. Given that CQL is supposed
> to be SQL like, I think the assumption around the behaviour of now() was a
> fair one to make.
>
> I definitely agree that raising a jira ticket would be a great place to
> discuss what the behaviour of now() should be for Cassandra. Personally I
> would be in favour of seeing the deterministic component (the actual time
> part) being the same across multiple calls in the one statement or multiple
> statements in a batch.
>
> Cassandra documentation does not make any claims as to how now() works
> within a single statement and reading the code it shows the intent is to
> work like sysdate() from MySQL rather than now(). One of the identified
> dangers of making cql similar to sql is that, while yes it aids adoption,
> users will find that SQL like things don't behave as expected. Of course as
> a user, one shouldn't have to read the source code to determine correct
> behaviour.
>
> Given that a timeuuid is made up of deterministic and (pseudo)
> non-deterministic components I can see why this issue has been largely
> ignored and hasn't had a chance for the behaviour to be formally defined
> (you would expect now to return the same time in the one statement despite
> multiple calls, but you wouldn't expect the same behaviour for say a call
> to rand()).
>
>
>
>
>
>
>
> On Wed, 30 Nov 2016 at 19:54 Cody Yancey  wrote:
>
> This is not a bug, and in fact changing it would be a serious bug.
>
> False. Absolutely no consumer would be broken by a change to guarantee an
> identical time component that isn't broken already, for the simple reason
> your code already has to handle that case, as it is in fact the majority
> case RIGHT NOW. Users can hit this bug, in production, because unit tests
> might not experienced it! The time component should be the time that the
> command was processed by the coordinator node.
>
>  would one expect a java/py/bash script that loops
>
> Individual Cassandra writes (which is what OP is referring to
> specifically) are not loops. They are in almost every case atomic
> operations that either 

Re: Why does `now()` produce different times within the same query?

[Marko Švaljek]

One millisecond is not an issue in most of Internet of Things projects out
there. There are lots of connection related things that add far more
latency to the requests than that. Especially if you take into account the
time it takes for the data to actually come to a cassandra node in the
background etc. I'm simply not aware of any larger projects where edge
devices write directly to cassandra.

Requests almost always come in to some sort of gateway before that. The
usual pattern is storing the timestamp measured on the device (if it even
has own clock) and timestamp when it was received on the platform side.
Having two same timestamps on millisecond level in one insert statement
generated by now() simply doesn't add that much to the table.

Only case that comes to my mind would be when there is time series
bucketing of inserts and placing measurements in partitions based on some
sort of a mapping function with the results of now() but then again this is
usually done on the server side, I'm not sure it would be best practice to
do it within the insert.

Even if it would be done that way, analytics (be it near real time or
batch) usually takes that kind of things into account and compensates -
reports rarely show millisecond level dynamics.

In the end it just wouldn't be a good idea to change behaviour of a
function being around for quite some time.


@msvaljek 

2016-12-01 18:10 GMT+01:00 Cody Yancey :

> On Thu, Dec 1, 2016 at 11:09 AM Sylvain Lebresne 
> wrote:
>
>> there is much much more trivial solution: generate it client side. The
>> `now()` function is a small convenience but there is nothing you cannot do
>> without it client side
>>
>
> Please see my post above as to why this is a bad idea for inserts based on
> request time where knowing the time the request was made is actually
> important.
>
> Cody
>
>>
>

empty buckets with STCS

[John Sanda]

I have 2.2.1 Cassandra node that does not appear to be compacting
SSTables.  The table is currently configured with STCS. I turned on some
debug logging and when the compaction checks run, they log:

Compaction buckets are []

I have been going over SizeTieredCompactionStrategy.java and looking in
particular at the getNextBackgroundTables method since that is where the
above message is logged. Assuming I am reading the logs correctly, what if
anything besides all SSTables being blacklisted would result in empty
buckets? As I understand the code if an SSTable falls outside the threshold
of all existing buckets, it will be added to a new bucket.


- John

Re: Cassandra 2.x Stability

[Kai Wang]

I have been running 2.2.6 in production. As of today I would still pick it
over 3.x for production.

On Nov 30, 2016 5:42 AM, "Shalom Sagges"  wrote:

> Hi Everyone,
>
> I'm about to upgrade our 2.0.14 version to a newer 2.x version.
> At first I thought of upgrading to 2.2.8, but I'm not sure how stable it
> is, as I understand the 2.2 version was supposed to be a sort of beta
> version for 3.0 feature-wise, whereas 3.0 upgrade will mainly handle the
> storage modifications (please correct me if I'm wrong).
>
> So my question is, if I need a 2.x version (can't upgrade to 3 due to
> client considerations), which one should I choose, 2.1.x or 2.2.x? (I'm
> don't require any new features available in 2.2).
>
> Thanks!
>
> Shalom Sagges
> DBA
> T: +972-74-700-4035 <+972%2074-700-4035>
>  
>  We Create Meaningful Connections
>
> 
>
>
> This message may contain confidential and/or privileged information.
> If you are not the addressee or authorized to receive this on behalf of
> the addressee you must not use, copy, disclose or take action based on this
> message or any information herein.
> If you have received this message in error, please advise the sender
> immediately by reply email and delete this message. Thank you.
>

Re: Why does `now()` produce different times within the same query?

[Sylvain Lebresne]

One can of course always open a JIRA, but I'm going to strongly disagree
with a
change here (outside of a documentation one that is).

The now() function is a timeuuid generator, and it thus generates a unique
timeuuid on every call, as specified by the timeuuid spec. I'll note that
document lists it under "Timeuuid functions", and has sentences like
"the value returned by now() is guaranteed to be unique", so while I'm sure
the
documentation can be further clarified, I think it's pretty clear it's not
the
now() of SQL, and getting unique values on every call shouldn't be *that*
surprising.

Also, now() was primarily meant for use on timeuuid clustering columns for a
time-series like table, something like:
  CREATE TABLE ts (
k int,
t timeuuid,
v text,
PRIMARY KEY (k, t)
  )
and if you use it multiple times in a batch, this would look something like:
  BEGIN BATCH
INSERT INTO ts (k, t, v) VALUES (0, now(), 'foo');
INSERT INTO ts (k, t, v) VALUES (0, now(), 'bar');
  APPLY BATCH
and you definitively want that to insert 2 "events", not just one.

This is also why changing the behavior of this method *would* be a breaking
change.

Another reason this work the way it is is that functions in CQL are just
that,
functions. Each execution is unique and they have no notion of being
executed in
the same statement/batch/whatever. I actually think this is sensible,
assuming
one stops being obsessed with what other databases that aren't Apache
Cassandra
do.

I will note that Ben seems to suggest keeping the return of now() unique
across
call while keeping the time component equals, thus varying the rest of the
uuid
bytes. However:
 - I'm starting to wonder what this would buy us. Why would someone be super
   confused by the time changing across calls (in a single
statement/batch), but
   be totally not confused by the actual full return to not be equal? And
how is
   that actually useful: you're having different result anyway and you're
   letting the server pick the timestamp in the first place, so you're
probably
   not caring about milliseconds precision of that timestamp in the first
place.
 - This would basically be a violation of the timeuuid spec
 - This would be a big pain in the code and make of now() a special case
among functions. I'm unconvinced special cases are making things easier
in general.

So I'm all for improving the documentation if this confuses users due to
expectations (mistakenly) carried from prior experiences, and please
feel free to open a JIRA for that. I'm a lot less in agreement that there is
something wrong with the way the function behave in principle.

> I can see why this issue has been largely ignored and hasn't had a chance
for
> the behaviour to be formally defined

Don't make too much assumptions. The behavior is perfectly well defined:
now()
is a "normal" function and is evaluated whenever it's called according to
the
timeuuid spec (or as close to it as we can make it).

On Thu, Dec 1, 2016 at 7:25 AM, Benjamin Roth 
wrote:

> Great comment. +1
>
> Am 01.12.2016 06:29 schrieb "Ben Bromhead" :
>
>> tl;dr +1 yup raise a jira to discuss how now() should behave in a single
>> statement (and possible extend to batch statements).
>>
>> The values of now should be the same if you assume that now() works like
>> it does in relational databases such as postgres or mysql, however at the
>> moment it instead works like sysdate() in mysql. Given that CQL is supposed
>> to be SQL like, I think the assumption around the behaviour of now() was a
>> fair one to make.
>>
>> I definitely agree that raising a jira ticket would be a great place to
>> discuss what the behaviour of now() should be for Cassandra. Personally I
>> would be in favour of seeing the deterministic component (the actual time
>> part) being the same across multiple calls in the one statement or multiple
>> statements in a batch.
>>
>> Cassandra documentation does not make any claims as to how now() works
>> within a single statement and reading the code it shows the intent is to
>> work like sysdate() from MySQL rather than now(). One of the identified
>> dangers of making cql similar to sql is that, while yes it aids adoption,
>> users will find that SQL like things don't behave as expected. Of course as
>> a user, one shouldn't have to read the source code to determine correct
>> behaviour.
>>
>> Given that a timeuuid is made up of deterministic and (pseudo)
>> non-deterministic components I can see why this issue has been largely
>> ignored and hasn't had a chance for the behaviour to be formally defined
>> (you would expect now to return the same time in the one statement despite
>> multiple calls, but you wouldn't expect the same behaviour for say a call
>> to rand()).
>>
>>
>>
>>
>>
>>
>>
>> On Wed, 30 Nov 2016 at 19:54 Cody Yancey  wrote:
>>
>>> This is not a bug, and in fact changing it would be a serious bug.
>>>
>>> 

Re: Sanity checks to run post restore data?

[Benjamin Roth]

Restoring SSTables is not trivial in terms of restoring a consistent state.
It depends on your failure scenario.

- Do you want to recover from unintentionally deleted data? Then it is
possible that you restore the data on one node, run a repair and the data
is deleted again because the tombstones are newer and you just resurrect
your data again. So - you really need to know what you are doing.
- Do you want to recover from complete loss? Maybe its easier to set up a
backup DC.
- You want to resume a dead node? Just rebuild / replace it.
- You truncated a table and want to restore it? Then restoring backed up
SSTables may work as disaster recovery. Although you will lose data between
backup + truncate.

Does that answer your question or did you mean sth completely different?

2016-11-30 21:07 GMT+01:00 Varun Gupta :

> Hi,
>
> We are periodically backing up sstables, and need to learn, what sanity
> checks should be performed after restoring them?
>
> Thanks,
> Varun
>



-- 
Benjamin Roth
Prokurist

Jaumo GmbH · www.jaumo.com
Wehrstraße 46 · 73035 Göppingen · Germany
Phone +49 7161 304880-6 · Fax +49 7161 304880-1
AG Ulm · HRB 731058 · Managing Director: Jens Kammerer