Re: How to insert "Empty" timeuuid by Cql

[Stefania Alborghetti]

You're correct, cassandra 2.1 is still using protocol version 3. You need
at least version 2.2.

On Thu, Oct 20, 2016 at 11:18 AM, Lijun Huang  wrote:

> Thanks Stefania, we haven't tried before, and I think the version is not
> matched, we are still using,
> [cqlsh 4.1.1 | Cassandra 2.1.11 | CQL spec 3.1.1 | Thrift protocol 19.39.0]
>
> On Thu, Oct 20, 2016 at 10:33 AM, Stefania Alborghetti <
> stefania.alborghe...@datastax.com> wrote:
>
>> Have you already tried using unset values?
>>
>> http://www.datastax.com/dev/blog/datastax-java-driver-3-0-0-
>> released#unset-values
>>
>> They are only available starting with protocol version 4 however.
>>
>> On Thu, Oct 20, 2016 at 10:19 AM, Lijun Huang  wrote:
>>
>>> Hi Vladimir,
>>>
>>> Indeed, that's a little weird, I think it is like a empty string: '' but
>>> is a timeuuid value. We have many such records that inserted by Astyanax
>>> API, when we select it in cqlsh, it is like as below, note the column4 is
>>> timeuuid, it is not null or some value, just "empty".
>>>
>>> key  | column1  | column2 | column3 | column4 | value
>>> --+++
>>> ++--
>>> test by thrift | accessState |  |  |
>>>| 0x5
>>>
>>> But when we use Cql, we couldn't set this empty value, it is null or
>>> explicit value, like below,
>>>
>>> key  | column1  | column2 | column3 | column4  | value
>>> --+---+-+---
>>> --+--+--
>>>  test by cql   | accessState |  | |  null
>>>   | 0x5
>>>
>>> key  | column1  | column2 | column3 |
>>> column4  | value
>>> ---+--+--+--
>>> --+-
>>> --+-
>>>  test by cql   | accessState |  | |
>>> 4a528300-95cb-11e6-8650-0242f5eaa8c3| 0x5
>>>
>>> I don't know whether you could understand now, if not I could provide
>>> some code related to Astyanax. Really appreciate your help.
>>>
>>>
>>> On Wed, Oct 19, 2016 at 9:53 PM, Vladimir Yudovin 
>>> wrote:
>>>
 Hi,

 what does it exactly mean 'empty timeuuid'?  UUID takes 16 bytes for
 storage, so it should be either null, or some value. Do you mean 'zero'
 UUID?

 Best regards, Vladimir Yudovin,

 *Winguzone  - Hosted Cloud
 CassandraLaunch your cluster in minutes.*


  On Wed, 19 Oct 2016 09:16:29 -0400*coderhlj >* wrote 

 Hi all,

 We use Cassandra 2.1.11 in our product, and we update the Java Drive
 from Astyanax(Thrift API) to DataStax Java Driver(Cql) recently, but we
 encounter a difficult issue as following, please help us, thanks in 
 advance.

 Previously we were using Astyanax API, and we can insert empty timeuuid
 like below, but now we can only insert null timeuuid by cql command but not
 empty one. Is there any cql function to insert an empty timeuuid like
 by Astyanax?
 And this cause a tough problem is that we can not delete the record by
 specifying the primary key, like:
 *delete from "Foo" where column1='test' and column2='accessState' and
 column3='' and column4=(need fill empty uuid here) IF EXISTS;*

 key  | column1  | column2 | column3 | column4 | value
 -+-+-+-
 +-+--
 test by thrift | accessState |  |  |
| 0x5

 key  | column1  | column2 | column3 | column4  | value
 -+-+-+-+--+-
 -
  test by cql   | accessState |  | |  null
 | 0x5


 cqlsh:StorageOS> desc table "Foo";

 CREATE TABLE "Foo" (
   key text,
   column1 text,
   column2 text,
   column3 text,
   column4 timeuuid,
   value blob,
   PRIMARY KEY (key, column1, column2, column3, column4)
 ) WITH COMPACT STORAGE AND
   bloom_filter_fp_chance=0.01 AND
   caching='{"keys":"ALL", "rows_per_partition":"NONE"}' AND
   comment='' AND
   dclocal_read_repair_chance=0.10 AND
   gc_grace_seconds=432000 AND
   read_repair_chance=0.00 AND
   default_time_to_live=0 AND
   speculative_retry='NONE' AND
   memtable_flush_period_in_ms=0 AND
   compaction={'class': 'SizeTieredCompactionStrategy'} AND
   compression={'sstable_compression': 'LZ4Compressor'};

 --
 Thanks,
 Lijun Huang



>>>
>>>
>>> --
>>> Best regards,
>>> 

Re: non incremental repairs with cassandra 2.2+

[Alexander Dejanovski]

Hi Kurt,

we're not actually.
Reaper performs full repair by subrange but does incremental repair on all
ranges at once, node by node.
Subrange is incompatible with incremental repair anyway.

Cheers,

On Thu, Oct 20, 2016 at 5:24 AM kurt Greaves  wrote:

>
> On 19 October 2016 at 17:13, Alexander Dejanovski 
> wrote:
>
> There aren't that many tools I know to orchestrate repairs and we maintain
> a fork of Reaper, that was made by Spotify, and handles incremental repair
> : https://github.com/thelastpickle/cassandra-reaper
>
>
> Looks like you're using subranges with incremental repairs. This will
> generate a lot of anticompactions as you'll only repair a portion of the
> SSTables. You should use forceRepairAsync for incremental repairs so that
> it's possible for the repair to act on the whole SSTable, minimising
> anticompactions.
>
> Kurt Greaves
> k...@instaclustr.com
> www.instaclustr.com
>
-- 
-
Alexander Dejanovski
France
@alexanderdeja

Consultant
Apache Cassandra Consulting
http://www.thelastpickle.com

Re: non incremental repairs with cassandra 2.2+

[kurt Greaves]

On 19 October 2016 at 17:13, Alexander Dejanovski 
wrote:

> There aren't that many tools I know to orchestrate repairs and we maintain
> a fork of Reaper, that was made by Spotify, and handles incremental repair
> : https://github.com/thelastpickle/cassandra-reaper


Looks like you're using subranges with incremental repairs. This will
generate a lot of anticompactions as you'll only repair a portion of the
SSTables. You should use forceRepairAsync for incremental repairs so that
it's possible for the repair to act on the whole SSTable, minimising
anticompactions.

Kurt Greaves
k...@instaclustr.com
www.instaclustr.com

Re: How to insert "Empty" timeuuid by Cql

[Lijun Huang]

Thanks Stefania, we haven't tried before, and I think the version is not
matched, we are still using,
[cqlsh 4.1.1 | Cassandra 2.1.11 | CQL spec 3.1.1 | Thrift protocol 19.39.0]

On Thu, Oct 20, 2016 at 10:33 AM, Stefania Alborghetti <
stefania.alborghe...@datastax.com> wrote:

> Have you already tried using unset values?
>
> http://www.datastax.com/dev/blog/datastax-java-driver-3-0-
> 0-released#unset-values
>
> They are only available starting with protocol version 4 however.
>
> On Thu, Oct 20, 2016 at 10:19 AM, Lijun Huang  wrote:
>
>> Hi Vladimir,
>>
>> Indeed, that's a little weird, I think it is like a empty string: '' but
>> is a timeuuid value. We have many such records that inserted by Astyanax
>> API, when we select it in cqlsh, it is like as below, note the column4 is
>> timeuuid, it is not null or some value, just "empty".
>>
>> key  | column1  | column2 | column3 | column4 | value
>> --+++
>> ++--
>> test by thrift | accessState |  |  |  |
>> 0x5
>>
>> But when we use Cql, we couldn't set this empty value, it is null or
>> explicit value, like below,
>>
>> key  | column1  | column2 | column3 | column4  | value
>> --+---+-+---
>> --+--+--
>>  test by cql   | accessState |  | |  null
>>   | 0x5
>>
>> key  | column1  | column2 | column3 |
>> column4  | value
>> ---+--+--+--
>> --+-
>> --+-
>>  test by cql   | accessState |  | |
>> 4a528300-95cb-11e6-8650-0242f5eaa8c3| 0x5
>>
>> I don't know whether you could understand now, if not I could provide
>> some code related to Astyanax. Really appreciate your help.
>>
>>
>> On Wed, Oct 19, 2016 at 9:53 PM, Vladimir Yudovin 
>> wrote:
>>
>>> Hi,
>>>
>>> what does it exactly mean 'empty timeuuid'?  UUID takes 16 bytes for
>>> storage, so it should be either null, or some value. Do you mean 'zero'
>>> UUID?
>>>
>>> Best regards, Vladimir Yudovin,
>>>
>>> *Winguzone  - Hosted Cloud
>>> CassandraLaunch your cluster in minutes.*
>>>
>>>
>>>  On Wed, 19 Oct 2016 09:16:29 -0400*coderhlj >> >* wrote 
>>>
>>> Hi all,
>>>
>>> We use Cassandra 2.1.11 in our product, and we update the Java Drive
>>> from Astyanax(Thrift API) to DataStax Java Driver(Cql) recently, but we
>>> encounter a difficult issue as following, please help us, thanks in advance.
>>>
>>> Previously we were using Astyanax API, and we can insert empty timeuuid
>>> like below, but now we can only insert null timeuuid by cql command but not
>>> empty one. Is there any cql function to insert an empty timeuuid like
>>> by Astyanax?
>>> And this cause a tough problem is that we can not delete the record by
>>> specifying the primary key, like:
>>> *delete from "Foo" where column1='test' and column2='accessState' and
>>> column3='' and column4=(need fill empty uuid here) IF EXISTS;*
>>>
>>> key  | column1  | column2 | column3 | column4 | value
>>> -+-+-+-
>>> +-+--
>>> test by thrift | accessState |  |  |
>>>| 0x5
>>>
>>> key  | column1  | column2 | column3 | column4  | value
>>> -+-+-+-+--+-
>>> -
>>>  test by cql   | accessState |  | |  null
>>>   | 0x5
>>>
>>>
>>> cqlsh:StorageOS> desc table "Foo";
>>>
>>> CREATE TABLE "Foo" (
>>>   key text,
>>>   column1 text,
>>>   column2 text,
>>>   column3 text,
>>>   column4 timeuuid,
>>>   value blob,
>>>   PRIMARY KEY (key, column1, column2, column3, column4)
>>> ) WITH COMPACT STORAGE AND
>>>   bloom_filter_fp_chance=0.01 AND
>>>   caching='{"keys":"ALL", "rows_per_partition":"NONE"}' AND
>>>   comment='' AND
>>>   dclocal_read_repair_chance=0.10 AND
>>>   gc_grace_seconds=432000 AND
>>>   read_repair_chance=0.00 AND
>>>   default_time_to_live=0 AND
>>>   speculative_retry='NONE' AND
>>>   memtable_flush_period_in_ms=0 AND
>>>   compaction={'class': 'SizeTieredCompactionStrategy'} AND
>>>   compression={'sstable_compression': 'LZ4Compressor'};
>>>
>>> --
>>> Thanks,
>>> Lijun Huang
>>>
>>>
>>>
>>
>>
>> --
>> Best regards,
>> Lijun Huang
>>
>
>
>
> --
>
>
> Stefania Alborghetti
>
> |+852 6114 9265| stefania.alborghe...@datastax.com
>



-- 
Best regards,
Lijun Huang

Re: How to insert "Empty" timeuuid by Cql

[Stefania Alborghetti]

Have you already tried using unset values?

http://www.datastax.com/dev/blog/datastax-java-driver-3-0-0-released#unset-values

They are only available starting with protocol version 4 however.

On Thu, Oct 20, 2016 at 10:19 AM, Lijun Huang  wrote:

> Hi Vladimir,
>
> Indeed, that's a little weird, I think it is like a empty string: '' but
> is a timeuuid value. We have many such records that inserted by Astyanax
> API, when we select it in cqlsh, it is like as below, note the column4 is
> timeuuid, it is not null or some value, just "empty".
>
> key  | column1  | column2 | column3 | column4 | value
> --+++
> ++--
> test by thrift | accessState |  |  |  |
> 0x5
>
> But when we use Cql, we couldn't set this empty value, it is null or
> explicit value, like below,
>
> key  | column1  | column2 | column3 | column4  | value
> --+---+-+---
> --+--+--
>  test by cql   | accessState |  | |  null |
> 0x5
>
> key  | column1  | column2 | column3 |
> column4  | value
> ---+--+--+--
> --+-
> --+-
>  test by cql   | accessState |  | |
> 4a528300-95cb-11e6-8650-0242f5eaa8c3| 0x5
>
> I don't know whether you could understand now, if not I could provide some
> code related to Astyanax. Really appreciate your help.
>
>
> On Wed, Oct 19, 2016 at 9:53 PM, Vladimir Yudovin 
> wrote:
>
>> Hi,
>>
>> what does it exactly mean 'empty timeuuid'?  UUID takes 16 bytes for
>> storage, so it should be either null, or some value. Do you mean 'zero'
>> UUID?
>>
>> Best regards, Vladimir Yudovin,
>>
>> *Winguzone  - Hosted Cloud
>> CassandraLaunch your cluster in minutes.*
>>
>>
>>  On Wed, 19 Oct 2016 09:16:29 -0400*coderhlj > >* wrote 
>>
>> Hi all,
>>
>> We use Cassandra 2.1.11 in our product, and we update the Java Drive from
>> Astyanax(Thrift API) to DataStax Java Driver(Cql) recently, but we
>> encounter a difficult issue as following, please help us, thanks in advance.
>>
>> Previously we were using Astyanax API, and we can insert empty timeuuid
>> like below, but now we can only insert null timeuuid by cql command but not
>> empty one. Is there any cql function to insert an empty timeuuid like by
>> Astyanax?
>> And this cause a tough problem is that we can not delete the record by
>> specifying the primary key, like:
>> *delete from "Foo" where column1='test' and column2='accessState' and
>> column3='' and column4=(need fill empty uuid here) IF EXISTS;*
>>
>> key  | column1  | column2 | column3 | column4 | value
>> -+-+-+-
>> +-+--
>> test by thrift | accessState |  |  |  |
>> 0x5
>>
>> key  | column1  | column2 | column3 | column4  | value
>> -+-+-+-+--+-
>> -
>>  test by cql   | accessState |  | |  null
>>   | 0x5
>>
>>
>> cqlsh:StorageOS> desc table "Foo";
>>
>> CREATE TABLE "Foo" (
>>   key text,
>>   column1 text,
>>   column2 text,
>>   column3 text,
>>   column4 timeuuid,
>>   value blob,
>>   PRIMARY KEY (key, column1, column2, column3, column4)
>> ) WITH COMPACT STORAGE AND
>>   bloom_filter_fp_chance=0.01 AND
>>   caching='{"keys":"ALL", "rows_per_partition":"NONE"}' AND
>>   comment='' AND
>>   dclocal_read_repair_chance=0.10 AND
>>   gc_grace_seconds=432000 AND
>>   read_repair_chance=0.00 AND
>>   default_time_to_live=0 AND
>>   speculative_retry='NONE' AND
>>   memtable_flush_period_in_ms=0 AND
>>   compaction={'class': 'SizeTieredCompactionStrategy'} AND
>>   compression={'sstable_compression': 'LZ4Compressor'};
>>
>> --
>> Thanks,
>> Lijun Huang
>>
>>
>>
>
>
> --
> Best regards,
> Lijun Huang
>



-- 


Stefania Alborghetti

|+852 6114 9265| stefania.alborghe...@datastax.com

Re: How to insert "Empty" timeuuid by Cql

[Lijun Huang]

Hi Vladimir,

Indeed, that's a little weird, I think it is like a empty string: '' but is
a timeuuid value. We have many such records that inserted by Astyanax API,
when we select it in cqlsh, it is like as below, note the column4 is
timeuuid, it is not null or some value, just "empty".

key  | column1  | column2 | column3 | column4 | value
--+++
++--
test by thrift | accessState |  |  |  |
0x5

But when we use Cql, we couldn't set this empty value, it is null or
explicit value, like below,

key  | column1  | column2 | column3 | column4  | value
--+---+--
---+-+--+--
 test by cql   | accessState |  | |  null |
0x5

key  | column1  | column2 | column3 |
column4  | value
---+--+--
++---+-

 test by cql   | accessState |  | |
4a528300-95cb-11e6-8650-0242f5eaa8c3| 0x5

I don't know whether you could understand now, if not I could provide some
code related to Astyanax. Really appreciate your help.


On Wed, Oct 19, 2016 at 9:53 PM, Vladimir Yudovin 
wrote:

> Hi,
>
> what does it exactly mean 'empty timeuuid'?  UUID takes 16 bytes for
> storage, so it should be either null, or some value. Do you mean 'zero'
> UUID?
>
> Best regards, Vladimir Yudovin,
>
> *Winguzone  - Hosted Cloud
> CassandraLaunch your cluster in minutes.*
>
>
>  On Wed, 19 Oct 2016 09:16:29 -0400*coderhlj  >* wrote 
>
> Hi all,
>
> We use Cassandra 2.1.11 in our product, and we update the Java Drive from
> Astyanax(Thrift API) to DataStax Java Driver(Cql) recently, but we
> encounter a difficult issue as following, please help us, thanks in advance.
>
> Previously we were using Astyanax API, and we can insert empty timeuuid
> like below, but now we can only insert null timeuuid by cql command but not
> empty one. Is there any cql function to insert an empty timeuuid like by
> Astyanax?
> And this cause a tough problem is that we can not delete the record by
> specifying the primary key, like:
> *delete from "Foo" where column1='test' and column2='accessState' and
> column3='' and column4=(need fill empty uuid here) IF EXISTS;*
>
> key  | column1  | column2 | column3 | column4 | value
> -+-+-+-
> +-+--
> test by thrift | accessState |  |  |  |
> 0x5
>
> key  | column1  | column2 | column3 | column4  | value
> -+-+-+-+--+-
> -
>  test by cql   | accessState |  | |  null |
> 0x5
>
>
> cqlsh:StorageOS> desc table "Foo";
>
> CREATE TABLE "Foo" (
>   key text,
>   column1 text,
>   column2 text,
>   column3 text,
>   column4 timeuuid,
>   value blob,
>   PRIMARY KEY (key, column1, column2, column3, column4)
> ) WITH COMPACT STORAGE AND
>   bloom_filter_fp_chance=0.01 AND
>   caching='{"keys":"ALL", "rows_per_partition":"NONE"}' AND
>   comment='' AND
>   dclocal_read_repair_chance=0.10 AND
>   gc_grace_seconds=432000 AND
>   read_repair_chance=0.00 AND
>   default_time_to_live=0 AND
>   speculative_retry='NONE' AND
>   memtable_flush_period_in_ms=0 AND
>   compaction={'class': 'SizeTieredCompactionStrategy'} AND
>   compression={'sstable_compression': 'LZ4Compressor'};
>
> --
> Thanks,
> Lijun Huang
>
>
>


-- 
Best regards,
Lijun Huang

Re: Introducing Cassandra 3.7 LTS

[kurt Greaves]

On 19 October 2016 at 21:07, sfesc...@gmail.com  wrote:

> Wow, thank you for doing this. This sentiment regarding stability seems to
> be widespread. Is the team reconsidering the whole tick-tock cadence? If
> not, I would add my voice to those asking that it is revisited.


There has certainly been discussion regarding the tick-tock cadence, and it
seems safe to say it will change. There hasn't been any official
announcement yet, however.

Kurt Greaves
k...@instaclustr.com
www.instaclustr.com

[no subject]

[Anseh Danesharasteh]

unsubscribe

Re: Introducing Cassandra 3.7 LTS

[Jonathan Haddad]

Awesome!

On Wed, Oct 19, 2016 at 2:57 PM Jason J. W. Williams <
jasonjwwilli...@gmail.com> wrote:

> +1
>
> On Wed, Oct 19, 2016 at 2:07 PM, sfesc...@gmail.com 
> wrote:
>
> Wow, thank you for doing this. This sentiment regarding stability seems to
> be widespread. Is the team reconsidering the whole tick-tock cadence? If
> not, I would add my voice to those asking that it is revisited.
>
> Steve
>
> On Wed, Oct 19, 2016 at 1:00 PM Matija Gobec  wrote:
>
> Hi Ben,
>
> Thanks for this awesome contribution. I'm eager to give it a try and test
> it out.
>
> Best,
> Matija
>
> On Wed, Oct 19, 2016 at 8:55 PM, Ben Bromhead  wrote:
>
> Hi All
>
> I am proud to announce we are making available our production build of
> Cassandra 3.7 that we run at Instaclustr (both for ourselves and our
> customers). Our release of Cassandra 3.7 includes a number of backported
> patches from later versions of Cassandra e.g. 3.8 and 3.9 but doesn't
> include the new features of these releases.
>
> You can find our release of Cassandra 3.7 LTS on github here (
> https://github.com/instaclustr/cassandra). You can read more of our
> thinking and how this applies to our managed service here (
> https://www.instaclustr.com/blog/2016/10/19/patched-cassandra-3-7/).
>
> We also have an expanded FAQ about why and how we are approaching 3.x in
> this manner (https://github.com/instaclustr/cassandra#cassandra-37-lts),
> however I've included the top few question and answers below:
>
> *Is this a fork?*
> No, This is just Cassandra with a different release cadence for those who
> want 3.x features but are slightly more risk averse than the current
> schedule allows.
>
> *Why not just use the official release?*
> With the 3.x tick-tock branch we have encountered more instability than
> with the previous release cadence. We feel that releasing new features
> every other release makes it very hard for operators to stabilize their
> production environment without bringing in brand new features that are not
> battle tested. With the release of Cassandra 3.8 and 3.9 simultaneously the
> bug fix branch included new and real-world untested features, specifically
> CDC. We have decided to stick with Cassandra 3.7 and instead backport
> critical issues and maintain it ourselves rather than trying to stick with
> the current Apache Cassandra release cadence.
>
> *Why backport?*
> At Instaclustr we support and run a number of different versions of Apache
> Cassandra on behalf of our customers. Over the course of managing Cassandra
> for our customers we often encounter bugs. There are existing patches for
> some of them, others we patch ourselves. Generally, if we can, we try to
> wait for the next official Apache Cassandra release, however in the need to
> ensure our customers remain stable and running we will sometimes backport
> bugs and write our own hotfixes (which are also submitted back to the
> community).
>
> *Why release it?*
> A number of our customers and people in the community have asked if we
> would make this available, which we are more than happy to do so. This
> repository represents what Instaclustr runs in production for Cassandra 3.7
> and this is our way of helping the community get a similar level of
> stability as what you would get from our managed service.
>
> Cheers
>
> Ben
>
>
>
> --
> Ben Bromhead
> CTO | Instaclustr 
> +1 650 284 9692
> Managed Cassandra / Spark on AWS, Azure and Softlayer
>
>
>
>

Re: Introducing Cassandra 3.7 LTS

[Jason J. W. Williams]

+1

On Wed, Oct 19, 2016 at 2:07 PM, sfesc...@gmail.com 
wrote:

> Wow, thank you for doing this. This sentiment regarding stability seems to
> be widespread. Is the team reconsidering the whole tick-tock cadence? If
> not, I would add my voice to those asking that it is revisited.
>
> Steve
>
> On Wed, Oct 19, 2016 at 1:00 PM Matija Gobec  wrote:
>
>> Hi Ben,
>>
>> Thanks for this awesome contribution. I'm eager to give it a try and test
>> it out.
>>
>> Best,
>> Matija
>>
>> On Wed, Oct 19, 2016 at 8:55 PM, Ben Bromhead 
>> wrote:
>>
>> Hi All
>>
>> I am proud to announce we are making available our production build of
>> Cassandra 3.7 that we run at Instaclustr (both for ourselves and our
>> customers). Our release of Cassandra 3.7 includes a number of backported
>> patches from later versions of Cassandra e.g. 3.8 and 3.9 but doesn't
>> include the new features of these releases.
>>
>> You can find our release of Cassandra 3.7 LTS on github here (
>> https://github.com/instaclustr/cassandra). You can read more of our
>> thinking and how this applies to our managed service here (
>> https://www.instaclustr.com/blog/2016/10/19/patched-cassandra-3-7/).
>>
>> We also have an expanded FAQ about why and how we are approaching 3.x in
>> this manner (https://github.com/instaclustr/cassandra#cassandra-37-lts),
>> however I've included the top few question and answers below:
>>
>> *Is this a fork?*
>> No, This is just Cassandra with a different release cadence for those who
>> want 3.x features but are slightly more risk averse than the current
>> schedule allows.
>>
>> *Why not just use the official release?*
>> With the 3.x tick-tock branch we have encountered more instability than
>> with the previous release cadence. We feel that releasing new features
>> every other release makes it very hard for operators to stabilize their
>> production environment without bringing in brand new features that are not
>> battle tested. With the release of Cassandra 3.8 and 3.9 simultaneously the
>> bug fix branch included new and real-world untested features, specifically
>> CDC. We have decided to stick with Cassandra 3.7 and instead backport
>> critical issues and maintain it ourselves rather than trying to stick with
>> the current Apache Cassandra release cadence.
>>
>> *Why backport?*
>> At Instaclustr we support and run a number of different versions of
>> Apache Cassandra on behalf of our customers. Over the course of managing
>> Cassandra for our customers we often encounter bugs. There are existing
>> patches for some of them, others we patch ourselves. Generally, if we can,
>> we try to wait for the next official Apache Cassandra release, however in
>> the need to ensure our customers remain stable and running we will
>> sometimes backport bugs and write our own hotfixes (which are also
>> submitted back to the community).
>>
>> *Why release it?*
>> A number of our customers and people in the community have asked if we
>> would make this available, which we are more than happy to do so. This
>> repository represents what Instaclustr runs in production for Cassandra 3.7
>> and this is our way of helping the community get a similar level of
>> stability as what you would get from our managed service.
>>
>> Cheers
>>
>> Ben
>>
>>
>>
>> --
>> Ben Bromhead
>> CTO | Instaclustr 
>> +1 650 284 9692
>> Managed Cassandra / Spark on AWS, Azure and Softlayer
>>
>>
>>

Re: Introducing Cassandra 3.7 LTS

[sfesc...@gmail.com]

Wow, thank you for doing this. This sentiment regarding stability seems to
be widespread. Is the team reconsidering the whole tick-tock cadence? If
not, I would add my voice to those asking that it is revisited.

Steve

On Wed, Oct 19, 2016 at 1:00 PM Matija Gobec  wrote:

> Hi Ben,
>
> Thanks for this awesome contribution. I'm eager to give it a try and test
> it out.
>
> Best,
> Matija
>
> On Wed, Oct 19, 2016 at 8:55 PM, Ben Bromhead  wrote:
>
> Hi All
>
> I am proud to announce we are making available our production build of
> Cassandra 3.7 that we run at Instaclustr (both for ourselves and our
> customers). Our release of Cassandra 3.7 includes a number of backported
> patches from later versions of Cassandra e.g. 3.8 and 3.9 but doesn't
> include the new features of these releases.
>
> You can find our release of Cassandra 3.7 LTS on github here (
> https://github.com/instaclustr/cassandra). You can read more of our
> thinking and how this applies to our managed service here (
> https://www.instaclustr.com/blog/2016/10/19/patched-cassandra-3-7/).
>
> We also have an expanded FAQ about why and how we are approaching 3.x in
> this manner (https://github.com/instaclustr/cassandra#cassandra-37-lts),
> however I've included the top few question and answers below:
>
> *Is this a fork?*
> No, This is just Cassandra with a different release cadence for those who
> want 3.x features but are slightly more risk averse than the current
> schedule allows.
>
> *Why not just use the official release?*
> With the 3.x tick-tock branch we have encountered more instability than
> with the previous release cadence. We feel that releasing new features
> every other release makes it very hard for operators to stabilize their
> production environment without bringing in brand new features that are not
> battle tested. With the release of Cassandra 3.8 and 3.9 simultaneously the
> bug fix branch included new and real-world untested features, specifically
> CDC. We have decided to stick with Cassandra 3.7 and instead backport
> critical issues and maintain it ourselves rather than trying to stick with
> the current Apache Cassandra release cadence.
>
> *Why backport?*
> At Instaclustr we support and run a number of different versions of Apache
> Cassandra on behalf of our customers. Over the course of managing Cassandra
> for our customers we often encounter bugs. There are existing patches for
> some of them, others we patch ourselves. Generally, if we can, we try to
> wait for the next official Apache Cassandra release, however in the need to
> ensure our customers remain stable and running we will sometimes backport
> bugs and write our own hotfixes (which are also submitted back to the
> community).
>
> *Why release it?*
> A number of our customers and people in the community have asked if we
> would make this available, which we are more than happy to do so. This
> repository represents what Instaclustr runs in production for Cassandra 3.7
> and this is our way of helping the community get a similar level of
> stability as what you would get from our managed service.
>
> Cheers
>
> Ben
>
>
>
> --
> Ben Bromhead
> CTO | Instaclustr 
> +1 650 284 9692
> Managed Cassandra / Spark on AWS, Azure and Softlayer
>
>
>

Re: Introducing Cassandra 3.7 LTS

[Matija Gobec]

Hi Ben,

Thanks for this awesome contribution. I'm eager to give it a try and test
it out.

Best,
Matija

On Wed, Oct 19, 2016 at 8:55 PM, Ben Bromhead  wrote:

> Hi All
>
> I am proud to announce we are making available our production build of
> Cassandra 3.7 that we run at Instaclustr (both for ourselves and our
> customers). Our release of Cassandra 3.7 includes a number of backported
> patches from later versions of Cassandra e.g. 3.8 and 3.9 but doesn't
> include the new features of these releases.
>
> You can find our release of Cassandra 3.7 LTS on github here (
> https://github.com/instaclustr/cassandra). You can read more of our
> thinking and how this applies to our managed service here (
> https://www.instaclustr.com/blog/2016/10/19/patched-cassandra-3-7/).
>
> We also have an expanded FAQ about why and how we are approaching 3.x in
> this manner (https://github.com/instaclustr/cassandra#cassandra-37-lts),
> however I've included the top few question and answers below:
>
> *Is this a fork?*
> No, This is just Cassandra with a different release cadence for those who
> want 3.x features but are slightly more risk averse than the current
> schedule allows.
>
> *Why not just use the official release?*
> With the 3.x tick-tock branch we have encountered more instability than
> with the previous release cadence. We feel that releasing new features
> every other release makes it very hard for operators to stabilize their
> production environment without bringing in brand new features that are not
> battle tested. With the release of Cassandra 3.8 and 3.9 simultaneously the
> bug fix branch included new and real-world untested features, specifically
> CDC. We have decided to stick with Cassandra 3.7 and instead backport
> critical issues and maintain it ourselves rather than trying to stick with
> the current Apache Cassandra release cadence.
>
> *Why backport?*
> At Instaclustr we support and run a number of different versions of Apache
> Cassandra on behalf of our customers. Over the course of managing Cassandra
> for our customers we often encounter bugs. There are existing patches for
> some of them, others we patch ourselves. Generally, if we can, we try to
> wait for the next official Apache Cassandra release, however in the need to
> ensure our customers remain stable and running we will sometimes backport
> bugs and write our own hotfixes (which are also submitted back to the
> community).
>
> *Why release it?*
> A number of our customers and people in the community have asked if we
> would make this available, which we are more than happy to do so. This
> repository represents what Instaclustr runs in production for Cassandra 3.7
> and this is our way of helping the community get a similar level of
> stability as what you would get from our managed service.
>
> Cheers
>
> Ben
>
>
>
> --
> Ben Bromhead
> CTO | Instaclustr 
> +1 650 284 9692
> Managed Cassandra / Spark on AWS, Azure and Softlayer
>

Introducing Cassandra 3.7 LTS

[Ben Bromhead]

Hi All

I am proud to announce we are making available our production build of
Cassandra 3.7 that we run at Instaclustr (both for ourselves and our
customers). Our release of Cassandra 3.7 includes a number of backported
patches from later versions of Cassandra e.g. 3.8 and 3.9 but doesn't
include the new features of these releases.

You can find our release of Cassandra 3.7 LTS on github here (
https://github.com/instaclustr/cassandra). You can read more of our
thinking and how this applies to our managed service here (
https://www.instaclustr.com/blog/2016/10/19/patched-cassandra-3-7/).

We also have an expanded FAQ about why and how we are approaching 3.x in
this manner (https://github.com/instaclustr/cassandra#cassandra-37-lts),
however I've included the top few question and answers below:

*Is this a fork?*
No, This is just Cassandra with a different release cadence for those who
want 3.x features but are slightly more risk averse than the current
schedule allows.

*Why not just use the official release?*
With the 3.x tick-tock branch we have encountered more instability than
with the previous release cadence. We feel that releasing new features
every other release makes it very hard for operators to stabilize their
production environment without bringing in brand new features that are not
battle tested. With the release of Cassandra 3.8 and 3.9 simultaneously the
bug fix branch included new and real-world untested features, specifically
CDC. We have decided to stick with Cassandra 3.7 and instead backport
critical issues and maintain it ourselves rather than trying to stick with
the current Apache Cassandra release cadence.

*Why backport?*
At Instaclustr we support and run a number of different versions of Apache
Cassandra on behalf of our customers. Over the course of managing Cassandra
for our customers we often encounter bugs. There are existing patches for
some of them, others we patch ourselves. Generally, if we can, we try to
wait for the next official Apache Cassandra release, however in the need to
ensure our customers remain stable and running we will sometimes backport
bugs and write our own hotfixes (which are also submitted back to the
community).

*Why release it?*
A number of our customers and people in the community have asked if we
would make this available, which we are more than happy to do so. This
repository represents what Instaclustr runs in production for Cassandra 3.7
and this is our way of helping the community get a similar level of
stability as what you would get from our managed service.

Cheers

Ben



-- 
Ben Bromhead
CTO | Instaclustr 
+1 650 284 9692
Managed Cassandra / Spark on AWS, Azure and Softlayer

Cassandra Read / Write Benchmarks with Stress - Public listing

[Bhuvan Rawal]

Hi,

Is there any public listing of cassandra performance test results with
cstar or cassandra-stress for read and write, with mention of
configurations modified from default and cassandra version.

It would be useful to not redo and do optimisations for cassandra wrt
Threadpools / JVM tuning / Caching which have already been tried before and
can be observed from past results for a faster approach to best tuning.

In case If such a thing does not exist currently, I propose a listing
possibly on Cassandra Wiki with a format say:

*# CPU* *# RAM* *# Nodes* *Latencies* *Throughput* *Con. Reads* *Con.
Writes* *Con. Compactors* *JVM Strategy* *Heap Size* *New Gen* *Other JVM
Params* *Link to Images / Detailed Blog*

Kindly suggest.

Thanks & Regards,
Bhuvan

ApacheCon is now less than a month away!

[Rich Bowen]

Dear Apache Enthusiast,

ApacheCon Sevilla is now less than a month out, and we need your help
getting the word out. Please tell your colleagues, your friends, and
members of related technical communities, about this event. Rates go up
November 3rd, so register today!

ApacheCon, and Apache Big Data, are the official gatherings of the
Apache Software Foundation, and one of the best places in the world to
meet other members of your project community, gain deeper knowledge
about your favorite Apache projects, learn about the ASF. Your project
doesn't live in a vacuum - it's part of a larger family of projects that
have a shared set of values, as well as a shared governance model. And
many of our project have an overlap in developers, in communities, and
in subject matter, making ApacheCon a great place for cross-pollination
of ideas and of communities.

Some highlights of these events will be:

* Many of our board members and project chairs will be present
* The lightning talks are a great place to hear, and give, short
presentations about what you and other members of the community are
working on
* The key signing gets you linked into the web of trust, and better
able to verify our software releases
* Evening receptions and parties where you can meet community
members in a less formal setting
* The State of the Feather, where you can learn what the ASF has
done in the last year, and what's coming next year
* BarCampApache, an informal unconference-style event, is another
venue for discussing your projects at the ASF

We have a great schedule lined up, covering the wide range of ASF
projects, including:

* CI and CD at Scale: Scaling Jenkins with Docker and Apache Mesos -
Carlos Sanchez
* Inner sourcing 101 - Jim Jagielski
* Java Memory Leaks in Modular Environments - Mark Thomas

ApacheCon/Apache Big Data will be held in Sevilla, Spain, at the Melia
Sevilla, November 14th through 18th. You can find out more at
http://apachecon.com/  Other ways to stay up to date with ApacheCon are:

* Follow us on Twitter at @apachecon
* Join us on IRC, at #apachecon on the Freenode IRC network
* Join the apachecon-discuss mailing list by sending email to
apachecon-discuss-subscr...@apache.org
* Or contact me directly at rbo...@apache.org with questions,
comments, or to volunteer to help

See you in Sevilla!

-- 
Rich Bowen: VP, Conferences
rbo...@apache.org
http://apachecon.com/
@apachecon

Re: non incremental repairs with cassandra 2.2+

[Kant Kodali]

Sorry I shouldn't have said adding a node. Sometimes data seems to be corrupted 
or inconsistent in which case would like to run a repair. 

Sent from my iPhone

> On Oct 19, 2016, at 10:10 AM, Sean Bridges  
> wrote:
> 
> Thanks, we will try that.
> 
> Sean
> 
>> On 16-10-19 09:34 AM, Alexander Dejanovski wrote:
>> Hi Sean,
>> 
>> you should be able to do that by running subrange repairs, which is the only 
>> type of repair that wouldn't trigger anticompaction AFAIK.
>> Beware that now you will have sstables marked as repaired and others marked 
>> as unrepaired, which will never be compacted   together.
>> You might want to flag all sstables as unrepaired before moving on, if you 
>> do not intend to switch to incremental repair for now.
>> 
>> Cheers,
>> 
>>> On Wed, Oct 19, 2016 at 6:31 PM Sean Bridges  
>>> wrote:
>>> Hey,
>>> 
>>> We are upgrading from cassandra 2.1 to cassandra 2.2.  
>>> 
>>> With cassandra 2.1 we would periodically repair all nodes, using the -pr 
>>> flag.  
>>> 
>>> With cassandra 2.2, the same repair takes a very long time, as cassandra 
>>> does an anti compaction after the repair.  This anti compaction causes most 
>>> (all?) the sstables to be rewritten.  Is there a way to do full repairs 
>>> without continually anti compacting?  If we do a full repair on 
>>> each node with the -pr flag, will subsequent full repairs also force 
>>> anti compacting most (all?) sstables?  
>>> 
>>> Thanks,
>>> 
>>> Sean
>> 
>> -- 
>> -
>> Alexander Dejanovski
>> France
>> @alexanderdeja
>> 
>> Consultant
>> Apache Cassandra Consulting
>> http://www.thelastpickle.com
> 
> 
> -- 
> Sean Bridges
> 
> senior systems architect
> Global Relay 
> 
> sean.brid...@globalrelay.net 
> 
> 866.484.6630 
> New York | Chicago | Vancouver | London (+44.0800.032.9829) | Singapore 
> (+65.3158.1301) 
> Global Relay Archive supports email, instant messaging, BlackBerry, 
> Bloomberg, Thomson Reuters, Pivot, YellowJacket, LinkedIn, Twitter, Facebook 
> and more. 
> 
> Ask about Global Relay Message - The Future of Collaboration in the Financial 
> Services World
> 
> All email sent to or from this address will be retained by Global Relay's 
> email archiving system. This message is intended only for the use of the 
> individual or entity to which it is addressed, and may contain information 
> that is privileged, confidential, and exempt from disclosure under applicable 
> law. Global Relay will not be liable for any compliance or technical 
> information provided herein. All trademarks are the property of their 
> respective owners.
> 

Re: non incremental repairs with cassandra 2.2+

[Alexander Dejanovski]

There aren't that many tools I know to orchestrate repairs and we maintain
a fork of Reaper, that was made by Spotify, and handles incremental repair
: https://github.com/thelastpickle/cassandra-reaper

We just added Cassandra as storage back end (only postgres currently) in
one of the branches, which should soon be merged to master.

Le mer. 19 oct. 2016 19:03, Kant Kodali  a écrit :

Also any suggestions on a tool to orchestrate the incremental repair? Like
say most commonly used

Sent from my iPhone

On Oct 19, 2016, at 9:54 AM, Alexander Dejanovski 
wrote:

Hi Kant,

subrange is a form of full repair, so it will just split the repair process
in smaller yet sequential pieces of work (repair is started giving a start
and end token). Overall, you should not expect improvements other than
having less overstreaming and better chances of success if your cluster is
dense.

You can try to use incremental repair if you know what the caveats are and
use a proper tool to orchestrate it, that would save you from repairing all
10TB each time.
CASSANDRA-12580 might help too as Romain showed us :
https://www.mail-archive.com/user@cassandra.apache.org/msg49344.html

Cheers,



On Wed, Oct 19, 2016 at 6:42 PM Kant Kodali  wrote:

Another question on a same note would be what would be the fastest way to
do repairs of size 10TB cluster ? Full repairs are taking days. So among
repair parallel or repair sub range which is faster in the case of say
adding a new node to the cluster?

Sent from my iPhone

On Oct 19, 2016, at 9:30 AM, Sean Bridges 
wrote:

Hey,

We are upgrading from cassandra 2.1 to cassandra 2.2.

With cassandra 2.1 we would periodically repair all nodes, using the -pr
flag.

With cassandra 2.2, the same repair takes a very long time, as cassandra
does an anti compaction after the repair.  This anti compaction causes most
(all?) the sstables to be rewritten.  Is there a way to do full repairs
without continually anti compacting?  If we do a full repair on each node
with the -pr flag, will subsequent full repairs also force anti compacting
most (all?) sstables?

Thanks,

Sean

-- 
-
Alexander Dejanovski
France
@alexanderdeja

Consultant
Apache Cassandra Consulting
http://www.thelastpickle.com

-- 
-
Alexander Dejanovski
France
@alexanderdeja

Consultant
Apache Cassandra Consulting
http://www.thelastpickle.com

Unsubscribe

[Anseh Danesharasteh]

Unsubscribe

Re: non incremental repairs with cassandra 2.2+

[Sean Bridges]

Thanks, we will try that.

Sean

On 16-10-19 09:34 AM, Alexander Dejanovski wrote:

Hi Sean,

you should be able to do that by running subrange repairs, which is 
the only type of repair that wouldn't trigger anticompaction AFAIK.
Beware that now you will have sstables marked as repaired and others 
marked as unrepaired, which will never be compacted together.
You might want to flag all sstables as unrepaired before moving on, if 
you do not intend to switch to incremental repair for now.


Cheers,

On Wed, Oct 19, 2016 at 6:31 PM Sean Bridges 
> 
wrote:


Hey,

We are upgrading from cassandra 2.1 to cassandra 2.2.

With cassandra 2.1 we would periodically repair all nodes, using
the -pr flag.

With cassandra 2.2, the same repair takes a very long time, as
cassandra does an anti compaction after the repair. This anti
compaction causes most (all?) the sstables to be rewritten.  Is
there a way to do full repairs without continually anti
compacting?  If we do a full repair on each node with the -pr
flag, will subsequent full repairs also force anti compacting most
(all?) sstables?

Thanks,

Sean

--
-
Alexander Dejanovski
France
@alexanderdeja

Consultant
Apache Cassandra Consulting
http://www.thelastpickle.com 



--

Sean Bridges

senior systems architect
Global Relay

_sean.bridges@globalrelay.net_ 

*866.484.6630 *
New York | Chicago | Vancouver | London (+44.0800.032.9829) | Singapore 
(+65.3158.1301)


Global Relay Archive supports email, instant messaging, BlackBerry, 
Bloomberg, Thomson Reuters, Pivot, YellowJacket, LinkedIn, Twitter, 
Facebook and more.


Ask about *_Global Relay Message_* 
 - The Future of 
Collaboration in the Financial Services World


All email sent to or from this address will be retained by Global 
Relay's email archiving system. This message is intended only for the 
use of the individual or entity to which it is addressed, and may 
contain information that is privileged, confidential, and exempt from 
disclosure under applicable law. Global Relay will not be liable for any 
compliance or technical information provided herein. All trademarks are 
the property of their respective owners.

Re: non incremental repairs with cassandra 2.2+

[Alexander Dejanovski]

Can you explain why you would want to run repair for new nodes?

Aren't you talking about bootstrap, which is not related to repair actually?

Le mer. 19 oct. 2016 18:57, Kant Kodali  a écrit :

> Thanks! How do I do an incremental repair when I add a new node?
>
> Sent from my iPhone
>
> On Oct 19, 2016, at 9:54 AM, Alexander Dejanovski 
> wrote:
>
> Hi Kant,
>
> subrange is a form of full repair, so it will just split the repair
> process in smaller yet sequential pieces of work (repair is started giving
> a start and end token). Overall, you should not expect improvements other
> than having less overstreaming and better chances of success if your
> cluster is dense.
>
> You can try to use incremental repair if you know what the caveats are and
> use a proper tool to orchestrate it, that would save you from repairing all
> 10TB each time.
> CASSANDRA-12580 might help too as Romain showed us :
> https://www.mail-archive.com/user@cassandra.apache.org/msg49344.html
>
> Cheers,
>
>
>
> On Wed, Oct 19, 2016 at 6:42 PM Kant Kodali  wrote:
>
> Another question on a same note would be what would be the fastest way to
> do repairs of size 10TB cluster ? Full repairs are taking days. So among
> repair parallel or repair sub range which is faster in the case of say
> adding a new node to the cluster?
>
> Sent from my iPhone
>
> On Oct 19, 2016, at 9:30 AM, Sean Bridges 
> wrote:
>
> Hey,
>
> We are upgrading from cassandra 2.1 to cassandra 2.2.
>
> With cassandra 2.1 we would periodically repair all nodes, using the -pr
> flag.
>
> With cassandra 2.2, the same repair takes a very long time, as cassandra
> does an anti compaction after the repair.  This anti compaction causes most
> (all?) the sstables to be rewritten.  Is there a way to do full repairs
> without continually anti compacting?  If we do a full repair on each node
> with the -pr flag, will subsequent full repairs also force anti compacting
> most (all?) sstables?
>
> Thanks,
>
> Sean
>
> --
> -
> Alexander Dejanovski
> France
> @alexanderdeja
>
> Consultant
> Apache Cassandra Consulting
> http://www.thelastpickle.com
>
> --
-
Alexander Dejanovski
France
@alexanderdeja

Consultant
Apache Cassandra Consulting
http://www.thelastpickle.com

Re: non incremental repairs with cassandra 2.2+

[Kant Kodali]

Also any suggestions on a tool to orchestrate the incremental repair? Like say 
most commonly used 

Sent from my iPhone

> On Oct 19, 2016, at 9:54 AM, Alexander Dejanovski  
> wrote:
> 
> Hi Kant,
> 
> subrange is a form of full repair, so it will just split the repair process 
> in smaller yet sequential pieces of work (repair is started giving a start 
> and end token). Overall, you should not expect improvements other than having 
> less overstreaming and better chances of success if your cluster is dense.
> 
> You can try to use incremental repair if you know what the caveats are and 
> use a proper tool to orchestrate it, that would save you from repairing all 
> 10TB each time.
> CASSANDRA-12580 might help too as Romain showed us : 
> https://www.mail-archive.com/user@cassandra.apache.org/msg49344.html
> 
> Cheers,
> 
> 
> 
> On Wed, Oct 19, 2016 at 6:42 PM Kant Kodali  wrote:
> Another question on a same note would be what would be the fastest way to do 
> repairs of size 10TB cluster ? Full repairs are taking days. So among repair 
> parallel or repair sub range which is faster in the case of say adding a new 
> node to the cluster?
> 
> Sent from my iPhone
> 
>> On Oct 19, 2016, at 9:30 AM, Sean Bridges  
>> wrote:
>> 
>> Hey,
>> 
>> We are upgrading from cassandra 2.1 to cassandra 2.2.  
>> 
>> With cassandra 2.1 we would periodically repair all nodes, using the -pr 
>> flag.  
>> 
>> With cassandra 2.2, the same repair takes a very long time, as cassandra 
>> does an anti compaction after the repair.  This anti compaction causes most 
>> (all?) the sstables to be rewritten.  Is there a way to do full repairs 
>> without continually anti compacting?  If we do a full repair on each node 
>> with the -pr flag, will subsequent full repairs also force anti 
>> compacting most (all?) sstables?  
>> 
>> Thanks,
>> 
>> Sean
> 
> -- 
> -
> Alexander Dejanovski
> France
> @alexanderdeja
> 
> Consultant
> Apache Cassandra Consulting
> http://www.thelastpickle.com

Re: non incremental repairs with cassandra 2.2+

[Kant Kodali]

Thanks! How do I do an incremental repair when I add a new node?

Sent from my iPhone

> On Oct 19, 2016, at 9:54 AM, Alexander Dejanovski  
> wrote:
> 
> Hi Kant,
> 
> subrange is a form of full repair, so it will just split the repair process 
> in smaller yet sequential pieces of work (repair is started giving a start 
> and end token). Overall, you should not expect improvements other than having 
> less overstreaming and better chances of success if your cluster is dense.
> 
> You can try to use incremental repair if you know what the caveats are and 
> use a proper tool to orchestrate it, that would save you from repairing all 
> 10TB each time.
> CASSANDRA-12580 might help too as Romain showed us : 
> https://www.mail-archive.com/user@cassandra.apache.org/msg49344.html
> 
> Cheers,
> 
> 
> 
> On Wed, Oct 19, 2016 at 6:42 PM Kant Kodali  wrote:
> Another question on a same note would be what would be the fastest way to do 
> repairs of size 10TB cluster ? Full repairs are taking days. So among repair 
> parallel or repair sub range which is faster in the case of say adding a new 
> node to the cluster?
> 
> Sent from my iPhone
> 
>> On Oct 19, 2016, at 9:30 AM, Sean Bridges  
>> wrote:
>> 
>> Hey,
>> 
>> We are upgrading from cassandra 2.1 to cassandra 2.2.  
>> 
>> With cassandra 2.1 we would periodically repair all nodes, using the -pr 
>> flag.  
>> 
>> With cassandra 2.2, the same repair takes a very long time, as cassandra 
>> does an anti compaction after the repair.  This anti compaction causes most 
>> (all?) the sstables to be rewritten.  Is there a way to do full repairs 
>> without continually anti compacting?  If we do a full repair on each node 
>> with the -pr flag, will subsequent full repairs also force anti 
>> compacting most (all?) sstables?  
>> 
>> Thanks,
>> 
>> Sean
> 
> -- 
> -
> Alexander Dejanovski
> France
> @alexanderdeja
> 
> Consultant
> Apache Cassandra Consulting
> http://www.thelastpickle.com

Re: non incremental repairs with cassandra 2.2+

[Alexander Dejanovski]

Hi Kant,

subrange is a form of full repair, so it will just split the repair process
in smaller yet sequential pieces of work (repair is started giving a start
and end token). Overall, you should not expect improvements other than
having less overstreaming and better chances of success if your cluster is
dense.

You can try to use incremental repair if you know what the caveats are and
use a proper tool to orchestrate it, that would save you from repairing all
10TB each time.
CASSANDRA-12580 might help too as Romain showed us :
https://www.mail-archive.com/user@cassandra.apache.org/msg49344.html

Cheers,



On Wed, Oct 19, 2016 at 6:42 PM Kant Kodali  wrote:

Another question on a same note would be what would be the fastest way to
do repairs of size 10TB cluster ? Full repairs are taking days. So among
repair parallel or repair sub range which is faster in the case of say
adding a new node to the cluster?

Sent from my iPhone

On Oct 19, 2016, at 9:30 AM, Sean Bridges 
wrote:

Hey,

We are upgrading from cassandra 2.1 to cassandra 2.2.

With cassandra 2.1 we would periodically repair all nodes, using the -pr
flag.

With cassandra 2.2, the same repair takes a very long time, as cassandra
does an anti compaction after the repair.  This anti compaction causes most
(all?) the sstables to be rewritten.  Is there a way to do full repairs
without continually anti compacting?  If we do a full repair on each node
with the -pr flag, will subsequent full repairs also force anti compacting
most (all?) sstables?

Thanks,

Sean

-- 
-
Alexander Dejanovski
France
@alexanderdeja

Consultant
Apache Cassandra Consulting
http://www.thelastpickle.com

Re: non incremental repairs with cassandra 2.2+

[Kant Kodali]

Another question on a same note would be what would be the fastest way to do 
repairs of size 10TB cluster ? Full repairs are taking days. So among repair 
parallel or repair sub range which is faster in the case of say adding a new 
node to the cluster?

Sent from my iPhone

> On Oct 19, 2016, at 9:30 AM, Sean Bridges  
> wrote:
> 
> Hey,
> 
> We are upgrading from cassandra 2.1 to cassandra 2.2.  
> 
> With cassandra 2.1 we would periodically repair all nodes, using the -pr 
> flag.  
> 
> With cassandra 2.2, the same repair takes a very long time, as cassandra does 
> an anti compaction after the repair.  This anti compaction causes most (all?) 
> the sstables to be rewritten.  Is there a way to do full repairs without 
> continually anti compacting?  If we do a full repair on each node with the 
> -pr flag, will subsequent full repairs also force anti compacting most (all?) 
> sstables?  
> 
> Thanks,
> 
> Sean

Re: non incremental repairs with cassandra 2.2+

[Alexander Dejanovski]

Hi Sean,

you should be able to do that by running subrange repairs, which is the
only type of repair that wouldn't trigger anticompaction AFAIK.
Beware that now you will have sstables marked as repaired and others marked
as unrepaired, which will never be compacted together.
You might want to flag all sstables as unrepaired before moving on, if you
do not intend to switch to incremental repair for now.

Cheers,

On Wed, Oct 19, 2016 at 6:31 PM Sean Bridges 
wrote:

> Hey,
>
> We are upgrading from cassandra 2.1 to cassandra 2.2.
>
> With cassandra 2.1 we would periodically repair all nodes, using the -pr
> flag.
>
> With cassandra 2.2, the same repair takes a very long time, as cassandra
> does an anti compaction after the repair.  This anti compaction causes most
> (all?) the sstables to be rewritten.  Is there a way to do full repairs
> without continually anti compacting?  If we do a full repair on each node
> with the -pr flag, will subsequent full repairs also force anti compacting
> most (all?) sstables?
>
> Thanks,
>
> Sean
>
-- 
-
Alexander Dejanovski
France
@alexanderdeja

Consultant
Apache Cassandra Consulting
http://www.thelastpickle.com

non incremental repairs with cassandra 2.2+

[Sean Bridges]

Hey,

We are upgrading from cassandra 2.1 to cassandra 2.2.

With cassandra 2.1 we would periodically repair all nodes, using the -pr 
flag.


With cassandra 2.2, the same repair takes a very long time, as cassandra 
does an anti compaction after the repair.  This anti compaction causes 
most (all?) the sstables to be rewritten.  Is there a way to do full 
repairs without continually anti compacting?  If we do a full repair on 
each node with the -pr flag, will subsequent full repairs also force 
anti compacting most (all?) sstables?


Thanks,

Sean

Re: How to insert "Empty" timeuuid by Cql

[Vladimir Yudovin]

Hi,



what does it exactly mean 'empty timeuuid'?  UUID takes 16 bytes for storage, 
so it should be either null, or some value. Do you mean 'zero' UUID?



Best regards, Vladimir Yudovin, 

Winguzone - Hosted Cloud Cassandra
Launch your cluster in minutes.





 On Wed, 19 Oct 2016 09:16:29 -0400coderhlj coder...@gmail.com 
wrote 




Hi all,



We use Cassandra 2.1.11 in our product, and we update the Java Drive from 
Astyanax(Thrift API) to DataStax Java Driver(Cql) recently, but we encounter a 
difficult issue as following, please help us, thanks in advance.



Previously we were using Astyanax API, and we can insert empty timeuuid like 
below, but now we can only insert null timeuuid by cql command but not empty 
one. Is there any cql function to insert an empty timeuuid like by Astyanax?

And this cause a tough problem is that we can not delete the record by 
specifying the primary key, like:

delete from "Foo" where column1='test' and column2='accessState' and column3='' 
and column4=(need fill empty uuid here) IF EXISTS;



key  | column1  | column2 | column3 | column4 | value

-+-+-+- 
+-+--

test by thrift | accessState |  |  |  | 0x5




key  | column1  | column2 | column3 | column4  | value

-+-+-+-+--+--

 test by cql   | accessState |  | |  null | 0x5






cqlsh:StorageOS desc table "Foo";  



CREATE TABLE "Foo" (

  key text,

  column1 text,

  column2 text,

  column3 text,

  column4 timeuuid,

  value blob,

  PRIMARY KEY (key, column1, column2, column3, column4)

) WITH COMPACT STORAGE AND

  bloom_filter_fp_chance=0.01 AND

  caching='{"keys":"ALL", "rows_per_partition":"NONE"}' AND

  comment='' AND

  dclocal_read_repair_chance=0.10 AND

  gc_grace_seconds=432000 AND

  read_repair_chance=0.00 AND

  default_time_to_live=0 AND

  speculative_retry='NONE' AND

  memtable_flush_period_in_ms=0 AND

  compaction={'class': 'SizeTieredCompactionStrategy'} AND

  compression={'sstable_compression': 'LZ4Compressor'};




--

Thanks,

Lijun Huang

How to insert "Empty" timeuuid by Cql

[coderhlj]

Hi all,

We use Cassandra 2.1.11 in our product, and we update the Java Drive from
Astyanax(Thrift API) to DataStax Java Driver(Cql) recently, but we
encounter a difficult issue as following, please help us, thanks in advance.

Previously we were using Astyanax API, and we can insert empty timeuuid
like below, but now we can only insert null timeuuid by cql command but not
empty one. Is there any cql function to insert an empty timeuuid like by
Astyanax?
And this cause a tough problem is that we can not delete the record by
specifying the primary key, like:
*delete from "Foo" where column1='test' and column2='accessState' and
column3='' and column4=(need fill empty uuid here) IF EXISTS;*

key  | column1  | column2 | column3 | column4 | value
-+-+-+-
+-+--
test by thrift | accessState |  |  |  |
0x5

key  | column1  | column2 | column3 | column4  | value
-+-+-+-+--+-
-
 test by cql   | accessState |  | |  null |
0x5


cqlsh:StorageOS> desc table "Foo";

CREATE TABLE "Foo" (
  key text,
  column1 text,
  column2 text,
  column3 text,
  column4 timeuuid,
  value blob,
  PRIMARY KEY (key, column1, column2, column3, column4)
) WITH COMPACT STORAGE AND
  bloom_filter_fp_chance=0.01 AND
  caching='{"keys":"ALL", "rows_per_partition":"NONE"}' AND
  comment='' AND
  dclocal_read_repair_chance=0.10 AND
  gc_grace_seconds=432000 AND
  read_repair_chance=0.00 AND
  default_time_to_live=0 AND
  speculative_retry='NONE' AND
  memtable_flush_period_in_ms=0 AND
  compaction={'class': 'SizeTieredCompactionStrategy'} AND
  compression={'sstable_compression': 'LZ4Compressor'};

--
Thanks,
Lijun Huang

Re: WARN [SharedPool-Worker-3] AbstractTracingAwareExecutorService.java

[Pranay akula]

Thanks Doan, will try to enable slow query for app side and will let you
know if it helps or not.


Thanks
James.

On Wed, Oct 19, 2016 at 2:59 AM, DuyHai Doan  wrote:

> "which appilcation is trying to write large writes and to which keyspace
> and table it is trying to write ??"
>
> --> I don't think there is something such as audit trail for each CQL
> query.
>
> You may want to enable slow query logger on the client side (driver side)
> and bet on the fact that the queries with a huge payload are "slower" than
> normal queries ...
>
> On Tue, Oct 18, 2016 at 6:20 PM, James Joseph 
> wrote:
>
>>
>> I have seen the following warn in system.log, as a temporary turn around
>> i  increased commitlog_size in cassandra.yaml to 64, but how can i trace it
>> down ??? which appilcation is trying to write large writes and to which
>> keyspace and table it is trying to write ??
>>
>>
>> WARN  [SharedPool-Worker-3] 2016-10-05 03:46:22,363
>>  AbstractTracingAwareExecutorService.java:169 - Uncaught exception on
>> thread Thread[SharedPool-Worker-3,5,main]: {}
>> java.lang.IllegalArgumentException: Mutation of 19711728 bytes is too
>> large for the maxiumum size of 16777216
>>   at org.apache.cassandra.db.commitlog.CommitLog.add(CommitLog.java:221)
>> ~[cassandra-all-2.1.8.689.jar:2.1.8.689]
>>   at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:383)
>> ~[cassandra-all-2.1.8.689.jar:2.1.8.689]
>>   at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:363)
>> ~[cassandra-all-2.1.8.689.jar:2.1.8.689]
>>   at org.apache.cassandra.db.Mutation.apply(Mutation.java:214)
>> ~[cassandra-all-2.1.8.689.jar:2.1.8.689]
>>   at 
>> org.apache.cassandra.db.MutationVerbHandler.doVerb(MutationVerbHandler.java:54)
>> ~[cassandra-all-2.1.8.689.jar:2.1.8.689]
>>   at 
>> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:64)
>> ~[cassandra-all-2.1.8.689.jar:2.1.8.689]
>>   at java.util.concurrent.Executors$RunnableAdapter.call(Unknown
>> Source) ~[na:1.8.0_92]
>>   at org.apache.cassandra.concurrent.AbstractTracingAwareExecutor
>> Service$FutureTask.run(AbstractTracingAwareExecutorService.java:164)
>> ~[cassandra-all-2.1.8.689.jar:2.1.8.689]
>>   at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105)
>> [cassandra-all-2.1.8.689.jar:2.1.8.689]
>>   at java.lang.Thread.run(Unknown Source) [na:1.8.0_92]
>>
>>
>> Thanks
>> James.
>>
>>
>

Re: problem starting incremental repair using TheLastPicke Reaper

[Alexander Dejanovski]

Abhishek,

can you file an issue on our github repo so that we can further discuss
this ? https://github.com/thelastpickle/cassandra-reaper/issues

Thanks,

On Wed, Oct 19, 2016 at 1:20 PM Abhishek Aggarwal <
abhishek.aggarwa...@snapdeal.com> wrote:

> Hi Alex,
>
> that i already did and it worked but my question is if the passed value of
> incremental repair flag is different from the existing value  then it
> should allow to create new repair_unit instead of getting repair_unit based
> on cluster name/ keyspace /column combination.
>
> and also if i delete the repair_unit then due to referential constraints i
> need to delete repair_segment and repair_run as well which will delete the
> run history corresponds to the repaid_unit.
>
> Abhishek Aggarwal
>
> *Senior Software Engineer*
> *M*: +91 8861212073 <+91%2088612%2012073> , 8588840304
> *T*: 0124 6600600 *EXT*: 12128
> ASF Center -A, ASF Center Udyog Vihar Phase IV,
> Download Our App
> [image: A]
> 
>  [image:
> A]
> 
>  [image:
> W]
> 
>
> On Wed, Oct 19, 2016 at 4:44 PM, Alexander Dejanovski <
> a...@thelastpickle.com> wrote:
>
> Hi Abhishek,
>
> This shows you have two repair units for the same keyspace/table with
> different incremental repair settings.
> Can you delete your prior repair run (the one with incremental repair set
> to false) and then create the new one with incremental repair set to true ?
>
> Let me know how that works,
>
>
> On Wed, Oct 19, 2016 at 10:45 AM Abhishek Aggarwal <
> abhishek.aggarwa...@snapdeal.com> wrote:
>
>
> is there a way to start the incremental repair using the reaper. we
> completed full repair successfully and after that i tried to run the
> incremental run but getting the below error.
>
>
> A repair run already exist for the same cluster/keyspace/table but with a
> different incremental repair value.Requested value: true | Existing value:
> false
>
>
> --
> -
> Alexander Dejanovski
> France
> @alexanderdeja
>
> Consultant
> Apache Cassandra Consulting
> http://www.thelastpickle.com
>
>
> --
-
Alexander Dejanovski
France
@alexanderdeja

Consultant
Apache Cassandra Consulting
http://www.thelastpickle.com

Re: problem starting incremental repair using TheLastPicke Reaper

[Abhishek Aggarwal]

Hi Alex,

that i already did and it worked but my question is if the passed value of
incremental repair flag is different from the existing value  then it
should allow to create new repair_unit instead of getting repair_unit based
on cluster name/ keyspace /column combination.

and also if i delete the repair_unit then due to referential constraints i
need to delete repair_segment and repair_run as well which will delete the
run history corresponds to the repaid_unit.

Abhishek Aggarwal

*Senior Software Engineer*
*M*: +91 8861212073 , 8588840304
*T*: 0124 6600600 *EXT*: 12128
ASF Center -A, ASF Center Udyog Vihar Phase IV,
Download Our App
[image: A]

[image:
A]

[image:
W]


On Wed, Oct 19, 2016 at 4:44 PM, Alexander Dejanovski <
a...@thelastpickle.com> wrote:

> Hi Abhishek,
>
> This shows you have two repair units for the same keyspace/table with
> different incremental repair settings.
> Can you delete your prior repair run (the one with incremental repair set
> to false) and then create the new one with incremental repair set to true ?
>
> Let me know how that works,
>
>
> On Wed, Oct 19, 2016 at 10:45 AM Abhishek Aggarwal <
> abhishek.aggarwa...@snapdeal.com> wrote:
>
>>
>> is there a way to start the incremental repair using the reaper. we
>> completed full repair successfully and after that i tried to run the
>> incremental run but getting the below error.
>>
>>
>> A repair run already exist for the same cluster/keyspace/table but with a
>> different incremental repair value.Requested value: true | Existing value:
>> false
>>
>>
>> --
> -
> Alexander Dejanovski
> France
> @alexanderdeja
>
> Consultant
> Apache Cassandra Consulting
> http://www.thelastpickle.com
>

Re: problem starting incremental repair using TheLastPicke Reaper

[Alexander Dejanovski]

Hi Abhishek,

This shows you have two repair units for the same keyspace/table with
different incremental repair settings.
Can you delete your prior repair run (the one with incremental repair set
to false) and then create the new one with incremental repair set to true ?

Let me know how that works,


On Wed, Oct 19, 2016 at 10:45 AM Abhishek Aggarwal <
abhishek.aggarwa...@snapdeal.com> wrote:

>
> is there a way to start the incremental repair using the reaper. we
> completed full repair successfully and after that i tried to run the
> incremental run but getting the below error.
>
>
> A repair run already exist for the same cluster/keyspace/table but with a
> different incremental repair value.Requested value: true | Existing value:
> false
>
>
> --
-
Alexander Dejanovski
France
@alexanderdeja

Consultant
Apache Cassandra Consulting
http://www.thelastpickle.com

Re: Is there any way to throttle the memtable flushing throughput?

[Satoshi Hikida]

Hi, Ben

Thank you for your reply.

> The AWS instance type you are using is not appropriate for a production
workload
I agree with you. I use it for a just verification of the C* behavior.

So I really want to understand the actual mechanism of the write request
 blocking. I would appreciate if you could give me more advice.

> The small amount of memory on the node could also mean your flush writers
are getting backed up (blocked)
Does it mean the flush writer thread is blocked? or write request (from
client) is blocked?
Why the small amount of memory leads to flush writers block?

Anyway, I'll change the some configurations of my node according to your
advice.
Thanks a lot.

Regards,
Satoshi

On Wed, Oct 12, 2016 at 4:27 AM, Ben Bromhead  wrote:

> A few thoughts on the larger problem at hand.
>
> The AWS instance type you are using is not appropriate for a production
> workload. Also with memtable flushes that cause spiky write throughput it
> sounds like your commitlog is on the same disk as your data directory,
> combined with the use of non-SSD EBS I'm not surprised this is happening.
> The small amount of memory on the node could also mean your flush writers
> are getting backed up (blocked), possibly causing JVM heap pressure and
> other fun things (you can check this with nodetool tpstats).
>
> Before you get into tuning memtable flushing I would do the following:
>
>- Reset your commitlog_sync settings back to default
>- Use an EC2 instance type with at least 15GB of memory, 4 cores and
>is EBS optimized (dedicated EBS bandwidth)
>- Use gp2 or io2 EBS volumes
>- Put your commitlog on a separate EBS volume.
>- Make sure your memtable_flush_writers are not being blocked, if so
>increase the number of flush writers (no more than # of cores)
>- Optimize your read_ahead_kb size and compression_chunk_length to
>keep those EBS reads as small as possible.
>
> Once you have fixed the above, memtable flushing should not be an issue.
> Even if you can't/don't want to upgrade the instance type, the other steps
> will help things.
>
> Ben
>
> On Tue, 11 Oct 2016 at 10:23 Satoshi Hikida  wrote:
>
>> Hi,
>>
>> I'm investigating the read/write performance of the C* (Ver. 2.2.8).
>> However, I have an issue about memtable flushing which forces the spiky
>> write throughput. And then it affects the latency of the client's requests.
>>
>> So I want to know the answers for the following questions.
>>
>> 1. Is there any way that throttling the write throughput of the memtable
>> flushing? If it exists, how can I do that?
>> 2. Is there any way to reduce the spike of the write bandwidth during the
>> memtable flushing?
>>(I'm in trouble because the delay of the request increases when the
>> spike of the write bandwidth occurred)
>>
>> I'm using one C* node for this investigation. And C* runs on an EC2
>> instance (2vCPU, 4GB memory), In addition, I attach two magnetic disks to
>> the instance, one stores system data(root file system.(/)), the other
>> stores C* data (data files and commit logs).
>>
>> I also changed a few configurations.
>> - commitlog_sync: batch
>> - commitlog_sync_batch_window_in_ms: 2
>> (Using default value for the other configurations)
>>
>>
>> Regards,
>> Satoshi
>>
>> --
> Ben Bromhead
> CTO | Instaclustr 
> +1 650 284 9692
> Managed Cassandra / Spark on AWS, Azure and Softlayer
>

Re: quick question

[Kant Kodali]

I mean disk/cpu/network usage but I understand what Dynamic snitch does!



On Wed, Oct 19, 2016 at 3:34 AM, Vladimir Yudovin 
wrote:

> What exactly do you mean by "resource usage"? If you mean "data size on
> disk" - no.
> If you mean "current CPU usage" - it depends on query. Modify query should
> be be sent to all nodes owning specific partition key.
> For read queries see http://www.datastax.com/dev/
> blog/dynamic-snitching-in-cassandra-past-present-and-future
>
> Best regards, Vladimir Yudovin,
>
> *Winguzone  - Hosted Cloud
> CassandraLaunch your cluster in minutes.*
>
>
>  On Wed, 19 Oct 2016 06:14:27 -0400*Kant Kodali  >* wrote 
>
> can Cassandra cluster direct or load balance the requests by detecting the
> resource usage of a particular node?
>
>
>

Re: quick question

[Vladimir Yudovin]

What exactly do you mean by "resource usage"? If you mean "data size on disk" - 
no.

If you mean "current CPU usage" - it depends on query. Modify query should be 
be sent to all nodes owning specific partition key.

For read queries see 
http://www.datastax.com/dev/blog/dynamic-snitching-in-cassandra-past-present-and-future



Best regards, Vladimir Yudovin, 

Winguzone - Hosted Cloud Cassandra
Launch your cluster in minutes.





 On Wed, 19 Oct 2016 06:14:27 -0400Kant Kodali k...@peernova.com 
wrote 




can Cassandra cluster direct or load balance the requests by detecting the 
resource usage of a particular node? 

Re: quick question

[DuyHai Doan]

The coordinator can optimize latency for a SELECT by asking data from the
lowest-latency replica using DynamicSnitch. It's not really load balancing
per se but it's the closest idea.

quick question

[Kant Kodali]

can Cassandra cluster direct or load balance the requests by detecting the
resource usage of a particular node?

problem starting incremental repair using TheLastPicke Reaper

[Abhishek Aggarwal]

is there a way to start the incremental repair using the reaper. we
completed full repair successfully and after that i tried to run the
incremental run but getting the below error.


A repair run already exist for the same cluster/keyspace/table but with a
different incremental repair value.Requested value: true | Existing value:
false

Re: Speeding up schema generation during tests

[DuyHai Doan]

Ohh didn't know such system property exist, nice idea!

On Wed, Oct 19, 2016 at 9:40 AM, horschi  wrote:

> Have you tried starting Cassandra with -Dcassandra.unsafesystem=true ?
>
>
> On Wed, Oct 19, 2016 at 9:31 AM, DuyHai Doan  wrote:
>
>> As I said, when I bootstrap the server and create some keyspace,
>> sometimes the schema is not fully initialized and when the test code tried
>> to insert data, it fails.
>>
>> I did not have time to dig into the source code to find the root cause,
>> maybe it's something really stupid and simple to fix. If you want to
>> investigate and try out my CassandraDaemon server, I'd be happy to get
>> feedbacks
>>
>> On Wed, Oct 19, 2016 at 9:22 AM, Ali Akhtar  wrote:
>>
>>> Thanks. I've disabled durable writes but this is still pretty slow
>>> (about 10 seconds).
>>>
>>> What issues did you run into with your impl?
>>>
>>> On Wed, Oct 19, 2016 at 12:15 PM, DuyHai Doan 
>>> wrote:
>>>
 There is a lot of pre-flight checks when starting the cassandra server
 and they took time.

 For integration testing, I have developped a modified CassandraDeamon
 here that remove pretty most of those checks:

 https://github.com/doanduyhai/Achilles/blob/master/achilles-
 embedded/src/main/java/info/archinnov/achilles/embedded/Achi
 llesCassandraDaemon.java

 The problem is that I felt into weird scenarios where creating a
 keyspace wasn't created in timely manner so I just stop using this impl for
 the moment, just look at it and do whatever you want.

 Another idea for testing is to disable durable write to speed up
 mutation (CREATE KEYSPACE ... WITH durable_write=false)

 On Wed, Oct 19, 2016 at 3:24 AM, Ali Akhtar 
 wrote:

> Is there a way to speed up the creation of keyspace + tables during
> integration tests? I am using an RF of 1, with SimpleStrategy, but it 
> still
> takes upto 10-15 seconds.
>


>>>
>>
>

Re: Speeding up schema generation during tests

[Ali Akhtar]

Horschi, you are the hero gotham deserves.

Test time reduced from 10 seconds to 800 ms

On Wed, Oct 19, 2016 at 12:40 PM, horschi  wrote:

> Have you tried starting Cassandra with -Dcassandra.unsafesystem=true ?
>
>
> On Wed, Oct 19, 2016 at 9:31 AM, DuyHai Doan  wrote:
>
>> As I said, when I bootstrap the server and create some keyspace,
>> sometimes the schema is not fully initialized and when the test code tried
>> to insert data, it fails.
>>
>> I did not have time to dig into the source code to find the root cause,
>> maybe it's something really stupid and simple to fix. If you want to
>> investigate and try out my CassandraDaemon server, I'd be happy to get
>> feedbacks
>>
>> On Wed, Oct 19, 2016 at 9:22 AM, Ali Akhtar  wrote:
>>
>>> Thanks. I've disabled durable writes but this is still pretty slow
>>> (about 10 seconds).
>>>
>>> What issues did you run into with your impl?
>>>
>>> On Wed, Oct 19, 2016 at 12:15 PM, DuyHai Doan 
>>> wrote:
>>>
 There is a lot of pre-flight checks when starting the cassandra server
 and they took time.

 For integration testing, I have developped a modified CassandraDeamon
 here that remove pretty most of those checks:

 https://github.com/doanduyhai/Achilles/blob/master/achilles-
 embedded/src/main/java/info/archinnov/achilles/embedded/Achi
 llesCassandraDaemon.java

 The problem is that I felt into weird scenarios where creating a
 keyspace wasn't created in timely manner so I just stop using this impl for
 the moment, just look at it and do whatever you want.

 Another idea for testing is to disable durable write to speed up
 mutation (CREATE KEYSPACE ... WITH durable_write=false)

 On Wed, Oct 19, 2016 at 3:24 AM, Ali Akhtar 
 wrote:

> Is there a way to speed up the creation of keyspace + tables during
> integration tests? I am using an RF of 1, with SimpleStrategy, but it 
> still
> takes upto 10-15 seconds.
>


>>>
>>
>

Re: Speeding up schema generation during tests

[horschi]

Have you tried starting Cassandra with -Dcassandra.unsafesystem=true ?


On Wed, Oct 19, 2016 at 9:31 AM, DuyHai Doan  wrote:

> As I said, when I bootstrap the server and create some keyspace, sometimes
> the schema is not fully initialized and when the test code tried to insert
> data, it fails.
>
> I did not have time to dig into the source code to find the root cause,
> maybe it's something really stupid and simple to fix. If you want to
> investigate and try out my CassandraDaemon server, I'd be happy to get
> feedbacks
>
> On Wed, Oct 19, 2016 at 9:22 AM, Ali Akhtar  wrote:
>
>> Thanks. I've disabled durable writes but this is still pretty slow (about
>> 10 seconds).
>>
>> What issues did you run into with your impl?
>>
>> On Wed, Oct 19, 2016 at 12:15 PM, DuyHai Doan 
>> wrote:
>>
>>> There is a lot of pre-flight checks when starting the cassandra server
>>> and they took time.
>>>
>>> For integration testing, I have developped a modified CassandraDeamon
>>> here that remove pretty most of those checks:
>>>
>>> https://github.com/doanduyhai/Achilles/blob/master/achilles-
>>> embedded/src/main/java/info/archinnov/achilles/embedded/Achi
>>> llesCassandraDaemon.java
>>>
>>> The problem is that I felt into weird scenarios where creating a
>>> keyspace wasn't created in timely manner so I just stop using this impl for
>>> the moment, just look at it and do whatever you want.
>>>
>>> Another idea for testing is to disable durable write to speed up
>>> mutation (CREATE KEYSPACE ... WITH durable_write=false)
>>>
>>> On Wed, Oct 19, 2016 at 3:24 AM, Ali Akhtar 
>>> wrote:
>>>
 Is there a way to speed up the creation of keyspace + tables during
 integration tests? I am using an RF of 1, with SimpleStrategy, but it still
 takes upto 10-15 seconds.

>>>
>>>
>>
>

Re: Speeding up schema generation during tests

[DuyHai Doan]

As I said, when I bootstrap the server and create some keyspace, sometimes
the schema is not fully initialized and when the test code tried to insert
data, it fails.

I did not have time to dig into the source code to find the root cause,
maybe it's something really stupid and simple to fix. If you want to
investigate and try out my CassandraDaemon server, I'd be happy to get
feedbacks

On Wed, Oct 19, 2016 at 9:22 AM, Ali Akhtar  wrote:

> Thanks. I've disabled durable writes but this is still pretty slow (about
> 10 seconds).
>
> What issues did you run into with your impl?
>
> On Wed, Oct 19, 2016 at 12:15 PM, DuyHai Doan 
> wrote:
>
>> There is a lot of pre-flight checks when starting the cassandra server
>> and they took time.
>>
>> For integration testing, I have developped a modified CassandraDeamon
>> here that remove pretty most of those checks:
>>
>> https://github.com/doanduyhai/Achilles/blob/master/achilles-
>> embedded/src/main/java/info/archinnov/achilles/embedded/Achi
>> llesCassandraDaemon.java
>>
>> The problem is that I felt into weird scenarios where creating a keyspace
>> wasn't created in timely manner so I just stop using this impl for the
>> moment, just look at it and do whatever you want.
>>
>> Another idea for testing is to disable durable write to speed up mutation
>> (CREATE KEYSPACE ... WITH durable_write=false)
>>
>> On Wed, Oct 19, 2016 at 3:24 AM, Ali Akhtar  wrote:
>>
>>> Is there a way to speed up the creation of keyspace + tables during
>>> integration tests? I am using an RF of 1, with SimpleStrategy, but it still
>>> takes upto 10-15 seconds.
>>>
>>
>>
>

Re: Speeding up schema generation during tests

[Ali Akhtar]

Thanks. I've disabled durable writes but this is still pretty slow (about
10 seconds).

What issues did you run into with your impl?

On Wed, Oct 19, 2016 at 12:15 PM, DuyHai Doan  wrote:

> There is a lot of pre-flight checks when starting the cassandra server and
> they took time.
>
> For integration testing, I have developped a modified CassandraDeamon here
> that remove pretty most of those checks:
>
> https://github.com/doanduyhai/Achilles/blob/master/achilles-
> embedded/src/main/java/info/archinnov/achilles/embedded/
> AchillesCassandraDaemon.java
>
> The problem is that I felt into weird scenarios where creating a keyspace
> wasn't created in timely manner so I just stop using this impl for the
> moment, just look at it and do whatever you want.
>
> Another idea for testing is to disable durable write to speed up mutation
> (CREATE KEYSPACE ... WITH durable_write=false)
>
> On Wed, Oct 19, 2016 at 3:24 AM, Ali Akhtar  wrote:
>
>> Is there a way to speed up the creation of keyspace + tables during
>> integration tests? I am using an RF of 1, with SimpleStrategy, but it still
>> takes upto 10-15 seconds.
>>
>
>

Re: Speeding up schema generation during tests

[DuyHai Doan]

There is a lot of pre-flight checks when starting the cassandra server and
they took time.

For integration testing, I have developped a modified CassandraDeamon here
that remove pretty most of those checks:

https://github.com/doanduyhai/Achilles/blob/master/achilles-embedded/src/main/java/info/archinnov/achilles/embedded/AchillesCassandraDaemon.java

The problem is that I felt into weird scenarios where creating a keyspace
wasn't created in timely manner so I just stop using this impl for the
moment, just look at it and do whatever you want.

Another idea for testing is to disable durable write to speed up mutation
(CREATE KEYSPACE ... WITH durable_write=false)

On Wed, Oct 19, 2016 at 3:24 AM, Ali Akhtar  wrote:

> Is there a way to speed up the creation of keyspace + tables during
> integration tests? I am using an RF of 1, with SimpleStrategy, but it still
> takes upto 10-15 seconds.
>

Re: WARN [SharedPool-Worker-3] AbstractTracingAwareExecutorService.java

[DuyHai Doan]

"which appilcation is trying to write large writes and to which keyspace
and table it is trying to write ??"

--> I don't think there is something such as audit trail for each CQL
query.

You may want to enable slow query logger on the client side (driver side)
and bet on the fact that the queries with a huge payload are "slower" than
normal queries ...

On Tue, Oct 18, 2016 at 6:20 PM, James Joseph 
wrote:

>
> I have seen the following warn in system.log, as a temporary turn around i
>  increased commitlog_size in cassandra.yaml to 64, but how can i trace it
> down ??? which appilcation is trying to write large writes and to which
> keyspace and table it is trying to write ??
>
>
> WARN  [SharedPool-Worker-3] 2016-10-05 03:46:22,363
>  AbstractTracingAwareExecutorService.java:169 - Uncaught exception on
> thread Thread[SharedPool-Worker-3,5,main]: {}
> java.lang.IllegalArgumentException: Mutation of 19711728 bytes is too
> large for the maxiumum size of 16777216
>   at org.apache.cassandra.db.commitlog.CommitLog.add(CommitLog.java:221)
> ~[cassandra-all-2.1.8.689.jar:2.1.8.689]
>   at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:383)
> ~[cassandra-all-2.1.8.689.jar:2.1.8.689]
>   at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:363)
> ~[cassandra-all-2.1.8.689.jar:2.1.8.689]
>   at org.apache.cassandra.db.Mutation.apply(Mutation.java:214)
> ~[cassandra-all-2.1.8.689.jar:2.1.8.689]
>   at 
> org.apache.cassandra.db.MutationVerbHandler.doVerb(MutationVerbHandler.java:54)
> ~[cassandra-all-2.1.8.689.jar:2.1.8.689]
>   at 
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:64)
> ~[cassandra-all-2.1.8.689.jar:2.1.8.689]
>   at java.util.concurrent.Executors$RunnableAdapter.call(Unknown
> Source) ~[na:1.8.0_92]
>   at org.apache.cassandra.concurrent.AbstractTracingAwareExecutor
> Service$FutureTask.run(AbstractTracingAwareExecutorService.java:164)
> ~[cassandra-all-2.1.8.689.jar:2.1.8.689]
>   at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105)
> [cassandra-all-2.1.8.689.jar:2.1.8.689]
>   at java.lang.Thread.run(Unknown Source) [na:1.8.0_92]
>
>
> Thanks
> James.
>
>