Re: Gossip 2.0

[Eric Evans]

On Thu, Sep 1, 2016 at 7:02 AM, Jason Brown  wrote:
> have opened up CASSANDRA-12345...

Nice; What did you do, camp on the "create" button until after 12344
was submitted? :)

-- 
Eric Evans
john.eric.ev...@gmail.com

Re: Gossip 2.0

[Jeremiah D Jordan]

He denied it when I asked him that earlier.  But we know he did.
http://wilderness.apache.org/channels/?f=cassandra-dev/2016-09-01#1472732219 


> On Sep 1, 2016, at 11:02 AM, Eric Evans  wrote:
> 
> On Thu, Sep 1, 2016 at 7:02 AM, Jason Brown  wrote:
>> have opened up CASSANDRA-12345...
> 
> Nice; What did you do, camp on the "create" button until after 12344
> was submitted? :)
> 
> -- 
> Eric Evans
> john.eric.ev...@gmail.com

Gossip 2.0

[Jason Brown]

Hi all,

One of the problems I been thinking about for quite a while is how to
improve the efficacy of cassandra's gossip, especially when dealing with
large clusters (greater than 1,000 nodes). By reducing gossip overhead we
can implement new features that require data to be disseminated
cluster-wide and we can support those features to thousands of nodes. To
that end, I've done much research and thinking, and have opened up
CASSANDRA-12345 as a parent ticket for this work. I'd like to invite the
community to discuss the ideas, the approach, and hopefully dig into the
implementations that I've posted in the child tickets.

Thanks,

-Jason

Gossip Behavioral Difference between C* 2.0 and C* 2.1

[Michael Fong]

Hi,

We recently discovered that there are some differences in gossip behavior 
between C* 2.0 and C* 2.1. In some cases of network instability or a node 
reboot, we can observe some behavioral differences from Cassandra/system.log.

2.0.17
We can observe this log of similar pattern in log :
DEBUG [RequestResponseStage:3] 2016-04-19 11:18:18,332 Gossiper.java (line 977) 
removing expire time for endpoint : /192.168.88.34
INFO [RequestResponseStage:3] 2016-04-19 11:18:18,333 Gossiper.java (line 978) 
InetAddress /192.168.88.34 is now UP
DEBUG [RequestResponseStage:4] 2016-04-19 11:18:18,335 Gossiper.java (line 977) 
removing expire time for endpoint : /192.168.88.34
INFO [RequestResponseStage:4] 2016-04-19 11:18:18,335 Gossiper.java (line 978) 
InetAddress /192.168.88.34 is now UP
DEBUG [RequestResponseStage:3] 2016-04-19 11:18:18,335 Gossiper.java (line 977) 
removing expire time for endpoint : /192.168.88.34
INFO [RequestResponseStage:3] 2016-04-19 11:18:18,335 Gossiper.java (line 978) 
InetAddress /192.168.88.34 is now UP


It seems the longer for the node to regain connection (or reboot), the more 
accumulated gossip message, and the more gossip message will appear afterwards.

However,  in 2.1, we do not observe this kind of behavior any more. There seems 
to be some fundamental changes on gossip protocol. Did anyone also observe the 
similar pattern, or could kindly point out which changes (JIRA #) that made of 
this improvement?

Thanks in advanced!

Sincerely,

Michael Fong

RE: Cassandra 2.0.x OOM during startsup - schema version inconsistency after reboot

[Michael Fong]

Hi Alain,

Thanks for your reply.

We understood that there is a chance that this would be left unresolved, since 
we are really way behind the official Cassandra releases.

Here is what have further found for the OOM issue, which seems to be related to 
# of gossip message accumulated on a live node waiting to connect to the 
rebooted node. Once that node is rebooted, all the gossip message floods into 
each other, triggers StorageService.onAlive() and schedules a schema pull on 
demand. In our case, schema version sometimes is different after reboot. When 
that happens, schema-exchange storm begins.

Also, thanks for your tip on sharing the SOP on stopping an ode, here is what 
we have for our stop procedure:
Disable thrift
Disable Binary
Wait 10s
Disable gossip
Drain
Kill 

Any thought on this to be further improved?

Thanks!

Sincerely,

Michael Fong


From: Alain RODRIGUEZ [mailto:arodr...@gmail.com]
Sent: Wednesday, May 11, 2016 10:01 PM
To: u...@cassandra.apache.org
Cc: dev@cassandra.apache.org
Subject: Re: Cassandra 2.0.x OOM during startsup - schema version inconsistency 
after reboot

Hi Michaels :-),

My guess is this ticket will be closed with a "Won't Fix" resolution.

Cassandra 2.0 is no longer supported and I have seen tickets being rejected 
like CASSANDRA-10510<https://issues.apache.org/jira/browse/CASSANDRA-10510>.

Would you like to upgrade to 2.1.last and see if you still have the issue?

About your issue, do you stop your node using a command like the following one?

nodetool disablethrift && nodetool disablebinary && sleep 5 && nodetool 
disablegossip && sleep 10 && nodetool drain && sleep 10 && sudo service 
cassandra stop

or even flushing:

nodetool disablethrift && nodetool disablebinary && sleep 5 && nodetool 
disablegossip && sleep 10 && nodetool flush && nodetool drain && sleep 10 && 
sudo service cassandra stop

Are commitlogs empty when you start cassandra?

C*heers,

---
Alain Rodriguez - al...@thelastpickle.com<mailto:al...@thelastpickle.com>
France

The Last Pickle - Apache Cassandra Consulting
http://www.thelastpickle.com

2016-05-11 5:35 GMT+02:00 Michael Fong 
<michael.f...@ruckuswireless.com<mailto:michael.f...@ruckuswireless.com>>:
Hi,

Thanks for your recommendation.
I also opened a ticket to keep track @ 
https://issues.apache.org/jira/browse/CASSANDRA-11748
Hope this could brought someone's attention to take a look. Thanks.

Sincerely,

Michael Fong

-Original Message-
From: Michael Kjellman 
[mailto:mkjell...@internalcircle.com<mailto:mkjell...@internalcircle.com>]
Sent: Monday, May 09, 2016 11:57 AM
To: dev@cassandra.apache.org<mailto:dev@cassandra.apache.org>
Cc: u...@cassandra.apache.org<mailto:u...@cassandra.apache.org>
Subject: Re: Cassandra 2.0.x OOM during startsup - schema version inconsistency 
after reboot

I'd recommend you create a JIRA! That way you can get some traction on the 
issue. Obviously an OOM is never correct, even if your process is wrong in some 
way!

Best,
kjellman

Sent from my iPhone

> On May 8, 2016, at 8:48 PM, Michael Fong 
> <michael.f...@ruckuswireless.com<mailto:michael.f...@ruckuswireless.com>> 
> wrote:
>
> Hi, all,
>
>
> Haven't heard any responses so far, and this isue has troubled us for quite 
> some time. Here is another update:
>
> We have noticed several times that The schema version may change after 
> migration and reboot:
>
> Here is the scenario:
>
> 1.   Two node cluster (1 & 2).
>
> 2.   There are some schema changes, i.e. create a few new columnfamily. 
> The cluster will wait until both nodes have schema version in sync (describe 
> cluster) before moving on.
>
> 3.   Right before node2 is rebooted, the schema version is consistent; 
> however, after ndoe2 reboots and starts servicing, the MigrationManager would 
> gossip different schema version.
>
> 4.   Afterwards, both nodes starts exchanging schema  message 
> indefinitely until one of the node dies.
>
> We currently suspect the change of schema is due to replying the old entry in 
> commit log. We wish to continue dig further, but need experts help on this.
>
> I don't know if anyone has seen this before, or if there is anything wrong 
> with our migration flow though..
>
> Thanks in advance.
>
> Best regards,
>
>
> Michael Fong
>
> From: Michael Fong 
> [mailto:michael.f...@ruckuswireless.com<mailto:michael.f...@ruckuswireless.com>]
> Sent: Thursday, April 21, 2016 6:41 PM
> To: u...@cassandra.apache.org<mailto:u...@cassandra.apache.org>; 
> dev@cassandra.apache.org<mailto:dev@cassandra.apache.org>
> Subject: RE: Cassandra 2.0.x OOM during bootstrap
>
> Hi,

Re: Cassandra 2.0.x OOM during startsup - schema version inconsistency after reboot

[Alain RODRIGUEZ]

Hi Michaels :-),

My guess is this ticket will be closed with a "Won't Fix" resolution.

Cassandra 2.0 is no longer supported and I have seen tickets being rejected
like CASSANDRA-10510 <https://issues.apache.org/jira/browse/CASSANDRA-10510>
.

Would you like to upgrade to 2.1.last and see if you still have the issue?

About your issue, do you stop your node using a command like the following
one?

nodetool disablethrift && nodetool disablebinary && sleep 5 && nodetool
disablegossip && sleep 10 && nodetool drain && sleep 10 && sudo service
cassandra stop

or even flushing:

nodetool disablethrift && nodetool disablebinary && sleep 5 && nodetool
disablegossip && sleep 10 && nodetool flush && nodetool drain && sleep 10
&& sudo service cassandra stop

Are commitlogs empty when you start cassandra?

C*heers,

---
Alain Rodriguez - al...@thelastpickle.com
France

The Last Pickle - Apache Cassandra Consulting
http://www.thelastpickle.com

2016-05-11 5:35 GMT+02:00 Michael Fong <michael.f...@ruckuswireless.com>:

> Hi,
>
> Thanks for your recommendation.
> I also opened a ticket to keep track @
> https://issues.apache.org/jira/browse/CASSANDRA-11748
> Hope this could brought someone's attention to take a look. Thanks.
>
> Sincerely,
>
> Michael Fong
>
> -Original Message-
> From: Michael Kjellman [mailto:mkjell...@internalcircle.com]
> Sent: Monday, May 09, 2016 11:57 AM
> To: dev@cassandra.apache.org
> Cc: u...@cassandra.apache.org
> Subject: Re: Cassandra 2.0.x OOM during startsup - schema version
> inconsistency after reboot
>
> I'd recommend you create a JIRA! That way you can get some traction on the
> issue. Obviously an OOM is never correct, even if your process is wrong in
> some way!
>
> Best,
> kjellman
>
> Sent from my iPhone
>
> > On May 8, 2016, at 8:48 PM, Michael Fong <
> michael.f...@ruckuswireless.com> wrote:
> >
> > Hi, all,
> >
> >
> > Haven't heard any responses so far, and this isue has troubled us for
> quite some time. Here is another update:
> >
> > We have noticed several times that The schema version may change after
> migration and reboot:
> >
> > Here is the scenario:
> >
> > 1.   Two node cluster (1 & 2).
> >
> > 2.   There are some schema changes, i.e. create a few new
> columnfamily. The cluster will wait until both nodes have schema version in
> sync (describe cluster) before moving on.
> >
> > 3.   Right before node2 is rebooted, the schema version is
> consistent; however, after ndoe2 reboots and starts servicing, the
> MigrationManager would gossip different schema version.
> >
> > 4.   Afterwards, both nodes starts exchanging schema  message
> indefinitely until one of the node dies.
> >
> > We currently suspect the change of schema is due to replying the old
> entry in commit log. We wish to continue dig further, but need experts help
> on this.
> >
> > I don't know if anyone has seen this before, or if there is anything
> wrong with our migration flow though..
> >
> > Thanks in advance.
> >
> > Best regards,
> >
> >
> > Michael Fong
> >
> > From: Michael Fong [mailto:michael.f...@ruckuswireless.com]
> > Sent: Thursday, April 21, 2016 6:41 PM
> > To: u...@cassandra.apache.org; dev@cassandra.apache.org
> > Subject: RE: Cassandra 2.0.x OOM during bootstrap
> >
> > Hi, all,
> >
> > Here is some more information on before the OOM happened on the rebooted
> node in a 2-node test cluster:
> >
> >
> > 1.   It seems the schema version has changed on the rebooted node
> after reboot, i.e.
> > Before reboot,
> > Node 1: DEBUG [MigrationStage:1] 2016-04-19 11:09:42,326
> > MigrationManager.java (line 328) Gossiping my schema version
> > 4cb463f8-5376-3baf-8e88-a5cc6a94f58f
> > Node 2: DEBUG [MigrationStage:1] 2016-04-19 11:09:42,122
> > MigrationManager.java (line 328) Gossiping my schema version
> > 4cb463f8-5376-3baf-8e88-a5cc6a94f58f
> >
> > After rebooting node 2,
> > Node 2: DEBUG [main] 2016-04-19 11:18:18,016 MigrationManager.java
> > (line 328) Gossiping my schema version
> > f5270873-ba1f-39c7-ab2e-a86db868b09b
> >
> >
> >
> > 2.   After reboot, both nods repeatedly send MigrationTask to each
> other - we suspect it is related to the schema version (Digest) mismatch
> after Node 2 rebooted:
> > The node2  keeps submitting the migration task over 100+ times to the
> other node.
> > INFO [GossipStag

RE: Cassandra 2.0.x OOM during startsup - schema version inconsistency after reboot

[Michael Fong]

Hi,

Thanks for your recommendation. 
I also opened a ticket to keep track @ 
https://issues.apache.org/jira/browse/CASSANDRA-11748
Hope this could brought someone's attention to take a look. Thanks.

Sincerely,

Michael Fong

-Original Message-
From: Michael Kjellman [mailto:mkjell...@internalcircle.com] 
Sent: Monday, May 09, 2016 11:57 AM
To: dev@cassandra.apache.org
Cc: u...@cassandra.apache.org
Subject: Re: Cassandra 2.0.x OOM during startsup - schema version inconsistency 
after reboot

I'd recommend you create a JIRA! That way you can get some traction on the 
issue. Obviously an OOM is never correct, even if your process is wrong in some 
way!

Best,
kjellman 

Sent from my iPhone

> On May 8, 2016, at 8:48 PM, Michael Fong <michael.f...@ruckuswireless.com> 
> wrote:
> 
> Hi, all,
> 
> 
> Haven't heard any responses so far, and this isue has troubled us for quite 
> some time. Here is another update:
> 
> We have noticed several times that The schema version may change after 
> migration and reboot:
> 
> Here is the scenario:
> 
> 1.   Two node cluster (1 & 2).
> 
> 2.   There are some schema changes, i.e. create a few new columnfamily. 
> The cluster will wait until both nodes have schema version in sync (describe 
> cluster) before moving on.
> 
> 3.   Right before node2 is rebooted, the schema version is consistent; 
> however, after ndoe2 reboots and starts servicing, the MigrationManager would 
> gossip different schema version.
> 
> 4.   Afterwards, both nodes starts exchanging schema  message 
> indefinitely until one of the node dies.
> 
> We currently suspect the change of schema is due to replying the old entry in 
> commit log. We wish to continue dig further, but need experts help on this.
> 
> I don't know if anyone has seen this before, or if there is anything wrong 
> with our migration flow though..
> 
> Thanks in advance.
> 
> Best regards,
> 
> 
> Michael Fong
> 
> From: Michael Fong [mailto:michael.f...@ruckuswireless.com]
> Sent: Thursday, April 21, 2016 6:41 PM
> To: u...@cassandra.apache.org; dev@cassandra.apache.org
> Subject: RE: Cassandra 2.0.x OOM during bootstrap
> 
> Hi, all,
> 
> Here is some more information on before the OOM happened on the rebooted node 
> in a 2-node test cluster:
> 
> 
> 1.   It seems the schema version has changed on the rebooted node after 
> reboot, i.e.
> Before reboot,
> Node 1: DEBUG [MigrationStage:1] 2016-04-19 11:09:42,326 
> MigrationManager.java (line 328) Gossiping my schema version 
> 4cb463f8-5376-3baf-8e88-a5cc6a94f58f
> Node 2: DEBUG [MigrationStage:1] 2016-04-19 11:09:42,122 
> MigrationManager.java (line 328) Gossiping my schema version 
> 4cb463f8-5376-3baf-8e88-a5cc6a94f58f
> 
> After rebooting node 2,
> Node 2: DEBUG [main] 2016-04-19 11:18:18,016 MigrationManager.java 
> (line 328) Gossiping my schema version 
> f5270873-ba1f-39c7-ab2e-a86db868b09b
> 
> 
> 
> 2.   After reboot, both nods repeatedly send MigrationTask to each other 
> - we suspect it is related to the schema version (Digest) mismatch after Node 
> 2 rebooted:
> The node2  keeps submitting the migration task over 100+ times to the other 
> node.
> INFO [GossipStage:1] 2016-04-19 11:18:18,261 Gossiper.java (line 1011) 
> Node /192.168.88.33 has restarted, now UP INFO [GossipStage:1] 
> 2016-04-19 11:18:18,262 TokenMetadata.java (line 414) Updating 
> topology for /192.168.88.33 INFO [GossipStage:1] 2016-04-19 
> 11:18:18,263 StorageService.java (line 1544) Node /192.168.88.33 state 
> jump to normal INFO [GossipStage:1] 2016-04-19 11:18:18,264 
> TokenMetadata.java (line 414) Updating topology for /192.168.88.33 DEBUG 
> [GossipStage:1] 2016-04-19 11:18:18,265 MigrationManager.java (line 102) 
> Submitting migration task for /192.168.88.33 DEBUG [GossipStage:1] 2016-04-19 
> 11:18:18,265 MigrationManager.java (line 102) Submitting migration task for 
> /192.168.88.33 DEBUG [MigrationStage:1] 2016-04-19 11:18:18,268 
> MigrationTask.java (line 62) Can't send schema pull request: node 
> /192.168.88.33 is down.
> DEBUG [MigrationStage:1] 2016-04-19 11:18:18,268 MigrationTask.java (line 62) 
> Can't send schema pull request: node /192.168.88.33 is down.
> DEBUG [RequestResponseStage:1] 2016-04-19 11:18:18,353 Gossiper.java 
> (line 977) removing expire time for endpoint : /192.168.88.33 INFO 
> [RequestResponseStage:1] 2016-04-19 11:18:18,353 Gossiper.java (line 
> 978) InetAddress /192.168.88.33 is now UP DEBUG 
> [RequestResponseStage:1] 2016-04-19 11:18:18,353 MigrationManager.java 
> (line 102) Submitting migration task for /192.168.88.33 DEBUG 
> [RequestResponseStage:1] 2016-04-19 11:18:18,355 Gossiper.java (line 
> 977) removing expi

Re: Cassandra 2.0.x OOM during startsup - schema version inconsistency after reboot

[Michael Kjellman]

I'd recommend you create a JIRA! That way you can get some traction on the 
issue. Obviously an OOM is never correct, even if your process is wrong in some 
way!

Best,
kjellman 

Sent from my iPhone

> On May 8, 2016, at 8:48 PM, Michael Fong <michael.f...@ruckuswireless.com> 
> wrote:
> 
> Hi, all,
> 
> 
> Haven't heard any responses so far, and this isue has troubled us for quite 
> some time. Here is another update:
> 
> We have noticed several times that The schema version may change after 
> migration and reboot:
> 
> Here is the scenario:
> 
> 1.   Two node cluster (1 & 2).
> 
> 2.   There are some schema changes, i.e. create a few new columnfamily. 
> The cluster will wait until both nodes have schema version in sync (describe 
> cluster) before moving on.
> 
> 3.   Right before node2 is rebooted, the schema version is consistent; 
> however, after ndoe2 reboots and starts servicing, the MigrationManager would 
> gossip different schema version.
> 
> 4.   Afterwards, both nodes starts exchanging schema  message 
> indefinitely until one of the node dies.
> 
> We currently suspect the change of schema is due to replying the old entry in 
> commit log. We wish to continue dig further, but need experts help on this.
> 
> I don't know if anyone has seen this before, or if there is anything wrong 
> with our migration flow though..
> 
> Thanks in advance.
> 
> Best regards,
> 
> 
> Michael Fong
> 
> From: Michael Fong [mailto:michael.f...@ruckuswireless.com]
> Sent: Thursday, April 21, 2016 6:41 PM
> To: u...@cassandra.apache.org; dev@cassandra.apache.org
> Subject: RE: Cassandra 2.0.x OOM during bootstrap
> 
> Hi, all,
> 
> Here is some more information on before the OOM happened on the rebooted node 
> in a 2-node test cluster:
> 
> 
> 1.   It seems the schema version has changed on the rebooted node after 
> reboot, i.e.
> Before reboot,
> Node 1: DEBUG [MigrationStage:1] 2016-04-19 11:09:42,326 
> MigrationManager.java (line 328) Gossiping my schema version 
> 4cb463f8-5376-3baf-8e88-a5cc6a94f58f
> Node 2: DEBUG [MigrationStage:1] 2016-04-19 11:09:42,122 
> MigrationManager.java (line 328) Gossiping my schema version 
> 4cb463f8-5376-3baf-8e88-a5cc6a94f58f
> 
> After rebooting node 2,
> Node 2: DEBUG [main] 2016-04-19 11:18:18,016 MigrationManager.java (line 328) 
> Gossiping my schema version f5270873-ba1f-39c7-ab2e-a86db868b09b
> 
> 
> 
> 2.   After reboot, both nods repeatedly send MigrationTask to each other 
> - we suspect it is related to the schema version (Digest) mismatch after Node 
> 2 rebooted:
> The node2  keeps submitting the migration task over 100+ times to the other 
> node.
> INFO [GossipStage:1] 2016-04-19 11:18:18,261 Gossiper.java (line 1011) Node 
> /192.168.88.33 has restarted, now UP
> INFO [GossipStage:1] 2016-04-19 11:18:18,262 TokenMetadata.java (line 414) 
> Updating topology for /192.168.88.33
> INFO [GossipStage:1] 2016-04-19 11:18:18,263 StorageService.java (line 1544) 
> Node /192.168.88.33 state jump to normal
> INFO [GossipStage:1] 2016-04-19 11:18:18,264 TokenMetadata.java (line 414) 
> Updating topology for /192.168.88.33
> DEBUG [GossipStage:1] 2016-04-19 11:18:18,265 MigrationManager.java (line 
> 102) Submitting migration task for /192.168.88.33
> DEBUG [GossipStage:1] 2016-04-19 11:18:18,265 MigrationManager.java (line 
> 102) Submitting migration task for /192.168.88.33
> DEBUG [MigrationStage:1] 2016-04-19 11:18:18,268 MigrationTask.java (line 62) 
> Can't send schema pull request: node /192.168.88.33 is down.
> DEBUG [MigrationStage:1] 2016-04-19 11:18:18,268 MigrationTask.java (line 62) 
> Can't send schema pull request: node /192.168.88.33 is down.
> DEBUG [RequestResponseStage:1] 2016-04-19 11:18:18,353 Gossiper.java (line 
> 977) removing expire time for endpoint : /192.168.88.33
> INFO [RequestResponseStage:1] 2016-04-19 11:18:18,353 Gossiper.java (line 
> 978) InetAddress /192.168.88.33 is now UP
> DEBUG [RequestResponseStage:1] 2016-04-19 11:18:18,353 MigrationManager.java 
> (line 102) Submitting migration task for /192.168.88.33
> DEBUG [RequestResponseStage:1] 2016-04-19 11:18:18,355 Gossiper.java (line 
> 977) removing expire time for endpoint : /192.168.88.33
> INFO [RequestResponseStage:1] 2016-04-19 11:18:18,355 Gossiper.java (line 
> 978) InetAddress /192.168.88.33 is now UP
> DEBUG [RequestResponseStage:1] 2016-04-19 11:18:18,355 MigrationManager.java 
> (line 102) Submitting migration task for /192.168.88.33
> DEBUG [RequestResponseStage:2] 2016-04-19 11:18:18,355 Gossiper.java (line 
> 977) removing expire time for endpoint : /192.168.88.33
> INFO [RequestResponseStage:2] 2016-04-19 11:18:18,355 Gos

RE: Cassandra 2.0.x OOM during startsup - schema version inconsistency after reboot

[Michael Fong]

Hi, all,


Haven't heard any responses so far, and this isue has troubled us for quite 
some time. Here is another update:

We have noticed several times that The schema version may change after 
migration and reboot:

Here is the scenario:

1.   Two node cluster (1 & 2).

2.   There are some schema changes, i.e. create a few new columnfamily. The 
cluster will wait until both nodes have schema version in sync (describe 
cluster) before moving on.

3.   Right before node2 is rebooted, the schema version is consistent; 
however, after ndoe2 reboots and starts servicing, the MigrationManager would 
gossip different schema version.

4.   Afterwards, both nodes starts exchanging schema  message indefinitely 
until one of the node dies.

We currently suspect the change of schema is due to replying the old entry in 
commit log. We wish to continue dig further, but need experts help on this.

I don't know if anyone has seen this before, or if there is anything wrong with 
our migration flow though..

Thanks in advance.

Best regards,


Michael Fong

From: Michael Fong [mailto:michael.f...@ruckuswireless.com]
Sent: Thursday, April 21, 2016 6:41 PM
To: u...@cassandra.apache.org; dev@cassandra.apache.org
Subject: RE: Cassandra 2.0.x OOM during bootstrap

Hi, all,

Here is some more information on before the OOM happened on the rebooted node 
in a 2-node test cluster:


1.   It seems the schema version has changed on the rebooted node after 
reboot, i.e.
Before reboot,
Node 1: DEBUG [MigrationStage:1] 2016-04-19 11:09:42,326 MigrationManager.java 
(line 328) Gossiping my schema version 4cb463f8-5376-3baf-8e88-a5cc6a94f58f
Node 2: DEBUG [MigrationStage:1] 2016-04-19 11:09:42,122 MigrationManager.java 
(line 328) Gossiping my schema version 4cb463f8-5376-3baf-8e88-a5cc6a94f58f

After rebooting node 2,
Node 2: DEBUG [main] 2016-04-19 11:18:18,016 MigrationManager.java (line 328) 
Gossiping my schema version f5270873-ba1f-39c7-ab2e-a86db868b09b



2.   After reboot, both nods repeatedly send MigrationTask to each other - 
we suspect it is related to the schema version (Digest) mismatch after Node 2 
rebooted:
The node2  keeps submitting the migration task over 100+ times to the other 
node.
INFO [GossipStage:1] 2016-04-19 11:18:18,261 Gossiper.java (line 1011) Node 
/192.168.88.33 has restarted, now UP
INFO [GossipStage:1] 2016-04-19 11:18:18,262 TokenMetadata.java (line 414) 
Updating topology for /192.168.88.33
INFO [GossipStage:1] 2016-04-19 11:18:18,263 StorageService.java (line 1544) 
Node /192.168.88.33 state jump to normal
INFO [GossipStage:1] 2016-04-19 11:18:18,264 TokenMetadata.java (line 414) 
Updating topology for /192.168.88.33
DEBUG [GossipStage:1] 2016-04-19 11:18:18,265 MigrationManager.java (line 102) 
Submitting migration task for /192.168.88.33
DEBUG [GossipStage:1] 2016-04-19 11:18:18,265 MigrationManager.java (line 102) 
Submitting migration task for /192.168.88.33
DEBUG [MigrationStage:1] 2016-04-19 11:18:18,268 MigrationTask.java (line 62) 
Can't send schema pull request: node /192.168.88.33 is down.
DEBUG [MigrationStage:1] 2016-04-19 11:18:18,268 MigrationTask.java (line 62) 
Can't send schema pull request: node /192.168.88.33 is down.
DEBUG [RequestResponseStage:1] 2016-04-19 11:18:18,353 Gossiper.java (line 977) 
removing expire time for endpoint : /192.168.88.33
INFO [RequestResponseStage:1] 2016-04-19 11:18:18,353 Gossiper.java (line 978) 
InetAddress /192.168.88.33 is now UP
DEBUG [RequestResponseStage:1] 2016-04-19 11:18:18,353 MigrationManager.java 
(line 102) Submitting migration task for /192.168.88.33
DEBUG [RequestResponseStage:1] 2016-04-19 11:18:18,355 Gossiper.java (line 977) 
removing expire time for endpoint : /192.168.88.33
INFO [RequestResponseStage:1] 2016-04-19 11:18:18,355 Gossiper.java (line 978) 
InetAddress /192.168.88.33 is now UP
DEBUG [RequestResponseStage:1] 2016-04-19 11:18:18,355 MigrationManager.java 
(line 102) Submitting migration task for /192.168.88.33
DEBUG [RequestResponseStage:2] 2016-04-19 11:18:18,355 Gossiper.java (line 977) 
removing expire time for endpoint : /192.168.88.33
INFO [RequestResponseStage:2] 2016-04-19 11:18:18,355 Gossiper.java (line 978) 
InetAddress /192.168.88.33 is now UP
DEBUG [RequestResponseStage:2] 2016-04-19 11:18:18,356 MigrationManager.java 
(line 102) Submitting migration task for /192.168.88.33
.


On the otherhand, Node 1 keeps updating its gossip information, followed by 
receiving and submitting migrationTask afterwards:
DEBUG [RequestResponseStage:3] 2016-04-19 11:18:18,332 Gossiper.java (line 977) 
removing expire time for endpoint : /192.168.88.34
INFO [RequestResponseStage:3] 2016-04-19 11:18:18,333 Gossiper.java (line 978) 
InetAddress /192.168.88.34 is now UP
DEBUG [RequestResponseStage:4] 2016-04-19 11:18:18,335 Gossiper.java (line 977) 
removing expire time for endpoint : /192.168.88.34
INFO [RequestResponseStage:4] 2016-04-19 11:18:18,335 Gossiper.java (line 978) 
InetAdd

RE: Cassandra 2.0.x OOM during bootstrap

[Michael Fong]

 in advanced!

Sincerely,

Michael Fong

From: Michael Fong [mailto:michael.f...@ruckuswireless.com]
Sent: Wednesday, April 20, 2016 10:43 AM
To: u...@cassandra.apache.org; dev@cassandra.apache.org
Subject: Cassandra 2.0.x OOM during bootstrap

Hi, all,

We have recently encountered a Cassandra OOM issue when Cassandra is brought up 
sometimes (but not always) in our 4-node cluster test bed.

After analyzing the heap dump, we could find the Internal-Response thread pool 
(JMXEnabledThreadPoolExecutor) is filled with thounds of 
'org.apache.cassandra.net.MessageIn' objects, and occupy > 2 gigabytes of heap 
memory.

According to the documents on internet, it seems internal-response thread pool 
is somewhat related to schema-checking. Has anyone encountered similar issue 
before?

We are using Cassandra 2.0.17 and JDK 1.8. Thanks in advance!

Sincerely,

Michael Fong

Cassandra 2.0.x OOM during bootstrap

[Michael Fong]

Hi, all,

We have recently encountered a Cassandra OOM issue when Cassandra is brought up 
sometimes (but not always) in our 4-node cluster test bed.

After analyzing the heap dump, we could find the Internal-Response thread pool 
(JMXEnabledThreadPoolExecutor) is filled with thounds of 
'org.apache.cassandra.net.MessageIn' objects, and occupy > 2 gigabytes of heap 
memory.

According to the documents on internet, it seems internal-response thread pool 
is somewhat related to schema-checking. Has anyone encountered similar issue 
before?

We are using Cassandra 2.0.17 and JDK 1.8. Thanks in advance!

Sincerely,

Michael Fong

Modeling nested collection with C* 2.0

[Ahmed Eljami]

​Hi,

I need your help for modeling a nested collection with cassanrda2.0 (UDT no,
no fozen)

My users table contains emails by type, each type of email contains multiple
emails.

Example:
Type: pro. emails: {a...@mail.com, b...@mail.com ...}

Type: private. emails: {c...@mail.com, d...@mail.com}
.

The user table also contains addresses, address type with fields.

Example:

Type: Pro. address {Street= aaa, number = 123, apartment = bbb}

Type: Private. address {Street = bbb, number = 123, apartment = kkk }

I am looking for a solution to store all these columns in one table.

Thank you.

Re: Modeling nested collection with C* 2.0

[Ryan Svihla]

Ahmed,

Just using text and serializing as Json is the easy way and a common approach.

However, this list is for Cassandra commiter discussion, please be so kind as 
to use the regular user list for data modeling questions or for any future 
responses to this email thread.


Regards,
Ryan Svihla

> On Jan 28, 2016, at 7:28 AM, Ahmed Eljami  wrote:
> 
> ​Hi,
> 
> I need your help for modeling a nested collection with cassanrda2.0 (UDT no,
> no fozen)
> 
> My users table contains emails by type, each type of email contains multiple
> emails.
> 
> Example:
> Type: pro. emails: {a...@mail.com, b...@mail.com ...}
> 
> Type: private. emails: {c...@mail.com, d...@mail.com}
> .
> 
> The user table also contains addresses, address type with fields.
> 
> Example:
> 
> Type: Pro. address {Street= aaa, number = 123, apartment = bbb}
> 
> Type: Private. address {Street = bbb, number = 123, apartment = kkk }
> 
> I am looking for a solution to store all these columns in one table.
> 
> Thank you.

Committing to 2.0 and merging up without touching 2.1.0

[Brandon Williams]

Committers,

Since we've had the 2.1.0 branch around a bit longer than we normally would
this time around, we're starting to run into situations where we want to
commit something to 2.0 and 2.1, but skip 2.1.0.  Here's the process to
correctly do that:

First, commit to 2.0 (obviously).

Next, merge 2.0 into 2.1, so the commit is in there too.

Here's the slightly tricky part:

Merge 2.0 into 2.1.0, with the '-s ours' flag. git merge -s ours
cassandra-2.0  This means 2.1.0 won't change at all, but will record the
merge.

Finally, merge 2.1.0 into 2.1, again with -s ours, so nothing changes in
2.1 and it keeps the commit.

After this of course you can merge to trunk normally.

Hope that helps,
-Brandon

Re: Cassandra 2.0 with Hadoop 2.x?

[Ben Coverston]

Clint, I'm hoping that
https://issues.apache.org/jira/browse/CASSANDRA-5201 will
make it into Cassandra 2.0, as it is a general purpose solution that will
work for both mapred, and mapreduce based code.


On Wed, Feb 5, 2014 at 1:33 PM, Clint Kelly clint.ke...@gmail.com wrote:

 One other question: Has anyone considered taking a crack at modifying
 the CqlPagingRecordReader to use the DataStax java driver?  I am not
 sure how that kind of cross-pollination of different projects works,
 but I thought the paging mechanisms in the DataStax driver might help
 simplify the CqlPagingRecordReader code.

 Best regards,
 Clint

 On Tue, Feb 4, 2014 at 12:49 PM, Clint Kelly clint.ke...@gmail.com
 wrote:
  Folks,
 
  Has anyone out there used Cassandra 2.0 with Hadoop 2.x?  I saw this
  discussion on the Cassandra JIRA:
 
  https://issues.apache.org/jira/browse/CASSANDRA-5201
 
  but the fix (https://github.com/michaelsembwever/cassandra-hadoop)
  referenced in the thread is for Cassandra 1.2.
 
  I put together a similar patch for Cassandra 2.0 for anyone who is
 interested:
 
  https://github.com/wibiclint/cassandra2-hadoop2
 
  but I'm wondering if there is a more official solution to this
  problem.  Any help would be appreciated.  Thanks!
 
  Best regards,
  Clint




-- 
Ben Coverston
DataStax -- The Apache Cassandra Company

Re: Cassandra 2.0 with Hadoop 2.x?

[Clint Kelly]

One other question: Has anyone considered taking a crack at modifying
the CqlPagingRecordReader to use the DataStax java driver?  I am not
sure how that kind of cross-pollination of different projects works,
but I thought the paging mechanisms in the DataStax driver might help
simplify the CqlPagingRecordReader code.

Best regards,
Clint

On Tue, Feb 4, 2014 at 12:49 PM, Clint Kelly clint.ke...@gmail.com wrote:
 Folks,

 Has anyone out there used Cassandra 2.0 with Hadoop 2.x?  I saw this
 discussion on the Cassandra JIRA:

 https://issues.apache.org/jira/browse/CASSANDRA-5201

 but the fix (https://github.com/michaelsembwever/cassandra-hadoop)
 referenced in the thread is for Cassandra 1.2.

 I put together a similar patch for Cassandra 2.0 for anyone who is interested:

 https://github.com/wibiclint/cassandra2-hadoop2

 but I'm wondering if there is a more official solution to this
 problem.  Any help would be appreciated.  Thanks!

 Best regards,
 Clint

Cassandra 2.0 with Hadoop 2.x?

[Clint Kelly]

Folks,

Has anyone out there used Cassandra 2.0 with Hadoop 2.x?  I saw this
discussion on the Cassandra JIRA:

https://issues.apache.org/jira/browse/CASSANDRA-5201

but the fix (https://github.com/michaelsembwever/cassandra-hadoop)
referenced in the thread is for Cassandra 1.2.

I put together a similar patch for Cassandra 2.0 for anyone who is interested:

https://github.com/wibiclint/cassandra2-hadoop2

but I'm wondering if there is a more official solution to this
problem.  Any help would be appreciated.  Thanks!

Best regards,
Clint

Cassandra 2.0 new features ?

[Emalayan Vairavanathan]

Hi All,

Can anyone tell me about the new features that are going to come in Cassandra 
2.0 ?

Thank you
Emalayan

Re: Cassandra 2.0 new features ?

[Tomaz Muraus]

Check CHANGES file (
https://github.com/apache/cassandra/blob/trunk/CHANGES.txt) and / or JIRA (
https://issues.apache.org/jira/browse/CASSANDRA).

On Wed, Jun 12, 2013 at 3:48 PM, Emalayan Vairavanathan 
svemala...@yahoo.com wrote:

 Hi All,

 Can anyone tell me about the new features that are going to come in
 Cassandra 2.0 ?

 Thank you
 Emalayan

Re: Cassandra 2.0 new features ?

[Emalayan Vairavanathan]

Great. Thank you Tomaz.



 From: Tomaz Muraus to...@apache.org
To: dev@cassandra.apache.org; Emalayan Vairavanathan svemala...@yahoo.com 
Sent: Wednesday, 12 June 2013 4:39 PM
Subject: Re: Cassandra 2.0 new features ?
 


Check CHANGES file (https://github.com/apache/cassandra/blob/trunk/CHANGES.txt) 
and / or JIRA (https://issues.apache.org/jira/browse/CASSANDRA).


On Wed, Jun 12, 2013 at 3:48 PM, Emalayan Vairavanathan svemala...@yahoo.com 
wrote:

Hi All,

Can anyone tell me about the new features that are going to come in Cassandra 
2.0 ?

Thank you
Emalayan

Re: Proposal: require Java7 for Cassandra 2.0

[aaron morton]

+1
-
Aaron Morton
Freelance Cassandra Developer
New Zealand

@aaronmorton
http://www.thelastpickle.com

On 7/02/2013, at 11:21 AM, Jonathan Ellis jbel...@gmail.com wrote:

 Java 6 EOL is this month.  Java 7 will be two years old when C* 2.0
 comes out (July).  Anecdotally, a bunch of people are running C* on
 Java7 with no issues, except for the Snappy-on-OS-X problem (which
 will be moot if LZ4 becomes our default, as looks likely).
 
 Upgrading to Java7 lets us take advantage of new (two year old)
 features as well as simplifying interoperability with other
 dependencies, e.g., Jetty's BlockingArrayQueue requires java7.
 
 Thoughts?
 
 -- 
 Jonathan Ellis
 Project Chair, Apache Cassandra
 co-founder, http://www.datastax.com
 @spyced

Re: Proposal: require Java7 for Cassandra 2.0

[Vijay]

+1

Regards,
/VJ


On Wed, Feb 6, 2013 at 2:21 PM, Jonathan Ellis jbel...@gmail.com wrote:

 Java 6 EOL is this month.  Java 7 will be two years old when C* 2.0
 comes out (July).  Anecdotally, a bunch of people are running C* on
 Java7 with no issues, except for the Snappy-on-OS-X problem (which
 will be moot if LZ4 becomes our default, as looks likely).

 Upgrading to Java7 lets us take advantage of new (two year old)
 features as well as simplifying interoperability with other
 dependencies, e.g., Jetty's BlockingArrayQueue requires java7.

 Thoughts?

 --
 Jonathan Ellis
 Project Chair, Apache Cassandra
 co-founder, http://www.datastax.com
 @spyced

Re: Proposal: require Java7 for Cassandra 2.0

[Anton Prakash]

unsubscribe


On Thu, Feb 7, 2013 at 12:58 PM, Ajith Kannan kann...@mail.montclair.eduwrote:

 +1

 On 2/6/2013 9:13 PM, Jake Luciani wrote:

 +1

 On Wed, Feb 6, 2013 at 5:32 PM, Gary Dusbabekgdusba...@gmail.com
  wrote:

  +1


 On Wed, Feb 6, 2013 at 4:21 PM, Jonathan Ellisjbel...@gmail.com
  wrote:

  Java 6 EOL is this month.  Java 7 will be two years old when C* 2.0
 comes out (July).  Anecdotally, a bunch of people are running C* on
 Java7 with no issues, except for the Snappy-on-OS-X problem (which
 will be moot if LZ4 becomes our default, as looks likely).

 Upgrading to Java7 lets us take advantage of new (two year old)
 features as well as simplifying interoperability with other
 dependencies, e.g., Jetty's BlockingArrayQueue requires java7.

 Thoughts?

 --
 Jonathan Ellis
 Project Chair, Apache Cassandra
 co-founder, http://www.datastax.com
 @spyced

Re: Proposal: require Java7 for Cassandra 2.0

[Eric Evans]

http://i.imgur.com/kQWV2.gif

On Thu, Feb 7, 2013 at 9:38 PM, Anton Prakash
anton.prak...@cloudtalk.com wrote:
 unsubscribe


 On Thu, Feb 7, 2013 at 12:58 PM, Ajith Kannan 
 kann...@mail.montclair.eduwrote:

 +1

 On 2/6/2013 9:13 PM, Jake Luciani wrote:

 +1

 On Wed, Feb 6, 2013 at 5:32 PM, Gary Dusbabekgdusba...@gmail.com
  wrote:

  +1


 On Wed, Feb 6, 2013 at 4:21 PM, Jonathan Ellisjbel...@gmail.com
  wrote:

  Java 6 EOL is this month.  Java 7 will be two years old when C* 2.0
 comes out (July).  Anecdotally, a bunch of people are running C* on
 Java7 with no issues, except for the Snappy-on-OS-X problem (which
 will be moot if LZ4 becomes our default, as looks likely).

 Upgrading to Java7 lets us take advantage of new (two year old)
 features as well as simplifying interoperability with other
 dependencies, e.g., Jetty's BlockingArrayQueue requires java7.

 Thoughts?

 --
 Jonathan Ellis
 Project Chair, Apache Cassandra
 co-founder, http://www.datastax.com
 @spyced







--
Eric Evans
Acunu | http://www.acunu.com | @acunu

Re: Proposal: require Java7 for Cassandra 2.0

[Edward Capriolo]

Counter proposal java 8 and closures. Jk
On Thursday, February 7, 2013, Carl Yeksigian c...@yeksigian.com wrote:
 +1


 On Wed, Feb 6, 2013 at 5:21 PM, Jonathan Ellis jbel...@gmail.com wrote:

 Java 6 EOL is this month.  Java 7 will be two years old when C* 2.0
 comes out (July).  Anecdotally, a bunch of people are running C* on
 Java7 with no issues, except for the Snappy-on-OS-X problem (which
 will be moot if LZ4 becomes our default, as looks likely).

 Upgrading to Java7 lets us take advantage of new (two year old)
 features as well as simplifying interoperability with other
 dependencies, e.g., Jetty's BlockingArrayQueue requires java7.

 Thoughts?

 --
 Jonathan Ellis
 Project Chair, Apache Cassandra
 co-founder, http://www.datastax.com
 @spyced

Re: Proposal: require Java7 for Cassandra 2.0

[Pavel Yaskevich]

+1 

-- 
Pavel Yaskevich


On Wednesday, February 6, 2013 at 2:21 PM, Jonathan Ellis wrote:

 Java 6 EOL is this month. Java 7 will be two years old when C* 2.0
 comes out (July). Anecdotally, a bunch of people are running C* on
 Java7 with no issues, except for the Snappy-on-OS-X problem (which
 will be moot if LZ4 becomes our default, as looks likely).
 
 Upgrading to Java7 lets us take advantage of new (two year old)
 features as well as simplifying interoperability with other
 dependencies, e.g., Jetty's BlockingArrayQueue requires java7.
 
 Thoughts?
 
 -- 
 Jonathan Ellis
 Project Chair, Apache Cassandra
 co-founder, http://www.datastax.com
 @spyced
 
 

Re: Proposal: require Java7 for Cassandra 2.0

[Aleksey Yeschenko]

+1

-- 
AY


On Thursday, February 7, 2013 at 01:24 AM, Pavel Yaskevich wrote:

 +1 
 
 -- 
 Pavel Yaskevich
 
 
 On Wednesday, February 6, 2013 at 2:21 PM, Jonathan Ellis wrote:
 
  Java 6 EOL is this month. Java 7 will be two years old when C* 2.0
  comes out (July). Anecdotally, a bunch of people are running C* on
  Java7 with no issues, except for the Snappy-on-OS-X problem (which
  will be moot if LZ4 becomes our default, as looks likely).
  
  Upgrading to Java7 lets us take advantage of new (two year old)
  features as well as simplifying interoperability with other
  dependencies, e.g., Jetty's BlockingArrayQueue requires java7.
  
  Thoughts?
  
  -- 
  Jonathan Ellis
  Project Chair, Apache Cassandra
  co-founder, http://www.datastax.com
  @spyced
  
 
 
 

Re: Proposal: require Java7 for Cassandra 2.0

[Michael Kjellman]

There seems to be a lot of confusion on this topic in the C* Community.
Most people look at the Datastax docs that recommend Java 6 and ask why
that recommendation is there. Threads never seem to finish with a clear
definitive answer. There seem to be about 3-4 of these for the past few
months on the Users list.

I love the idea of requiring Java 7 explicitly + clarifying the project's
stance on the JVM requirement overall.

Might also want to lock it down to something like Java 7 u8 (u13 might be
even nicer given 2.0's timeframe) given the early instability in Java7's
early releases (and I guess allow but maybe warn for OpenJDK7?).

My 2-cents.

Best,
Michael

On 2/6/13 2:21 PM, Jonathan Ellis jbel...@gmail.com wrote:

Java 6 EOL is this month.  Java 7 will be two years old when C* 2.0
comes out (July).  Anecdotally, a bunch of people are running C* on
Java7 with no issues, except for the Snappy-on-OS-X problem (which
will be moot if LZ4 becomes our default, as looks likely).

Upgrading to Java7 lets us take advantage of new (two year old)
features as well as simplifying interoperability with other
dependencies, e.g., Jetty's BlockingArrayQueue requires java7.

Thoughts?

-- 
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder, http://www.datastax.com
@spyced

Re: Proposal: require Java7 for Cassandra 2.0

[Brandon Williams]

On Wed, Feb 6, 2013 at 4:21 PM, Jonathan Ellis jbel...@gmail.com wrote:
 Java 6 EOL is this month.  Java 7 will be two years old when C* 2.0
 comes out (July).  Anecdotally, a bunch of people are running C* on
 Java7 with no issues, except for the Snappy-on-OS-X problem (which
 will be moot if LZ4 becomes our default, as looks likely).

 Upgrading to Java7 lets us take advantage of new (two year old)
 features as well as simplifying interoperability with other
 dependencies, e.g., Jetty's BlockingArrayQueue requires java7.

 Thoughts?

We already do quite a bit of jdk version/type detection in
cassandra-env.sh so we can catch it early if they try to use java6 and
give a meaningful error, so this sounds good to me.

-Brandon

Re: Proposal: require Java7 for Cassandra 2.0

[Jason Brown]

+1


On Wed, Feb 6, 2013 at 2:29 PM, Brandon Williams dri...@gmail.com wrote:

 On Wed, Feb 6, 2013 at 4:21 PM, Jonathan Ellis jbel...@gmail.com wrote:
  Java 6 EOL is this month.  Java 7 will be two years old when C* 2.0
  comes out (July).  Anecdotally, a bunch of people are running C* on
  Java7 with no issues, except for the Snappy-on-OS-X problem (which
  will be moot if LZ4 becomes our default, as looks likely).
 
  Upgrading to Java7 lets us take advantage of new (two year old)
  features as well as simplifying interoperability with other
  dependencies, e.g., Jetty's BlockingArrayQueue requires java7.
 
  Thoughts?

 We already do quite a bit of jdk version/type detection in
 cassandra-env.sh so we can catch it early if they try to use java6 and
 give a meaningful error, so this sounds good to me.

 -Brandon

Re: Proposal: require Java7 for Cassandra 2.0

[Gary Dusbabek]

+1


On Wed, Feb 6, 2013 at 4:21 PM, Jonathan Ellis jbel...@gmail.com wrote:

 Java 6 EOL is this month.  Java 7 will be two years old when C* 2.0
 comes out (July).  Anecdotally, a bunch of people are running C* on
 Java7 with no issues, except for the Snappy-on-OS-X problem (which
 will be moot if LZ4 becomes our default, as looks likely).

 Upgrading to Java7 lets us take advantage of new (two year old)
 features as well as simplifying interoperability with other
 dependencies, e.g., Jetty's BlockingArrayQueue requires java7.

 Thoughts?

 --
 Jonathan Ellis
 Project Chair, Apache Cassandra
 co-founder, http://www.datastax.com
 @spyced

Re: Proposal: require Java7 for Cassandra 2.0

[Jake Luciani]

+1

On Wed, Feb 6, 2013 at 5:32 PM, Gary Dusbabek gdusba...@gmail.com wrote:

 +1


 On Wed, Feb 6, 2013 at 4:21 PM, Jonathan Ellis jbel...@gmail.com wrote:

  Java 6 EOL is this month.  Java 7 will be two years old when C* 2.0
  comes out (July).  Anecdotally, a bunch of people are running C* on
  Java7 with no issues, except for the Snappy-on-OS-X problem (which
  will be moot if LZ4 becomes our default, as looks likely).
 
  Upgrading to Java7 lets us take advantage of new (two year old)
  features as well as simplifying interoperability with other
  dependencies, e.g., Jetty's BlockingArrayQueue requires java7.
 
  Thoughts?
 
  --
  Jonathan Ellis
  Project Chair, Apache Cassandra
  co-founder, http://www.datastax.com
  @spyced
 




-- 
http://twitter.com/tjake

Re: Proposal: require Java7 for Cassandra 2.0

[Sylvain Lebresne]

+1

--
Sylvain


On Wed, Feb 6, 2013 at 11:21 PM, Jonathan Ellis jbel...@gmail.com wrote:

 Java 6 EOL is this month.  Java 7 will be two years old when C* 2.0
 comes out (July).  Anecdotally, a bunch of people are running C* on
 Java7 with no issues, except for the Snappy-on-OS-X problem (which
 will be moot if LZ4 becomes our default, as looks likely).

 Upgrading to Java7 lets us take advantage of new (two year old)
 features as well as simplifying interoperability with other
 dependencies, e.g., Jetty's BlockingArrayQueue requires java7.

 Thoughts?

 --
 Jonathan Ellis
 Project Chair, Apache Cassandra
 co-founder, http://www.datastax.com
 @spyced

Re: 2.0

[Jason Brown]

- world

Hi Jonathan,

This topic may have been discussed elsewhere, or my memory is worse off
than I thought, but what is our long term vision for thrift support?
Admittedly, I need to learn much more about the binary CQL protocol, and I
understand Ed's concerns, as well (more acutely now) about existing
installations, but we probably wouldn't have dreamt up a new client
interface/protocol if we went planning, at some point, on retiring the old
one. And, also, I missed the Avro debate from the past, so I'm not sure how
much that affects current and future thinking.

After raising the issue here on the dev list, it certainly seems like 2.0
is premature for a full-on switch over, and Ed raised some interesting
metrics to consider when we could declare the CQL protocol as 'accepted'.
I'm curious as to how you are seeing it roll out.

Thanks for your time,

-Jason





On Fri, Nov 30, 2012 at 2:49 PM, Jonathan Ellis jbel...@gmail.com wrote:

 As attractive as it would be to clean house, I think we owe it to our
 users to keep Thrift around for the forseeable future rather than
 orphan all Thrift-using applications (which is virtually everyone) on
 1.2.

 On Sat, Dec 1, 2012 at 7:33 AM, Jason Brown jasedbr...@gmail.com wrote:
  Hi Jonathan,
 
  I'm in favor of paying off the technical debt, as well, and I wonder if
  there is value in removing support for thrift with 2.0? We're currently
 in
  'do as little as possible' mode with thrift, so should we aggressively
 cast
  it off and push the binary CQL protocol? Seems like a jump to '2.0',
 along
  with the other initiatives, would be a reasonable time/milestone to do
 so.
 
  Thanks,
 
  -Jason
 
 
  On Fri, Nov 30, 2012 at 12:12 PM, Jonathan Ellis jbel...@gmail.com
 wrote:
 
  The more I think about it, the more I think we should call 1.2-next,
  2.0.  I'd like to spend some time paying off our technical debt:
 
  - replace supercolumns with composites (CASSANDRA-3237)
  - rewrite counters (CASSANDRA-4775)
  - improve storage engine support for wide rows
  - better stage management to improve latency (disruptor? lightweight
  threads?  custom executor + queue?)
  - improved repair (CASSANDRA-3362, 2699)
 
  Of course, we're planning some new features as well:
  - triggers (CASSANDRA-1311)
  - improved query fault tolerance (CASSANDRA-4705)
  - row size limits (CASSANDRA-3929)
  - cql3 integration for hadoop (CASSANDRA-4421)
  - improved caching (CASSANDRA-1956, 2864)
 
  --
  Jonathan Ellis
  Project Chair, Apache Cassandra
  co-founder, http://www.datastax.com
  @spyced
 



 --
 Jonathan Ellis
 Project Chair, Apache Cassandra
 co-founder, http://www.datastax.com
 @spyced

Re: 2.0

[Jason Brown]

Oops, meant to address this specifically to Jonathan, but since I've
confused 'reply' with 'forward'. my apologies for any extra noise on this
topic.


On Mon, Dec 3, 2012 at 9:25 AM, Jason Brown jasedbr...@gmail.com wrote:

 - world

 Hi Jonathan,

 This topic may have been discussed elsewhere, or my memory is worse off
 than I thought, but what is our long term vision for thrift support?
 Admittedly, I need to learn much more about the binary CQL protocol, and I
 understand Ed's concerns, as well (more acutely now) about existing
 installations, but we probably wouldn't have dreamt up a new client
 interface/protocol if we went planning, at some point, on retiring the old
 one. And, also, I missed the Avro debate from the past, so I'm not sure how
 much that affects current and future thinking.

 After raising the issue here on the dev list, it certainly seems like 2.0
 is premature for a full-on switch over, and Ed raised some interesting
 metrics to consider when we could declare the CQL protocol as 'accepted'.
 I'm curious as to how you are seeing it roll out.

 Thanks for your time,

 -Jason





 On Fri, Nov 30, 2012 at 2:49 PM, Jonathan Ellis jbel...@gmail.com wrote:

 As attractive as it would be to clean house, I think we owe it to our
 users to keep Thrift around for the forseeable future rather than
 orphan all Thrift-using applications (which is virtually everyone) on
 1.2.

 On Sat, Dec 1, 2012 at 7:33 AM, Jason Brown jasedbr...@gmail.com wrote:
  Hi Jonathan,
 
  I'm in favor of paying off the technical debt, as well, and I wonder if
  there is value in removing support for thrift with 2.0? We're currently
 in
  'do as little as possible' mode with thrift, so should we aggressively
 cast
  it off and push the binary CQL protocol? Seems like a jump to '2.0',
 along
  with the other initiatives, would be a reasonable time/milestone to do
 so.
 
  Thanks,
 
  -Jason
 
 
  On Fri, Nov 30, 2012 at 12:12 PM, Jonathan Ellis jbel...@gmail.com
 wrote:
 
  The more I think about it, the more I think we should call 1.2-next,
  2.0.  I'd like to spend some time paying off our technical debt:
 
  - replace supercolumns with composites (CASSANDRA-3237)
  - rewrite counters (CASSANDRA-4775)
  - improve storage engine support for wide rows
  - better stage management to improve latency (disruptor? lightweight
  threads?  custom executor + queue?)
  - improved repair (CASSANDRA-3362, 2699)
 
  Of course, we're planning some new features as well:
  - triggers (CASSANDRA-1311)
  - improved query fault tolerance (CASSANDRA-4705)
  - row size limits (CASSANDRA-3929)
  - cql3 integration for hadoop (CASSANDRA-4421)
  - improved caching (CASSANDRA-1956, 2864)
 
  --
  Jonathan Ellis
  Project Chair, Apache Cassandra
  co-founder, http://www.datastax.com
  @spyced
 



 --
 Jonathan Ellis
 Project Chair, Apache Cassandra
 co-founder, http://www.datastax.com
 @spyced

Re: 2.0

[Drew Kutcharian]

I agree with Edward here. We use Thrift too and we haven't really found a good 
enough reason to move to CQL3.

-- Drew

On Dec 1, 2012, at 10:24 AM, Edward Capriolo edlinuxg...@gmail.com wrote:

 I do not understand why everyone wants to force this issue on removing
 thrift. If cql, cql sparse tables and the new transport are better people
 will naturally begin to use them, but as it stands now I see the it
 this way:
 
 Thrift still has more clients for more languages, thrift has more higher
 level clients for more languages.
 Thrift has Hadoop support hive support and pig support in the wild.
 Thrift has third party tools like Orm tools, support for tools like flume.
 
 Most of cql3 features like collections do not work with compact tables,
 and compact tables are much more space efficient then their cql3 sparse
 counterparts, composite rows with UTf column names, blank rows, etc.
 Cql3 binary client is only available for in beta stage for a few languages.
 
 So the project can easily remove thrift today but until a majority of the
 tooling by the community adopts the transport and for the most part cqls
 sparse tables it is not going to mean anything. Many people already have
 code live in production working fine with the old toolset and will be
 unwilling to convert something just because
 
 Think about it like this a company like mine that already has something in
 production. Even if you could convince me us that cql native transport was
 better, which by the way no one has showed me a vast performance reason to
 this point, they still may not want to invest the resources to convert
 their app. Many companies endured the painful transition from Cassandra 0.6
 to Cassandra 0.7 conversion and they are not eagerly going to entertain
 another change which is mostly cosmetic.
 
 Also I find issues like this extremely frustrating.
 https://issues.apache.org/jira/browse/CASSANDRA-4924
 
 It seems like the project is drawing a hard line in the sand dividing
 people. Is it the case that cql3's sparse tables can't be accessed
 by thrift, or is it the case that no one wants to make this happen? Like is
 it technically impossible? It seems not to me in Cassandra
 Row key, column, and value are all still byte arrays right? So I do not see
 why thrift users need to be locked out of them. Just like composites we
 will figure out how to pack the bytes.
 
 I hope that we can stop talking about removing thrift until there is some
 consensus between active users that it is not in use anymore.
 This consensus is not as simple as n committers saying that something is
 technically not needed anymore. It has to look at the users, the number of
 clients, the number of languages, the number of high level tools available.
 In the mean time when issues like 4924 pop up it would be better if people
 tried to find solutions for maximum forward and backward compatibility
 instead of drawing a line and trying to shut thrift users out of things.
 
 Avro was much the same way . I had a spirited debate on irc and got
 basicallly insulted because i belived thrift was not dead. The glory of
 avro never came true because it really did not work for clients outside a
 few languages. Cql and the binary transport has to pass this same litmus
 test. Let it gain momentum and have rock solid clients for 5 languages and
 have higher level tools written on top of it then its easy to say thrift is
 not needed anymore.
 
 
 On Saturday, December 1, 2012, Sylvain Lebresne wrote:
 
 I agree on 2.0.
 
 For the thrift part, we've said clearly that we wouldn't remove it any time
 soon so let's stick to that. Besides, I would agree it's too soon anyway.
 What we can do however in the relatively short term on that front, is to
 pull thrift in it's own jar (we've almost removed all internal dependencies
 on thrift, and the few remaining ones will be easy to kill) and make that
 jar optional if you don't want to use it.
 
 --
 Sylvain
 
 
 On Sat, Dec 1, 2012 at 2:52 AM, Ray Slakinski 
 ray.slakin...@gmail.comjavascript:;
 wrote:
 
 I agree, I don't think its a great idea to drop thrift until the back
 end tools are 100% compatible and have some level of agreement from the
 major users of
 Cassandra.
 
 Paying off technical dept though I'm all for, and I think its key to the
 long term success of the application. Right now Supercolumns to someone
 new coming to the system might think Hey, these things look great. Lets
 use them and in a few months time hate all things that are cassandra.
 
 Ray Slakinski
 
 On 12/01, Jonathan Ellis wrote:
 As attractive as it would be to clean house, I think we owe it to our
 users to keep Thrift around for the forseeable future rather than
 orphan all Thrift-using applications (which is virtually everyone) on
 1.2.
 
 On Sat, Dec 1, 2012 at 7:33 AM, Jason Brown 
 jasedbr...@gmail.comjavascript:;
 
 wrote:
 Hi Jonathan,
 
 I'm in favor of paying off the technical debt, as well, and I wonder
 if
 there is value

Re: 2.0

[Brandon Williams]

On Sat, Dec 1, 2012 at 12:24 PM, Edward Capriolo edlinuxg...@gmail.com wrote:
 I do not understand why everyone wants to force this issue on removing
 thrift.

I'm -1 on removing thrift, and by my count, that would put us at -3
binding if it ever came to vote, so let's consider this proposition
closed and move on to other more constructive points.

-Brandon

Re: 2.0

[Sylvain Lebresne]

I agree on 2.0.

For the thrift part, we've said clearly that we wouldn't remove it any time
soon so let's stick to that. Besides, I would agree it's too soon anyway.
What we can do however in the relatively short term on that front, is to
pull thrift in it's own jar (we've almost removed all internal dependencies
on thrift, and the few remaining ones will be easy to kill) and make that
jar optional if you don't want to use it.

--
Sylvain


On Sat, Dec 1, 2012 at 2:52 AM, Ray Slakinski ray.slakin...@gmail.comwrote:

 I agree, I don't think its a great idea to drop thrift until the back
 end tools are 100% compatible and have some level of agreement from the
 major users of
 Cassandra.

 Paying off technical dept though I'm all for, and I think its key to the
 long term success of the application. Right now Supercolumns to someone
 new coming to the system might think Hey, these things look great. Lets
 use them and in a few months time hate all things that are cassandra.

 Ray Slakinski

 On 12/01, Jonathan Ellis wrote:
  As attractive as it would be to clean house, I think we owe it to our
  users to keep Thrift around for the forseeable future rather than
  orphan all Thrift-using applications (which is virtually everyone) on
  1.2.
 
  On Sat, Dec 1, 2012 at 7:33 AM, Jason Brown jasedbr...@gmail.com
 wrote:
   Hi Jonathan,
  
   I'm in favor of paying off the technical debt, as well, and I wonder if
   there is value in removing support for thrift with 2.0? We're
 currently in
   'do as little as possible' mode with thrift, so should we aggressively
 cast
   it off and push the binary CQL protocol? Seems like a jump to '2.0',
 along
   with the other initiatives, would be a reasonable time/milestone to do
 so.
  
   Thanks,
  
   -Jason
  
  
   On Fri, Nov 30, 2012 at 12:12 PM, Jonathan Ellis jbel...@gmail.com
 wrote:
  
   The more I think about it, the more I think we should call 1.2-next,
   2.0.  I'd like to spend some time paying off our technical debt:
  
   - replace supercolumns with composites (CASSANDRA-3237)
   - rewrite counters (CASSANDRA-4775)
   - improve storage engine support for wide rows
   - better stage management to improve latency (disruptor? lightweight
   threads?  custom executor + queue?)
   - improved repair (CASSANDRA-3362, 2699)
  
   Of course, we're planning some new features as well:
   - triggers (CASSANDRA-1311)
   - improved query fault tolerance (CASSANDRA-4705)
   - row size limits (CASSANDRA-3929)
   - cql3 integration for hadoop (CASSANDRA-4421)
   - improved caching (CASSANDRA-1956, 2864)
  
   --
   Jonathan Ellis
   Project Chair, Apache Cassandra
   co-founder, http://www.datastax.com
   @spyced
  
 
 
 
  --
  Jonathan Ellis
  Project Chair, Apache Cassandra
  co-founder, http://www.datastax.com
  @spyced

Re: 2.0

[Jason Brown]

Hi Jonathan,

I'm in favor of paying off the technical debt, as well, and I wonder if
there is value in removing support for thrift with 2.0? We're currently in
'do as little as possible' mode with thrift, so should we aggressively cast
it off and push the binary CQL protocol? Seems like a jump to '2.0', along
with the other initiatives, would be a reasonable time/milestone to do so.

Thanks,

-Jason


On Fri, Nov 30, 2012 at 12:12 PM, Jonathan Ellis jbel...@gmail.com wrote:

 The more I think about it, the more I think we should call 1.2-next,
 2.0.  I'd like to spend some time paying off our technical debt:

 - replace supercolumns with composites (CASSANDRA-3237)
 - rewrite counters (CASSANDRA-4775)
 - improve storage engine support for wide rows
 - better stage management to improve latency (disruptor? lightweight
 threads?  custom executor + queue?)
 - improved repair (CASSANDRA-3362, 2699)

 Of course, we're planning some new features as well:
 - triggers (CASSANDRA-1311)
 - improved query fault tolerance (CASSANDRA-4705)
 - row size limits (CASSANDRA-3929)
 - cql3 integration for hadoop (CASSANDRA-4421)
 - improved caching (CASSANDRA-1956, 2864)

 --
 Jonathan Ellis
 Project Chair, Apache Cassandra
 co-founder, http://www.datastax.com
 @spyced

Re: 2.0

[Edward Capriolo]

Good idea. Lets remove thrift, CQL3 is still beta, but I am willing to
upgrade to a version that removes thrift. Then when all our clients can not
connect they will be forced to get with the program.

On Fri, Nov 30, 2012 at 5:33 PM, Jason Brown jasedbr...@gmail.com wrote:

 Hi Jonathan,

 I'm in favor of paying off the technical debt, as well, and I wonder if
 there is value in removing support for thrift with 2.0? We're currently in
 'do as little as possible' mode with thrift, so should we aggressively cast
 it off and push the binary CQL protocol? Seems like a jump to '2.0', along
 with the other initiatives, would be a reasonable time/milestone to do so.

 Thanks,

 -Jason


 On Fri, Nov 30, 2012 at 12:12 PM, Jonathan Ellis jbel...@gmail.com
 wrote:

  The more I think about it, the more I think we should call 1.2-next,
  2.0.  I'd like to spend some time paying off our technical debt:
 
  - replace supercolumns with composites (CASSANDRA-3237)
  - rewrite counters (CASSANDRA-4775)
  - improve storage engine support for wide rows
  - better stage management to improve latency (disruptor? lightweight
  threads?  custom executor + queue?)
  - improved repair (CASSANDRA-3362, 2699)
 
  Of course, we're planning some new features as well:
  - triggers (CASSANDRA-1311)
  - improved query fault tolerance (CASSANDRA-4705)
  - row size limits (CASSANDRA-3929)
  - cql3 integration for hadoop (CASSANDRA-4421)
  - improved caching (CASSANDRA-1956, 2864)
 
  --
  Jonathan Ellis
  Project Chair, Apache Cassandra
  co-founder, http://www.datastax.com
  @spyced
 

Re: 2.0

[Jonathan Ellis]

As attractive as it would be to clean house, I think we owe it to our
users to keep Thrift around for the forseeable future rather than
orphan all Thrift-using applications (which is virtually everyone) on
1.2.

On Sat, Dec 1, 2012 at 7:33 AM, Jason Brown jasedbr...@gmail.com wrote:
 Hi Jonathan,

 I'm in favor of paying off the technical debt, as well, and I wonder if
 there is value in removing support for thrift with 2.0? We're currently in
 'do as little as possible' mode with thrift, so should we aggressively cast
 it off and push the binary CQL protocol? Seems like a jump to '2.0', along
 with the other initiatives, would be a reasonable time/milestone to do so.

 Thanks,

 -Jason


 On Fri, Nov 30, 2012 at 12:12 PM, Jonathan Ellis jbel...@gmail.com wrote:

 The more I think about it, the more I think we should call 1.2-next,
 2.0.  I'd like to spend some time paying off our technical debt:

 - replace supercolumns with composites (CASSANDRA-3237)
 - rewrite counters (CASSANDRA-4775)
 - improve storage engine support for wide rows
 - better stage management to improve latency (disruptor? lightweight
 threads?  custom executor + queue?)
 - improved repair (CASSANDRA-3362, 2699)

 Of course, we're planning some new features as well:
 - triggers (CASSANDRA-1311)
 - improved query fault tolerance (CASSANDRA-4705)
 - row size limits (CASSANDRA-3929)
 - cql3 integration for hadoop (CASSANDRA-4421)
 - improved caching (CASSANDRA-1956, 2864)

 --
 Jonathan Ellis
 Project Chair, Apache Cassandra
 co-founder, http://www.datastax.com
 @spyced




-- 
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder, http://www.datastax.com
@spyced

Re: 2.0

[Jason Brown]

My hope is that after 1.2 (i.e. by the time we're 2.0'ing), the binary CQL
protocol is out of beta :).

On Fri, Nov 30, 2012 at 2:39 PM, Edward Capriolo edlinuxg...@gmail.comwrote:

 Good idea. Lets remove thrift, CQL3 is still beta, but I am willing to
 upgrade to a version that removes thrift. Then when all our clients can not
 connect they will be forced to get with the program

Re: 2.0

[Vijay]

+1

Regards,
/VJ



On Fri, Nov 30, 2012 at 12:12 PM, Jonathan Ellis jbel...@gmail.com wrote:

 The more I think about it, the more I think we should call 1.2-next,
 2.0.  I'd like to spend some time paying off our technical debt:

 - replace supercolumns with composites (CASSANDRA-3237)
 - rewrite counters (CASSANDRA-4775)
 - improve storage engine support for wide rows
 - better stage management to improve latency (disruptor? lightweight
 threads?  custom executor + queue?)
 - improved repair (CASSANDRA-3362, 2699)

 Of course, we're planning some new features as well:
 - triggers (CASSANDRA-1311)
 - improved query fault tolerance (CASSANDRA-4705)
 - row size limits (CASSANDRA-3929)
 - cql3 integration for hadoop (CASSANDRA-4421)
 - improved caching (CASSANDRA-1956, 2864)

 --
 Jonathan Ellis
 Project Chair, Apache Cassandra
 co-founder, http://www.datastax.com
 @spyced

Re: 2.0

[Bill de hÓra]

I'm not thrilled with Thrift, but I'd like to see and hear more real 
world use of CQL first (Avro all the things is not that long ago).


That said, a major rev /could/ do this - not start the thrift server by 
default.  It's then a hoop jump to enable it via nodetool/yaml, and 
signals to the client communities and user base, where the project 
intends to go.


Bill


On 30/11/12 22:49, Jonathan Ellis wrote:

As attractive as it would be to clean house, I think we owe it to our
users to keep Thrift around for the forseeable future rather than
orphan all Thrift-using applications (which is virtually everyone) on
1.2.

On Sat, Dec 1, 2012 at 7:33 AM, Jason Brown jasedbr...@gmail.com wrote:

Hi Jonathan,

I'm in favor of paying off the technical debt, as well, and I wonder if
there is value in removing support for thrift with 2.0? We're currently in
'do as little as possible' mode with thrift, so should we aggressively cast
it off and push the binary CQL protocol? Seems like a jump to '2.0', along
with the other initiatives, would be a reasonable time/milestone to do so.

Thanks,

-Jason


On Fri, Nov 30, 2012 at 12:12 PM, Jonathan Ellis jbel...@gmail.com wrote:


The more I think about it, the more I think we should call 1.2-next,
2.0.  I'd like to spend some time paying off our technical debt:

- replace supercolumns with composites (CASSANDRA-3237)
- rewrite counters (CASSANDRA-4775)
- improve storage engine support for wide rows
- better stage management to improve latency (disruptor? lightweight
threads?  custom executor + queue?)
- improved repair (CASSANDRA-3362, 2699)

Of course, we're planning some new features as well:
- triggers (CASSANDRA-1311)
- improved query fault tolerance (CASSANDRA-4705)
- row size limits (CASSANDRA-3929)
- cql3 integration for hadoop (CASSANDRA-4421)
- improved caching (CASSANDRA-1956, 2864)

--
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder, http://www.datastax.com
@spyced

Re: 2.0

[Ray Slakinski]

I agree, I don't think its a great idea to drop thrift until the back
end tools are 100% compatible and have some level of agreement from the major 
users of
Cassandra. 

Paying off technical dept though I'm all for, and I think its key to the
long term success of the application. Right now Supercolumns to someone
new coming to the system might think Hey, these things look great. Lets
use them and in a few months time hate all things that are cassandra.

Ray Slakinski

On 12/01, Jonathan Ellis wrote:
 As attractive as it would be to clean house, I think we owe it to our
 users to keep Thrift around for the forseeable future rather than
 orphan all Thrift-using applications (which is virtually everyone) on
 1.2.
 
 On Sat, Dec 1, 2012 at 7:33 AM, Jason Brown jasedbr...@gmail.com wrote:
  Hi Jonathan,
 
  I'm in favor of paying off the technical debt, as well, and I wonder if
  there is value in removing support for thrift with 2.0? We're currently in
  'do as little as possible' mode with thrift, so should we aggressively cast
  it off and push the binary CQL protocol? Seems like a jump to '2.0', along
  with the other initiatives, would be a reasonable time/milestone to do so.
 
  Thanks,
 
  -Jason
 
 
  On Fri, Nov 30, 2012 at 12:12 PM, Jonathan Ellis jbel...@gmail.com wrote:
 
  The more I think about it, the more I think we should call 1.2-next,
  2.0.  I'd like to spend some time paying off our technical debt:
 
  - replace supercolumns with composites (CASSANDRA-3237)
  - rewrite counters (CASSANDRA-4775)
  - improve storage engine support for wide rows
  - better stage management to improve latency (disruptor? lightweight
  threads?  custom executor + queue?)
  - improved repair (CASSANDRA-3362, 2699)
 
  Of course, we're planning some new features as well:
  - triggers (CASSANDRA-1311)
  - improved query fault tolerance (CASSANDRA-4705)
  - row size limits (CASSANDRA-3929)
  - cql3 integration for hadoop (CASSANDRA-4421)
  - improved caching (CASSANDRA-1956, 2864)
 
  --
  Jonathan Ellis
  Project Chair, Apache Cassandra
  co-founder, http://www.datastax.com
  @spyced
 
 
 
 
 -- 
 Jonathan Ellis
 Project Chair, Apache Cassandra
 co-founder, http://www.datastax.com
 @spyced