Re: apache-cassandra 2.2.8 rpm

[Carlos Rolo]

I would recommend migrate to a higher version of Apache Cassandra. Since
Datastax always push some extra patches in their distribution. So I would
go 2.2.8 -> 2.2.9+ at least. Since it's a minor upgrade I would read this
https://github.com/apache/cassandra/blob/cassandra-2.2/NEWS.txt and upgrade
to the 2.2.12.

[image: Pythian]
*Carlos Rolo* | Open Source Consultant | [image: LinkedIn]
<https://www.linkedin.com/in/carlosjuzarterolo/>
*m* +351 918 918 100
r...@pythian.com   *www.pythian.com*
<https://www.google.com/url?q=https%3A%2F%2Fwww.pythian.com=D=1=AFQjCNHhR4YJfBb19QxglicHut6aTAjXyQ>
[image: Pythian]
<https://www.google.com/url?q=https%3A%2F%2Fwww.pythian.com%2Femail-footer-click=D=1=AFQjCNF7Ld7zJGpBUtvj3Lum--bqwUvvog>

On Tue, Jun 5, 2018 at 3:49 PM, ukevg...@gmail.com 
wrote:

>
>
> On 2018/06/05 14:28:20, Nicolas Guyomar 
> wrote:
> > Hi,
> >
> > I believe this rpm was built by Datastax right ?
> > https://rpm.datastax.com/community/noarch/ is what you seem to be
> looking
> > for
> > Otherwise newest rpm are here :
> > https://www.apache.org/dist/cassandra/redhat/22x/
> >
> > On 5 June 2018 at 16:21, ukevg...@gmail.com  wrote:
> >
> > > Hi everybody,
> > >
> > > I am not able to find an RPM package for apache-cassandra 2.2.8
> > >
> > > Is there anyone who can share a link I really couldn't find it.
> > >
> > > Thank you
> > >
> > > Ev
> > >
> > > -
> > > To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
> > > For additional commands, e-mail: user-h...@cassandra.apache.org
> > >
> > >
> I am trying to migrate from Datastax to apache cassandra.
> I already have datastax 2.2.8 installed just trying to migrate to Apache
> cassandra 2.2.8
>
> -
> To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: user-h...@cassandra.apache.org
>
>

-- 


--

Re: cassandra repair takes ages

[Carlos Rolo]

Hello,

I just stated that if you use QUORUM or in fact using ALL, since you're
running ONE, this is a non-issue.

Regarding incremental repairs you can read here:
http://thelastpickle.com/blog/2017/12/14/should-you-use-incremental-repair.html

You can't run repair -pr simultaneously. You can try to use a tool like
Reaper to better manage and schedule repairs, but I doubt it will speed up
a lot.

Regards,

Carlos Juzarte Rolo
Cassandra Consultant / Datastax Certified Architect / Cassandra MVP

Pythian - Love your data

rolo@pythian | Twitter: @cjrolo | Skype: cjr2k3 | Linkedin:
*linkedin.com/in/carlosjuzarterolo
<http://linkedin.com/in/carlosjuzarterolo>*
Mobile: +351 918 918 100
www.pythian.com

On Sun, Apr 22, 2018 at 11:39 AM, Nuno Cervaens - Hoist Group - Portugal <
nuno.cerva...@hoistgroup.com> wrote:

> Hi Carlos,
>
>
> Thanks for the reply.
>
> Isnt the consistency level defined per session? All my session, being for
> read or write as defaulted to ONE.
>
>
> Movid to SSD is for sure an obvious improvement but not possible at the
> moment.
>
> My goal is to really spend the lowest time possible on running a repair
> throughout all the nodes.
>
> Are there any more downsides to run nodetool repair -pr simultaneously on
> each node, besides the cpu and mem overload?
>
> Also if someone can clarify about the safety of an incremental repair.
>
>
> thanks,
>
> nuno
> --
> *From:* Carlos Rolo <r...@pythian.com>
> *Sent:* Friday, April 20, 2018 4:55:21 PM
> *To:* user@cassandra.apache.org
> *Subject:* Re: cassandra repair takes ages
>
> Changing the datadrives to SSD would help to speed up the repairs.
>
> Also don't run 3 node, RF2. That makes Quorum = All.
>
> Regards,
>
> Carlos Juzarte Rolo
> Cassandra Consultant / Datastax Certified Architect / Cassandra MVP
>
> Pythian - Love your data
>
> rolo@pythian | Twitter: @cjrolo | Skype: cjr2k3 | Linkedin:
> *linkedin.com/in/carlosjuzarterolo
> <http://linkedin.com/in/carlosjuzarterolo>*
> Mobile: +351 918 918 100
> www.pythian.com
>
> On Fri, Apr 20, 2018 at 4:42 PM, Nuno Cervaens - Hoist Group - Portugal <
> nuno.cerva...@hoistgroup.com> wrote:
>
> Hello,
>
> I have a 3 node cluster with RF 2 and using STCS. I use SSDs for
> commitlogs and HDDs for data. Apache Cassandra version is 3.11.2.
> I basically have a huge keyspace ('newts' from opennms) and a big keyspace
> ('opspanel'). Here's a summary of the 'du' output for one node (which is
> more or less the same for each node):
>
> 51G ./data/opspanel
> 776G ./data/newts/samples-00ae9420ea0711e5a39bbd7839a19930
> 776G ./data/newts
>
> My issue is that running a 'nodetool repair -pr' takes one day an a half
> per node and as I want to store daily snapshots (for the past 7 days), I
> dont see how I can do this as repairs take too long.
> For example I see huge compactions and validations that take lots of hours
> (compactionstats taken at different times):
>
> id   compaction type keyspace
> table   completedtotalunit  progress
> 7125eb20-446b-11e8-a57d-f36e88375e31 Compaction  newtssamples
> 294177987449 835153786347 bytes 35,22%
>
> id   compaction type keyspace
> table   completedtotalunit  progress
> 6aa5ce51-4425-11e8-a7c1-572dede7e4d6 Anticompaction after repair
> newtssamples 581839334815 599408876344 bytes 97,07%
>
> id   compaction type keyspace
> table   completedtotalunit  progress
> 69976700-43e2-11e8-a7c1-572dede7e4d6 Validation  newtssamples
> 63249761990  826302170493 bytes 7,65%
> 69973ff0-43e2-11e8-a7c1-572dede7e4d6 Validation  newtssamples
> 102513762816 826302170600 bytes 12,41%
>
> Is there something I can do to improve the situation?
>
> Also, is an incremental repair (apparently nodetool's default) safe? As I
> see in the datastax documentation that the incremental should not be used,
> only the full. Can you please clarify?
>
> Thanks for the feedback.
> Nuno
>
>
>
> --
>
>
>
>

-- 


--

Re: cassandra repair takes ages

[Carlos Rolo]

Changing the datadrives to SSD would help to speed up the repairs.

Also don't run 3 node, RF2. That makes Quorum = All.

Regards,

Carlos Juzarte Rolo
Cassandra Consultant / Datastax Certified Architect / Cassandra MVP

Pythian - Love your data

rolo@pythian | Twitter: @cjrolo | Skype: cjr2k3 | Linkedin:
*linkedin.com/in/carlosjuzarterolo
*
Mobile: +351 918 918 100
www.pythian.com

On Fri, Apr 20, 2018 at 4:42 PM, Nuno Cervaens - Hoist Group - Portugal <
nuno.cerva...@hoistgroup.com> wrote:

> Hello,
>
> I have a 3 node cluster with RF 2 and using STCS. I use SSDs for
> commitlogs and HDDs for data. Apache Cassandra version is 3.11.2.
> I basically have a huge keyspace ('newts' from opennms) and a big keyspace
> ('opspanel'). Here's a summary of the 'du' output for one node (which is
> more or less the same for each node):
>
> 51G ./data/opspanel
> 776G ./data/newts/samples-00ae9420ea0711e5a39bbd7839a19930
> 776G ./data/newts
>
> My issue is that running a 'nodetool repair -pr' takes one day an a half
> per node and as I want to store daily snapshots (for the past 7 days), I
> dont see how I can do this as repairs take too long.
> For example I see huge compactions and validations that take lots of hours
> (compactionstats taken at different times):
>
> id   compaction type keyspace
> table   completedtotalunit  progress
> 7125eb20-446b-11e8-a57d-f36e88375e31 Compaction  newtssamples
> 294177987449 835153786347 bytes 35,22%
>
> id   compaction type keyspace
> table   completedtotalunit  progress
> 6aa5ce51-4425-11e8-a7c1-572dede7e4d6 Anticompaction after repair
> newtssamples 581839334815 599408876344 bytes 97,07%
>
> id   compaction type keyspace
> table   completedtotalunit  progress
> 69976700-43e2-11e8-a7c1-572dede7e4d6 Validation  newtssamples
> 63249761990  826302170493 bytes 7,65%
> 69973ff0-43e2-11e8-a7c1-572dede7e4d6 Validation  newtssamples
> 102513762816 826302170600 bytes 12,41%
>
> Is there something I can do to improve the situation?
>
> Also, is an incremental repair (apparently nodetool's default) safe? As I
> see in the datastax documentation that the incremental should not be used,
> only the full. Can you please clarify?
>
> Thanks for the feedback.
> Nuno
>

-- 


--

Re: Latest version and Features

[Carlos Rolo]

Thanks for the heads-up. I will update the post.

Regards,

Carlos Juzarte Rolo
Cassandra Consultant / Datastax Certified Architect / Cassandra MVP

Pythian - Love your data

rolo@pythian | Twitter: @cjrolo | Skype: cjr2k3 | Linkedin:
*linkedin.com/in/carlosjuzarterolo
<http://linkedin.com/in/carlosjuzarterolo>*
Mobile: +351 918 918 100
www.pythian.com

On Thu, Apr 12, 2018 at 5:02 AM, Michael Shuler <mich...@pbandjelly.org>
wrote:

> On 04/11/2018 06:12 PM, Carlos Rolo wrote:
> >
> > I blogged about this decision recently
> > here: https://blog.pythian.com/what-cassandra-version-should-i-use-2018/
>
> s/it the fact/is the fact/ typo, and possibly not 100% correct on the
> statement in that sentence.
>
> There are commits since the last 2.1 & 2.2 releases. Generally, we'll do
> a last release with any fixes on the branch, before shuttering
> development on older branches. 1.0 and 1.1 had a few commits after the
> last releases, but 1.2 and 2.0 both had final releases with any bug
> fixes we had in-tree. I expect we'll do the same with 2.1 and 2.2 to
> wrap things up nicely.
>
> --
> Warm regards,
> Michael
>
> -
> To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: user-h...@cassandra.apache.org
>
>

-- 


--

Re: Latest version and Features

[Carlos Rolo]

If you are on 3.1.0 I would move forward to 3.11.2.

I blogged about this decision recently here:
https://blog.pythian.com/what-cassandra-version-should-i-use-2018/

Regards,

Carlos Juzarte Rolo
Cassandra Consultant / Datastax Certified Architect / Cassandra MVP

Pythian - Love your data

rolo@pythian | Twitter: @cjrolo | Skype: cjr2k3 | Linkedin:
*linkedin.com/in/carlosjuzarterolo
*
Mobile: +351 918 918 100
www.pythian.com

On Wed, Apr 11, 2018 at 4:27 PM, Nicolas Guyomar 
wrote:

> Everything is in the same document, you have a "New features" section plus
> an "Upgrading" one
>
> On 11 April 2018 at 17:24, Abdul Patel  wrote:
>
>> Nicolas,
>> I do see all new features but instructions for upgrade are mentioned in
>> next section ..not sure if i missed it ..can you share that section?
>>
>>
>> On Wednesday, April 11, 2018, Abdul Patel  wrote:
>>
>>> Thanks .this is perfect
>>>
>>> On Wednesday, April 11, 2018, Nicolas Guyomar 
>>> wrote:
>>>
 Sorry, I should have give you this link instead :
 https://github.com/apache/cassandra/blob/trunk/NEWS.txt

 You'll find everything you need IMHO

 On 11 April 2018 at 17:05, Abdul Patel  wrote:

> Thanks.
>
> Is the upgrade process staright forward do we have any documentation
> to upgrade?
>
>
> On Wednesday, April 11, 2018, Jonathan Haddad 
> wrote:
>
>> Move to the latest 3.0, or if you're feeling a little more
>> adventurous, 3.11.2.
>>
>> 4.0 discussion is happening now, nothing is decided.
>>
>> On Wed, Apr 11, 2018 at 7:35 AM Abdul Patel 
>> wrote:
>>
>>> Hi All,
>>>
>>> Our company is planning for upgrading cassandra to maitain the audit
>>> gudilines for patch cycle.
>>> We are currently on 3.1.0, whats the latest stable version and what
>>> are the new features?
>>> Will it be better to wait for 4.0? Any news on what will be new
>>> features in 4.0 ?
>>>
>>

>

-- 


--

Re: Is this SSTable restore merging scenario possible ?

[Carlos Rolo]

As said before, as long as you rename the UUIDs to match you should be good.

The Production "win out" depends on the timestamps. In Cassandra last write
wins, so as long as, for the same row, the production timestamps are more
recent than the secondary cluster, the production data would "win over".

Regards,

Carlos Juzarte Rolo
Cassandra Consultant / Datastax Certified Architect / Cassandra MVP

Pythian - Love your data

rolo@pythian | Twitter: @cjrolo | Skype: cjr2k3 | Linkedin:
*linkedin.com/in/carlosjuzarterolo
*
Mobile: +351 918 918 100
www.pythian.com

On Wed, Mar 21, 2018 at 2:04 PM, Rahul Singh 
wrote:

> If its not on the same “cluster” and you are not using something like
> OpsCenter, the snapshotted files will have a diferent schema UUID for each
> entity. If you rename the files to have the matching UUID in the file
> names, then you should be able to do what you are talking about.
>
> On Mar 21, 2018, 4:50 AM -0500, Andrew Voumard ,
> wrote:
>
> Hi All,
>
> I am using Cassandra 3.10
>
> I would like to know if the following SSTable row level merging scenario
> is possible:
>
> 1. On a Production Cluster
> - Take a full snapshot on every node
>
> 2. On a new, empty Secondary Cluster with the same topology
> - Create a matching schema (keyspaces + tables as the production cluster).
> Note: The schema is very simple: No secondary or SASI indices, materialized
> views, etc.
> - Restore the full snapshots from production on each corresponding
> secondary node
> - INSERT rows into tables on the secondary node, but with keys that are
> guaranteed to be different to any INSERTs that were restored from the
> production cluster
> - UPDATE some of the existing rows in the secondary cluster (these are
> rows originally from the production cluster)
>
> 3. On the Production Cluster
> - Apply INSERTs, UPDATEs and DELETEs to tables
> - Take an incremental snapshot on every node
>
> 4. On the Secondary Cluster
> - Attempt to restore the incremental snapshots from the Production Cluster
> on each corresponding secondary cluster node, using nodetool refresh
>
> Question: Will Step 4 be successful ?, reiterating that:
> - The INSERTs will not have conflicting keys (as noted above)
> - The Production Cluster may have made UPDATES to rows that the Secondary
> Cluster has also made UPDATEs to
> - The Production Cluster may have DELETEd rows which the Secondary Cluster
> has UPDATED in the meantime
>
> If it works, I would expect the changes from Production to "win out" over
> any made independently on the Secondary Cluster.
>
> Many Thanks for any help you can provide
>
>
>

-- 


--

Re: Is there any limit in the number of partitions that a table can have

[Carlos Rolo]

Great explanation, thanks Jeff!

On 7 Mar 2018 17:49, "Javier Pareja" <pareja.jav...@gmail.com> wrote:

> Thank you for your time Jeff, very helpful.I  couldn't find anything out
> there about the subject and I suspected that this could be the case.
>
> Regarding the clustering key in this case:
> Back in the RDBMS world, you will always assign a sequential (or as
> sequential as possible) clustering key to a table to minimize fragmentation
> and increase the speed of the insertions. In the Cassandra world, does the
> same apply to the clustering key? For example, is it a good idea to assign
> a UUID to a clustering key, or would a timestamp be a better choice? I am
> thinking that partitions need to keep some sort of binary index for the
> clustering keys and for relatively large partitions it can be relatively
> expensive to maintain.
>
> F Javier Pareja
>
> On Wed, Mar 7, 2018 at 5:20 PM, Jeff Jirsa <jji...@gmail.com> wrote:
>
>>
>>
>> On Wed, Mar 7, 2018 at 7:13 AM, Carlos Rolo <r...@pythian.com> wrote:
>>
>>> Hi Jeff,
>>>
>>> Could you expand: "Tables without clustering keys are often deceptively
>>> expensive to compact, as a lot of work (relative to the other cell
>>> boundaries) happens on partition boundaries." This is something I didn't
>>> know and highly interesting to know more about!
>>>
>>>
>>>
>> We do a lot "by partition". We build column indexes by partition. We
>> update the partition index on each partition. We invalidate key cache by
>> partition. They're not super expensive, but they take time, and tables with
>> tiny partitions can actually be slower to compact.
>>
>> There's no magic cutoff where it does/doesn't make sense, my comment is
>> mostly a warning that the edges of the "normal" use cases tend to be less
>> optimized than the common case. Having a table with a hundred billion
>> records, where the key is numeric and the value is a single byte (let's say
>> you're keeping track of whether or not a specific sensor has ever detected
>> some magic event, and you have 100B sensors, that table will be close to
>> the worst-case example of this behavior).
>>
>
>

-- 


--

Re: Is there any limit in the number of partitions that a table can have

[Carlos Rolo]

Hi Jeff,

Could you expand: "Tables without clustering keys are often deceptively
expensive to compact, as a lot of work (relative to the other cell
boundaries) happens on partition boundaries." This is something I didn't
know and highly interesting to know more about!

--
Carlos Rolo

On Wed, Mar 7, 2018 at 2:36 PM, Jeff Jirsa <jji...@gmail.com> wrote:

> There is no limit
>
> The token range of murmur3 is 2^64, but Cassandra properly handles token
> overlaps (we use a key that’s effectively a tuple of the token/hash and the
> underlying key itself), so having more than 2^64 partitions won’t hurt
> anything in theory
>
> That said, having that many partitions would be an incredibly huge data
> set, and unless modeled properly, would be very likely to be unwieldy in
> practice.
>
> Tables without clustering keys are often deceptively expensive to compact,
> as a lot of work (relative to the other cell boundaries) happens on
> partition boundaries.
>
> --
> Jeff Jirsa
>
>
> > On Mar 7, 2018, at 3:06 AM, Javier Pareja <pareja.jav...@gmail.com>
> wrote:
> >
> > Hello all,
> >
> > I have been trying to find an answer to the following but I have had no
> luck so far:
> > Is there any limit to the number of partitions that a table can have?
> > Let's say a table has a partition key an no clustering key, is there a
> recommended limit on the number of values that this partition key can have?
> Is it recommended to have a clustering key to reduce this number by storing
> several rows in each partition instead of one row per partition.
> >
> > Regards,
> >
> > F Javier Pareja
>
> -
> To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: user-h...@cassandra.apache.org
>
>

-- 


--

Re: [External] Re: Whch version is the best version to run now?

[Carlos Rolo]

Hello,

Our 5 cents. Either 3.0.16 or 3.11.x
We are really happy with the way 3.11.1/2 is behaving.
We still have a lot of really well behaving Clusters in 2.1/2.2 latest.


Regards,

Carlos Juzarte Rolo
Cassandra Consultant / Datastax Certified Architect / Cassandra MVP

Pythian - Love your data

rolo@pythian | Twitter: @cjrolo | Skype: cjr2k3 | Linkedin:
*linkedin.com/in/carlosjuzarterolo
*
Mobile: +351 918 918 100
www.pythian.com

On Tue, Mar 6, 2018 at 2:05 PM, Alain RODRIGUEZ  wrote:

> Hello Tom,
>
> It's good to hear this kind of feedbacks,
>
> Thanks for sharing.
>
> 3.11.x seems to get more love from the community wrt patches. This is why
>> I'd recommend 3.11.x for new projects.
>>
>
> I also agree with this analysis.
>
> Stay away from any of the 2.x series, they're going EOL soonish and the
>> newer versions are very stable.
>>
>
> +1 here as well. Maybe add that 3.11.x, that is described as 'very stable'
> above, aims at stabilizing Cassandra after the tick-tock releases and is a
> 'bug fix' series and brings features developed during this period, even
> though it is needed to be careful with of some the new features, even in
> latest 3.11.x versions.
>
> I did not work that much with it yet, but I think I would pick 3.11.2 as
> well for a new cluster at the moment.
>
> C*heers,
>
> ---
> Alain Rodriguez - @arodream - al...@thelastpickle.com
> France / Spain
>
> The Last Pickle - Apache Cassandra Consulting
> http://www.thelastpickle.com
>
>
> 2018-03-05 12:39 GMT+00:00 Tom van der Woerdt <
> tom.vanderwoe...@booking.com>:
>
>> We run on the order of a thousand Cassandra nodes in production. Most of
>> that is 3.0.16, but new clusters are defaulting to 3.11.2 and some older
>> clusters have been upgraded to it as well.
>>
>> All of the bugs I encountered in 3.11.x were also seen in 3.0.x, but
>> 3.11.x seems to get more love from the community wrt patches. This is why
>> I'd recommend 3.11.x for new projects.
>>
>> Stay away from any of the 2.x series, they're going EOL soonish and the
>> newer versions are very stable.
>>
>> Tom van der Woerdt
>> Site Reliability Engineer
>>
>> Booking.com B.V.
>> Vijzelstraat 66
>> -80
>> Amsterdam 1017HL Netherlands
>> [image: Booking.com] 
>> The world's #1 accommodation site
>> 43 languages, 198+ offices worldwide, 120,000+ global destinations,
>> 1,550,000+ room nights booked every day
>> No booking fees, best price always guaranteed
>> Subsidiary of Booking Holdings Inc. (NASDAQ: BKNG)
>>
>> On Sat, Mar 3, 2018 at 12:25 AM, Jeff Jirsa  wrote:
>>
>>> I’d personally be willing to run 3.0.16
>>>
>>> 3.11.2 or 3 whatever should also be similar, but I haven’t personally
>>> tested it at any meaningful scale
>>>
>>>
>>> --
>>> Jeff Jirsa
>>>
>>>
>>> On Mar 2, 2018, at 2:37 PM, Kenneth Brotman <
>>> kenbrot...@yahoo.com.INVALID> wrote:
>>>
>>> Seems like a lot of people are running old versions of Cassandra.  What
>>> is the best version, most reliable stable version to use now?
>>>
>>>
>>>
>>> Kenneth Brotman
>>>
>>>
>>
>

-- 


--

Re: Cassandra Summit 2019 / Cassandra Summit 2018

[Carlos Rolo]

Hello all,

I'm interested planning/organizing a small kinda of NGCC in Lisbon,
Portugal in late May early June. Just waiting for the venue to confirm
possible dates.

Would be a 1day event kinda last year, is this something people would be
interested? I can push a google form for accessing the interest today.


Regards,

Carlos Juzarte Rolo
Cassandra Consultant / Datastax Certified Architect / Cassandra MVP

Pythian - Love your data

rolo@pythian | Twitter: @cjrolo | Skype: cjr2k3 | Linkedin:
*linkedin.com/in/carlosjuzarterolo
*
Mobile: +351 918 918 100
www.pythian.com

On Tue, Feb 27, 2018 at 11:39 AM, Kenneth Brotman <
kenbrot...@yahoo.com.invalid> wrote:

> Event planning is fun as long as you can pace it out properly.  Once you
> set a firm date for an event the pressure on you to keep everything on
> track is nerve racking.  To do something on the order of Cassandra Summit
> 2016, I think we are should plan for 2020.  It’s too late for 2018 and even
> trying to meet the timeline for everything that would have to come together
> makes 2019 too nerve racking a target date.  The steps should be:
>
> Form a planning committee
>
> Bring potential sponsors into the planning early
>
> Select an event planning vendor to guide us and to do the
> heavy lifting for us
>
>
>
> In the meantime, we could have a World-wide Distributed Asynchronous
> Cassandra Convention which offers four benefits:
>
> It allows us to address the fact that we are a world-wide
> group that needs a way to reach everyone in a way where no one is
> geographically disadvantaged
>
> No travel time, no travel expenses and no ticket fees
> makes it accessible to a lot of people that otherwise would have to miss out
>
> The lower production costs and simpler administrative workload allows us
> to reach implementation sooner
>
> It’s cutting edge, world class innovation like Cassandra
>
>
>
> Kenneth Brotman
>
>
>
> *From:* Jeff Jirsa [mailto:jji...@gmail.com]
> *Sent:* Monday, February 26, 2018 9:38 PM
> *To:* cassandra
> *Subject:* Re: Cassandra Summit 2019 / Cassandra Summit 2018
>
>
>
> Instaclustr sponsored the 2017 NGCC (Next Gen Cassandra Conference), which
> was developer/development focused (vs user focused).
>
>
>
> For 2018, we're looking at options for both a developer conference and a
> user conference. There's a lot of logistics involved, and I think it's
> fairly obvious that most of the PMC members aren't professional event
> planners, so it's possible that either/both conferences may not happen, but
> we're doing our best to try to put something together.
>
>
>
>
>
> On Mon, Feb 26, 2018 at 3:00 PM, Rahul Singh 
> wrote:
>
> I think some of the Instaclustr folks had done one last year which I
> really wanted to go to.. Distributed / Async both would be easier to get
> people to write papers, make slides, do youtube videos with.. and then we
> could do a virtual web conf of the best submissions.
>
>
> On Feb 26, 2018, 1:04 PM -0600, Kenneth Brotman <
> kenbrot...@yahoo.com.invalid>, wrote:
>
> Is there any planning yet for a Cassandra Summit 2019 or Cassandra Summit
> 2018 (probably too late)?
>
>
>
> Is there a planning committee?
>
>
>
> Who wants there to be a Cassandra Summit 2019 and who thinks there is a
> better way?
>
>
>
> We could try a Cassandra Distributed Summit 2019 where we meet virtually
> and perhaps asynchronously, but there would be a lot more energy and
> bonding if it’s not virtual.  I’m up for any of these.
>
>
>
> Kenneth Brotman
>
>
>
>
>
>
>

-- 


--

Re: What happens if multiple processes send create table if not exist statement to cassandra?

[Carlos Rolo]

Don't do that. Worst case you might get different schemas in flight and no
agreement on your cluster.  If you are already doing that, check "nodetool
describecluster" after you do that.

Like Jeff said, it is likely to cause problems.

Regards,

Carlos Juzarte Rolo
Cassandra Consultant / Datastax Certified Architect / Cassandra MVP

Pythian - Love your data

rolo@pythian | Twitter: @cjrolo | Skype: cjr2k3 | Linkedin:
*linkedin.com/in/carlosjuzarterolo
*
Mobile: +351 918 918 100
www.pythian.com

On Sat, Jan 27, 2018 at 7:25 PM, Jeff Jirsa  wrote:

> It’s not LWT. Don’t do programmatic schema changes that can race, it’s
> likely to cause problems
>
>
> --
> Jeff Jirsa
>
>
> > On Jan 27, 2018, at 10:19 AM, Kant Kodali  wrote:
> >
> > Hi All,
> >
> > What happens if multiple processes send create table if not exist
> statement to cassandra? will there be any data corruption or any other
> issues if I send "create table if not exist" request often?
> >
> > I dont see any entry in system.paxos table so is it fair to say "IF NOT
> EXISTS" doesn't automatically imply LWT?
> >
> > Thanks!
>
> -
> To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: user-h...@cassandra.apache.org
>
>

-- 


--

RE: Meltdown/Spectre Linux patch - Performance impact on Cassandra?

[Carlos Rolo]

We are seeing almost no impact on Azure (+1 or 2%). Non-patched OS.

On AWS we don't have any solid data yet.

On 13 Jan 2018 09:46, "Steinmaurer, Thomas" <
thomas.steinmau...@dynatrace.com> wrote:

Hello Ben,



thanks for the notice. Similar here + others reporting as well:
https://blog.appoptics.com/visualizing-meltdown-aws/





Regards,

Thomas



*From:* Ben Slater [mailto:ben.sla...@instaclustr.com]
*Sent:* Freitag, 12. Jänner 2018 23:37

*To:* user@cassandra.apache.org
*Subject:* Re: Meltdown/Spectre Linux patch - Performance impact on
Cassandra?



We’re seeing evidence across our fleet that AWS has rolled something out in
the last 24 hours that has significantly reduce the performance impacts -
back pretty close to pre-patch levels. Yet to see if the impacts come back
with o/s patching on top of the improved hypervisor.



Cheers

Ben







On Thu, 11 Jan 2018 at 05:32 Jon Haddad  wrote:

For what it’s worth, we (TLP) just posted some results comparing pre and
post meltdown statistics: http://thelastpickle.com/blog/2018/
01/10/meltdown-impact-on-latency.html





On Jan 10, 2018, at 1:57 AM, Steinmaurer, Thomas <
thomas.steinmau...@dynatrace.com> wrote:



m4.xlarge do have PCID to my knowledge, but possibly we need a rather new
kernel 4.14. But I fail to see how this could help anyway, cause this looks
highly Amazon Hypervisor patch related and we do not have the production
instances patched at OS/VM level (yet).



Thomas



*From:* Dor Laor [mailto:d...@scylladb.com ]
*Sent:* Dienstag, 09. Jänner 2018 19:30
*To:* user@cassandra.apache.org
*Subject:* Re: Meltdown/Spectre Linux patch - Performance impact on
Cassandra?



Make sure you pick instances with PCID cpu capability, their TLB overhead
flush

overhead is much smaller



On Tue, Jan 9, 2018 at 2:04 AM, Steinmaurer, Thomas <
thomas.steinmau...@dynatrace.com> wrote:

Quick follow up.



Others in AWS reporting/seeing something similar, e.g.:https://twitter.com/
BenBromhead/status/950245250504601600



So, while we have seen an relative CPU increase of ~ 50% since Jan 4, 2018,
we now also have applied a kernel update at OS/VM level on a single node
(loadtest and not production though), thus more or less double patched now.
Additional CPU impact by OS/VM level kernel patching is more or less
negligible, so looks highly Hypervisor related.



Regards,

Thomas



*From:* Steinmaurer, Thomas [mailto:thomas.steinmau...@dynatrace.com]
*Sent:* Freitag, 05. Jänner 2018 12:09
*To:* user@cassandra.apache.org
*Subject:* Meltdown/Spectre Linux patch - Performance impact on Cassandra?



Hello,



has anybody already some experience/results if a patched Linux kernel
regarding Meltdown/Spectre is affecting performance of Cassandra negatively?



In production, all nodes running in AWS with m4.xlarge, we see up to a 50%
relative (e.g. AVG CPU from 40% => 60%) CPU increase since Jan 4, 2018,
most likely correlating with Amazon finished patching the underlying
Hypervisor infrastructure …



Anybody else seeing a similar CPU increase?



Thanks,

Thomas



The contents of this e-mail are intended for the named addressee only. It
contains information that may be confidential. Unless you are the named
addressee or an authorized designee, you may not copy or use it, or
disclose it to anyone else. If you received it in error please notify us
immediately and then destroy it. Dynatrace Austria GmbH (registration
number FN 91482h) is a company registered in Linz whose registered office
is at 4040 Linz, Austria, Freistädterstraße 313


The contents of this e-mail are intended for the named addressee only. It
contains information that may be confidential. Unless you are the named
addressee or an authorized designee, you may not copy or use it, or
disclose it to anyone else. If you received it in error please notify us
immediately and then destroy it. Dynatrace Austria GmbH (registration
number FN 91482h) is a company registered in Linz whose registered office
is at 4040 Linz, Austria, Freistädterstraße 313




The contents of this e-mail are intended for the named addressee only. It
contains information that may be confidential. Unless you are the named
addressee or an authorized designee, you may not copy or use it, or
disclose it to anyone else. If you received it in error please notify us
immediately and then destroy it. Dynatrace Austria GmbH (registration
number FN 91482h) is a company registered in Linz whose registered office
is at 4040 Linz, Austria, Freistädterstraße 313






-- 

*Ben Slater*
*Chief Product Officer*
* *

   

Re: Stable Cassandra 3.x version for production

[Carlos Rolo]

DSE is already pushing 3.11 in 5.1

If you're going into 3.x either 3.11.1 or 3.0.15. I would recommend 3.11.1
to take advantage of a couple of features (ex: Slow query log) that you
don't have on 3.0.

If you're stable on 2.x and you don't need any new feature, you can leave
it there safely (although the latest releases are probably the last ones
you will see).

Regards,

Carlos Juzarte Rolo
Cassandra Consultant / Datastax Certified Architect / Cassandra MVP

Pythian - Love your data

rolo@pythian | Twitter: @cjrolo | Skype: cjr2k3 | Linkedin:
*linkedin.com/in/carlosjuzarterolo
*
Mobile: +351 918 918 100
www.pythian.com

On Tue, Nov 7, 2017 at 1:35 PM, Herbert Fischer <
herbert.fisc...@crossengage.io> wrote:

> I know that people usually prefers to use the 3.0.x branch because that's
> the one that is underneath DSE.
>
> I've never heard of anyone using Cassandra > 3.0.x on production.
>
>
>
> On Tue, Nov 7, 2017 at 11:29 AM, shini gupta 
> wrote:
>
>>
>>
>> Hi
>>
>> Which version of Cassandra 3.x is stable and production-ready?
>>
>> Regards
>>
>
>
>
> --
> Herbert Fischer | Senior IT Architect
> CrossEngage GmbH | Bertha-Benz Straße 5 | 10557 Berlin
> 
>
> E-Mail: herbert.fisc...@crossengage.io
> Web: www.crossengage.io
>
> Amtsgericht Berlin-Charlottenburg | HRB 169537 B
> Geschäftsführer: Dr. Markus Wübben, Manuel Hinz | USt-IdNr.: DE301504202
>

-- 


--

Re: Schema Mismatch Issue in Production

[Carlos Rolo]

Which version are you running? I got stuck in a similar situation (With a
lot more nodes) and the only way to make it good was to stop the whole
cluster, start nodes 1 by 1.



Regards,

Carlos Juzarte Rolo
Cassandra Consultant / Datastax Certified Architect / Cassandra MVP

Pythian - Love your data

rolo@pythian | Twitter: @cjrolo | Skype: cjr2k3 | Linkedin:
*linkedin.com/in/carlosjuzarterolo
*
Mobile: +351 918 918 100
www.pythian.com

On Thu, Oct 12, 2017 at 5:53 AM, Pradeep Chhetri 
wrote:

> Hello everyone,
>
> We had some issues yesterday in our 3 nodes cluster where the application
> tried to create the same table twice quickly and cluster became unstable.
>
> Temporarily, we reduced it to single node cluster which gave us some
> relief.
>
> Now when we are trying to bootstrap a new node and add it to cluster.
> we're seeing schema mismatch issue.
>
> # nodetool status
> Datacenter: datacenter1
> ===
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> --  AddressLoad   Tokens   Owns (effective)  Host ID
>  Rack
> UN  10.42.247.173  3.07 GiB   256  100.0%
> dffc39e5-d4ba-4b10-872e-0e3cc10f5e08  rack1
> UN  10.42.209.245  2.25 GiB   256  100.0%
> 9b99d5d8-818e-4741-9533-259d0fc0e16d  rack1
>
> root@cassandra-2:~# nodetool describecluster
> Cluster Information:
> Name: sa-cassandra
> Snitch: org.apache.cassandra.locator.DynamicEndpointSnitch
> Partitioner: org.apache.cassandra.dht.Murmur3Partitioner
> Schema versions:
> e2275d0f-a5fc-39d9-8f11-268b5e9dc295: [10.42.209.245]
>
> 5f5f66f5-d6aa-3b90-b674-e08811d4d412: [10.42.247.173]
>
> Freshly bootstrapped node - 10.42.247.173
> Single node from original cluster - 10.42.209.245
>
> I read https://docs.datastax.com/en/dse-trblshoot/doc/troubles
> hooting/schemaDisagree.html and tried restarting the new node but it
> didnt help.
>
> Please do suggest. We are facing this issue in production.
>
> Thank you.
>

-- 


--

Re: Materialized views stability

[Carlos Rolo]

I've been dealing with MV extensively, and I second Blake. MVs are not
suitable for production. Unless you're ready for the pain (The out of sync
is a major pain point), I would not go that way.

Regards,

Carlos Juzarte Rolo
Cassandra Consultant / Datastax Certified Architect / Cassandra MVP

Pythian - Love your data

rolo@pythian | Twitter: @cjrolo | Skype: cjr2k3 | Linkedin:
*linkedin.com/in/carlosjuzarterolo
*
Mobile: +351 918 918 100
www.pythian.com

On Mon, Oct 2, 2017 at 4:50 PM, Blake Eggleston 
wrote:

> Hi Hannu,
>
> There are more than a few committers that don't think MVs are currently
> suitable for production use. I'm not involved with MV development, so this
> may not be 100% accurate, but the problems as I understand them are:
>
> There's no way to determine if a view is out of sync with the base table.
> If you do determine that a view is out of sync, the only way to fix it is
> to drop and rebuild the view.
> There are liveness issues with updates being reflected in the view.
>
> Any one of these issues makes it difficult to recommend for general
> application development. I'd say that if you're not super familiar with
> their shortcomings and are confident you can fit your use case in them,
> you're probably better off not using them.
>
> Thanks,
>
> Blake
>
> On October 2, 2017 at 6:55:52 AM, Hannu Kröger (hkro...@gmail.com) wrote:
>
> Hello,
>
> I have seen some discussions around Materialized Views and stability of
> that functionality.
>
> There are some open issues around repairs:
> https://issues.apache.org/jira/browse/CASSANDRA-13810?
> jql=project%20%3D%20CASSANDRA%20AND%20issuetype%20%3D%20Bug%
> 20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%
> 20Reopened%2C%20%22Patch%20Available%22%2C%20Testing%
> 2C%20%22Ready%20to%20Commit%22%2C%20%22Awaiting%20Feedback%22)%20AND%
> 20component%20%3D%20%22Materialized%20Views%22
>
> Is it so that the current problems are mostly related to incremental
> repairs or are there also other major concerns why some people don’t
> consider them to be safe for production use?
>
> Cheers,
> Hannu
>
>

-- 


--

Re: Reaper 0.7 is released!

[Carlos Rolo]

Thanks a lot for the release!

On 27 Sep 2017 10:21 pm, "Jon Haddad"  wrote:

> We’ve discussed it, and it’s in GitHub, but we haven’t scheduled it yet.
> For now we’re trying to make it easier to use, cut down on it’s query
> count, and ensure what’s there is solid.  The team (Mick, Alex, Anthony)
> has done a lot of refactoring to make the codebase easier to work on, build
> and deploy, but there’s still a bit more to go before we’re ready to move
> onto other areas.
>
> Jon
>
>
> On Sep 27, 2017, at 4:04 PM, Dominik Petrovic  INVALID> wrote:
>
> @Jon, do you have in the pipeline to add support for cleanup job?
>
>
>
> Wednesday, September 27, 2017 10:33 AM -07:00 from Aiman Parvaiz <
> ai...@steelhouse.com>:
>
> Thanks!! Love Reaper :)
>
> Sent from my iPhone
>
> On Sep 27, 2017, at 10:01 AM, Jon Haddad  wrote:
>
> Hey folks,
>
> We (The Last Pickle) are proud to announce the release of Reaper 0.7!  In
> this release we've added support to run Reaper across multiple data centers
> as well as supporting Reaper failover when using the Cassandra storage
> backend.
>
> You can grab DEB, RPM and tarballs off the downloads page:
> http://cassandra-reaper.io/docs/download/
>
> We've made significant improvements to the docs section of the site as
> well, with a slew of other improvements in the works.
>
> The Reaper user mailing list is located here, for questions and feedback:
> https://groups.google.com/forum/#!forum/tlp-apache-cassandra-reaper-users
>
> Thanks,
> Jon
>
>
>
> --
> Dominik Petrovic
>
>
>

-- 


--

Re: Self-healing data integrity?

[Carlos Rolo]

Wouldn't be easier for

1) The CRC to be checked by the sender, and don't send if it doesn't match?

2) And once the stream ends, you could compare the 2 CRCs to see if
something got weird during transfer?

Also you could implement this in 2 pieces instead of reviewing the
streaming architecture as a whole. I have no familiarity with Cassandra
code for making this assumptions, so just wanting to contribute (And
actually trying to implement at least the first part).

Regards,

Carlos Juzarte Rolo
Cassandra Consultant / Datastax Certified Architect / Cassandra MVP

Pythian - Love your data

rolo@pythian | Twitter: @cjrolo | Skype: cjr2k3 | Linkedin:
*linkedin.com/in/carlosjuzarterolo
*
Mobile: +351 918 918 100
www.pythian.com

On Mon, Sep 11, 2017 at 9:12 AM, DuyHai Doan  wrote:

> Agree
>
>  A tricky detail about streaming is that:
>
> 1) On the sender side, the node just send the SSTable (without any other
> components like CRC files, partition index, partition summary etc...)
> 2) The sender does not even bother to de-serialize the SSTable data, it is
> just sending the stream of bytes by reading directly SSTables content from
> disk
> 3) On the receiver side, the node receives the bytes stream and needs to
> serialize it in memory to rebuild all the SSTable components (CRC files,
> partition index, partition summary ...)
>
> So the consequences are:
>
> a. there is a bottleneck on receiving side because of serialization
> b. if there is a bit rot in SSTables, since CRC files are not sent, no
> chance to detect it from receiving side
> c. if we want to include CRC checks in the streaming path, it's a whole
> review of the streaming architecture, not only adding some feature
>
> On Sat, Sep 9, 2017 at 10:06 PM, Jeff Jirsa  wrote:
>
>> (Which isn't to say that someone shouldn't implement this; they should,
>> and there's probably a JIRA to do so already written, but it's a project of
>> volunteers, and nobody has volunteered to do the work yet)
>>
>> --
>> Jeff Jirsa
>>
>>
>> On Sep 9, 2017, at 12:59 PM, Jeff Jirsa  wrote:
>>
>> There is, but they aren't consulted on the streaming paths (only on
>> normal reads)
>>
>>
>> --
>> Jeff Jirsa
>>
>>
>> On Sep 9, 2017, at 12:02 PM, DuyHai Doan  wrote:
>>
>> Jeff,
>>
>>  With default compression enabled on each table, isn't there CRC files
>> created along side with SSTables that can help detecting bit-rot ?
>>
>>
>> On Sat, Sep 9, 2017 at 7:50 PM, Jeff Jirsa  wrote:
>>
>>> Cassandra doesn't do that automatically - it can guarantee consistency
>>> on read or write via ConsistencyLevel on each query, and it can run active
>>> (AntiEntropy) repairs. But active repairs must be scheduled (by human or
>>> cron or by third party script like http://cassandra-reaper.io/), and to
>>> be pedantic, repair only fixes consistency issue, there's some work to be
>>> done to properly address/support fixing corrupted replicas (for example,
>>> repair COULD send a bit flip from one node to all of the others)
>>>
>>>
>>>
>>> --
>>> Jeff Jirsa
>>>
>>>
>>> On Sep 9, 2017, at 1:07 AM, Ralph Soika  wrote:
>>>
>>> Hi,
>>>
>>> I am searching for a big data storage solution for the Imixs-Workflow
>>> project. I started with Hadoop until I became aware of the
>>> 'small-file-problem'. So I am considering using Cassandra now.
>>>
>>> But Hadoop has one important feature for me. The replicator continuously
>>> examines whether data blocks are consistent across all datanodes. This will
>>> detect disk errors and automatically move data from defective blocks to
>>> working blocks. I think this is called 'self-healing mechanism'.
>>>
>>> Is there a similar feature in Cassandra too?
>>>
>>>
>>> Thanks for help
>>>
>>> Ralph
>>>
>>>
>>>
>>> --
>>>
>>>
>>
>

-- 


--

Re: Manual repair not showing in the log.

[Carlos Rolo]

Can you check if you have any validation compaction running in nodetool
compactionstats?

On 7 Sep 2017 7:56 pm, "Mark Furlong"  wrote:

I have started a repair and I received the message ‘Starting repair command
#1, repairing 25301 ranges for keyspace x (parallelism=PARALLEL,
full=true). When I look in the log for antientropy repairs I do not see
anything for this keyspace. Expecting to see messages for the merkle trees
on each column family in the keyspace… nothing.



The repair appears to not be doing anything, what is it stuck on?





*Mark Furlong*

Sr. Database Administrator

*mfurl...@ancestry.com *
M: 801-859-7427

O: 801-705-7115

1300 W Traverse Pkwy


Lehi, UT 84043






​[image: http://c.mfcreative.com/mars/email/shared-icon/sig-logo.gif]

-- 


--

Re: truncate table in C* 3.11.0

[Carlos Rolo]

If you waited less than 60s, no warning/error was issued. Do the following:

* Check if all nodes are up (truncate fails if not)
* Check if you got a snapshot generated (unless you have auto_snapshot
disabled)
* Check if you have still the sstables in the directories (you shouldn't)

If it didn't happen, open a ticket.

Tested right now on a 3 nodes 3.11.0 cluster and it worked fine. So if you
can trace the query to provide additional detail, it would help.




Regards,

Carlos Juzarte Rolo
Cassandra Consultant / Datastax Certified Architect / Cassandra MVP

Pythian - Love your data

rolo@pythian | Twitter: @cjrolo | Skype: cjr2k3 | Linkedin:
*linkedin.com/in/carlosjuzarterolo
*
Mobile: +351 918 918 100
www.pythian.com

On Thu, Sep 7, 2017 at 10:07 AM, Cogumelos Maravilha <
cogumelosmaravi...@sapo.pt> wrote:

> Hi list,
>
> Using cqlsh:
>
> consistency all;
>
> select count(*) table1;
>
> 219871
>
> truncate table1;
>
> select count(*) table1;
>
> 219947
>
>
> There is a consumer reading data from kafka and inserting in C* but the
> rate is around 50 inserts/minute.
>
>
> Cheers
>
>
>
> -
> To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: user-h...@cassandra.apache.org
>
>

-- 


--

Re: Cassandra Setup Question

[Carlos Rolo]

Use networktopologystrategy as replication strategy and make sure you have
dc1: 3 and dc2: 3.

This way you have 3 replicas in each DC.



On 23 Aug 2017 12:53, "Jonathan Baynes" 
wrote:

> Hi Community,
>
>
>
> Quick question regarding Replication Factor.
>
>
>
> In my Production Environment I currently have 6 nodes this will grow to 10
> shortly), over 2 datacentres, so currently 3 in each, we want to have
> Active/Passive setup so the Client will only speak to DC 1 via load
> balancing policy, but we want the data to replicate across both DCs , so
> that in the event of a DC failure the second DC acts as our DR DC and has
>  all the data as well.
>
>
>
> The Consistency will be Quorum, but replication Factor does that need to
> be 3 or 6.
>
>
>
> Thanks
>
> J
>
>
>
> *Jonathan Baynes*
>
> DBA
> Tradeweb Europe Limited
>
> Moor Place  •  1 Fore Street Avenue  •  London EC2Y 9DT
> P +44 (0)20 77760988 <+44%2020%207776%200988>  •  F +44 (0)20 7776 3201
> <+44%2020%207776%203201>  •  M +44 (0) xx
>
> jonathan.bay...@tradeweb.com
>
>
>
> [image: cid:image001.jpg@01CD26AD.4165F110] 
> follow us:  *[image: cid:image002.jpg@01CD26AD.4165F110]*
>    [image:
> cid:image003.jpg@01CD26AD.4165F110] 
>
> —
>
> A leading marketplace  for
> electronic fixed income, derivatives and ETF trading
>
>
>
> 
>
> This e-mail may contain confidential and/or privileged information. If you
> are not the intended recipient (or have received this e-mail in error)
> please notify the sender immediately and destroy it. Any unauthorized
> copying, disclosure or distribution of the material in this e-mail is
> strictly forbidden. Tradeweb reserves the right to monitor all e-mail
> communications through its networks. If you do not wish to receive
> marketing emails about our products / services, please let us know by
> contacting us, either by email at contac...@tradeweb.com or by writing to
> us at the registered office of Tradeweb in the UK, which is: Tradeweb
> Europe Limited (company number 3912826), 1 Fore Street Avenue London EC2Y
> 9DT. To see our privacy policy, visit our website @ www.tradeweb.com.
>

-- 


--

Re: Adding a new node with the double of disk space

[Carlos Rolo]

I would preferably spin 2 JVMs inside the same hardware (if you double
everything) than having to deal with what Jeff stated.

Also certain operations are not really found of a large number of vnodes
(eg. repair). There was a lot of improvements in the 3.x release cycle, but
I do still tend to reduce vnodes number and not increase.

Regards,

Carlos Juzarte Rolo
Cassandra Consultant / Datastax Certified Architect / Cassandra MVP

Pythian - Love your data

rolo@pythian | Twitter: @cjrolo | Skype: cjr2k3 | Linkedin:
*linkedin.com/in/carlosjuzarterolo
<http://linkedin.com/in/carlosjuzarterolo>*
Mobile: +351 918 918 100
www.pythian.com

On Thu, Aug 17, 2017 at 9:19 PM, Jeff Jirsa <jji...@gmail.com> wrote:

> If you really double the hardware in every way, it's PROBABLY reasonable
> to double num_tokens. It won't be quite the same as doubling
> all-the-things, because you still have a single JVM, and you'll still have
> to deal with GC as you're now reading twice as much and generating twice as
> much garbage, but you can probably adjust the tuning of the heap to
> compensate.
>
>
>
> On Thu, Aug 17, 2017 at 1:00 PM, Kevin O'Connor <ke...@reddit.com.invalid>
> wrote:
>
>> Are you saying if a node had double the hardware capacity in every way it
>> would be a bad idea to up num_tokens? I thought that was the whole idea of
>> that setting though?
>>
>> On Thu, Aug 17, 2017 at 9:52 AM, Carlos Rolo <r...@pythian.com> wrote:
>>
>>> No.
>>>
>>> If you would double all the hardware on that node vs the others would
>>> still be a bad idea.
>>> Keep the cluster uniform vnodes wise.
>>>
>>> Regards,
>>>
>>> Carlos Juzarte Rolo
>>> Cassandra Consultant / Datastax Certified Architect / Cassandra MVP
>>>
>>> Pythian - Love your data
>>>
>>> rolo@pythian | Twitter: @cjrolo | Skype: cjr2k3 | Linkedin:
>>> *linkedin.com/in/carlosjuzarterolo
>>> <http://linkedin.com/in/carlosjuzarterolo>*
>>> Mobile: +351 918 918 100 <+351%20918%20918%20100>
>>> www.pythian.com
>>>
>>> On Thu, Aug 17, 2017 at 5:47 PM, Cogumelos Maravilha <
>>> cogumelosmaravi...@sapo.pt> wrote:
>>>
>>>> Hi all,
>>>>
>>>> I need to add a new node to my cluster but this time the new node will
>>>> have the double of disk space comparing to the other nodes.
>>>>
>>>> I'm using the default vnodes (num_tokens: 256). To fully use the disk
>>>> space in the new node I just have to configure num_tokens: 512?
>>>>
>>>> Thanks in advance.
>>>>
>>>>
>>>>
>>>> -
>>>> To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
>>>> For additional commands, e-mail: user-h...@cassandra.apache.org
>>>>
>>>>
>>>
>>> --
>>>
>>>
>>>
>>>
>>
>

-- 


--

Re: Adding a new node with the double of disk space

[Carlos Rolo]

No.

If you would double all the hardware on that node vs the others would still
be a bad idea.
Keep the cluster uniform vnodes wise.

Regards,

Carlos Juzarte Rolo
Cassandra Consultant / Datastax Certified Architect / Cassandra MVP

Pythian - Love your data

rolo@pythian | Twitter: @cjrolo | Skype: cjr2k3 | Linkedin:
*linkedin.com/in/carlosjuzarterolo
*
Mobile: +351 918 918 100
www.pythian.com

On Thu, Aug 17, 2017 at 5:47 PM, Cogumelos Maravilha <
cogumelosmaravi...@sapo.pt> wrote:

> Hi all,
>
> I need to add a new node to my cluster but this time the new node will
> have the double of disk space comparing to the other nodes.
>
> I'm using the default vnodes (num_tokens: 256). To fully use the disk
> space in the new node I just have to configure num_tokens: 512?
>
> Thanks in advance.
>
>
>
> -
> To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: user-h...@cassandra.apache.org
>
>

-- 


--

Re: c* updates not getting reflected.

[Carlos Rolo]

What consistency are you using on those queries?

On 11 Jul 2017 19:09, "techpyaasa ."  wrote:

> Hi,
>
> We have a table with following schema:
>
> CREATE TABLE ks1.cf1 ( pid bigint, cid bigint, resp_json text, status int,
> PRIMARY KEY (pid, cid) ) WITH CLUSTERING ORDER BY (cid ASC) with LCS
> compaction strategy.
>
> We make very frequent updates to this table with query like
>
> UPDATE ks1.cf1 SET status = 0 where pid=1 and cid=1;
> UPDATE ks1.cf1 SET resp_json='' where uid=1 and mid=1;
>
>
> Now we seeing a strange issue like sometimes status column or resp_json
> column value not getting updated when we query using SELECT query.
>
> We are not seeing any exceptions though during UPDATE query executions.
> And also is there any way to make sure that last UPDATE was success??
>
> We are using c* - 2.1.17 , datastax java driver 2.1.18.
>
> Can someone point out what the issue is or anybody faced such strange
> issue?
>
> Any help is appreciated.
>
> Thanks in advance
> TechPyaasa
>

-- 


--

Re: Reaper v0.6.1 released

[Carlos Rolo]

Great!

Thanks a lot!

Regards,

Carlos Juzarte Rolo
Cassandra Consultant / Datastax Certified Architect / Cassandra MVP

Pythian - Love your data

rolo@pythian | Twitter: @cjrolo | Skype: cjr2k3 | Linkedin:
*linkedin.com/in/carlosjuzarterolo
*
Mobile: +351 918 918 100
www.pythian.com

On Thu, Jun 15, 2017 at 7:56 AM, Aiman Parvaiz  wrote:

> Great work!! Thanks
>
> Sent from my iPhone
>
> On Jun 14, 2017, at 11:30 PM, Shalom Sagges 
> wrote:
>
> That's awesome!! Thanks for contributing! 
>
>
> Shalom Sagges
> DBA
> T: +972-74-700-4035 <+972%2074-700-4035>
>  
>  We Create Meaningful Connections
>
>
>
> On Thu, Jun 15, 2017 at 2:32 AM, Jonathan Haddad 
> wrote:
>
>> Hey folks!
>>
>> I'm proud to announce the 0.6.1 release of the Reaper project, the open
>> source repair management tool for Apache Cassandra.
>>
>> This release improves the Cassandra backend significantly, making it a
>> first class citizen for storing repair schedules and managing repair
>> progress.  It's no longer necessary to manage a PostgreSQL DB in addition
>> to your Cassandra DB.
>>
>> We've been very active since we forked the original Spotify repo.  Since
>> this time we've added:
>>
>> * A native Cassandra backend
>> * Support for versions > 2.0
>> * Merged in the WebUI, maintained by Stefan Podkowinski (
>> https://github.com/spodkowinski/cassandra-reaper-ui)
>> * Support for incremental repair (probably best to avoid till Cassandra
>> 4.0, see CASSANDRA-9143)
>>
>> We're excited to continue making improvements past the original intent of
>> the project.  With the lack of Cassandra 3.0 support in OpsCenter, there's
>> a gap that needs to be filled for tools that help with managing a cluster.
>> Alex Dejanovski showed me a prototype he recently put together for a really
>> nice view into cluster health.  We're also looking to add in support common
>> cluster operations like snapshots, upgradesstables, cleanup, and setting
>> options at runtime.
>>
>> Grab it here: https://github.com/thelastpickle/cassandra-reaper
>>
>> Feedback / bug reports / ideas are very much appreciated.
>>
>> We have a dedicated, low traffic ML here: https://groups.google.com/foru
>> m/#!forum/tlp-apache-cassandra-reaper-users
>>
>> Jon Haddad
>> Principal Consultant, The Last Pickle
>> http://thelastpickle.com/
>>
>
>
> This message may contain confidential and/or privileged information.
> If you are not the addressee or authorized to receive this on behalf of
> the addressee you must not use, copy, disclose or take action based on this
> message or any information herein.
> If you have received this message in error, please advise the sender
> immediately by reply email and delete this message. Thank you.
>
>

-- 


--

Re: Stable version apache cassandra 3.X /3.0.X

[Carlos Rolo]

On sync in Jon.

Only go 3.0.x if you REALLY need something from there (ex: MV) even then,
be carefull.

3.x wait for 3.11.x. 3.10 if you REALLY need something from there right now.

Latest 2.2.x or 2.1.x if you are just doing baseline Cassandra and need the
stability.

Regards,

Carlos Juzarte Rolo
Cassandra Consultant / Datastax Certified Architect / Cassandra MVP

Pythian - Love your data

rolo@pythian | Twitter: @cjrolo | Skype: cjr2k3 | Linkedin:
*linkedin.com/in/carlosjuzarterolo
*
Mobile: +351 918 918 100
www.pythian.com

On Wed, May 31, 2017 at 5:48 PM, Jonathan Haddad  wrote:

> I really wouldn't go by the tick tock blog post, considering tick tock is
> dead.
>
> I'm still not wild about putting any 3.0 or 3.x into production.  3.0
> removed off heap memtables and there have been enough bugs in the storage
> engine that I'm still wary.  My hope is to see 3.11.x get enough bug fixes
> to where most people just skip 3.0 altogether.  I'm not sure if we're there
> yet though.
>
>
> On Wed, May 31, 2017 at 9:43 AM Junaid Nasir  wrote:
>
>> as mentioned here http://www.datastax.com/dev/blog/cassandra-2-2-3-0-
>> and-beyond
>>
>>> Under normal conditions, we will NOT release 3.x.y stability releases
>>> for x > 0.  That is, we will have a traditional 3.0.y stability series, but
>>> the odd-numbered bugfix-only releases will fill that role for the tick-tock
>>> series — recognizing that occasionally we will need to be flexible enough
>>> to release an emergency fix in the case of a critical bug or security
>>> vulnerability.
>>> We do recognize that it will take some time for tick-tock releases to
>>> deliver production-level stability, which is why we will continue to
>>> deliver 2.2.y and 3.0.y bugfix releases.  (But if we do demonstrate that
>>> tick-tock can deliver the stability we want, there will be no need for a
>>> 4.0.y bugfix series, only 4.x tick-tock.)
>>
>>
>> On Wed, May 31, 2017 at 9:02 PM, pabbireddy avinash <
>> pabbireddyavin...@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> We are planning to deploy a cassandra production cluster on 3.X /3.0.X .
>>> Please let us know if there is any stable version  in 3.X/3.0.X that we
>>> could deploy in production .
>>>
>>> Regards,
>>> Avinash.
>>>
>>
>>

-- 


--

Re: Slowness in C* cluster after implementing multiple network interface configuration.

[Carlos Rolo]

It might be a bug.
Cassandra, AFAIK, scans those files for changes and updates the topology
(So you don't need a restart if you change the files). It might be the case
that the absence of the file, is still noticed by Cassandra even if it is
not really used.

I can do a small test to confirm, if so, it is a question of "expected
behaviour" (as in, always leave the file there) vs Bug (It shouldn't care
for files it doesn't use).

If you can always reproduce, feel free to Open a JIRA.

Thanks for the description.

Regards,

Carlos Juzarte Rolo
Cassandra Consultant / Datastax Certified Architect / Cassandra MVP

Pythian - Love your data

rolo@pythian | Twitter: @cjrolo | Skype: cjr2k3 | Linkedin:
*linkedin.com/in/carlosjuzarterolo
*
Mobile: +351 918 918 100
www.pythian.com

On Wed, May 24, 2017 at 8:12 AM, Prakash Chauhan <
prakash.chau...@ericsson.com> wrote:

> Hi All,
>
>
>
> We have a new observation.
>
>
>
> Earlier for implementing multiple network interfaces, we were deleting
> *cassandra-topologies.properties* in the last step (Steps are mentioned
> in mail trail).
>
> The rationale was that because we are using altogether a new
> endpoint_snitch , we don’t require cassandra-topologies.properties file
> anymore.
>
>
>
> Now we have observed that if we don’t delete cassandra-topologies.properties,
> the slowness is not there in the cluster (Even with multiple restarts)
>
>
>
> Is there some relationship between *GossipingPropertyFileSnitch* and
> *cassandra-topologies.properties* ?
>
>
>
> As per my knowledge,  *cassandra-topologies.properties* file is only used
> as a fallback while doing snitch migration. If that’s the case, why does
> Cassandra becomes slow with time ( and after doing multiple restarts )
> after deleting cassandra-topologies.properties ?
>
>
>
>
>
>
>
>
>
> Regards,
>
> Prakash Chauhan.
>
>
>
> *From:* Cogumelos Maravilha [mailto:cogumelosmaravi...@sapo.pt]
> *Sent:* Wednesday, May 24, 2017 12:15 AM
> *To:* user@cassandra.apache.org
> *Subject:* Re: Slowness in C* cluster after implementing multiple network
> interface configuration.
>
>
>
> Hi,
>
> I never used version 2.0.x but I think port 7000 isn't enough.
>
> Try enable:
>
> 7000 inter-node
>
> 7001 SSL inter-node
>
> 9042 CQL
>
> 9160 Thrift is enable in that version
>
>
>
> And
>
> In Cassandra.yaml, add property “broadcast_address”.  = local ipv4
>
> In Cassandra.yaml, change “listen_address” to private IP. = local ipv4
>
>
>
> As a starting point.
>
>
>
> Cheers.
>
>
>
> On 22-05-2017 12:36, Prakash Chauhan wrote:
>
> Hi All ,
>
>
>
> Need Help !!!
>
>
>
> *Setup Details:*
>
> Cassandra 2.0.14
>
> Geo Red setup
>
> · DC1 - 3 nodes
>
> · DC2 - 3 nodes
>
>
>
>
>
> We were trying to implement multiple network interfaces with Cassandra
> 2.0.14
>
> After doing all the steps mentioned in DataStax doc
> http://docs.datastax.com/en/archived/cassandra/2.0/
> cassandra/configuration/configMultiNetworks.html, we observed that nodes
> were not able to see each other (checked using nodetool status).
>
>
>
> To resolve this issue, we followed the comment
> 
> mentioned in the JIRA : CASSANDRA-9748
> 
>
>
>
> Exact steps that we followed are :
>
> 
>
> *1.   *Stop Cassandra
>
> *2.   *Add rule to “iptables” to forward all packets on the public
> interface to the private interface.
>
>
>
> COMMAND: # iptables -t nat -A PREROUTING -p tcp -m tcp -d 
> --dport 7000 -j DNAT --to-destination :7000
>
>
>
> *3.   *In Cassandra.yaml, add property “broadcast_address”.
>
> *4.   *In Cassandra.yaml, change “listen_address” to private IP.
>
> *5.   *Clear the data from directory “peers”.
>
> *6.   *Change Snitch to GossipingPropertyFileSnitch.
>
> *7.   *Append following property to the file 
> “/etc/cassandra/conf/cassandra-env.sh”
> to purge gossip state.
>
> JVM_OPTS="$JVM_OPTS -Dcassandra.load_ring_state=false"
>
>
>
> *8.   *Start Cassandra
>
> *9.   *After node has been started, remove following property from
> the file “/etc/cassandra/conf/cassandra-env.sh” (previously added in step
> 7)
>
> JVM_OPTS="$JVM_OPTS -Dcassandra.load_ring_state=false"
>
> *10.   *Delete file “/etc/cassandra/conf/cassandra-topology.properties”
>
>
>
>
>
> Now We have an observation that after multiple restarts of Cassandra on
> multiple nodes, slowness is observed in the cluster.
>
> The problem gets resolved when we revert the steps mentioned above.
>
>
>
> *Do u think there is any step that can cause the problem ?*
>
> We are suspecting Step 2(iptable rule) but not very sure about it.
>
>
>
>
>
> Regards,
>
> Prakash Chauhan.
>
>
>

-- 


--

Re: Cassandra Schema version mismatch

[Carlos Rolo]

Sometimes "resetlocalschema" (happened to me, didn't check why) will not
work, and you need to stop the offending nodes and bring them back one by
one.. That solved the issue.

In the cases I've seen this happened the clusters in question where either:

a) A couple of nodes down (cloud provider caused a partial outage) with an
application that would create/change tables every now and then
b) Massive GC pauses around the cluster with a schema change being tried
over and over.

CQLSH stopped working, your nodes died? GC Pauses? Operator mistake?

Regards,

Carlos Juzarte Rolo
Cassandra Consultant / Datastax Certified Architect / Cassandra MVP

Pythian - Love your data

rolo@pythian | Twitter: @cjrolo | Skype: cjr2k3 | Linkedin:
*linkedin.com/in/carlosjuzarterolo
*
Mobile: +351 918 918 100
www.pythian.com

On Sat, May 6, 2017 at 1:07 PM, Nitan Kainth  wrote:

> Thank you Jeff.
>
> James,
>
> We started getting insufficient replica errors. Cqlsh stopped working for
> two nodes .
>
> Sent from my iPhone
>
> > On May 5, 2017, at 7:13 PM, James Rothering 
> wrote:
> >
> > I've heard about this ... how did the problem present itself?
> >
> > Sent from my iPhone
> >
> >> On May 5, 2017, at 3:17 PM, Jeff Jirsa  wrote:
> >>
> >>
> >>
> >>> On 2017-05-05 11:00 (-0700), Nitan Kainth  wrote:
> >>> Hi Experts,
> >>>
> >>> We found schema version mismatch in our cluster. We fixed it by
> bouncing C* on nodes where version was mismatched. Can someone suggest,
> what are the possible reasons for this? We are trying to figure out the
> root cause.
> >>>
> >>
> >> Do all of your versions match? You didn't accidentally upgrade half the
> cluster did you?
> >>
> >>
> >>
> >> -
> >> To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
> >> For additional commands, e-mail: user-h...@cassandra.apache.org
> >>
> >
> > -
> > To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: user-h...@cassandra.apache.org
> >
>
> -
> To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: user-h...@cassandra.apache.org
>
>

-- 


--

Re: Cassandra Schema version mismatch

[Carlos Rolo]

Are you changing the schema in a dynamic fashion? If you get problems
(network, gc pauses, etc) during the schema changes it might lead to that.

Regards,

Carlos Juzarte Rolo
Cassandra Consultant / Datastax Certified Architect / Cassandra MVP

Pythian - Love your data

rolo@pythian | Twitter: @cjrolo | Skype: cjr2k3 | Linkedin:
*linkedin.com/in/carlosjuzarterolo
*
Mobile: +351 918 918 100
www.pythian.com

On Fri, May 5, 2017 at 7:00 PM, Nitan Kainth  wrote:

> Hi Experts,
>
> We found schema version mismatch in our cluster. We fixed it by bouncing
> C* on nodes where version was mismatched. Can someone suggest, what are the
> possible reasons for this? We are trying to figure out the root cause.
>
> thank you!
> -
> To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: user-h...@cassandra.apache.org
>
>

-- 


--

Re: cassandra OOM

[Carlos Rolo]

To add some contribution to this thread, we have seen both cases. CMS
easily outperforming G1 for the same Heapsize and the inverse too. On the
same cluster different workloads (datacenter based) we have both collectors
because of performance based on the workload.

It would be good to colect this information out and do a talk/blog, but for
a later time.

Regards,

Carlos Juzarte Rolo
Cassandra Consultant / Datastax Certified Architect / Cassandra MVP

Pythian - Love your data

rolo@pythian | Twitter: @cjrolo | Skype: cjr2k3 | Linkedin:
*linkedin.com/in/carlosjuzarterolo
*
Mobile: +351 918 918 100
www.pythian.com

On Tue, Apr 25, 2017 at 6:47 PM, Durity, Sean R  wrote:

> We have seen much better stability (and MUCH less GC pauses) from G1 with
> a variety of heap sizes. I don’t even consider CMS any more.
>
>
>
>
>
> Sean Durity
>
>
>
> *From:* Gopal, Dhruva [mailto:dhruva.go...@aspect.com]
> *Sent:* Tuesday, April 04, 2017 5:34 PM
> *To:* user@cassandra.apache.org
> *Subject:* Re: cassandra OOM
>
>
>
> Thanks, that’s interesting – so CMS is a better option for
> stability/performance? We’ll try this out in our cluster.
>
>
>
> *From: *Alexander Dejanovski 
> *Reply-To: *"user@cassandra.apache.org" 
> *Date: *Monday, April 3, 2017 at 10:31 PM
> *To: *"user@cassandra.apache.org" 
> *Subject: *Re: cassandra OOM
>
>
>
> Hi,
>
>
>
> we've seen G1GC going OOM on production clusters (repeatedly) with a 16GB
> heap when the workload is intense, and given you're running on m4.2xl I
> wouldn't go over 16GB for the heap.
>
>
>
> I'd suggest to revert back to CMS, using a 16GB heap and up to 6GB of new
> gen. You can use 5 as MaxTenuringThreshold as an initial value and activate
> GC logging to fine tune the settings afterwards.
>
>
>
> FYI CMS tends to perform better than G1 even though it's a little bit
> harder to tune.
>
>
>
> Cheers,
>
>
>
> On Mon, Apr 3, 2017 at 10:54 PM Gopal, Dhruva 
> wrote:
>
> 16 Gig heap, with G1. Pertinent info from jvm.options below (we’re using
> m2.2xlarge instances in AWS):
>
>
>
>
>
> #
>
> # HEAP SETTINGS #
>
> #
>
>
>
> # Heap size is automatically calculated by cassandra-env based on this
>
> # formula: max(min(1/2 ram, 1024MB), min(1/4 ram, 8GB))
>
> # That is:
>
> # - calculate 1/2 ram and cap to 1024MB
>
> # - calculate 1/4 ram and cap to 8192MB
>
> # - pick the max
>
> #
>
> # For production use you may wish to adjust this for your environment.
>
> # If that's the case, uncomment the -Xmx and Xms options below to override
> the
>
> # automatic calculation of JVM heap memory.
>
> #
>
> # It is recommended to set min (-Xms) and max (-Xmx) heap sizes to
>
> # the same value to avoid stop-the-world GC pauses during resize, and
>
> # so that we can lock the heap in memory on startup to prevent any
>
> # of it from being swapped out.
>
> -Xms16G
>
> -Xmx16G
>
>
>
> # Young generation size is automatically calculated by cassandra-env
>
> # based on this formula: min(100 * num_cores, 1/4 * heap size)
>
> #
>
> # The main trade-off for the young generation is that the larger it
>
> # is, the longer GC pause times will be. The shorter it is, the more
>
> # expensive GC will be (usually).
>
> #
>
> # It is not recommended to set the young generation size if using the
>
> # G1 GC, since that will override the target pause-time goal.
>
> # More info: http://www.oracle.com/technetwork/articles/java/
> g1gc-1984535.html
> 
>
> #
>
> # The example below assumes a modern 8-core+ machine for decent
>
> # times. If in doubt, and if you do not particularly want to tweak, go
>
> # 100 MB per physical CPU core.
>
> #-Xmn800M
>
>
>
> #
>
> #  GC SETTINGS  #
>
> #
>
>
>
> ### CMS Settings
>
>
>
> #-XX:+UseParNewGC
>
> #-XX:+UseConcMarkSweepGC
>
> #-XX:+CMSParallelRemarkEnabled
>
> #-XX:SurvivorRatio=8
>
> #-XX:MaxTenuringThreshold=1
>
> #-XX:CMSInitiatingOccupancyFraction=75
>
> #-XX:+UseCMSInitiatingOccupancyOnly
>
> #-XX:CMSWaitDuration=1
>
> #-XX:+CMSParallelInitialMarkEnabled
>
> #-XX:+CMSEdenChunksRecordAlways
>
> # some JVMs will fill up their heap when accessed via JMX, see
> CASSANDRA-6541
>
> #-XX:+CMSClassUnloadingEnabled
>
>
>
> ### G1 Settings (experimental, comment previous section and uncomment
> section below to enable)
>
>
>
> ## Use the Hotspot garbage-first collector.
>
> -XX:+UseG1GC
>
> #
>
> ## Have the JVM do less remembered set work during STW, instead
>
> ## preferring concurrent GC. Reduces p99.9 latency.
>
> -XX:G1RSetUpdatingPauseTimePercent=5
>
> #
>
> ## 

Re: Drop tables takes too long

[Carlos Rolo]

You have 4800 Tables in total? That is a lot of tables, plus MVs? or MVs
are already considered in the 60*80 account?

I would recommend to reduce the table number. Other thing is that you need
to check your log file for GC Pauses, and how long those pauses take.

You also might need to increase the node count if you're resource
constrained.

Regards,

Carlos Juzarte Rolo
Cassandra Consultant / Datastax Certified Architect / Cassandra MVP

Pythian - Love your data

rolo@pythian | Twitter: @cjrolo | Skype: cjr2k3 | Linkedin:
*linkedin.com/in/carlosjuzarterolo
*
Mobile: +351 918 918 100
www.pythian.com

On Thu, Apr 20, 2017 at 11:10 AM, Bohdan Tantsiura 
wrote:

> Hi,
>
> We are using cassandra 3.10 in a 10 nodes cluster with replication = 3.
> MAX_HEAP_SIZE=64GB on all nodes, G1 GC is used. We have about 60 keyspaces
> with about 80 tables in each keyspace. We had to delete three tables and
> two materialized views from each keyspace. It began to take more and more
> time for each next keyspace (for some keyspaces it took about 30 minutes)
> and then failed with "Cannot achieve consistency level ALL". After
> restarting the same repeated. It seems that cassandra hangs on GC. How that
> can be solved?
>
> Thanks
>

-- 


--

Re: WriteTimeoutException with LWT after few milliseconds

[Carlos Rolo]

You can try to use TRACING to debug the situation, but for a LWT to fail so
fast, the most probable cause is what you stated: "It is possible that
there are concurrent inserts on the same PK - actually thats the reason why
I use LWTs." AKA, someone inserted first.

Regards,

Carlos Juzarte Rolo
Cassandra Consultant / Datastax Certified Architect / Cassandra MVP

Pythian - Love your data

rolo@pythian | Twitter: @cjrolo | Skype: cjr2k3 | Linkedin:
*linkedin.com/in/carlosjuzarterolo
*
Mobile: +351 918 918 100
www.pythian.com

On Wed, Apr 12, 2017 at 3:51 PM, Roland Otta 
wrote:

> sorry .. ignore my comment ...
>
> i missed your comment that the record is in the table ...
>
> On Wed, 2017-04-12 at 16:48 +0200, Roland Otta wrote:
>
> Hi Benjamin,
>
> its unlikely that i can assist you .. but nevertheless ... i give it a try
> ;-)
>
> whats your consistency level for the insert?
> what if one ore more nodes are marked down and proper consistency cant be
> achieved?
> of course the error message does not indicate that problem (as it says its
> a timeout)... but in that case you would get an instant error for inserts.
> wouldn't you?
>
> br,
> roland
>
>
>
> On Wed, 2017-04-12 at 15:09 +0200, benjamin roth wrote:
>
> Hi folks,
>
> Can someone explain why that occurs?
>
> Write timeout after 0.006s
> Query: 'INSERT INTO log_moment_import ("source", "reference", "user_id",
> "moment_id", "date", "finished") VALUES (3, '1305821272790495', 65675537,
> 0, '2017-04-12 13:00:51', NULL) IF NOT EXISTS
> Primary key and parition key is source + reference
> Message: Operation timed out - received only 1 responses.
>
> This appears every now and then in the log. When I check the for the
> record in the table, it is there.
> I could explain that, if the WTE occured after the configured write
> timeout but it happens withing a few milliseconds.
> Is this caused by lock contention? It is possible that there are
> concurrent inserts on the same PK - actually thats the reason why I use
> LWTs.
>
> Thanks!
>
>

-- 


--

Re: too many compactions pending and compaction is slow on few tables

[Carlos Rolo]

Is not a good idea to do LCS on spinning. Change to STCS, and reduce the
compactors to 2 (if you have more than 2). Check if that helps.

On Apr 7, 2017 20:18, "Matija Gobec"  wrote:

> It does as the "new" data, even if the values are the same, has new write
> time timestamp.
> Spinning disks are hard to run LCS on. Do you maybe have some kind of non
> stripe raid in place?
>
> On Fri, Apr 7, 2017 at 8:46 PM, Giri P  wrote:
>
>> Does LCS try compacting already compacted files if it see same key loaded
>> again ?
>>
>> On Fri, Apr 7, 2017 at 11:39 AM, Giri P  wrote:
>>
>>> cassandra version : 2.1
>>> volume : initially loading 28 days worth of data around 1 TB and then we
>>>  process hourly
>>> load: only cassandra running on nodes
>>> disks: spinning disks
>>>
>>>
>>>
>>> On Fri, Apr 7, 2017 at 11:27 AM, Jonathan Haddad 
>>> wrote:
>>>
 What version of Cassandra? How much data? How often are you reloading
 it? Is compaction throttled? What disks are you using? Any other load on
 the machine?
 On Fri, Apr 7, 2017 at 11:19 AM Giri P  wrote:

> Hi,
>
> we are continuously loading a table which has properties properties
> compaction strategy LCS and bloom filter off and compactions are not
> catching up . Even the compaction is running slow on that table even after
> we increases throughput and concurrent compactors.
>
> Can someone point me to what I should be looking to tune this ?
>
> Thanks
> Giri
>

>>>
>>
>

-- 


--

Re: Node always dieing

[Carlos Rolo]

i3 are having those issues more than the other instances it seems. Not the
first report I heard about.

Regards,

Carlos Juzarte Rolo
Cassandra Consultant / Datastax Certified Architect / Cassandra MVP

Pythian - Love your data

rolo@pythian | Twitter: @cjrolo | Skype: cjr2k3 | Linkedin:
*linkedin.com/in/carlosjuzarterolo
*
Mobile: +351 918 918 100
www.pythian.com

On Thu, Apr 6, 2017 at 5:36 PM, Cogumelos Maravilha <
cogumelosmaravi...@sapo.pt> wrote:

> Yes but this time I going to give lots of time between killing and pickup.
> Thanks a lot.
>
>
> On 04/06/2017 05:31 PM, Avi Kivity wrote:
>
> Your disk is bad.  Kill that instance and hope someone else gets it.
>
> On 04/06/2017 07:27 PM, Cogumelos Maravilha wrote:
>
> Interesting
>
> [  720.693768] blk_update_request: I/O error, dev nvme0n1, sector
> 1397303056
> [  750.698840] blk_update_request: I/O error, dev nvme0n1, sector
> 1397303080
> [ 1416.202103] blk_update_request: I/O error, dev nvme0n1, sector
> 1397303080
>
> On 04/06/2017 05:26 PM, Avi Kivity wrote:
>
> Is there anything in dmesg?
>
> On 04/06/2017 07:25 PM, Cogumelos Maravilha wrote:
>
> Now dies and restart (systemd) without logging why
>
> system.log
>
> INFO  [Native-Transport-Requests-2] 2017-04-06 16:06:55,362
> AuthCache.java:172 - (Re)initializing RolesCache (validity period
> /update interval/max entries) (2000/2000/1000)
> INFO  [main] 2017-04-06 16:17:42,535 YamlConfigurationLoader.java:89 -
> Configuration location: file:/etc/cassandra/cassandra.
> yaml
>
> debug.log
> DEBUG [GossipStage:1] 2017-04-06 16:16:56,272 FailureDetector.java:457 -
> Ignoring interval time of 2496703934 for /10.100.120.52
> DEBUG [GossipStage:1] 2017-04-06 16:16:59,090 FailureDetector.java:457 -
> Ignoring interval time of 2818071981 for /10.100.120.161
> INFO  [main] 2017-04-06 16:17:42,535 YamlConfigurationLoader.java:89 -
> Configuration location: file:/etc/cassandra/cassandra.yaml
> DEBUG [main] 2017-04-06 16:17:42,540 YamlConfigurationLoader.java:108 -
> Loading settings from file:/etc/cassandra/cassandra.yaml
>
>
> On 04/06/2017 04:18 PM, Cogumelos Maravilha wrote:
>
> find */mnt/cassandra/* \! -user cassandra
> nothing
>
> I've found some "strange" solutions on Internet
> chmod -R 2777 /tmp
> chmod -R 2775 cassandra folder
>
> Lets give some time to see the result
>
>
> On 04/06/2017 03:14 PM, Michael Shuler wrote:
>
> All it takes is one frustrated `sudo cassandra` run. Checking only the
> top level directory ownership is insufficient, since root could own
> files/dirs created below the top level. Find all files not owned by user
> cassandra:  `find */mnt/cassandra/* \! -user cassandra`
>
> Just another thought.
>
> --
> Michael
>
>
> On 04/06/2017 05:23 AM, Cogumelos Maravilha wrote:
>
> From cassandra.yaml:
>
> hints_directory: /mnt/cassandra/hints
> data_file_directories:
> - /mnt/cassandra/data
> commitlog_directory: /mnt/cassandra/commitlog
> saved_caches_directory: /mnt/cassandra/saved_caches
>
> drwxr-xr-x   3 cassandra cassandra   23 Apr  5 16:03 mnt/
>
> drwxr-xr-x 6 cassandra cassandra  68 Apr  5 16:17 ./
> drwxr-xr-x 3 cassandra cassandra  23 Apr  5 16:03 ../
> drwxr-xr-x 2 cassandra cassandra  80 Apr  6 10:07 commitlog/
> drwxr-xr-x 8 cassandra cassandra 124 Apr  5 16:17 data/
> drwxr-xr-x 2 cassandra cassandra  72 Apr  5 16:20 hints/
> drwxr-xr-x 2 cassandra cassandra  49 Apr  5 20:17 saved_caches/
>
> cassand+  2267 1 99 10:18 ?00:02:56 java
> -Xloggc:/var/log/cassandra/gc.log -ea -XX:+UseThreadPriorities -XX:Threa...
>
> /dev/mapper/um_vg-xfs_lv  885G   27G  858G   4% /mnt
>
> On /etc/security/limits.conf
>
> *   -   memlock  unlimited
> *   -  nofile  10
> *   -  nproc  32768
> *   -  as   unlimited
>
> On /etc/security/limits.d/cassandra.conf
>
> cassandra  -  memlock  unlimited
> cassandra  -  nofile   10
> cassandra  -  as   unlimited
> cassandra  -  nproc32768
>
> On /etc/sysctl.conf
>
> vm.max_map_count = 1048575
>
> On /etc/systcl.d/cassanda.conf
>
> vm.max_map_count = 1048575
> net.ipv4.tcp_keepalive_time=600
>
> On /etc/pam.d/su
> ...
> sessionrequired   pam_limits.so
> ...
>
> Distro is the currently Ubuntu LTS.
> Thanks
>
>
> On 04/06/2017 10:39 AM, benjamin roth wrote:
>
> Cassandra cannot write an SSTable to disk. Are you sure the
> disk/volume where SSTables reside (normally /var/lib/cassandra/data)
> is writeable for the CS user and has enough free space?
> The CDC warning also implies that.
> The other warnings indicate you are probably not running CS as root
> and you did not set an appropriate limit for max open files. Running
> out of open files can also be a reason for the IO error.
>
> 2017-04-06 11:34 GMT+02:00 Cogumelos Maravilha
>  
> >:
>
> Hi list,
>
> I'm using C* 3.10 in a 6 nodes cluster RF=2. All 

Re: Node always dieing

[Carlos Rolo]

There was some issue with the i3 instances and Cassandra. Did you had this
cluster running always on i3?

On Apr 6, 2017 13:06, "Cogumelos Maravilha" 
wrote:

> Limit Soft Limit   Hard Limit
> Units
> Max cpu time  unlimitedunlimited
> seconds
> Max file size unlimitedunlimited
> bytes
> Max data size unlimitedunlimited
> bytes
> Max stack size8388608  unlimited
> bytes
> Max core file size0unlimited
> bytes
> Max resident set  unlimitedunlimited
> bytes
> Max processes 122575   122575
> processes
> Max open files10   10
> files
> Max locked memory unlimitedunlimited
> bytes
> Max address space unlimitedunlimited
> bytes
> Max file locksunlimitedunlimited
> locks
> Max pending signals   122575   122575
> signals
> Max msgqueue size 819200   819200
> bytes
> Max nice priority 00
> Max realtime priority 00
> Max realtime timeout  unlimitedunlimitedus
> Please find something wrong there!
>
> Thanks.
>
> On 04/06/2017 11:50 AM, benjamin roth wrote:
>
> Limits: You should check them in /proc/$pid/limits
>
> 2017-04-06 12:48 GMT+02:00 Cogumelos Maravilha  >:
>
>> Yes C* is running as cassandra:
>>
>> cassand+  2267 1 99 10:18 ?00:02:56 java
>> -Xloggc:/var/log/cassandra/gc.log -ea -XX:+UseThreadPriorities
>> -XX:Threa...
>>
>> INFO  [main] 2017-04-06 10:35:42,956 Config.java:474 - Node
>> configuration:[allocate_tokens_for_keyspace=null;
>> authenticator=PasswordAuthenticator; authorizer=CassandraAuthorizer;
>> auto_bootstrap=true; auto_snapshot=true; back_pressure_enabled=false;
>> back_pressure_strategy=org.apache.cassandra.net.RateBasedBackPressure{high_ratio=0.9,
>> factor=5, flow=FAST}; batch_size_fail_threshold_in_kb=50;
>> batch_size_warn_threshold_in_kb=5; batchlog_replay_throttle_in_kb=1024;
>> broadcast_address=null; broadcast_rpc_address=null;
>> buffer_pool_use_heap_if_exhausted=true; cas_contention_timeout_in_ms=600;
>> cdc_enabled=false; cdc_free_space_check_interval_ms=250;
>> cdc_raw_directory=null; cdc_total_space_in_mb=0;
>> client_encryption_options=; cluster_name=company;
>> column_index_cache_size_in_kb=2; column_index_size_in_kb=64;
>> commit_failure_policy=ignore; commitlog_compression=null;
>> commitlog_directory=/mnt/cassandra/commitlog;
>> commitlog_max_compression_buffers_in_pool=3;
>> commitlog_periodic_queue_size=-1; commitlog_segment_size_in_mb=32;
>> commitlog_sync=periodic; commitlog_sync_batch_window_in_ms=NaN;
>> commitlog_sync_period_in_ms=1; commitlog_total_space_in_mb=null;
>> compaction_large_partition_warning_threshold_mb=100;
>> compaction_throughput_mb_per_sec=16; concurrent_compactors=null;
>> concurrent_counter_writes=32; concurrent_materialized_view_writes=32;
>> concurrent_reads=32; concurrent_replicates=null; concurrent_writes=32;
>> counter_cache_keys_to_save=2147483647; counter_cache_save_period=7200;
>> counter_cache_size_in_mb=null; counter_write_request_timeout_in_ms=600;
>> credentials_cache_max_entries=1000; credentials_update_interval_in_ms=-1;
>> credentials_validity_in_ms=2000; cross_node_timeout=false;
>> data_file_directories=[Ljava.lang.String;@223f3642;
>> disk_access_mode=auto; disk_failure_policy=ignore;
>> disk_optimization_estimate_percentile=0.95;
>> disk_optimization_page_cross_chance=0.1; disk_optimization_strategy=ssd;
>> dynamic_snitch=true; dynamic_snitch_badness_threshold=0.1;
>> dynamic_snitch_reset_interval_in_ms=60;
>> dynamic_snitch_update_interval_in_ms=100; 
>> enable_scripted_user_defined_functions=false;
>> enable_user_defined_functions=false; 
>> enable_user_defined_functions_threads=true;
>> encryption_options=null; endpoint_snitch=SimpleSnitch;
>> file_cache_size_in_mb=null; gc_log_threshold_in_ms=200;
>> gc_warn_threshold_in_ms=1000; hinted_handoff_disabled_datacenters=[];
>> hinted_handoff_enabled=true; hinted_handoff_throttle_in_kb=1024;
>> hints_compression=null; hints_directory=/mnt/cassandra/hints;
>> hints_flush_period_in_ms=1; incremental_backups=false;
>> index_interval=null; index_summary_capacity_in_mb=null;
>> index_summary_resize_interval_in_minutes=60; initial_token=null;
>> inter_dc_stream_throughput_outbound_megabits_per_sec=200;
>> inter_dc_tcp_nodelay=false; internode_authenticator=null;
>> internode_compression=dc; internode_recv_buff_size_in_bytes=0;
>> internode_send_buff_size_in_bytes=0; key_cache_keys_to_save=2147483647;
>> key_cache_save_period=14400; key_cache_size_in_mb=null;
>> listen_address=10.100.100.213; listen_interface=null;
>> listen_interface_prefer_ipv6=false; listen_on_broadcast_address=false;
>> 

Re: Archive node

[Carlos Rolo]

I would not suggest to do that, because the new "Archive" node would be a
new DC that you would need to build (Operational wise).

You could also snapshot the old one once it finishes and use SSTableloader
to push it into your Development DC. This way you have isolation from
Production. Plus no operational overhead.

Regards,

Carlos Juzarte Rolo
Cassandra Consultant / Datastax Certified Architect / Cassandra MVP

Pythian - Love your data

rolo@pythian | Twitter: @cjrolo | Skype: cjr2k3 | Linkedin:
*linkedin.com/in/carlosjuzarterolo
*
Mobile: +351 918 918 100
www.pythian.com

On Mon, Mar 6, 2017 at 10:43 AM, Gábor Auth  wrote:

> Hi,
>
> The background story: we are developing an MMO strategy game and every two
> week the game world ends and we are starting a new one with a slightly
> different new database scheme. So that, we have over ~100 keyspaces in our
> cluster and we want to archive the old schemes into a separated Cassandra
> node or something else to be available online for support of development.
> The archived keyspace is mostly read-only and rarely used (~once a year or
> less often).
>
> We've two DC Cassandra cluster with 4-4 nodes, the idea is the following:
> we add a new Cassandra node with DC name 'Archive' and change the
> replication factor of old keyspaces from {'class':
> 'NetworkTopologyStrategy', 'DC01': '3', 'DC02': '3'} to {'class':
> 'NetworkTopologyStrategy', 'Archive': '1'}, and repair the keyspace.
>
> What do you think? Any other idea? :)
>
> Bye,
> Gábor Auth
>
>

-- 


--

Re: Read exceptions after upgrading to 3.0.10

[Carlos Rolo]

By any chances are you using the PHP/C++ driver?

-- 


--

Re: Global TTL vs Insert TTL

[Carlos Rolo]

Awsome to know this!

Thanks Jon and DuyHai!

Regards,

Carlos Juzarte Rolo
Cassandra Consultant / Datastax Certified Architect / Cassandra MVP

Pythian - Love your data

rolo@pythian | Twitter: @cjrolo | Skype: cjr2k3 | Linkedin:
*linkedin.com/in/carlosjuzarterolo
<http://linkedin.com/in/carlosjuzarterolo>*
Mobile: +351 918 918 100
www.pythian.com

On Wed, Feb 1, 2017 at 6:57 PM, Jonathan Haddad <j...@jonhaddad.com> wrote:

> The optimization is there.  The entire sstable can be dropped but it's not
> because of the default TTL.  The default TTL only applies if a TTL isn't
> specified explicitly.  The default TTL can't be used to drop a table
> automatically since it can be overridden at insert time.  Check out this
> example.  The first insert uses the default TTL.  The second insert
> overrides the default.  Using the default TTL to drop the sstable would be
> pretty terrible in this case:
>
> CREATE TABLE test.b (
> k int PRIMARY KEY,
> v int
> ) WITH default_time_to_live = 1;
>
> insert into b (k, v) values (1, 1);
> cqlsh:test> select k, v, TTL(v) from b  where k = 1;
>
>  k | v | ttl(v)
> ---+---+
>  1 | 1 |   9943
>
> (1 rows)
>
> cqlsh:test> insert into b (k, v) values (2, 1) USING TTL ;
> cqlsh:test> select k, v, TTL(v) from b  where k = 2;
>
>  k | v | ttl(v)
> ---+---+--
>  2 | 1 | 9995
>
> (1 rows)
>
> TL;DR: The default TTL is there as a convenience so you don't have to keep
> the TTL in your code.  From a performance perspective it does not matter.
>
> Jon
>
>
> On Wed, Feb 1, 2017 at 10:39 AM DuyHai Doan <doanduy...@gmail.com> wrote:
>
>> I was referring to this JIRA https://issues.apache.
>> org/jira/browse/CASSANDRA-3974 when talking about dropping entire
>> SSTable at compaction time
>>
>> But the JIRA is pretty old and it is very possible that the optimization
>> is no longer there
>>
>>
>>
>> On Wed, Feb 1, 2017 at 6:53 PM, Jonathan Haddad <j...@jonhaddad.com>
>> wrote:
>>
>> This is incorrect, there's no optimization used that references the table
>> level TTL setting.   The max local deletion time is stored in table
>> metadata.  See 
>> org.apache.cassandra.io.sstable.metadata.StatsMetadata#maxLocalDeletionTime
>> in the Cassandra 3.0 branch.The default ttl is stored
>> here: org.apache.cassandra.schema.TableParams#defaultTimeToLive and is
>> never referenced during compaction.
>>
>> Here's an example from a table I created without a default TTL, you can
>> use the sstablemetadata tool to see:
>>
>> jhaddad@rustyrazorblade ~/dev/cassandra/data/data/test$
>> ../../../tools/bin/sstablemetadata a-7bca6b50e8a511e6869a5596edf4dd
>> 35/mc-1-big-Data.db
>> .
>> SSTable max local deletion time: 1485980862
>>
>> On Wed, Feb 1, 2017 at 6:59 AM DuyHai Doan <doanduy...@gmail.com> wrote:
>>
>> Global TTL is better than dynamic runtime TTL
>>
>> Why ?
>>
>>  Because Global TTL is a table property and Cassandra can perform
>> optimization when compacting.
>>
>> For example if it can see than the maxTimestamp of an SSTable is older
>> than the table Global TTL, the SSTable can be entirely dropped during
>> compaction
>>
>> Using dynamic TTL at runtime, since Cassandra doesn't how and cannot
>> track each individual TTL value, the previous optimization is not possible
>> (even if you always use the SAME TTL for all query, Cassandra is not
>> supposed to know that)
>>
>>
>>
>> On Wed, Feb 1, 2017 at 3:01 PM, Cogumelos Maravilha <
>> cogumelosmaravi...@sapo.pt> wrote:
>>
>> Thank you all, for your answers.
>>
>> On 02/01/2017 01:06 PM, Carlos Rolo wrote:
>>
>> To reinforce Alain statement:
>>
>> "I would say that the unsafe part is more about using C* 3.9" this is
>> key. You would be better on 3.0.x unless you need features on the 3.x
>> series.
>>
>> Regards,
>>
>> Carlos Juzarte Rolo
>> Cassandra Consultant / Datastax Certified Architect / Cassandra MVP
>>
>> Pythian - Love your data
>>
>> rolo@pythian | Twitter: @cjrolo | Skype: cjr2k3 | Linkedin:
>> *linkedin.com/in/carlosjuzarterolo
>> <http://linkedin.com/in/carlosjuzarterolo>*
>> Mobile: +351 918 918 100 <+351%20918%20918%20100>
>> www.pythian.com
>>
>> On Wed, Feb 1, 2017 at 8:32 AM, Alain RODRIGUEZ <arodr...@gmail.com>
>> wrote:
>>
>> Is it safe to use TWCS in C* 3.9?
>>
>>
>> I would say that the unsafe part is mo

Re: Global TTL vs Insert TTL

[Carlos Rolo]

To reinforce Alain statement:

"I would say that the unsafe part is more about using C* 3.9" this is key.
You would be better on 3.0.x unless you need features on the 3.x series.

Regards,

Carlos Juzarte Rolo
Cassandra Consultant / Datastax Certified Architect / Cassandra MVP

Pythian - Love your data

rolo@pythian | Twitter: @cjrolo | Skype: cjr2k3 | Linkedin:
*linkedin.com/in/carlosjuzarterolo
*
Mobile: +351 918 918 100
www.pythian.com

On Wed, Feb 1, 2017 at 8:32 AM, Alain RODRIGUEZ  wrote:

> Is it safe to use TWCS in C* 3.9?
>
>
> I would say that the unsafe part is more about using C* 3.9 than using
> TWCS in C*3.9 :-). I see no reason to say 3.9 would be specifically unsafe
> in C*3.9, but I might be missing something.
>
> Going from STCS to TWCS is often smooth, from LCS you might expect an
> extra load compacting a lot (all?) of the SSTable from what we saw from the
> field. In this case, be sure that your compaction options are safe enough
> to handle this.
>
> TWCS is even easier to use on C*3.0.8+ and C*3.8+ as it became the new
> default replacing TWCS, so no extra jar is needed, you can enable TWCS as
> any other default compaction strategy.
>
> C*heers,
> ---
> Alain Rodriguez - @arodream - al...@thelastpickle.com
> France
>
> The Last Pickle - Apache Cassandra Consulting
> http://www.thelastpickle.com
>
> 2017-01-31 23:29 GMT+01:00 Cogumelos Maravilha  >:
>
>> Hi Alain,
>>
>> Thanks for your response and the links.
>>
>> I've also checked "Time series data model and tombstones".
>>
>> Is it safe to use TWCS in C* 3.9?
>>
>> Thanks in advance.
>>
>> On 31-01-2017 11:27, Alain RODRIGUEZ wrote:
>>
>> Is there a overhead using line by line option or wasted disk space?
>>>
>>>  There is a very recent topic about that in the mailing list, look for "Time
>> series data model and tombstones". I believe DuyHai answer your question
>> there with more details :).
>>
>> *tl;dr:*
>>
>> Yes, if you know the TTL in advance, and it is fixed, you might want to
>> go with the table option instead of adding the TTL in each insert. Also you
>> might want consider using TWCS compaction strategy.
>>
>> Here are some blogposts my coworkers recently wrote about TWCS, it might
>> be useful:
>>
>> http://thelastpickle.com/blog/2016/12/08/TWCS-part1.html
>> http://thelastpickle.com/blog/2017/01/10/twcs-part2.html
>>
>> C*heers,
>> ---
>> Alain Rodriguez - @arodream - al...@thelastpickle.com
>> France
>>
>> The Last Pickle - Apache Cassandra Consulting
>> http://www.thelastpickle.com
>>
>>
>>
>> 2017-01-31 10:43 GMT+01:00 Cogumelos Maravilha <
>> cogumelosmaravi...@sapo.pt>:
>>
>>> Hi I'm just wondering what option is fastest:
>>>
>>> Global:*create table xxx (.**AND **default_time_to_live = **XXX**;**
>>> and**UPDATE xxx USING TTL XXX;*
>>>
>>> Line by line:
>>> *INSERT INTO xxx (...** USING TTL xxx;*
>>>
>>> Is there a overhead using line by line option or wasted disk space?
>>>
>>> Thanks in advance.
>>>
>>>
>>
>>
>

-- 


--

Re: Has anyone deployed a production cluster with less than 6 nodes per DC?

[Carlos Rolo]

It depends on a lot of factors.

What causes the cluster to get crazy? I/O, Network, CPU?

I manage clusters of all sizes (even 3 nodes per DC) but it all depends on
usage and configuration.

Regards,

Carlos

Regards,

Carlos Juzarte Rolo
Cassandra Consultant / Datastax Certified Architect / Cassandra MVP

Pythian - Love your data

rolo@pythian | Twitter: @cjrolo | Skype: cjr2k3 | Linkedin:
*linkedin.com/in/carlosjuzarterolo
*
Mobile: +351 918 918 100
www.pythian.com

On Mon, Dec 26, 2016 at 9:26 PM, Ney, Richard 
wrote:

> My company has a product we’re about to deploy into AWS with Cassandra
> setup as a two 3 node clusters in two availability zones (m4.2xlarge with 2
> 500GB EBS volumes per node). We’re doing over a million writes per hour
> with the cluster setup with R-2 and local quorum writes. We run
> successfully for several hours before Cassandra goes into the weeds and we
> start getting write timeouts to the point we must kill the Cassandra JVM
> processes to get the Cassandra cluster to restart. I keep raising to my
> upper management that the cluster is severely undersized but management is
> complaining that setting up 12 nodes is too expensive and to change the
> code to reduce load on Cassandra.
>
>
>
> So, the main question is “Is there any hope of success with a 3 node DC
> setup of Cassandra in production or are we on a fool’s errand?”
>
>
>
> *RICHARD NEY*
>
> TECHNICAL DIRECTOR, RESEARCH & DEVELOPMENT
>
> *+1 (978) 848.6640 <+1%20978-848-6640>* WORK
>
> *+1 (916) 846.2353 <+1%20916-846-2353> *MOBILE
>
> *UNITED STATES*
>
> *richard@aspect.com *
>
> *aspect.com *
>
>
>
> [image: mailSigLogo-rev.jpg]
> This email (including any attachments) is proprietary to Aspect Software,
> Inc. and may contain information that is confidential. If you have received
> this message in error, please do not read, copy or forward this message.
> Please notify the sender immediately, delete it from your system and
> destroy any copies. You may not further disclose or distribute this email
> or its attachments.
>

-- 


--

Re: Join_ring=false Use Cases

[Carlos Rolo]

Beware the Java Driver limitations around whitelisting IPs.

Works fine in Python.



Regards,

Carlos Juzarte Rolo
Cassandra Consultant / Datastax Certified Architect / Cassandra MVP

Pythian - Love your data

rolo@pythian | Twitter: @cjrolo | Skype: cjr2k3 | Linkedin:
*linkedin.com/in/carlosjuzarterolo
*
Mobile: +351 918 918 100
www.pythian.com

On Tue, Dec 20, 2016 at 9:25 PM, Matija Gobec  wrote:

> There is a talk from cassandra summit 2016 about coordinator nodes by Eric
> Lubow from SimpleReach. He explains how you can use that join_ring=false.
>
> On Tue, Dec 20, 2016 at 10:23 PM, kurt Greaves 
> wrote:
>
>> It seems that you're correct in saying that writes don't propagate to a
>> node that has join_ring set to false, so I'd say this is a flaw. In reality
>> I can't see many actual use cases in regards to node outages with the
>> current implementation. The main usage I'd think would be to have
>> additional coordinators for CPU heavy workloads.
>>
>> It seems to make it actually useful for repairs/outages we'd need to have
>> another option to turn on writes so that it behaved similarly to write
>> survey mode (but on already bootstrapped nodes).
>>
>> Is there a reason we don't have this already? Or does it exist somewhere
>> I'm not aware of?
>>
>> On 20 December 2016 at 17:40, Anuj Wadehra 
>> wrote:
>>
>>> No responses yet :)
>>>
>>> Any C* expert who could help on join_ring use case and the concern
>>> raised?
>>>
>>> Thanks
>>> Anuj
>>>
>>> On Tue, 13 Dec, 2016 at 11:31 PM, Anuj Wadehra
>>>  wrote:
>>> Hi,
>>>
>>> I need to understand the use case of join_ring=false in case of node
>>> outages. As per https://issues.apache.org/jira/browse/CASSANDRA-6961,
>>> you would want join_ring=false when you have to repair a node before
>>> bringing a node back after some considerable outage. The problem I see with
>>> join_ring=false is that unlike autobootstrap, the node will NOT accept
>>> writes while you are running repair on it. If a node was down for 5 hours
>>> and you bring it back with join_ring=false, repair the node for 7 hours and
>>> then make it join the ring, it will STILL have missed writes because while
>>> the time repair was running (7 hrs), writes only went to other others.
>>> So, if you want to make sure that reads served by the restored node at CL
>>> ONE will return consistent data after the node has joined, you wont get
>>> that as writes have been missed while the node is being repaired. And if
>>> you work with Read/Write CL=QUORUM, even if you bring back the node without
>>> join_ring=false, you would anyways get the desired consistency. So, how
>>> join_ring would provide any additional consistency in this case ??
>>>
>>> I can see join_ring=false useful only when I am restoring from Snapshot
>>> or bootstrapping and there are dropped mutations in my cluster which are
>>> not fixed by hinted handoff.
>>>
>>> For Example: 3 nodes A,B,C working at Read/Write CL QUORUM. Hinted
>>> Handoff=3 hrs.
>>> 10 AM Snapshot taken on all 3 nodes
>>> 11 AM: Node B goes down for 4 hours
>>> 3 PM: Node B comes up but data is not repaired. So, 1 hr of dropped
>>> mutations (2-3 PM) not fixed via Hinted Handoff.
>>> 5 PM: Node A crashes.
>>> 6 PM: Node A restored from 10 AM Snapshot, Node A started with
>>> join_ring=false, repaired and then joined the cluster.
>>>
>>> In above restore snapshot example, updates from 2-3 PM were outside
>>> hinted handoff window of 3 hours. Thus, node B wont get those updates.
>>> Node A data for 2-3 PM is already lost. So, 2-3 PM updates are only on one
>>> replica i.e. node C and minimum consistency needed is QUORUM so
>>> join_ring=false would help. But this is very specific use case.
>>>
>>> Thanks
>>> Anuj
>>>
>>>
>>
>

-- 


--

Re: Failure when setting up cassandra in cluster

[Carlos Rolo]

If Ryan answer doesn't help, post Cassandra version. There is a bug with
cql and some python version that would lead to that error.

Also, please post "nodetool status".

Regards,

Carlos Juzarte Rolo
Cassandra Consultant / Datastax Certified Architect / Cassandra MVP

Pythian - Love your data

rolo@pythian | Twitter: @cjrolo | Skype: cjr2k3 | Linkedin:
*linkedin.com/in/carlosjuzarterolo
*
Mobile: +351 918 918 100
www.pythian.com

On Mon, Aug 22, 2016 at 4:08 PM, Ryan Svihla  wrote:

> instead of 127.0.0.1 have you tried just passing the IP of the one of the
> nodes.
>
> On Mon, Aug 22, 2016 at 9:45 AM Raimund Klein 
> wrote:
>
>> Hello all,
>>
>> Sorry for reposting this, but I didn't receive any response. Can someone
>> help please?
>>
>> -- Forwarded message --
>> From: Raimund Klein 
>> Date: 2016-08-15 12:07 GMT+01:00
>> Subject: Failure when setting up cassandra in cluster
>> To: user@cassandra.apache.org
>>
>>
>> Hi all,
>>
>> Sorry if this is a fairly stupid question, but we've all only been
>> exposed to Cassandra very recently.
>>
>> We're trying to configure a 2-node cluster with non-default credentials.
>> Here's what I've been doing so far based on my understanding of the
>> documentation. The platform is RHEL 7:
>>
>>
>>1. Use an RPM I found with Datastax to perform a basic cassandra
>>installation.
>>2. Change the temporary directory in cassandra-env.sh, because nobody
>>is allowed to execute anything in /tmp.
>>3. In cassandra.yaml,
>>- change the cluster_name
>>- empty the listen_address entry
>>- define both VMs as seeds
>>4. Open port 7000 in the firewall.
>>5. Start cassandra.
>>6. In the cassandra.yaml, change to PasswordAuthenticator.
>>7. Run cqlsh -u cassandra -p cassandra -e "ALTER KEYSPACE system_auth
>>WITH REPLICATION = { 'class' : 'SimpleStrategy', 'replication_factor' : 2
>>};"
>>8. Restart cassandra
>>9. Perform 1-8 on the second node
>>10. To create a new user, run cqlsh -u cassandra -p cassandra
>>-e "CREATE USER ${CASSANDRA_USERNAME} WITH PASSWORD 
>> '${CASSANDRA_PASSWORD}'
>>SUPERUSER;"
>>
>> Step 10 fails with this error:
>>
>> Connection error: ('Unable to connect to any servers', {'127.0.0.1':
>> AuthenticationFailed(u'Failed to authenticate to 127.0.0.1: code=0100
>> [Bad credentials] 
>> message="org.apache.cassandra.exceptions.UnavailableException:
>> Cannot achieve consistency level QUORUM"',)})
>>
>>
>> What am I missing?
>>
>>
>> Cheers
>>
>> Raimund
>>
>>
>> --
> Regards,
>
> Ryan Svihla
>

-- 


--

Re: Support/Consulting companies

[Carlos Rolo]

https://www.pythian.com/

Regards,

Carlos Juzarte Rolo
Cassandra Consultant / Datastax Certified Architect / Cassandra MVP

Pythian - Love your data

rolo@pythian | Twitter: @cjrolo | Skype: cjr2k3 | Linkedin:
*linkedin.com/in/carlosjuzarterolo
*
Mobile: +351 918 918 100
www.pythian.com

On Sat, Aug 20, 2016 at 1:57 AM, Jim Ancona  wrote:

> There's also a list of companies that provide Cassandra-related services
> on the wiki:
>
> https://wiki.apache.org/cassandra/ThirdPartySupport
>
> Jim
>
> On Fri, Aug 19, 2016 at 3:37 PM, Chris Tozer 
> wrote:
>
>> Instaclustr ( Instaclustr.com ) also offers Cassandra consulting
>>
>>
>> On Friday, August 19, 2016,  wrote:
>>
>>> Yes, TLP is the place to go!
>>>
>>> Joe
>>>
>>> Sent from my iPhone
>>>
>>> On Aug 19, 2016, at 12:03 PM, Huang, Roger  wrote:
>>>
>>> http://thelastpickle.com/
>>>
>>>
>>>
>>>
>>>
>>> *From:* Roxy Ubi [mailto:roxy...@gmail.com]
>>> *Sent:* Friday, August 19, 2016 2:02 PM
>>> *To:* user@cassandra.apache.org
>>> *Subject:* Support/Consulting companies
>>>
>>>
>>>
>>> Howdy,
>>>
>>> I'm looking for a list of support or consulting companies that provide
>>> contracting services related to Cassandra.  Is there a comprehensive list
>>> somewhere?  Alternatively could you folks tell me who you use?
>>>
>>> Thanks in advance for any replies!
>>>
>>> Roxy
>>>
>>>
>>
>> --
>> Chris Tozer
>>
>> Instaclustr
>>
>> (408) 781-7914
>>
>> Spin Up a Free 14 Day Trial 
>>
>>
>

-- 


--

Re: IF EXISTS checks on all nodes?

[Carlos Rolo]

Hello,

As far as I know, lightweight transactions only apply to a single
partition, so in your case it will only execute on the nodes responsible
for that partition. And as a consequence, those nodes will all be in the
same state when the transaction ends (If it would apply).

Please refer to this blog post for more information about the LIghtweigth
transactions:
http://www.datastax.com/dev/blog/lightweight-transactions-in-cassandra-2-0



Regards,

Carlos Juzarte Rolo
Cassandra Consultant / Datastax Certified Architect / Cassandra MVP

Pythian - Love your data

rolo@pythian | Twitter: @cjrolo | Skype: cjr2k3 | Linkedin:
*linkedin.com/in/carlosjuzarterolo
*
Mobile: +351 918 918 100
www.pythian.com

On Thu, May 12, 2016 at 12:17 PM, Siddharth Verma <
verma.siddha...@snapdeal.com> wrote:

> Hi,
> If i have inconsistent data on nodes
> Scenario :
> I have 2 DCs each with 3 nodes
> and I have inconsistent data on them
>
> node 1,2,3,4,5 have
> P1,100,A,val1,w1
> P1,100,B,val2,w2
>
> node 6 has
> P1,100,A,val1,w1
> P1,100,B,val2,w2
> P1,200,C,val3,w3
> P1,200,D,val4,w4
>
> col1, col2, col3,col4,col5 in table
> Primary key (col1, col2, col3)
>
> Now i execute the query from CQLSH
> update mykeyspace.my_table_1 set col5 = 'w_x' where col1='P1' and col2=200
> and col3='C' IF EXISTS;
>
> Is it possible that
> node 1,2,3,4,5 will get the entry
> P1,200,C,null,w_x
>
> I.e. IF EXISTS is checked per node or only once and then execute on all?
>
> Thanks
> Siddharth Verma
>

-- 


--

Re: COPY TO export fails with

[Carlos Rolo]

Hello,

That is a lot of data to do an "COPY TO.

If you want a fast way to export, and you're fine with Java, you can use
Cassandra SSTableReader classes to read the sstables directly. Spark also
works.

Regards,

Carlos Juzarte Rolo
Cassandra Consultant / Datastax Certified Architect / Cassandra MVP

Pythian - Love your data

rolo@pythian | Twitter: @cjrolo | Skype: cjr2k3 | Linkedin:
*linkedin.com/in/carlosjuzarterolo
*
Mobile: +351 918 918 100
www.pythian.com

On Tue, May 10, 2016 at 9:33 AM, Matthias Niehoff <
matthias.nieh...@codecentric.de> wrote:

> sry, sent early..
>
> more errors:
>
> /export.cql:9:Error for (4549395184516451179, 4560441269902768904): 
> NoHostAvailable - ('Unable to complete the operation against any hosts', 
> {: ConnectionException('Host has been marked down 
> or removed',)}) (will try again later attempt 1 of 5)
> /export.cql:9:Error for (-2083690356124961461, -2068514534992400755): 
> NoHostAvailable - ('Unable to complete the operation against any hosts', {}) 
> (will try again later attempt 1 of 5)
> /export.cql:9:Error for (-4899866517058128956, -4897773268483324406): 
> NoHostAvailable - ('Unable to complete the operation against any hosts', {}) 
> (will try again later attempt 1 of 5)
> /export.cql:9:Error for (-1435092096023471089, -1434747957681478442): 
> NoHostAvailable - ('Unable to complete the operation against any hosts', {}) 
> (will try again later attempt 1 of 5)
> /export.cql:9:Error for (-2804962318029794069, -2783747272192843127): 
> NoHostAvailable - ('Unable to complete the operation against any hosts', {}) 
> (will try again later attempt 1 of 5)
> /export.cql:9:Error for (-5188633782964403059, -5149722481923709224): 
> NoHostAvailable - (‚Unable to complete the operation against any hosts', {}) 
> (will try again later attempt 1 of 5)
>
>
>
> It looks like the cluster can not handle export and the nodes cannot handle 
> the export.
>
> Is the cqlsh copy able to export this amount of data? or should other methods 
> be used (sstableloader, some custom code, spark…)
>
>
> Best regards
>
>
> 2016-05-10 10:29 GMT+02:00 Matthias Niehoff <
> matthias.nieh...@codecentric.de>:
>
>> Hi,
>>
>> i try to export data of a table (~15GB) using the cqlsh copy to. It fails
>> with „no host available“. If I try it with a smaller table everything works
>> fine.
>>
>> The statistics of the big table:
>>
>> SSTable count: 81
>> Space used (live): 14102945336
>> Space used (total): 14102945336
>> Space used by snapshots (total): 62482577
>> Off heap memory used (total): 16399540
>> SSTable Compression Ratio: 0.1863544514417909
>> Number of keys (estimate): 5034845
>> Memtable cell count: 5590
>> Memtable data size: 18579542
>> Memtable off heap memory used: 0
>> Memtable switch count: 72
>> Local read count: 0
>> Local read latency: NaN ms
>> Local write count: 139878
>> Local write latency: 0.023 ms
>> Pending flushes: 0
>> Bloom filter false positives: 0
>> Bloom filter false ratio: 0.0
>> Bloom filter space used: 6224240
>> Bloom filter off heap memory used: 6223592
>> Index summary off heap memory used: 1098860
>> Compression metadata off heap memory used: 9077088
>> Compacted partition minimum bytes: 373
>> Compacted partition maximum bytes: 1358102
>> Compacted partition mean bytes: 16252
>> Average live cells per slice (last five minutes): 0.0
>> Maximum live cells per slice (last five minutes): 0.0
>> Average tombstones per slice (last five minutes): 0.0
>> Maximum tombstones per slice (last five minutes): 0.0
>>
>>
>> Some of the errors:
>>
>> /export.cql:9:Error for (269754647900342974, 272655475232221549): 
>> OperationTimedOut - errors={}, last_host=10.1.12.89 (will try again later 
>> attempt 1 of 5)
>> /export.cql:9:Error for (-3191598516608295217, -3188807168672208162): 
>> OperationTimedOut - errors={}, last_host=10.1.12.89 (will try again later 
>> attempt 1 of 5)
>> /export.cql:9:Error for (-3066009427947359685, -3058745599093267591): 
>> OperationTimedOut - errors={}, last_host=10.1.8.5 (will try again later 
>> attempt 1 of 5)
>> /export.cql:9:Error for (-1737068099173540127, -1716693115263588178): 
>> OperationTimedOut - errors={}, last_host=10.1.8.5 (will try again later 
>> attempt 1 of 5)
>> /export.cql:9:Error for (-655042025062419794, -627527938552757160): 
>> OperationTimedOut - errors={}, last_host=10.1.12.89 (will try again later 
>> attempt 1 of 5)
>> /export.cql:9:Error for (2441403877625910843, 2445504271098651532): 
>> OperationTimedOut - 

Re: Changing snitch from PropertyFile to Gossip

[Carlos Rolo]

I just run it to be sure. Sometimes mistakes happen and it's a way to be
sure.
Em 25/04/2016 10:19, "Alain RODRIGUEZ" <arodr...@gmail.com> escreveu:

> Hi Carlos,
>
> Why running a repair there if the topology did not change? Is it as a best
> practice, just in case, or is there a specific reason?
>
> C*heers,
> ---
> Alain Rodriguez - al...@thelastpickle.com
> France
>
> The Last Pickle - Apache Cassandra Consulting
> http://www.thelastpickle.com
>
> 2016-04-24 15:44 GMT+02:00 Carlos Rolo <r...@pythian.com>:
>
>> As long as the topology doesn't change, yes. Repair once you finish.
>> Em 24/04/2016 13:23, "AJ" <qwerty15...@gmail.com> escreveu:
>>
>>> Is it possible to do this without down time i.e. run in mixed mode while
>>> doing a rolling upgrade?
>>
>>
>> --
>>
>>
>>
>>
>

-- 


--

Re: Changing snitch from PropertyFile to Gossip

[Carlos Rolo]

As long as the topology doesn't change, yes. Repair once you finish.
Em 24/04/2016 13:23, "AJ"  escreveu:

> Is it possible to do this without down time i.e. run in mixed mode while
> doing a rolling upgrade?

-- 


--

Re: Most stable version?

[Carlos Rolo]

I do expect 3 to get stable at some point, according to documentation it
will be the 3.0.x series. But the current 3.x tick-tock,  I would recommend
a jump into it when Datastax do it. Otherwise, maybe 4 might get stable and
we could be following similar releases cicles like some software out there,
even is stable (2 and 4) even is unstable (3 and 5). But this is my
guessing. Wait for a DSE release on 3.x and use that.

I had problems in earlier 2.2, 2.2.5 seems to be a solid release, but I
will wait for 2.2.6 before recommending for production. Just to be safe :)

Regards,

Carlos Juzarte Rolo
Cassandra Consultant / Datastax Certified Architect / Cassandra MVP

Pythian - Love your data

rolo@pythian | Twitter: @cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo
<http://linkedin.com/in/carlosjuzarterolo>*
Mobile: +351 91 891 81 00 | Tel: +1 613 565 8696 x1649
www.pythian.com

On Fri, Apr 22, 2016 at 6:42 PM, Jason Williams <jasonjwwilli...@gmail.com>
wrote:

> Hi Carlos,
>
> I read your blog post (actually almost everything I can find on tick
> tock). My understanding has been tick tock will be the only versioning
> going forward.
>
> Or are you suggesting at some point there will be a stable train for 3?
> (or that 3.x will be bumped to 4.0 when stable)?
>
> We're on 2.2.5 and haven't seen any major problems with it.
>
> -J
>
>
>
> Sent via iPhone
>
> On Apr 22, 2016, at 03:34, Carlos Rolo <r...@pythian.com> wrote:
>
> If you need SASI, you need to use 3.4+. 3.x will always be "unstable" (It
> is explained why in my blog post). You get those odd versions, but it is
> not a solid effort to stabilize the platform, otherwise devs would not jump
> to 3.6, and keep working on 3.5. And then you get 3.7, which might fix some
> issues of 3.4+, but next month you get 3.8 unstable again... I'm waiting to
> see where this is going. I only had bad experiences with 3.x series atm.
>
> If you want stability (and no new features), you would use 2.1.13.
>
> 2.2.x is kind of a mixed bag, no really huge improvements over 2.1.x
> series and it is still having some issues, so I would stick to 2.1.x
> series.
>
> Regards,
>
> Carlos Juzarte Rolo
> Cassandra Consultant / Datastax Certified Architect / Cassandra MVP
>
> Pythian - Love your data
>
> rolo@pythian | Twitter: @cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo
> <http://linkedin.com/in/carlosjuzarterolo>*
> Mobile: +351 91 891 81 00 | Tel: +1 613 565 8696 x1649
> www.pythian.com
>
> On Fri, Apr 22, 2016 at 10:16 AM, Jason Williams <
> jasonjwwilli...@gmail.com> wrote:
>
>> My reading of the tick-rock cycle, is that we've moved from a stable
>> train that receives mostly bug fixes until the next major stable, to one
>> where every odd minor version is a bug fix-only...likely mostly for the
>> previous even. The goal being a relatively continuously stable code base in
>> odd minor versions.
>>
>> In that environment where there is no "stable" train, would the right
>> approach be to pick the feature set needed and then choose the odd minor
>> where that feature set had been stable for 2-3 previous odd minors.
>>
>> For example, SASI was added in 3.4, so 3.5 is the first bug fix only (odd
>> minor) containing it. By the logic above you wouldn't want to use SASI in
>> production until 3.9 or later. Or is my logic about how to treat tick-tock
>> off base?
>>
>> -J
>>
>>
>> Sent via iPhone
>>
>> On Apr 22, 2016, at 01:46, Satoshi Hikida <sahik...@gmail.com> wrote:
>>
>> Hi,
>>
>> I'm also looking for the most stable version of the Cassandra, too. I
>> read Carlos's blog post. According to his article, I guess 2.1.x is the
>> most stable version, is it right? I prefer to use the most stable version
>> rather than many advanced features. For satisfy my purpose, should I use
>> 2.1.X? or latest 2.2.x is recommended?
>>
>> Currently I use 2.2.5, but is the latest 2.1.13 recommended for
>> production use?
>>
>> Regards,
>> Satoshi
>>
>>
>> On Mon, Apr 18, 2016 at 11:45 PM, Carlos Rolo <r...@pythian.com> wrote:
>>
>>> Sorry to resurrect this now, but I don't consider anything after 3.0.x
>>> stable.
>>>
>>> I wrote a blog post about this to be clear:
>>> https://www.pythian.com/blog/cassandra-version-production/
>>>
>>> Use it and pick a version based on your needs.
>>>
>>> Regards,
>>>
>>> Carlos Juzarte Rolo
>>> Cassandra Consultant / Datastax Certified Architect / Cassandra MVP
>>>
>>> Pythian - Love your data
>

Re: Most stable version?

[Carlos Rolo]

If you need SASI, you need to use 3.4+. 3.x will always be "unstable" (It
is explained why in my blog post). You get those odd versions, but it is
not a solid effort to stabilize the platform, otherwise devs would not jump
to 3.6, and keep working on 3.5. And then you get 3.7, which might fix some
issues of 3.4+, but next month you get 3.8 unstable again... I'm waiting to
see where this is going. I only had bad experiences with 3.x series atm.

If you want stability (and no new features), you would use 2.1.13.

2.2.x is kind of a mixed bag, no really huge improvements over 2.1.x series
and it is still having some issues, so I would stick to 2.1.x series.

Regards,

Carlos Juzarte Rolo
Cassandra Consultant / Datastax Certified Architect / Cassandra MVP

Pythian - Love your data

rolo@pythian | Twitter: @cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo
<http://linkedin.com/in/carlosjuzarterolo>*
Mobile: +351 91 891 81 00 | Tel: +1 613 565 8696 x1649
www.pythian.com

On Fri, Apr 22, 2016 at 10:16 AM, Jason Williams <jasonjwwilli...@gmail.com>
wrote:

> My reading of the tick-rock cycle, is that we've moved from a stable train
> that receives mostly bug fixes until the next major stable, to one where
> every odd minor version is a bug fix-only...likely mostly for the previous
> even. The goal being a relatively continuously stable code base in odd
> minor versions.
>
> In that environment where there is no "stable" train, would the right
> approach be to pick the feature set needed and then choose the odd minor
> where that feature set had been stable for 2-3 previous odd minors.
>
> For example, SASI was added in 3.4, so 3.5 is the first bug fix only (odd
> minor) containing it. By the logic above you wouldn't want to use SASI in
> production until 3.9 or later. Or is my logic about how to treat tick-tock
> off base?
>
> -J
>
>
> Sent via iPhone
>
> On Apr 22, 2016, at 01:46, Satoshi Hikida <sahik...@gmail.com> wrote:
>
> Hi,
>
> I'm also looking for the most stable version of the Cassandra, too. I read
> Carlos's blog post. According to his article, I guess 2.1.x is the most
> stable version, is it right? I prefer to use the most stable version rather
> than many advanced features. For satisfy my purpose, should I use 2.1.X? or
> latest 2.2.x is recommended?
>
> Currently I use 2.2.5, but is the latest 2.1.13 recommended for production
> use?
>
> Regards,
> Satoshi
>
>
> On Mon, Apr 18, 2016 at 11:45 PM, Carlos Rolo <r...@pythian.com> wrote:
>
>> Sorry to resurrect this now, but I don't consider anything after 3.0.x
>> stable.
>>
>> I wrote a blog post about this to be clear:
>> https://www.pythian.com/blog/cassandra-version-production/
>>
>> Use it and pick a version based on your needs.
>>
>> Regards,
>>
>> Carlos Juzarte Rolo
>> Cassandra Consultant / Datastax Certified Architect / Cassandra MVP
>>
>> Pythian - Love your data
>>
>> rolo@pythian | Twitter: @cjrolo | Linkedin: 
>> *linkedin.com/in/carlosjuzarterolo
>> <http://linkedin.com/in/carlosjuzarterolo>*
>> Mobile: +351 91 891 81 00 | Tel: +1 613 565 8696 x1649
>> www.pythian.com
>>
>> On Fri, Apr 15, 2016 at 12:44 PM, Jean Tremblay <
>> jean.tremb...@zen-innovations.com> wrote:
>>
>>> Thank you Jack.
>>> Jean
>>>
>>> On 14 Apr 2016, at 22:00 , Jack Krupansky <jack.krupan...@gmail.com>
>>> wrote:
>>>
>>> Normally, since 3.5 just came out, it would be wise to see if people
>>> report any problems over the next few weeks.
>>>
>>> But... the new tick-tock release process is designed to assure that
>>> these odd-numbered releases are only incremental bug fixes from the last
>>> even-numbered feature release, which was 3.4. So, 3.5 should be reasonably
>>> stable.
>>>
>>> That said, a bug-fix release of 3.0 is probably going to be more stable
>>> than a bug fix release of a more recent feature release (3.4).
>>>
>>> Usually it comes down to whether you need any of the new features or
>>> improvements in 3.x, or whether you might want to keep your chosen release
>>> in production for longer than the older 3.0 releases will be in production.
>>>
>>> Ultimately, this is a personality test: Are you adventuresome or
>>> conservative?
>>>
>>> To be clear, with the new tick-tock release scheme, 3.5 is designed to
>>> be a stable release.
>>>
>>> -- Jack Krupansky
>>>
>>> On Thu, Apr 14, 2016 at 3:23 PM, Jean Tremblay <
>>> jean.tremb...@zen-innovations.com> 

Re: Most stable version?

[Carlos Rolo]

Sorry to resurrect this now, but I don't consider anything after 3.0.x
stable.

I wrote a blog post about this to be clear:
https://www.pythian.com/blog/cassandra-version-production/

Use it and pick a version based on your needs.

Regards,

Carlos Juzarte Rolo
Cassandra Consultant / Datastax Certified Architect / Cassandra MVP

Pythian - Love your data

rolo@pythian | Twitter: @cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo
*
Mobile: +351 91 891 81 00 | Tel: +1 613 565 8696 x1649
www.pythian.com

On Fri, Apr 15, 2016 at 12:44 PM, Jean Tremblay <
jean.tremb...@zen-innovations.com> wrote:

> Thank you Jack.
> Jean
>
> On 14 Apr 2016, at 22:00 , Jack Krupansky 
> wrote:
>
> Normally, since 3.5 just came out, it would be wise to see if people
> report any problems over the next few weeks.
>
> But... the new tick-tock release process is designed to assure that these
> odd-numbered releases are only incremental bug fixes from the last
> even-numbered feature release, which was 3.4. So, 3.5 should be reasonably
> stable.
>
> That said, a bug-fix release of 3.0 is probably going to be more stable
> than a bug fix release of a more recent feature release (3.4).
>
> Usually it comes down to whether you need any of the new features or
> improvements in 3.x, or whether you might want to keep your chosen release
> in production for longer than the older 3.0 releases will be in production.
>
> Ultimately, this is a personality test: Are you adventuresome or
> conservative?
>
> To be clear, with the new tick-tock release scheme, 3.5 is designed to be
> a stable release.
>
> -- Jack Krupansky
>
> On Thu, Apr 14, 2016 at 3:23 PM, Jean Tremblay <
> jean.tremb...@zen-innovations.com> wrote:
>
>> Hi,
>> Could someone give his opinion on this?
>> What should be considered more stable, Cassandra 3.0.5 or Cassandra 3.5?
>>
>> Thank you
>> Jean
>>
>> > On 12 Apr,2016, at 07:00, Jean Tremblay <
>> jean.tremb...@zen-innovations.com> wrote:
>> >
>> > Hi,
>> > Which version of Cassandra should considered most stable in the version
>> 3?
>> > I see two main branch: the branch with the version 3.0.* and the
>> tick-tock one 3.*.*.
>> > So basically my question is: which one is most stable, version 3.0.5 or
>> version 3.3?
>> > I know odd versions in tick-took are bug fix.
>> > Thanks
>> > Jean
>>
>
>
>

-- 


--

Re: Do I have to use repair -inc with the option -par forcely?

[Carlos Rolo]

+1 on what Alain said, but I do think if you are high enough on a 2.1.x
(will look later) version you don't need to follow the documentation. It is
outdated. Run a full repair, the you can start incremental repairs since
the SSTables will have the metadata on them about the last repair.

 Wait someone to confirm this/or confirm the docs are correct.

Regards,

Carlos Juzarte Rolo
Cassandra Consultant / Datastax Certified Architect / Cassandra MVP

Pythian - Love your data

rolo@pythian | Twitter: @cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo
*
Mobile: +351 91 891 81 00 | Tel: +1 613 565 8696 x1649
www.pythian.com

On Tue, Feb 16, 2016 at 1:45 PM, Alain RODRIGUEZ  wrote:

> Hi,
>
> I am testing repairs repairs -inc -par and I can see that in all my nodes
>> the numbers of sstables explode to 5k from 5 sstables.
>
>
> This looks like a known issue, see
> https://issues.apache.org/jira/browse/CASSANDRA-10422
> Make sure your version is higher than 2.1.12, 2.2.4, 3.0.1, 3.1 to avoid
> this (and you are indeed facing CASSANDRA-10422).
> I am not sure you are facing this though, as you don't seem to be using
> subranges (nodetool repair -st  and -et options)
>
> *It is anyway to run repairs incrementals but not -par ?*
>>
>> I know that is it not possible to run sequential repair with incremental
>> repair at the same time.
>>
>
>
> From http://www.datastax.com/dev/blog/more-efficient-repairs
> "Incremental repairs can be opted into via the -inc option to nodetool
> repair. This is compatible with both sequential and parallel (-par)
> repair, e.g., bin/nodetool -par -inc  ."
> So you should be able to remove -par. Not sure this will solve your issue
> though.
>
>
> Did you respect this process to migrate to incremental repairs?
>
> https://docs.datastax.com/en/cassandra/2.1/cassandra/operations/opsRepairNodesMigration.html#opsRepairNodesMigration__ol_dxj_gp5_2s
>
> C*heers,
> -
> Alain Rodriguez
> France
>
> The Last Pickle
> http://www.thelastpickle.com
>
>
>
> 2016-02-10 17:45 GMT+01:00 Jean Carlo :
>
>> Hi guys; The question is on the subject.
>>
>> I am testing repairs repairs -inc -par and I can see that in all my nodes
>> the numbers of sstables explode to 5k from 5 sstables.
>>
>> I cannot permit this behaivor on my cluster in production.
>>
>> *It is anyway to run repairs incrementals but not -par ?*
>>
>> I know that is it not possible to run sequential repair with incremental
>> repair at the same time.
>>
>> Best regards
>>
>> Jean Carlo
>>
>> "The best way to predict the future is to invent it" Alan Kay
>>
>
>

-- 


--

Re: Latest stable release

[Carlos Rolo]

I honestly go with 2.1.13 unless you need the features on 2.2.x.

I would not recommend 3.x for now (unless you need the features).


Regards,

Carlos Juzarte Rolo
Cassandra Consultant / Datastax Certified Architect / Cassandra MVP

Pythian - Love your data

rolo@pythian | Twitter: @cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo
*
Mobile: +351 91 891 81 00 | Tel: +1 613 565 8696 x1649
www.pythian.com

On Mon, Feb 8, 2016 at 9:34 PM, Ravi Krishna 
wrote:

> We are starting a new project in Cassandra. Is 3.2 stable enough to be
> used in production. If not, which is the most stable version in 2.x.
>
> thanks.
>

-- 


--

Re: Cassandra is consuming a lot of disk space

[Carlos Rolo]

You can check if the snapshot exists in the snapshot folder.
Repairs stream sstables over, than can temporary increase disk space. But I
think Carlos Alonso might be correct. Running compactions might be the
issue.

Regards,

Carlos Juzarte Rolo
Cassandra Consultant

Pythian - Love your data

rolo@pythian | Twitter: @cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo
*
Mobile: +351 91 891 81 00 | Tel: +1 613 565 8696 x1649
www.pythian.com

On Wed, Jan 13, 2016 at 9:24 AM, Carlos Alonso  wrote:

> I'd have a look also at possible running compactions.
>
> If you have big column families with STCS then large compactions may be
> happening.
>
> Check it with nodetool compactionstats
>
> Carlos Alonso | Software Engineer | @calonso 
>
> On 13 January 2016 at 05:22, Kevin O'Connor  wrote:
>
>> Have you tried restarting? It's possible there's open file handles to
>> sstables that have been compacted away. You can verify by doing lsof and
>> grepping for DEL or deleted.
>>
>> If it's not that, you can run nodetool cleanup on each node to scan all
>> of the sstables on disk and remove anything that it's not responsible for.
>> Generally this would only work if you added nodes recently.
>>
>>
>> On Tuesday, January 12, 2016, Rahul Ramesh  wrote:
>>
>>> We have a 2 node Cassandra cluster with a replication factor of 2.
>>>
>>> The load factor on the nodes is around 350Gb
>>>
>>> Datacenter: Cassandra
>>> ==
>>> Address  RackStatus State   LoadOwns
>>>Token
>>>
>>> -5072018636360415943
>>> 172.31.7.91  rack1   Up Normal  328.5 GB100.00%
>>> -7068746880841807701
>>> 172.31.7.92  rack1   Up Normal  351.7 GB100.00%
>>> -5072018636360415943
>>>
>>> However,if I use df -h,
>>>
>>> /dev/xvdf   252G  223G   17G  94% /HDD1
>>> /dev/xvdg   493G  456G   12G  98% /HDD2
>>> /dev/xvdh   197G  167G   21G  90% /HDD3
>>>
>>>
>>> HDD1,2,3 contains only cassandra data. It amounts to close to 1Tb in one
>>> of the machine and in another machine it is close to 650Gb.
>>>
>>> I started repair 2 days ago, after running repair, the amount of disk
>>> space consumption has actually increased.
>>> I also checked if this is because of snapshots. nodetool listsnapshot
>>> intermittently lists a snapshot but it goes away after sometime.
>>>
>>> Can somebody please help me understand,
>>> 1. why so much disk space is consumed?
>>> 2. Why did it increase after repair?
>>> 3. Is there any way to recover from this state.
>>>
>>>
>>> Thanks,
>>> Rahul
>>>
>>>
>

-- 


--

Re: Unable to start one Cassandra node: OutOfMemoryError

[Carlos Rolo]

Dealt with that recently, and the only solution that made it work was to
increase heap sizes.



Regards,

Carlos Juzarte Rolo
Cassandra Consultant

Pythian - Love your data

rolo@pythian | Twitter: @cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo
*
Mobile: +351 91 891 81 00 | Tel: +1 613 565 8696 x1649
www.pythian.com

On Thu, Dec 10, 2015 at 10:14 PM, Mikhail Strebkov 
wrote:

> Jeff, CMS GC didn't help. Thinking about it, I don't see how can it help
> if there are 8GB of strongly reachable objects from the GC roots.
>
> Walsh, thanks for your suggestion, I checked the log and there are some
> compactions_in_progress but total size of those is ~300 MiB as far as I
> understand.
>
> Here is the log of the last unsuccessful start:
> https://gist.github.com/kluyg/7b9955d34def947f5e0a
>
>
> On Thu, Dec 10, 2015 at 2:09 AM, Walsh, Stephen 
> wrote:
>
>> 8GB is the max recommended for heap size and that’s if you have 32GB or
>> more available.
>>
>>
>>
>> We use 6GB on our 16GB machines and its very stable
>>
>>
>>
>> The out of memory could be coming from cassandra reloading
>> compactions_in_progress into memory, you can check this from the log files
>> if needs be.
>>
>> You can safely delete this folder inside the data directory.
>>
>>
>>
>> This can happen if you didn’t stop cassandra with a drain command and
>> wait for the compactions to finish.
>>
>> Last time we hit it – was due to testing HA when we forced killed an
>> entire cluster.
>>
>>
>>
>> Steve
>>
>>
>>
>>
>>
>>
>>
>> *From:* Jeff Jirsa [mailto:jeff.ji...@crowdstrike.com]
>> *Sent:* 10 December 2015 02:49
>> *To:* user@cassandra.apache.org
>> *Subject:* Re: Unable to start one Cassandra node: OutOfMemoryError
>>
>>
>>
>> 8G is probably too small for a G1 heap. Raise your heap or try CMS
>> instead.
>>
>>
>>
>> 71% of your heap is collections – may be a weird data model quirk, but
>> try CMS first and see if that behaves better.
>>
>>
>>
>>
>>
>>
>>
>> *From: *Mikhail Strebkov
>> *Reply-To: *"user@cassandra.apache.org"
>> *Date: *Wednesday, December 9, 2015 at 5:26 PM
>> *To: *"user@cassandra.apache.org"
>> *Subject: *Unable to start one Cassandra node: OutOfMemoryError
>>
>>
>>
>> Hi everyone,
>>
>>
>>
>> While upgrading our 5 machines cluster from DSE version 4.7.1 (Cassandra
>> 2.1.8) to DSE version: 4.8.2 (Cassandra 2.1.11)  one of the nodes can't
>> start with OutOfMemoryError.
>>
>> We're using HotSpot 64-Bit Server VM/1.8.0_45 and G1 garbage collector
>> with 8 GiB heap.
>>
>> Average node size is 300 GiB.
>>
>>
>>
>> I looked at the heap dump with YourKit profiler (www.yourkit.com) and it
>> was quite hard since it's so big, but can't get much out of it:
>> http://i.imgur.com/fIRImma.png
>>
>>
>>
>> As far as I understand the report, there are 1,332,812 instances of
>> org.apache.cassandra.db.Row which retain 8 GiB. I don't understand why all
>> of them are still strongly reachable?
>>
>>
>>
>> Please help me to debug this. I don't know even where to start.
>>
>> I feel very uncomfortable with 1 node running 4.8.2, 1 node down and 3
>> nodes running 4.7.1 at the same time.
>>
>>
>>
>> Thanks,
>>
>> Mikhail
>>
>>
>>
>>
>> This email (including any attachments) is proprietary to Aspect Software,
>> Inc. and may contain information that is confidential. If you have received
>> this message in error, please do not read, copy or forward this message.
>> Please notify the sender immediately, delete it from your system and
>> destroy any copies. You may not further disclose or distribute this email
>> or its attachments.
>>
>
>

-- 


--

Re: scylladb

[Carlos Rolo]

Sure! I have a lot of blog post on backlog to blog asap about this,
otherwise I would only share results mid 2016 :P

Regards,

Carlos Juzarte Rolo
Cassandra Consultant

Pythian - Love your data

rolo@pythian | Twitter: @cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo
<http://linkedin.com/in/carlosjuzarterolo>*
Mobile: +351 91 891 81 00 | Tel: +1 613 565 8696 x1649
www.pythian.com

On Wed, Nov 11, 2015 at 4:46 PM, Dani Traphagen <dani.trapha...@datastax.com
> wrote:

> Killer, @cjrolo. Will you update via this thread?
>
> On Wed, Nov 11, 2015 at 7:57 AM, Carlos Rolo <r...@pythian.com> wrote:
>
>> Not yet, but not far from doing it. No rain here yet! :)
>>
>> On a more serious tone, should be done before end of the Month.
>>
>> --
>>
>>
>>
>>
>
>
> --
> [image: datastax_logo.png] <http://www.datastax.com/>
>
> DANI TRAPHAGEN
>
> Technical Enablement Lead | dani.trapha...@datastax.com
>
> [image: twitter.png] <https://twitter.com/dtrapezoid> [image:
> linkedin.png] <https://www.linkedin.com/pub/dani-traphagen/31/93b/b85>
> <https://github.com/dtrapezoid>
>
>
>

-- 


--

Re: scylladb

[Carlos Rolo]

Not yet, but not far from doing it. No rain here yet! :)

On a more serious tone, should be done before end of the Month.

-- 


--

Re: scylladb

[Carlos Rolo]

I will not try until multi-DC is implemented. More than an month has passed
since I looked for it, so it could possibly be in place, if so I may take
some time to test it.

Regards,

Carlos Juzarte Rolo
Cassandra Consultant

Pythian - Love your data

rolo@pythian | Twitter: @cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo
*
Mobile: +351 91 891 81 00 | Tel: +1 613 565 8696 x1649
www.pythian.com

On Thu, Nov 5, 2015 at 9:37 AM, Jon Haddad 
wrote:

> Nope, no one I know.  Let me know if you try it I'd love to hear your
> feedback.
>
> > On Nov 5, 2015, at 9:22 AM, tommaso barbugli 
> wrote:
> >
> > Hi guys,
> >
> > did anyone already try Scylladb (yet another fastest NoSQL database in
> town) and has some thoughts/hands-on experience to share?
> >
> > Cheers,
> > Tommaso
>
>

-- 


--

Re: scylladb

[Carlos Rolo]

Something to do on a expected rainy weekend. Thanks for the information.

Regards,

Carlos Juzarte Rolo
Cassandra Consultant

Pythian - Love your data

rolo@pythian | Twitter: @cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo
<http://linkedin.com/in/carlosjuzarterolo>*
Mobile: +351 91 891 81 00 | Tel: +1 613 565 8696 x1649
www.pythian.com

On Thu, Nov 5, 2015 at 12:07 PM, Dani Traphagen <dani.trapha...@datastax.com
> wrote:

> As of two days ago, they say they've got it @cjrolo.
>
> https://github.com/scylladb/scylla/wiki/RELEASE-Scylla-0.11-Beta
>
>
> On Thursday, November 5, 2015, Carlos Rolo <r...@pythian.com> wrote:
>
>> I will not try until multi-DC is implemented. More than an month has
>> passed since I looked for it, so it could possibly be in place, if so I may
>> take some time to test it.
>>
>> Regards,
>>
>> Carlos Juzarte Rolo
>> Cassandra Consultant
>>
>> Pythian - Love your data
>>
>> rolo@pythian | Twitter: @cjrolo | Linkedin: 
>> *linkedin.com/in/carlosjuzarterolo
>> <http://linkedin.com/in/carlosjuzarterolo>*
>> Mobile: +351 91 891 81 00 | Tel: +1 613 565 8696 x1649
>> www.pythian.com
>>
>> On Thu, Nov 5, 2015 at 9:37 AM, Jon Haddad <jonathan.had...@gmail.com>
>> wrote:
>>
>>> Nope, no one I know.  Let me know if you try it I'd love to hear your
>>> feedback.
>>>
>>> > On Nov 5, 2015, at 9:22 AM, tommaso barbugli <tbarbu...@gmail.com>
>>> wrote:
>>> >
>>> > Hi guys,
>>> >
>>> > did anyone already try Scylladb (yet another fastest NoSQL database in
>>> town) and has some thoughts/hands-on experience to share?
>>> >
>>> > Cheers,
>>> > Tommaso
>>>
>>>
>>
>> --
>>
>>
>>
>>
>
> --
> Sent from mobile -- apologizes for brevity or errors.
>

-- 


--

Re: any update about CASSANDRA-10420

[Carlos Rolo]

Can you provide more information?

The description is generic, driver version and a test case that the devs
can use to reproduce the bug would be optimal.

I'm not involved in the development of cassandra, but that bug description
doesn't seem complete. Maybe it is a driver bug and not a Cassandra bug?



Regards,

Carlos Juzarte Rolo
Cassandra Consultant

Pythian - Love your data

rolo@pythian | Twitter: @cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo
*
Mobile: +351 91 891 81 00 | Tel: +1 613 565 8696 x1649
www.pythian.com

On Fri, Oct 16, 2015 at 10:16 AM, Lu, Boying  wrote:

> Hi, All,
>
>
>
> Can anyone give some suggestions about the
> https://issues.apache.org/jira/browse/CASSANDRA-10420 ?
>
>
>
> We are waiting for a solution of it.
>
>
>
> Thanks
>
>
>
> Boying
>

-- 


--

Re: Removed node is not completely removed

[Carlos Rolo]

Check system.peers table to see if the IP is still there. If so edit the
table and remove the offending IP.

You are probably running into this:
https://issues.apache.org/jira/browse/CASSANDRA-6053

Regards,

Carlos Juzarte Rolo
Cassandra Consultant

Pythian - Love your data

rolo@pythian | Twitter: cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo
*
Mobile: +31 6 159 61 814 | Tel: +1 613 565 8696 x1649
www.pythian.com

On Wed, Oct 14, 2015 at 12:26 PM, Tom van den Berge <
tom.vandenbe...@gmail.com> wrote:

> I have removed a node with nodetool removenode, which completed ok.
> Nodetool status does not list the node anymore.
>
> But since then, Im seeing messages in my other nodes log files referring
> to the removed node:
>
>  INFO [GossipStage:38] 2015-10-14 11:18:26,322 Gossiper.java (line 968)
> InetAddress /10.68.56.200 is now DOWN
>  INFO [GossipStage:38] 2015-10-14 11:18:26,324 StorageService.java (line
> 1891) Removing tokens [85070591730234615865843651857942052863] for /
> 10.68.56.200
>
>
> These two messages appear every minute.
> I've tried nodetool removenode again (Host ID not found) and removenode
> force (no token removals in process).
> The jmx unsafeAssassinateEndpoint gives a NullPointerException.
>
> What can I do to remove the node entirely?
>
>
>

-- 


--

Re: howto do sql query like in a relational database

[Carlos Rolo]

Hello Anton,

You need to look into Datastax Entreprise (DSE) Offering. It integrates
Solr search which allows you to do searches like the one you mention. There
are also some opensource projects doing this kind of integration, so its up
to you.

And as Oded mentioned Cassandra really shines on key queries.

Regards,

Carlos Juzarte Rolo
Cassandra Consultant

Pythian - Love your data

rolo@pythian | Twitter: cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo
http://linkedin.com/in/carlosjuzarterolo*
Mobile: +31 6 159 61 814 | Tel: +1 613 565 8696 x1649
www.pythian.com

On Wed, Jul 22, 2015 at 7:39 AM, Peer, Oded oded.p...@rsa.com wrote:

 Cassandra is a highly scalable, eventually consistent, distributed,
 structured key-value store http://wiki.apache.org/cassandra/
 It is intended for searching by key. It has more querying options but it
 really shines when querying by key.

 Not all databases offer the same functionality. Both a knife and a fork
 are eating utensils, but you wouldn't want to cut a tomato with a fork.
 There are text-indexing databases out there that might suit your needs
 better. Try elasticsearch.

 -Original Message-
 From: anton [mailto:anto...@gmx.de]
 Sent: Tuesday, July 21, 2015 7:54 PM
 To: user@cassandra.apache.org
 Subject: howto do sql query like in a relational database

 Hi,

 I have a simple (perhaps stupid) question.

 If I want to *search* data in cassandra, how could find in a text field
 all records which start with 'Cas'
 ( in sql I do select * from table where field like 'Cas%')

 I know that this is not directly possible.

  - But how is it possible?

  - Do nobody have the need to search text fragments,
and if not is there a small example to explain
*why* this is not needed?

 As far as I understand, databases are great for *searching* data.
 Concerning numerical data in cassandra I can use   = all that operators.

 Is cassandra intended to be used for mostly numerical data?

 I did not catch the point up to now, sorry.

  Anton




-- 


--

Re: Seed gossip version error

[Carlos Rolo]

That error should only occur when you have a mismatch between the Seed
version and the new node version. Are you sure all your nodes are running
in the same version?

Regards,

Carlos Juzarte Rolo
Cassandra Consultant

Pythian - Love your data

rolo@pythian | Twitter: cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo
http://linkedin.com/in/carlosjuzarterolo*
Mobile: +31 6 159 61 814 | Tel: +1 613 565 8696 x1649
www.pythian.com

On Tue, Jul 21, 2015 at 5:37 PM, DE VITO Dominique 
dominique.dev...@thalesgroup.com wrote:

 Hi Amlan,



 We have the same pb with Cassandra 2.1.5.



 I have no hint (yet) to follow.



 Did you found the root of this pb ?



 Thanks.



 Regards,

 Dominique





 [@@ THALES GROUP INTERNAL @@]



 *De :* Amlan Roy [mailto:amlan@cleartrip.com]
 *Envoyé :* mercredi 1 juillet 2015 12:46
 *À :* user@cassandra.apache.org
 *Objet :* Seed gossip version error



 Hi,



 I have a running cluster running with version 2.1.7. Two of the machines
 went down and they are not joining the cluster even after restart. I see
 the following WARN message in system.log in all the nodes:

 system.log:WARN  [
 MessagingService-Outgoing-cassandra2.cleartrip.com/172.18.3.32]
 2015-07-01 13:00:41,878 OutboundTcpConnection.java:414 - Seed gossip
 version is -2147483648; will not connect with that version



 Please let me know if you have faced the same problem.



 Regards,

 Amlan






-- 


--

Re: cassandra repair error

[Carlos Rolo]

Hello,

It seems that is a problem with Cassandra trying to delete a directory that
is not empty. First time seeing this error.
Are your harddrives with enough space and working correctly?

Regards,

Carlos Juzarte Rolo
Cassandra Consultant

Pythian - Love your data

rolo@pythian | Twitter: cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo
http://linkedin.com/in/carlosjuzarterolo*
Mobile: +31 6 159 61 814 | Tel: +1 613 565 8696 x1649
www.pythian.com

On Thu, Jul 16, 2015 at 2:45 PM, Modha, Digant 
digant.mo...@tdsecurities.com wrote:

 Hi,



 We’re using Cassandra 2.0.10 ( 2 DC, 3 Nodes each RF=3 for each DC).
 During one of the weekly repairs, we received the following error:



 ERROR [ValidationExecutor:1280] 2015-07-12 22:18:10,992 Validator.java
 (line 242) Failed creating a merkle tree for [repair
 #d2178ba0-2902-11e5-bd95-f14c61d86b85 on dmds/curve_dates,
 (-1942303675502999131,-1890400428284965630]], /  (see log for details)

 ERROR [ValidationExecutor:1280] 2015-07-12 22:18:10,992
 CassandraDaemon.java (line 199) Exception in thread
 Thread[ValidationExecutor:1280,1,main]

 FSWriteError in
 /apps/data/cassandra/dmds/data/dmds/curve_dates/snapshots/d2178ba0-2902-11e5-bd95-f14c61d86b85

   at
 org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:122)

   at
 org.apache.cassandra.io.util.FileUtils.deleteRecursive(FileUtils.java:384)

   at
 org.apache.cassandra.db.Directories.clearSnapshot(Directories.java:488)

   at
 org.apache.cassandra.db.ColumnFamilyStore.clearSnapshot(ColumnFamilyStore.java:1877)

   at
 org.apache.cassandra.db.compaction.CompactionManager.doValidationCompaction(CompactionManager.java:811)

   at
 org.apache.cassandra.db.compaction.CompactionManager.access$600(CompactionManager.java:63)

   at
 org.apache.cassandra.db.compaction.CompactionManager$8.call(CompactionManager.java:398)

   at java.util.concurrent.FutureTask.run(FutureTask.java:266)

   at
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

   at
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

   at java.lang.Thread.run(Thread.java:745)

 Caused by: java.nio.file.DirectoryNotEmptyException:
 /apps/data/cassandra/dmds/data/dmds/curve_dates/snapshots/d2178ba0-2902-11e5-bd95-f14c61d86b85

   at
 sun.nio.fs.UnixFileSystemProvider.implDelete(UnixFileSystemProvider.java:242)

   at
 sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:103)

   at java.nio.file.Files.delete(Files.java:1126)

   at
 org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:118)

   ... 10 more

 ERROR [ValidationExecutor:1280] 2015-07-12 22:18:10,993
 StorageService.java (line 364) Stopping gossiper

 WARN [ValidationExecutor:1280] 2015-07-12 22:18:10,993 StorageService.java
 (line 278) Stopping gossip by operator request

 INFO [ValidationExecutor:1280] 2015-07-12 22:18:10,993 Gossiper.java (line
 1279) Announcing shutdown

 Has anybody seen this error?  The drives are local. Once this happened,
 the other node performing  repair maxed out the CPU and cluster became
 unresponsive.





 Thanks,

 dm




 TD Securities disclaims any liability or losses either direct or
 consequential caused by the use of this information. This communication is
 for informational purposes only and is not intended as an offer or
 solicitation for the purchase or sale of any financial instrument or as an
 official confirmation of any transaction. TD Securities is neither making
 any investment recommendation nor providing any professional or advisory
 services relating to the activities described herein. All market prices,
 data and other information are not warranted as to completeness or accuracy
 and are subject to change without notice Any products described herein are
 (i) not insured by the FDIC, (ii) not a deposit or other obligation of, or
 guaranteed by, an insured depository institution and (iii) subject to
 investment risks, including possible loss of the principal amount invested.
 The information shall not be further distributed or duplicated in whole or
 in part by any means without the prior written consent of TD Securities. TD
 Securities is a trademark of The Toronto-Dominion Bank and represents TD
 Securities (USA) LLC and certain investment banking activities of The
 Toronto-Dominion Bank and its subsidiaries.


-- 


--

Re: Wrong peers

[Carlos Rolo]

There is a bug in Jira related to this, it is not a driver issue, is a
Cassandra issue. It is solved on 2.0.14 I think. I will post the ticket
once I find it.

Regards,

Carlos Juzarte Rolo
Cassandra Consultant

Pythian - Love your data

rolo@pythian | Twitter: cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo
http://linkedin.com/in/carlosjuzarterolo*
Mobile: +31 6 159 61 814 | Tel: +1 613 565 8696 x1649
www.pythian.com

On Mon, Jul 6, 2015 at 10:50 PM, Jeff Williams je...@wherethebitsroam.com
wrote:

 Anton,

 I have also seen this issue with decommissioned nodes remaining in the
 system.peers table.

 On the bright side, they can be safely removed from the system.peers table
 without issue. You will have to check every node in the cluster since this
 is a local setting per node.

 Jeff

 On 6 July 2015 at 22:45, nowarry nowa...@gmail.com wrote:

 Hey guys,

 I'm using Ruby driver( http://datastax.github.io/ruby-driver/ ) for
 backup scripts. I tried to discover all peers and got wrong peers that are
 different with nodetool status.

 =
 Status=Up/Down
 |/ State=Normal/Leaving/Joining/Moving
 --  Address   Load   Tokens  OwnsHost ID
   Rack
 UN  10.40.231.53  1.18 TB256 ?
 b2d877d7-f031-4190-8569-976bb0ce034f  RACK01
 UN  10.40.231.11  1.24 TB256 ?
 e15cda1c-65cc-40cb-b85c-c4bd665d02d7  RACK01

 cqlsh use system;
 cqlsh:system select peer from system.peers;

  peer
 --
  10.40.231.31
  10.40.231.53

 (2 rows)

 What to do with these old peers, whether they can be removed without
 consequences since they are not in production cluster? And how to keep up
 to date the peers?

 --
 Anton Koshevoy




-- 


--

Re: Error while adding a new node.

[Carlos Rolo]

Indeed you should upgrade to 2.1.7.

And then report if you are still facing problems. Versions up to 2.1.5 (in
the 2.1.x series) are not considered stable.

Regards,

Carlos Juzarte Rolo
Cassandra Consultant

Pythian - Love your data

rolo@pythian | Twitter: cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo
http://linkedin.com/in/carlosjuzarterolo*
Mobile: +31 6 159 61 814 | Tel: +1 613 565 8696 x1649
www.pythian.com

On Thu, Jul 2, 2015 at 11:40 AM, Neha Trivedi nehajtriv...@gmail.com
wrote:

 any help?

 On Thu, Jul 2, 2015 at 6:18 AM, Neha Trivedi nehajtriv...@gmail.com
 wrote:

 also:
 root@cas03:~# sudo service cassandra start
 root@cas03:~# lsof -n | grep java | wc -l
 5315
 root@cas03:~# lsof -n | grep java | wc -l
 977317
 root@cas03:~# lsof -n | grep java | wc -l
 880240
 root@cas03:~# lsof -n | grep java | wc -l
 882402


 On Wed, Jul 1, 2015 at 6:31 PM, Neha Trivedi nehajtriv...@gmail.com
 wrote:

 One of the column family has SStable count as under :
 SSTable count: 98506

 Can it be because of 2.1.3 version of cassandra..
 I found this : https://issues.apache.org/jira/browse/CASSANDRA-8964

 regards
 Neha


 On Wed, Jul 1, 2015 at 5:40 PM, Jason Wee peich...@gmail.com wrote:

 nodetool cfstats?

 On Wed, Jul 1, 2015 at 8:08 PM, Neha Trivedi nehajtriv...@gmail.com
 wrote:

 Hey..
 nodetool compactionstats
 pending tasks: 0

 no pending tasks.

 Dont have opscenter. how do I monitor sstables?


 On Wed, Jul 1, 2015 at 4:28 PM, Alain RODRIGUEZ arodr...@gmail.com
 wrote:

 You also might want to check if you have compactions pending
 (Opscenter / nodetool compactionstats).

 Also you can monitor the number of sstables.

 C*heers

 Alain

 2015-07-01 11:53 GMT+02:00 Neha Trivedi nehajtriv...@gmail.com:

 Thanks I will checkout.
 I increased the ulimit to 10, but I am getting the same error,
 but after a while.
 regards
 Neha


 On Wed, Jul 1, 2015 at 2:22 PM, Alain RODRIGUEZ arodr...@gmail.com
 wrote:

 Just check the process owner to be sure (top, htop, ps, ...)


 http://docs.datastax.com/en/cassandra/2.0/cassandra/install/installRecommendSettings.html#reference_ds_sxl_gf3_2k__user-resource-limits

 C*heers,

 Alain

 2015-07-01 7:33 GMT+02:00 Neha Trivedi nehajtriv...@gmail.com:

 Arun,
 I am logging on to Server as root and running (sudo service
 cassandra start)

 regards
 Neha

 On Wed, Jul 1, 2015 at 11:00 AM, Neha Trivedi 
 nehajtriv...@gmail.com wrote:

 Thanks Arun ! I will try and get back !

 On Wed, Jul 1, 2015 at 10:32 AM, Arun arunsi...@gmail.com
 wrote:

 Looks like you have too many open files issue. Increase the
 ulimit for the user.

  If you are starting the cassandra daemon using user cassandra,
 increase the ulimit for that user.


  On Jun 30, 2015, at 21:16, Neha Trivedi 
 nehajtriv...@gmail.com wrote:
 
  Hello,
  I have a 4 node cluster with SimpleSnitch.
  Cassandra :  Cassandra 2.1.3
 
  I am trying to add a new node (cassandra 2.1.7) and I get the
 following error.
 
  ERROR [STREAM-IN-] 2015-06-30 05:13:48,516
 JVMStabilityInspector.java:94 - JVM state determined to be unstable.
 Exiting forcefully due to:
  java.io.FileNotFoundException:
 /var/lib/cassandra/data/-Index.db (Too many open files)
 
  I increased the MAX_HEAP_SIZE then I get :
  ERROR [CompactionExecutor:9] 2015-06-30 23:31:44,792
 CassandraDaemon.java:223 - Exception in thread
 Thread[CompactionExecutor:9,1,main]
  java.lang.RuntimeException: java.io.FileNotFoundException:
 /var/lib/cassandra/data/-Data.db (Too many open files)
  at
 org.apache.cassandra.io.compress.CompressedThrottledReader.open(CompressedThrottledReader.java:52)
 ~[apache-cassandra-2.1.7.jar:2.1.7]
 
  Is it because of the different version of Cassandra (2.1.3 and
 2.17) ?
 
  regards
  N
 
 
 
 
 
 
 













-- 


--

Re: [MASSMAIL]Re: Error while adding a new node.

[Carlos Rolo]

Marco you should also avoid 2.1.5 and 2.1.6 because of
https://issues.apache.org/jira/browse/CASSANDRA-9549

I know (And often don't recommend last versions, I'm still recommending
2.0.x series unless someone is already in 2.1.x) but given the above bug,
2.1.7 is the best option.

Regards,

Carlos Juzarte Rolo
Cassandra Consultant

Pythian - Love your data

rolo@pythian | Twitter: cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo
http://linkedin.com/in/carlosjuzarterolo*
Mobile: +31 6 159 61 814 | Tel: +1 613 565 8696 x1649
www.pythian.com

On Thu, Jul 2, 2015 at 3:20 PM, Marcos Ortiz mlor...@uci.cu wrote:

  The recommended version to use is 2.1.5 because, like you Carlos said,
 2.1.6 and 2.1.7 are very new to consider them like
 stable.

 On 02/07/15 08:55, Carlos Rolo wrote:

  Indeed you should upgrade to 2.1.7.

  And then report if you are still facing problems. Versions up to 2.1.5
 (in the 2.1.x series) are not considered stable.

Regards,

  Carlos Juzarte Rolo
 Cassandra Consultant

 Pythian - Love your data

  rolo@pythian | Twitter: cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo
 http://linkedin.com/in/carlosjuzarterolo*
 Mobile: +31 6 159 61 814 | Tel: +1 613 565 8696 x1649
 www.pythian.com

 On Thu, Jul 2, 2015 at 11:40 AM, Neha Trivedi nehajtriv...@gmail.com
 wrote:

 any help?

 On Thu, Jul 2, 2015 at 6:18 AM, Neha Trivedi nehajtriv...@gmail.com
 wrote:

 also:
 root@cas03:~# sudo service cassandra start
 root@cas03:~# lsof -n | grep java | wc -l
 5315
 root@cas03:~# lsof -n | grep java | wc -l
 977317
 root@cas03:~# lsof -n | grep java | wc -l
 880240
 root@cas03:~# lsof -n | grep java | wc -l
 882402


 On Wed, Jul 1, 2015 at 6:31 PM, Neha Trivedi nehajtriv...@gmail.com
 wrote:

   One of the column family has SStable count as under :
 SSTable count: 98506

  Can it be because of 2.1.3 version of cassandra..
  I found this : https://issues.apache.org/jira/browse/CASSANDRA-8964

  regards
  Neha


 On Wed, Jul 1, 2015 at 5:40 PM, Jason Wee peich...@gmail.com wrote:

 nodetool cfstats?

 On Wed, Jul 1, 2015 at 8:08 PM, Neha Trivedi nehajtriv...@gmail.com
 wrote:

  Hey..
 nodetool compactionstats
 pending tasks: 0

  no pending tasks.

  Dont have opscenter. how do I monitor sstables?


 On Wed, Jul 1, 2015 at 4:28 PM, Alain RODRIGUEZ arodr...@gmail.com
 wrote:

 You also might want to check if you have compactions pending
 (Opscenter / nodetool compactionstats).

  Also you can monitor the number of sstables.

  C*heers

  Alain

 2015-07-01 11:53 GMT+02:00 Neha Trivedi nehajtriv...@gmail.com:

   Thanks I will checkout.
  I increased the ulimit to 10, but I am getting the same error,
 but after a while.
  regards
  Neha


 On Wed, Jul 1, 2015 at 2:22 PM, Alain RODRIGUEZ arodr...@gmail.com
  wrote:

  Just check the process owner to be sure (top, htop, ps, ...)


 http://docs.datastax.com/en/cassandra/2.0/cassandra/install/installRecommendSettings.html#reference_ds_sxl_gf3_2k__user-resource-limits

  C*heers,

  Alain

 2015-07-01 7:33 GMT+02:00 Neha Trivedi nehajtriv...@gmail.com:

   Arun,
  I am logging on to Server as root and running (sudo service
 cassandra start)

  regards
  Neha

 On Wed, Jul 1, 2015 at 11:00 AM, Neha Trivedi 
 nehajtriv...@gmail.com wrote:

 Thanks Arun ! I will try and get back !

 On Wed, Jul 1, 2015 at 10:32 AM, Arun arunsi...@gmail.com
 wrote:

 Looks like you have too many open files issue. Increase the
 ulimit for the user.

  If you are starting the cassandra daemon using user cassandra,
 increase the ulimit for that user.


  On Jun 30, 2015, at 21:16, Neha Trivedi 
 nehajtriv...@gmail.com wrote:
 
  Hello,
  I have a 4 node cluster with SimpleSnitch.
  Cassandra :  Cassandra 2.1.3
 
  I am trying to add a new node (cassandra 2.1.7) and I get the
 following error.
 
  ERROR [STREAM-IN-] 2015-06-30 05:13:48,516
 JVMStabilityInspector.java:94 - JVM state determined to be 
 unstable.
 Exiting forcefully due to:
  java.io.FileNotFoundException:
 /var/lib/cassandra/data/-Index.db (Too many open files)
 
  I increased the MAX_HEAP_SIZE then I get :
  ERROR [CompactionExecutor:9] 2015-06-30 23:31:44,792
 CassandraDaemon.java:223 - Exception in thread
 Thread[CompactionExecutor:9,1,main]
  java.lang.RuntimeException: java.io.FileNotFoundException:
 /var/lib/cassandra/data/-Data.db (Too many open files)
  at
 org.apache.cassandra.io.compress.CompressedThrottledReader.open(CompressedThrottledReader.java:52)
 ~[apache-cassandra-2.1.7.jar:2.1.7]
 
  Is it because of the different version of Cassandra (2.1.3
 and 2.17) ?
 
  regards
  N
 
 
 
 
 
 
 













 --




 --
 Marcos Ortiz http://about.me/marcosortiz, Sr. Product Manager (Data
 Infrastructure) at UCI
 @marcosluis2186 http://twitter.com/marcosluis2186




-- 


--

Re: compaction occurring very frequently

[Carlos Rolo]

Hi,

2.1.x does have compaction issues. Upgrading to 2.1.6 should help you. You
can also search this mailing list for some recommendations for that
specific problem since it is a recurrent issue.

Regards,

Carlos Juzarte Rolo
Cassandra Consultant

Pythian - Love your data

rolo@pythian | Twitter: cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo
http://linkedin.com/in/carlosjuzarterolo*
Mobile: +31 6 159 61 814 | Tel: +1 613 565 8696 x1649
www.pythian.com

On Thu, Jun 18, 2015 at 9:27 AM, Rahul Bhardwaj 
rahul.bhard...@indiamart.com wrote:

 Hi All,

 We have a cluster (2.1.2) of 3 nodes , There is one table which contains
 huge amount of data and having sstable count more than 1800 on all 3 nodes.

 We have noticed that suddently compaction for that CF occurs very
 frequently right after completion of previous compaction and also taking
 long time in compacting.

 Par New collection time and CMS collection time have also increased on
 those node.

 Please suggest what could be the problem.


 Regards:
 Rahul Bhardwaj


 Follow IndiaMART.com http://www.indiamart.com for latest updates on
 this and more: https://plus.google.com/+indiamart
 https://www.facebook.com/IndiaMART https://twitter.com/IndiaMART
 Mobile Channel:
 https://itunes.apple.com/WebObjects/MZStore.woa/wa/viewSoftware?id=668561641mt=8
 https://play.google.com/store/apps/details?id=com.indiamart.m
 http://m.indiamart.com/

 https://www.youtube.com/watch?v=DzORNbeSXN8list=PL2o4J51MqpL0mbue6kzDa6eymLVUXtlR1index=2
 Check out some smart tips to generate more business - *IndiaMART Learning
 Centre* http://corporate.indiamart.com/quick-learn/

-- 


--

Re: Replication in one datacetner only

[Carlos Rolo]

Yes, set the replication factor of the keyspace to only use the datacenter
you want.

More information here:
http://docs.datastax.com/en/cql/3.1/cql/cql_reference/create_keyspace_r.html

Regards,

Carlos Juzarte Rolo
Cassandra Consultant

Pythian - Love your data

rolo@pythian | Twitter: cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo
http://linkedin.com/in/carlosjuzarterolo*
Mobile: +31 6 159 61 814 | Tel: +1 613 565 8696 x1649
www.pythian.com

On Tue, Jun 16, 2015 at 5:07 PM, Willis Yonker wyon...@gmail.com wrote:


 Hi,

 I'm wondering if it would be possible to have a keyspace setup to
 replicate only within a single datacenter, regardless of how many other
 datacenters there are in the cluster.

 I have a case where some machines will be at a site that needs it's
 database replicated to all nodes at that site.  However, no other site will
 need that database nor do we really want to have it replicate to a remote
 site for backup or D/R.  Those nodes will need access to other keyspaces
 that are in the cluster though.

 Is there a way to do this?


-- 


--

Re: Missing data

[Carlos Rolo]

Hi Jean,

The problem of that Warning is that you are reading too many tombstones per
request.

If you do have Tombstones without doing DELETE it because you probably
TTL'ed the data when inserting (By mistake? Or did you set
default_time_to_live in your table?). You can use nodetool cfstats to see
how many tombstones per read slice you have. This is, probably, also the
cause of your missing data. Data was tombstoned, so it is not available.



Regards,

Carlos Juzarte Rolo
Cassandra Consultant

Pythian - Love your data

rolo@pythian | Twitter: cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo
http://linkedin.com/in/carlosjuzarterolo*
Mobile: +31 6 159 61 814 | Tel: +1 613 565 8696 x1649
www.pythian.com

On Mon, Jun 15, 2015 at 10:54 AM, Jean Tremblay 
jean.tremb...@zen-innovations.com wrote:

  Hi,

  I have reloaded the data in my cluster of 3 nodes RF: 2.
 I have loaded about 2 billion rows in one table.
 I use LeveledCompactionStrategy on my table.
 I use version 2.1.6.
 I use the default cassandra.yaml, only the ip address for seeds and
 throughput has been change.

  I loaded my data with simple insert statements. This took a bit more
 than one day to load the data… and one more day to compact the data on all
 nodes.
 For me this is quite acceptable since I should not be doing this again.
 I have done this with previous versions like 2.1.3 and others and I
 basically had absolutely no problems.

  Now I read the log files on the client side, there I see no warning and
 no errors.
 On the nodes side there I see many WARNING, all related with tombstones,
 but there are no ERRORS.

  My problem is that I see some *many missing records* in the DB, and I
 have never observed this with previous versions.

  1) Is this a know problem?
 2) Do you have any idea how I could track down this problem?
 3) What is the meaning of this WARNING (the only type of ERROR | WARN  I
 could find)?

  WARN  [SharedPool-Worker-2] 2015-06-15 10:12:00,866
 SliceQueryFilter.java:319 - Read 2990 live and 16016 tombstone cells in
 gttdata.alltrades_co_rep_pcode for key: D:07 (see
 tombstone_warn_threshold). 5000 columns were requested,
 slices=[388:201001-388:201412:!]


  4) Is it possible to have Tombstone when we make no DELETE statements?

  I’m lost…

  Thanks for your help.


-- 


--

Re: Question about nodetool status ... output

[Carlos Rolo]

Your data model also contributes to the balance (or lack of) of the
cluster. If you have a really bad data partitioning Cassandra will not do
any magic.

Regarding that cluster, I would decommission the x.52 node and add it again
with the correct configuration. After the bootstrap, run a cleanup. If is
still that off-balance, you need to look into your data model.

Regards,

Carlos Juzarte Rolo
Cassandra Consultant

Pythian - Love your data

rolo@pythian | Twitter: cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo
http://linkedin.com/in/carlosjuzarterolo*
Mobile: +31 6 159 61 814 | Tel: +1 613 565 8696 x1649
www.pythian.com

On Fri, Jun 12, 2015 at 11:58 AM, Jens Rantil jens.ran...@tink.se wrote:

 Hi,

 I have one node in my 5-node cluster that effectively owns 100% and it
 looks like my cluster is rather imbalanced. Is it common to have it this
 imbalanced for 4-5 nodes?

 My current output for a keyspace is:

 $ nodetool status myks
 Datacenter: Cassandra
 =
 Status=Up/Down
 |/ State=Normal/Leaving/Joining/Moving
 --  Address Load   Tokens  Owns (effective)  Host ID
 Rack
 UN  X.X.X.33  203.92 GB  256 41.3%
 871968c9-1d6b-4f06-ba90-8b3a8d92dcf0  RAC1
 UN  X.X.X.32  200.44 GB  256 34.2%
 d7cacd89-8613-4de5-8a5e-a2c53c41ea45  RAC1
 UN  X.X.X.51  197.17 GB  256 100.0%
  344b0adf-2b5d-47c8-8881-9a3f56be6f3b  RAC1
 UN  X.X.X.52  113.63 GB  1   46.3%
 55daa807-af49-44c5-9742-fe456df621a1  RAC1
 UN  X.X.X.31  204.49 GB  256 78.3%
 48cb0782-6c9a-4805-9330-38e192b6b680  RAC1

 My keyspace has RF=3 and originally I added X.X.X.52 (num_tokens=1 was a
 mistake) and then X.X.X.51. I haven't executed `nodetool cleanup` on any
 nodes yet.

 For the curious, the full ring can be found here:
 https://gist.github.com/JensRantil/57ee515e647e2f154779

 Cheers,
 Jens

 --
 Jens Rantil
 Backend engineer
 Tink AB

 Email: jens.ran...@tink.se
 Phone: +46 708 84 18 32
 Web: www.tink.se

 Facebook https://www.facebook.com/#!/tink.se Linkedin
 http://www.linkedin.com/company/2735919?trk=vsrp_companies_res_phototrkInfo=VSRPsearchId%3A1057023381369207406670%2CVSRPtargetId%3A2735919%2CVSRPcmpt%3Aprimary
  Twitter https://twitter.com/tink


-- 


--

Re: Lucene index plugin for Apache Cassandra

[Carlos Rolo]

Seems like an interesting tool!

What operational recommendations would you make to users of this tool
(Extra hardware capacity, extra metrics to monitor, etc)?

Regards,

Carlos Juzarte Rolo
Cassandra Consultant

Pythian - Love your data

rolo@pythian | Twitter: cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo
http://linkedin.com/in/carlosjuzarterolo*
Mobile: +31 6 159 61 814 | Tel: +1 613 565 8696 x1649
www.pythian.com

On Fri, Jun 12, 2015 at 11:07 AM, Andres de la Peña adelap...@stratio.com
wrote:

 Unfortunately, we don't have published any benchmarks yet, but we have
 plans to do it as soon as possible. However, you can expect a similar
 behavior as those of Elasticsearch or Solr, with some overhead due to the
 need for indexing both the Cassandra's row key and the partition's token.
 You can also take a look at this presentation
 http://planetcassandra.org/video-presentations/vp/cassandra-summit-europe-2014/vd/stratio-advanced-search-and-top-k-queries-in-cassandra/
 to see how cluster distribution is done.

 2015-06-12 0:45 GMT+02:00 Ben Bromhead b...@instaclustr.com:

 Looks awesome, do you have any examples/benchmarks of using these indexes
 for various cluster sizes e.g. 20 nodes, 60 nodes, 100s+?

 On 10 June 2015 at 09:08, Andres de la Peña adelap...@stratio.com
 wrote:

 Hi all,

 With the release of Cassandra 2.1.6, Stratio is glad to present its
 open source Lucene-based implementation of C* secondary indexes
 https://github.com/Stratio/cassandra-lucene-index as a plugin that
 can be attached to Apache Cassandra. Before the above changes, Lucene index
 was distributed inside a fork of Apache Cassandra, with all the
 difficulties implied. As of now, the fork is discontinued and new users
 should use the recently created plugin, which maintains all the features of 
 Stratio
 Cassandra https://github.com/Stratio/stratio-cassandra.



 Stratio's Lucene index extends Cassandra’s functionality to provide near
 real-time distributed search engine capabilities such as with ElasticSearch
 or Solr, including full text search capabilities, free multivariable
 search, relevance queries and field-based sorting. Each node indexes its
 own data, so high availability and scalability is guaranteed.


 We hope this will be useful to the Apache Cassandra community.


 Regards,

 --

 Andrés de la Peña


 http://www.stratio.com/
 Avenida de Europa, 26. Ática 5. 3ª Planta
 28224 Pozuelo de Alarcón, Madrid
 Tel: +34 91 352 59 42 // *@stratiobd https://twitter.com/StratioBD*




 --

 Ben Bromhead

 Instaclustr | www.instaclustr.com | @instaclustr
 http://twitter.com/instaclustr | (650) 284 9692




 --

 Andrés de la Peña


 http://www.stratio.com/
 Avenida de Europa, 26. Ática 5. 3ª Planta
 28224 Pozuelo de Alarcón, Madrid
 Tel: +34 91 352 59 42 // *@stratiobd https://twitter.com/StratioBD*


-- 


--

Re: Hundreds of sstables after every Repair

[Carlos Rolo]

Hello,

Do you have your clocks synced across your cluster? Are you using NTP and
have it properly configured?

Sometimes clock out of sync can trigger weird behaviour.

Regards,

Carlos Juzarte Rolo
Cassandra Consultant

Pythian - Love your data

rolo@pythian | Twitter: cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo
http://linkedin.com/in/carlosjuzarterolo*
Mobile: +31 6 159 61 814 | Tel: +1 613 565 8696 x1649
www.pythian.com

On Tue, Jun 9, 2015 at 5:11 PM, Anuj Wadehra anujw_2...@yahoo.co.in wrote:

 We were facing dropped mutations earlier and we increased flush writers.
 Now there are no dropped mutations in tpstats. To repair the damaged vnodes
 / inconsistent data we executed repair -pr on all nodes. Still, we see the
 same problem.

 When we analyze repair logs we see 2 strange things:

 1. Out of sync ranges for cf which are not being actively being
 written/updated while the repair is going on. When we repaired all data by
 repair -pr on all nodes, why out of sync data?

 2. For some cf , repair logs shows that all ranges are consistent. Still
 we get so many sstables created during repair. When everything is in sync ,
 why repair creates tiny sstables to repair data?

 Thanks
 Anuj Wadehra

 Sent from Yahoo Mail on Android
 https://overview.mail.yahoo.com/mobile/?.src=Android
 --
   *From*:Ken Hancock ken.hanc...@schange.com
 *Date*:Tue, 9 Jun, 2015 at 8:24 pm
 *Subject*:Re: Hundreds of sstables after every Repair

 I think this came up recently in another thread.  If you're getting large
 numbers of SSTables after repairs, that means that your nodes are diverging
 from the keys that they're supposed to be having.  Likely you're dropping
 mutations.  Do a nodetool tpstats on each of your nodes and look at the
 mutation droppped counters.  If you're seeing dropped message, my money you
 have a non-zero FlushWriter All time blocked stat which is causing
 mutations to be dropped.



 On Tue, Jun 9, 2015 at 10:35 AM, Anuj Wadehra anujw_2...@yahoo.co.in
 wrote:

 Any suggestions or comments on this one?

 Thanks
 Anuj Wadehra

 Sent from Yahoo Mail on Android
 https://overview.mail.yahoo.com/mobile/?.src=Android
 --
   *From*:Anuj Wadehra anujw_2...@yahoo.co.in
 *Date*:Sun, 7 Jun, 2015 at 1:54 am
 *Subject*:Hundreds of sstables after every Repair

 Hi,

 We are using 2.0.3 and vnodes. After every repair -pr operation  50+ tiny
 sstables( 10K) get created. And these sstables never get compacted due to
 coldness issue. I have raised
 https://issues.apache.org/jira/browse/CASSANDRA-9146 for this issue but
 I have been told to upgrade. Till we upgrade to latest 2.0.x , we are
 stuck. Upgrade takes time, testing and planning in Production systems :(

 I have observed that even if vnodes are NOT damaged, hundreds of tiny
 sstables are created during repair for a wide row CF. This is beyond my
 understanding. If everything is consistent, and for the entire repair
 process Cassandra is saying Endpoints /x.x.x.x and /x.x.x.y are consistent
 for CF. Whats the need of creating sstables?

 Is there any alternative to regular major compaction to deal with
 situation?


 Thanks
 Anuj Wadehra









-- 


--

Re: Ghost compaction process

[Carlos Rolo]

HI,

Is it 2.0.14 or 2.1.4? If you are on 2.1.4 I would recommend an upgrade to
2.1.5 regardless of that issue.

From the data you provide it is difficult to access what is the issue. If
you are running with RF=2 you can always add another node and kill that one
if that is the only node that shows that problem. With 40GB load is not a
big issue.

Regards,

Carlos Juzarte Rolo
Cassandra Consultant

Pythian - Love your data

rolo@pythian | Twitter: cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo
http://linkedin.com/in/carlosjuzarterolo*
Mobile: +31 6 159 61 814 | Tel: +1 613 565 8696 x1649
www.pythian.com

On Mon, Jun 8, 2015 at 4:04 AM, Arturas Raizys artu...@noantidot.com
wrote:

 Hi,

  Does `nodetool comactionstats` show nothing running as well? Also, for
  posterity what are some details of the setup (C* version, etc.)?

 `nodetool comactionstats` does not return anything, it just waits.
 If I do enable DEBUG logging, I see this line poping up while executing
 `nodetool compactionstats` :
 DEBUG [RMI TCP Connection(1856)-127.0.0.1] 2015-06-08 09:29:46,043
 LeveledManifest.java:680 - Estimating [0, 0, 0, 0, 0, 0, 0, 0, 0]
 compactions to do for system.paxos

 I'm running Cassandra 2.1.14 with 7 node cluster. We're using small VM
 with 8GB of ram and SSD. Our data size per node with RF=2 is ~40GB. Load
 is ~ 1000 writes/second. Most of the data TTL is 2weeks.


 Cheers,
 Arturas


-- 


--

Re: Reading too many tombstones

[Carlos Rolo]

The TTL data will only be removed after the gc_grace_seconds. So your data
with 30 days TTL will be still in Cassandra for 10 days more (40 in total).
Is your data being there for more than that? Otherwise it is expected
behaviour and probably you should do something on your data model to avoid
scanning tombstoned data.

Regards,

Carlos Juzarte Rolo
Cassandra Consultant

Pythian - Love your data

rolo@pythian | Twitter: cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo
http://linkedin.com/in/carlosjuzarterolo*
Mobile: +31 6 159 61 814 | Tel: +1 613 565 8696 x1649
www.pythian.com

On Thu, Jun 4, 2015 at 8:31 PM, Aiman Parvaiz ai...@flipagram.com wrote:

 yeah we don't update old data. One thing I am curious about is why are we
 running in to so many tombstones with compaction happening normally. Is
 compaction not removing tombstomes?


 On Thu, Jun 4, 2015 at 11:25 AM, Jonathan Haddad j...@jonhaddad.com
 wrote:

 DateTiered is fantastic if you've got time series, TTLed data.  That
 means no updates to old data.

 On Thu, Jun 4, 2015 at 10:58 AM Aiman Parvaiz ai...@flipagram.com
 wrote:

 Hi everyone,
 We are running a 10 node Cassandra 2.0.9 without vnode cluster. We are
 running in to a issue where we are reading too many tombstones and hence
 getting tons of WARN messages and some ERROR query aborted.

 cass-prod4 2015-06-04 14:38:34,307 WARN ReadStage:
 https://logentries.com/app/9f95dbd4#1998
 SliceQueryFilter.collectReducedColumns - Read 46 live and 1560 tombstoned
 cells in ABC.home_feed (see tombstone_warn_threshold). 100 columns was
 requested, slices= https://logentries.com/app/9f95dbd4#[-], delInfo=
 https://logentries.com/app/9f95dbd4#{deletedAt=
 https://logentries.com/app/9f95dbd4#-9223372036854775808,
 localDeletion= https://logentries.com/app/9f95dbd4#2147483647}

 cass-prod2 2015-05-31 12:55:55,331 ERROR ReadStage:
 https://logentries.com/app/9f95dbd4#1953
 SliceQueryFilter.collectReducedColumns - Scanned over 10 tombstones in
 ABC.home_feed; query aborted (see tombstone_fail_threshold)

 As you can see all of this is happening for CF home_feed. This CF is
 basically maintaining a feed with TTL set to 2592000 (30 days).
 gc_grace_seconds for this CF is 864000 and its SizeTieredCompaction.

 Repairs have been running regularly and automatic compactions are
 occurring normally too.

 I can definitely use some help here in how to tackle this issue.

 Up till now I have the following ideas:

 1) I can make gc_grace_seconds to 0 and then do a manual compaction for
 this CF and bump up the gc_grace again.

 2) Make gc_grace 0, run manual compaction on this CF and leave gc_grace
 to zero. In this case have to be careful in running repairs.

 3) I am also considering moving to DateTier Compaction.

 What would be the best approach here for my feed case. Any help is
 appreciated.

 Thanks







-- 


--

Re: How to interpret some GC logs

[Carlos Rolo]

GC Logs are a weird science. I use a couple of resources to get through
them. Regarding your question my 1.8.0_40 always have the first the -. I
greped through 2h of logs, and on a test environment.

I use the following set of options:

-XX:+PrintGCDetails
-XX:+PrintGCDateStamps
-XX:+PrintHeapAtGC
-XX:+PrintTenuringDistribution
-XX:+PrintGCApplicationStoppedTime
-XX:+PrintPromotionFailure
-XX:PrintFLSStatistics=1

I don't know if the following could help but I use this tool to visualize
my GC behaviour: https://github.com/chewiebug/GCViewer
If I need insight into the GC as it goes I use Java VisualVM.

Regards,

Carlos Juzarte Rolo
Cassandra Consultant

Pythian - Love your data

rolo@pythian | Twitter: cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo
http://linkedin.com/in/carlosjuzarterolo*
Mobile: +31 6 159 61 814 | Tel: +1 613 565 8696 x1649
www.pythian.com

On Tue, Jun 2, 2015 at 9:17 AM, Michał Łowicki mlowi...@gmail.com wrote:



 On Tue, Jun 2, 2015 at 9:06 AM, Sebastian Martinka 
 sebastian.marti...@mercateo.com wrote:

  this should help you:

 https://blogs.oracle.com/poonam/entry/understanding_cms_gc_logs


 I don't see there such format. Passed options related to GC are:

 -XX:+PrintGCDateStamps -Xloggc:/var/log/cassandra/gc.log




 Best Regards,
 Sebastian Martinka



 *Von:* Michał Łowicki [mailto:mlowi...@gmail.com]
 *Gesendet:* Montag, 1. Juni 2015 11:47
 *An:* user@cassandra.apache.org
 *Betreff:* How to interpret some GC logs



 Hi,



 Normally I get logs like:



 2015-06-01T09:19:50.610+: 4736.314: [GC 6505591K-4895804K(8178944K),
 0.0494560 secs]



 which is fine and understandable but occasionalIy I see something like:

 2015-06-01T09:19:50.661+: 4736.365: [GC 4901600K(8178944K), 0.0049600
 secs]

 How to interpret it? Does it miss only part before - so memory
 occupied before GC cycle?

 --

 BR,
 Michał Łowicki




 --
 BR,
 Michał Łowicki


-- 


--

Re: Multiple cassandra instances per physical node

[Carlos Rolo]

Hi,

I also advice against multiple instances on the same hardware. If you have
really big boxes why not virtualize?

Other option is experiment with CCM. Although there are some limitations
with CCM (ex: JNA is disabled)

If you follow up on this I would to hear how it went.
Em 21/05/2015 19:33, Dan Kinder dkin...@turnitin.com escreveu:

 Hi, I'd just like some clarity and advice regarding running multiple
 cassandra instances on a single large machine (big JBOD array, plenty of
 CPU/RAM).

 First, I am aware this was not Cassandra's original design, and doing this
 seems to unreasonably go against the commodity hardware intentions of
 Cassandra's design. In general it seems to be recommended against (at least
 as far as I've heard from @Rob Coli and others).

 However maybe this term commodity is changing... my hardware/ops team
 argues that due to cooling, power, and other datacenter costs, having
 slightly larger nodes (=32G RAM, =24 CPU, =8 disks JBOD) is actually a
 better price point. Now, I am not a hardware guy, so if this is not
 actually true I'd love to hear why, otherwise I pretty much need to take
 them at their word.

 Now, Cassandra features seemed to have improved such that JBOD works
 fairly well, but especially with memory/GC this seems to be reaching its
 limit. One Cassandra instance can only scale up so much.

 So my question is: suppose I take a 12 disk JBOD and run 2 Cassandra nodes
 (each with 5 data disks, 1 commit log disk) and either give each its own
 container  IP or change the listen ports. Will this work? What are the
 risks? Will/should Cassandra support this better in the future?


-- 


--

Re: minimum bandwidth requirement between two Geo Redundant sites of Cassandra database

[Carlos Rolo]

Hi,

I would not recommend anything below 1Gbps for the bandwidth. Latency try
to have it as low as you can.

Regards,

Carlos Juzarte Rolo
Cassandra Consultant

Pythian - Love your data

rolo@pythian | Twitter: cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo
http://linkedin.com/in/carlosjuzarterolo*
Mobile: +31 6 159 61 814 | Tel: +1 613 565 8696 x1649
www.pythian.com

On Tue, Apr 28, 2015 at 5:28 AM, Gaurav Bhatnagar gauravb...@gmail.com
wrote:

 Hi,
  Is there any minimum bandwidth requirement between two Geo Redundant
 data centres?
 What is the minimum latency that link between two Geo Redundant data
 centres should have to get best efficient operations?

 Regards,
 Gaurav


-- 


--

Re: New node got stuck joining the cluster after a while

[Carlos Rolo]

Hi,

The 2.1.x series is not recommeded for use, especially the first versions.
I would downgrade to 2.0.14 or if must stay on 2.1 upgrade your cluster to
2.1.4 or the imminent release of 2.1.5.

This mailing list as a few tips how to deal with the 2.1.x releases, but
the best way is indeed a downgrade or wait for the 2.1.5.

Regards,

Carlos Juzarte Rolo
Cassandra Consultant

Pythian - Love your data

rolo@pythian | Twitter: cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo
http://linkedin.com/in/carlosjuzarterolo*
Mobile: +31 6 159 61 814 | Tel: +1 613 565 8696 x1649
www.pythian.com

On Tue, Apr 28, 2015 at 3:30 AM, Analia Lorenzatto 
analialorenza...@gmail.com wrote:


 Hello guys,

 I have a cluster comprised of 2 nodes, configured with vnodes.  Using
 2.1.0-2 version of cassandra.

 And I am facing an issue when I want to joing a new node to the cluster.

 At first starting joining but then it got stuck:

 UN  1x.x.x.x  348.11 GB  256 100.0%  1c
 UN  1x.x.x.x  342.74 GB  256 100.0%  1c
 UJ  1x.x.x.x  26.86 GB   256 ?   1c


 I can see some errors on the already working nodes:

 *WARN  [SharedPool-Worker-7] 2015-04-27 17:41:16,060
 SliceQueryFilter.java:236 - Read 5001 live and 66548 tombstoned cells in
 usmc.userpixel (see tombstone_warn_threshol*
 *d). 5000 columns was requested, slices=[-],
 delInfo={deletedAt=-9223372036854775808, localDeletion=2147483647
 2147483647}*
 *WARN  [SharedPool-Worker-32] 2015-04-27 17:41:16,668
 SliceQueryFilter.java:236 - Read 2012 live and 30440 tombstoned cells in
 usmc.userpixel (see tombstone_warn_thresho*
 *ld). 5001 columns was requested,
 slices=[b6d051df-0a8f-4c13-b93c-1b4ff0d82b8d:date-],
 delInfo={deletedAt=-9223372036854775808, localDeletion=2147483647}*

 *ERROR [CompactionExecutor:35638] 2015-04-27 19:06:07,613
 CassandraDaemon.java:166 - Exception in thread
 Thread[CompactionExecutor:35638,1,main]*
 *java.lang.AssertionError: Memory was freed*
 *at
 org.apache.cassandra.io.util.Memory.checkPosition(Memory.java:281)
 ~[apache-cassandra-2.1.0.jar:2.1.0]*
 *at org.apache.cassandra.io.util.Memory.getInt(Memory.java:233)
 ~[apache-cassandra-2.1.0.jar:2.1.0]*
 *at
 org.apache.cassandra.io.sstable.IndexSummary.getPositionInSummary(IndexSummary.java:118)
 ~[apache-cassandra-2.1.0.jar:2.1.0]*
 *at
 org.apache.cassandra.io.sstable.IndexSummary.getKey(IndexSummary.java:123)
 ~[apache-cassandra-2.1.0.jar:2.1.0]*
 * at
 org.apache.cassandra.io.sstable.IndexSummary.binarySearch(IndexSummary.java:92)
 ~[apache-cassandra-2.1.0.jar:2.1.0]*
 *at
 org.apache.cassandra.io.sstable.SSTableReader.getSampleIndexesForRanges(SSTableReader.java:1209)
 ~[apache-cassandra-2.1.0.jar:2.1.0]*
 *at
 org.apache.cassandra.io.sstable.SSTableReader.estimatedKeysForRanges(SSTableReader.java:1165)
 ~[apache-cassandra-2.1.0.jar:2.1.0]*
 *at
 org.apache.cassandra.db.compaction.AbstractCompactionStrategy.worthDroppingTombstones(AbstractCompactionStrategy.java:328)
 ~[apache-cassandra-2.1.0.jar:2.1.0*
 *]*
 *at
 org.apache.cassandra.db.compaction.LeveledCompactionStrategy.findDroppableSSTable(LeveledCompactionStrategy.java:365)
 ~[apache-cassandra-2.1.0.jar:2.1.0]*
 *at
 org.apache.cassandra.db.compaction.LeveledCompactionStrategy.getMaximalTask(LeveledCompactionStrategy.java:127)
 ~[apache-cassandra-2.1.0.jar:2.1.0]*
 *at
 org.apache.cassandra.db.compaction.LeveledCompactionStrategy.getNextBackgroundTask(LeveledCompactionStrategy.java:112)
 ~[apache-cassandra-2.1.0.jar:2.1.0]*
 *at
 org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:229)
 ~[apache-cassandra-2.1.0.jar:2.1.0]*
 *at
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
 ~[na:1.7.0_51]*
 *at java.util.concurrent.FutureTask.run(FutureTask.java:262)
 ~[na:1.7.0_51]*
 *at
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
 ~[na:1.7.0_51]*
 *at
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
 [na:1.7.0_51]*
 *at java.lang.Thread.run(Thread.java:744) [na:1.7.0_51]*

 But I do not see any warning or error message in logs of the joining
 nodes.  I just see an exception there when I run: nodetool info:

 root@:~# nodetool info
 ID   : f5e49647-59fa-474f-b6af-9f65abc43581
 Gossip active: true
 Thrift active: false
 Native Transport active: false
 Load : 26.86 GB
 Generation No: 1430163258
 Uptime (seconds) : 18799
 Heap Memory (MB) : 4185.15 / 7566.00
 error: null
 -- StackTrace --
 java.lang.AssertionError
 at
 org.apache.cassandra.locator.TokenMetadata.getTokens(TokenMetadata.java:440)
 at
 org.apache.cassandra.service.StorageService.getTokens(StorageService.java:2079)
 at
 org.apache.cassandra.service.StorageService.getTokens(StorageService.java:2068)
 

Re: What is 'Read Reuqests' on OpsCenter exaclty?

[Carlos Rolo]

Let me try to reproduce your test and get back wiith some results.

Regards,

Carlos Juzarte Rolo
Cassandra Consultant

Pythian - Love your data

rolo@pythian | Twitter: cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo
http://linkedin.com/in/carlosjuzarterolo*
Mobile: +31 6 159 61 814 | Tel: +1 613 565 8696 x1649
www.pythian.com

On Fri, Apr 24, 2015 at 2:35 AM, Bongseo Jang grayce...@gmail.com wrote:

 Thanks a lot Carlos, Sebastian :-)

 My test was with 1 node/1 replica settings, on which I assumed client
 request = read request on the graph. Because there seems no read_repair and
 already CL=ONE in my case, I need more explanation, don't I? Or can any
 other internals be still involved?

 Do you have more suggestions? I want to design new test narrowing the gap
 on the suggestions.

 On 24 April 2015 at 00:23, Sebastian Estevez 
 sebastian.este...@datastax.com wrote:

 Carlos is right:

 *Read Requests* - The number of read requests per second on the
 coordinator nodes, analogous to client reads. Monitoring the number of
 requests over a given time period reveals system read workload and usage
 patterns.

 *Avg* - The average of values recorded during a time interval.

 A future version of OpsC will include tooltips with these descriptions
 for better clarity.
 On Apr 23, 2015 6:30 AM, Carlos Rolo r...@pythian.com wrote:

 Probably it takes in account the read repair, plus a read that have
 consistency != 1 will produce reads on other machines (which are taken in
 account). I don't know the internals of opscenter but I would assume that
 this is the case.

 If you want to test it further, disable read_repair, and make all your
 reads with CL=ONE. Then your client and Opscenter should match.

 PS: Speculative_retry could also send reads over to more machines.

 Regards,

 Carlos Juzarte Rolo
 Cassandra Consultant

 Pythian - Love your data

 rolo@pythian | Twitter: cjrolo | Linkedin: 
 *linkedin.com/in/carlosjuzarterolo
 http://linkedin.com/in/carlosjuzarterolo*
 Mobile: +31 6 159 61 814 | Tel: +1 613 565 8696 x1649
 www.pythian.com

 On Thu, Apr 23, 2015 at 10:34 AM, Bongseo Jang grayce...@gmail.com
 wrote:

 I have cassandra 2.1 + OpsCenter 5.1.1 and test them.

 When I monitored with opscenter 'read requests' graph, it seems the
 number on the graph is not what I expected, the number of client requests
 or responses.

 I recorded actual number of client request and compare it with graph,
 then found they're different. The number on the graph is about 4 times
 larger than what the client claimed.

 So, my question is what 'Read Reuqests' on OpsCenter counts exaclty ?

 Thanks !

 --
 Regards,
 Jang.

  a sound mind in a sound body



 --






 --
 Regards,
 Jang.

  a sound mind in a sound body


-- 


--

Re: What is 'Read Reuqests' on OpsCenter exaclty?

[Carlos Rolo]

Probably it takes in account the read repair, plus a read that have
consistency != 1 will produce reads on other machines (which are taken in
account). I don't know the internals of opscenter but I would assume that
this is the case.

If you want to test it further, disable read_repair, and make all your
reads with CL=ONE. Then your client and Opscenter should match.

PS: Speculative_retry could also send reads over to more machines.

Regards,

Carlos Juzarte Rolo
Cassandra Consultant

Pythian - Love your data

rolo@pythian | Twitter: cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo
http://linkedin.com/in/carlosjuzarterolo*
Mobile: +31 6 159 61 814 | Tel: +1 613 565 8696 x1649
www.pythian.com

On Thu, Apr 23, 2015 at 10:34 AM, Bongseo Jang grayce...@gmail.com wrote:

 I have cassandra 2.1 + OpsCenter 5.1.1 and test them.

 When I monitored with opscenter 'read requests' graph, it seems the number
 on the graph is not what I expected, the number of client requests or
 responses.

 I recorded actual number of client request and compare it with graph, then
 found they're different. The number on the graph is about 4 times larger
 than what the client claimed.

 So, my question is what 'Read Reuqests' on OpsCenter counts exaclty ?

 Thanks !

 --
 Regards,
 Jang.

  a sound mind in a sound body


-- 


--

Re: LCS Strategy, compaction pending tasks keep increasing

[Carlos Rolo]

Are you on version 2.1.x?

Regards,

Carlos Juzarte Rolo
Cassandra Consultant

Pythian - Love your data

rolo@pythian | Twitter: cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo
http://linkedin.com/in/carlosjuzarterolo*
Mobile: +31 6 159 61 814 | Tel: +1 613 565 8696 x1649
www.pythian.com

On Tue, Apr 21, 2015 at 1:06 PM, Anishek Agarwal anis...@gmail.com wrote:

 the some_bits column has about 14-15 bytes of data per key.

 On Tue, Apr 21, 2015 at 4:34 PM, Anishek Agarwal anis...@gmail.com
 wrote:

 Hello,

 I am inserting about 100 million entries via datastax-java driver to a
 cassandra cluster of 3 nodes.

 Table structure is as

 create keyspace test with replication = {'class':
 'NetworkTopologyStrategy', 'DC' : 3};

 CREATE TABLE test_bits(id bigint primary key , some_bits text) with
 gc_grace_seconds=0 and compaction = {'class': 'LeveledCompactionStrategy'}
 and compression={'sstable_compression' : ''};

 have 75 threads that are inserting data into the above table with each
 thread having non over lapping keys.

 I see that the number of pending tasks via nodetool compactionstats
 keeps increasing and looks like from nodetool cfstats test.test_bits has
 SSTTable levels as [154/4, 8, 0, 0, 0, 0, 0, 0, 0],

 Why is compaction not kicking in ?

 thanks
 anishek




-- 


--

Re: Adding nodes to existing cluster

[Carlos Rolo]

Start one node at a time. Wait 2 minutes before starting each node.


How much data and nodes you have already? Depending on that, the streaming
of data can stress on the resources you have.
I would recommend to start one and monitor, if things are ok, add another
one. And so on.

Regards,

Carlos Juzarte Rolo
Cassandra Consultant

Pythian - Love your data

rolo@pythian | Twitter: cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo
http://linkedin.com/in/carlosjuzarterolo*
Mobile: +31 6 159 61 814 | Tel: +1 613 565 8696 x1649
www.pythian.com

On Mon, Apr 20, 2015 at 11:02 AM, Or Sher or.sh...@gmail.com wrote:

 Hi all,
 In the near future I'll need to add more than 10 nodes to a 2.0.9
 cluster (using vnodes).
 I read this documentation on datastax website:

 http://docs.datastax.com/en/cassandra/2.0/cassandra/operations/ops_add_node_to_cluster_t.html

 In one point it says:
 If you are using racks, you can safely bootstrap two nodes at a time
 when both nodes are on the same rack.

 And in another is says:
 Start Cassandra on each new node. Allow two minutes between node
 initializations. You can monitor the startup and data streaming
 process using nodetool netstats.

 We're not using racks configuration and from reading this
 documentation I'm not really sure is it safe for us to bootstrap all
 nodes together (with two minutes between each other).
 I really hate the tought of doing it one by one, I assume it will take
 more than 6H per node.

 What do you say?
 --
 Or Sher


-- 


--

Re: Adding nodes to existing cluster

[Carlos Rolo]

Independent of the snitch, data needs to travel to the new nodes (plus all
the keyspace information that goes via gossip). So I won't bootstrap them
all at once, even if it is only for network traffic generated.

Don't forget to run cleanup on the old nodes once all nodes are in place to
reclaim disk space.

Regards,

Carlos Juzarte Rolo
Cassandra Consultant

Pythian - Love your data

rolo@pythian | Twitter: cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo
http://linkedin.com/in/carlosjuzarterolo*
Mobile: +31 6 159 61 814 | Tel: +1 613 565 8696 x1649
www.pythian.com

On Mon, Apr 20, 2015 at 1:58 PM, Or Sher or.sh...@gmail.com wrote:

 Thanks for the response.
 Sure we'll monitor as we're adding nodes.
 We're now using 6 nodes on each DC. (We have 2 DCs)
 Each node contains ~800GB

 Do you know how rack configurations are relevant here?
 Do you see any reason to bootstrap them one by one if we're not using
 rack awareness?


 On Mon, Apr 20, 2015 at 2:49 PM, Carlos Rolo r...@pythian.com wrote:
  Start one node at a time. Wait 2 minutes before starting each node.
 
 
  How much data and nodes you have already? Depending on that, the
 streaming
  of data can stress on the resources you have.
  I would recommend to start one and monitor, if things are ok, add another
  one. And so on.
 
  Regards,
 
  Carlos Juzarte Rolo
  Cassandra Consultant
 
  Pythian - Love your data
 
  rolo@pythian | Twitter: cjrolo | Linkedin:
 linkedin.com/in/carlosjuzarterolo
  Mobile: +31 6 159 61 814 | Tel: +1 613 565 8696 x1649
  www.pythian.com
 
  On Mon, Apr 20, 2015 at 11:02 AM, Or Sher or.sh...@gmail.com wrote:
 
  Hi all,
  In the near future I'll need to add more than 10 nodes to a 2.0.9
  cluster (using vnodes).
  I read this documentation on datastax website:
 
 
 http://docs.datastax.com/en/cassandra/2.0/cassandra/operations/ops_add_node_to_cluster_t.html
 
  In one point it says:
  If you are using racks, you can safely bootstrap two nodes at a time
  when both nodes are on the same rack.
 
  And in another is says:
  Start Cassandra on each new node. Allow two minutes between node
  initializations. You can monitor the startup and data streaming
  process using nodetool netstats.
 
  We're not using racks configuration and from reading this
  documentation I'm not really sure is it safe for us to bootstrap all
  nodes together (with two minutes between each other).
  I really hate the tought of doing it one by one, I assume it will take
  more than 6H per node.
 
  What do you say?
  --
  Or Sher
 
 
 
  --
 
 
 



 --
 Or Sher


-- 


--

Re: Best practice: Multiple clusters vs multiple tables in a single cluster?

[Carlos Rolo]

Adding a new keyspace should be perfectly fine. Unless you have completely
distinct workloads for the different keyspaces. Even so you can balanced
some stuff at keyspace/table level. But I would go with a new keyspace not
with a new cluster given the small size you say you have.

Regards,

Carlos Juzarte Rolo
Cassandra Consultant

Pythian - Love your data

rolo@pythian | Twitter: cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo
http://linkedin.com/in/carlosjuzarterolo*
Mobile: +31 6 159 61 814 | Tel: +1 613 565 8696 x1649
www.pythian.com

On Thu, Apr 2, 2015 at 3:06 PM, Ian Rose ianr...@fullstory.com wrote:

 Hi all -

 We currently have a single cassandra cluster that is dedicated to a
 relatively narrow purpose, with just 2 tables.  Soon we will need cassandra
 for another, unrelated, system, and my debate is whether to just add the
 new tables to our existing cassandra cluster or whether to spin up an
 entirely new, separate cluster for this new system.

 Does anyone have pros/cons to share on this?  It appears from watching
 talks and such online that the big users (e.g. Netflix, Spotify) tend to
 favor multiple, single-purpose clusters, and thus that was my initial
 preference.  But we are (for now) no where close to them in traffic so I'm
 wondering if running an entirely separate cluster would be a premature
 optimization which wouldn't pay for the (nontrivial) overhead in
 configuration management and ops.  While we are still small it might be
 much smarter to reuse our existing clusters so that I can get it done
 faster...

 Thanks!
 - Ian



-- 


--

Re: Replication to second data center with different number of nodes

[Carlos Rolo]

Sharing my experience here.

1) Never had any issues with different size DCs. If the hardware is the
same, keep the # to 256.
2) In most of the cases I keep the 256 vnodes and no performance problems
(when they are triggered, the cause is not the vnodes #)

Regards,

Carlos Juzarte Rolo
Cassandra Consultant

Pythian - Love your data

rolo@pythian | Twitter: cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo
http://linkedin.com/in/carlosjuzarterolo*
Tel: 1649
www.pythian.com

On Mon, Mar 30, 2015 at 6:31 AM, Anishek Agarwal anis...@gmail.com wrote:

 Colin,

 When you said larger number of tokens has Query performance hit, is it
 read or write performance. Also if you have any links you could share to
 shed some light on this it would be great.

 Thanks
 Anishek

 On Sun, Mar 29, 2015 at 2:20 AM, Colin Clark co...@clark.ws wrote:

 I typically use a # a lot lower than 256, usually less than 20 for
 num_tokens as a larger number has historically had a dramatic impact on
 query performance.
 —
 Colin Clark
 co...@clark.ws
 +1 612-859-6129
 skype colin.p.clark

 On Mar 28, 2015, at 3:46 PM, Eric Stevens migh...@gmail.com wrote:

 If you're curious about how Cassandra knows how to replicate data in the
 remote DC, it's the same as in the local DC, replication is independent in
 each, and you can even set a different replication strategy per keyspace
 per datacenter.  Nodes in each DC take up num_tokens positions on a ring,
 each partition key is mapped to a position on that ring, and whomever owns
 that part of the ring is the primary for that data.  Then (oversimplified)
 r-1 adjacent nodes become replicas for that same data.

 On Fri, Mar 27, 2015 at 6:55 AM, Sibbald, Charles 
 charles.sibb...@bskyb.com wrote:


 http://www.datastax.com/documentation/cassandra/2.0/cassandra/configuration/configCassandra_yaml_r.html?scroll=reference_ds_qfg_n1r_1k__num_tokens

  So go with a default 256, and leave initial token empty:

  num_tokens: 256

 # initial_token:


  Cassandra will always give each node the same number of tokens, the
 only time you might want to distribute this is if your instances are of
 different sizing/capability which is also a bad scenario.

   From: Björn Hachmann bjoern.hachm...@metrigo.de
 Reply-To: user@cassandra.apache.org user@cassandra.apache.org
 Date: Friday, 27 March 2015 12:11
 To: user user@cassandra.apache.org
 Subject: Re: Replication to second data center with different number of
 nodes


 2015-03-27 11:58 GMT+01:00 Sibbald, Charles charles.sibb...@bskyb.com:

 Cassandra’s Vnodes config


 ​Thank you. Yes, we are using vnodes! The num_token parameter controls
 the number of vnodes assigned to a specific node.​

  Might be I am seeing problems where are none.

  Let me rephrase my question: How does Cassandra know it has to
 replicate 1/3 of all keys to each single node in the second DC? I can see
 two ways:
  1. It has to be configured explicitly.
  2. It is derived from the number of nodes available in the data center
 at the time `nodetool rebuild` is started.

  Kind regards
 Björn
   Information in this email including any attachments may be
 privileged, confidential and is intended exclusively for the addressee. The
 views expressed may not be official policy, but the personal views of the
 originator. If you have received it in error, please notify the sender by
 return e-mail and delete it from your system. You should not reproduce,
 distribute, store, retransmit, use or disclose its contents to anyone.
 Please note we reserve the right to monitor all e-mail communication
 through our internal and external networks. SKY and the SKY marks are
 trademarks of Sky plc and Sky International AG and are used under licence.
 Sky UK Limited (Registration No. 2906991), Sky-In-Home Service Limited
 (Registration No. 2067075) and Sky Subscribers Services Limited
 (Registration No. 2340150) are direct or indirect subsidiaries of Sky plc
 (Registration No. 2247735). All of the companies mentioned in this
 paragraph are incorporated in England and Wales and share the same
 registered office at Grant Way, Isleworth, Middlesex TW7 5QD.






-- 


--

Re: Upgrade from 2.0.9 to 2.1.3

[Carlos Rolo]

I would not recommend an upgrade to 2.1.x for now. Do you have any specific
reason to upgrade?

For upgrading from 2.0.9 you can just do a direct upgrade.

Regards,

Carlos Juzarte Rolo
Cassandra Consultant

Pythian - Love your data

rolo@pythian | Twitter: cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo
http://linkedin.com/in/carlosjuzarterolo*
Tel: 1649
www.pythian.com

On Fri, Mar 6, 2015 at 3:03 PM, Fredrik Larsson Stigbäck 
fredrik.l.stigb...@sitevision.se wrote:

 What’s the recommended way of upgrading from 2.0.9 to 2.1.3?
 Is upgradeSSTables required?
 According to
 http://www.datastax.com/documentation/upgrade/doc/upgrade/cassandra/upgradeC_c.html
  it
 should be possible to just start the on 2.1.3 directly after 2.0.9.

 Regards
 Fredrik



-- 


--

Re: Write timeout under load but Read is fine

[Carlos Rolo]

What is the consistency level you are using?
Are you using Thrift or CQL?
Are you using SSDs?
Check if compactions are running when you get the timeouts.

Regards,

Carlos Juzarte Rolo
Cassandra Consultant

Pythian - Love your data

rolo@pythian | Twitter: cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo
http://linkedin.com/in/carlosjuzarterolo*
Tel: 1649
www.pythian.com

On Thu, Mar 5, 2015 at 9:51 PM, Jan cne...@yahoo.com wrote:

 Hello Jaydeep;

 Run *cassandra-stress* with R/W options enabled  for about the same time
 and check if you have dropped packets.
 It would eliminate the client as the source of the error  also give you a
 replicable tool to base subsequent tests/ findings.

 Jan/




   On Thursday, March 5, 2015 12:19 PM, Jaydeep Chovatia 
 chovatia.jayd...@gmail.com wrote:


 I have tried increasing timeout to 1 but no help. Also verified that
 there is no network lost packets.

 Jaydeep

 On Wed, Mar 4, 2015 at 12:19 PM, Jan cne...@yahoo.com wrote:

 HI Jaydeep;


- look at the i/o  on all three nodes
- Increase the write_request_timeout_in_ms: 1
- check the time-outs if any on the client inserting the Writes
- check the Network for  dropped/lost packets


 hope this helps
 Jan/



   On Wednesday, March 4, 2015 12:26 PM, Jaydeep Chovatia 
 chovatia.jayd...@gmail.com wrote:


 Hi,

 In my test program when I increase load then I keep getting few write
 timeout from Cassandra say every 10~15 mins. My read:write ratio is
 50:50. My reads are fine but only writes time out.

 Here is my Cassandra details:
 Version: 2.0.11
 Ring of 3 nodes with RF=3
 Node configuration: 24 core + 64GB RAM + 2TB

 write_request_timeout_in_ms: 5000, rest of Cassandra.yaml configuration
 is default

 I've also checked IO on Cassandra nodes and looks very low (around 5%).
 I've also checked Cassandra log file and do not see any GC happening. Also
 CPU on Cassandra is low (around 20%). I have 20GB data on each node.

 My test program creates connection to all three Cassandra nodes and sends
 read+write request randomly.

 Any idea what should I look for?

 Jaydeep





 --
 Jaydeep




-- 


--

Re: Unexplained query slowness

[Carlos Rolo]

You can use query tracing to check what is happening. Also you fire
jconsole/JavaVisualVM and push out some metrics like the 99th read Beans
for that column family.
A simpler check is using cfstats and look for weird numbers (high number
sstables, if you are deleting check how much tombstones per scan, etc).

Another is checking if compactions are not running when you query.
Opscenter can provide some graphs and help out.

Regards,

Carlos Juzarte Rolo
Cassandra Consultant

Pythian - Love your data

rolo@pythian | Twitter: cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo
http://linkedin.com/in/carlosjuzarterolo*
Tel: 1649
www.pythian.com

On Wed, Feb 25, 2015 at 4:32 PM, Robert Wille rwi...@fold3.com wrote:

 Our Cassandra database just rolled to live last night. I’m looking at our
 query performance, and overall it is very good, but perhaps 1 in 10,000
 queries takes several hundred milliseconds (up to a full second). I’ve
 grepped for GC in the system.log on all nodes, and there aren’t any recent
 GC events. I’m executing ~500 queries per second, which produces negligible
 load and CPU utilization. I have very minimal writes (one every few
 minutes). The slow queries are across the board. There isn’t one particular
 query that is slow.

 I’m running 2.0.12 with SSD’s. I’ve got a 10 node cluster with RF=3.

 I have no idea where to even begin to look. Any thoughts on where to start
 would be greatly appreciated.

 Robert



-- 


--

Re: Setting up JNA on CentOS 6.6. with cassandra20-2.0.12 and Oracle Java 1.7.0_75

[Carlos Rolo]

Hello,

I always install JNA into the lib directory of java itself

Since I normally have java in /opt/java I put the JNA into /opt/java/lib.

~$ grep  JNA /var/log/cassandra/system.log
INFO  HH:MM:SS JNA mlockall successful

Regards,

Carlos Juzarte Rolo
Cassandra Consultant

Pythian - Love your data

rolo@pythian | Twitter: cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo
http://linkedin.com/in/carlosjuzarterolo*
Tel: 1649
www.pythian.com

On Wed, Feb 25, 2015 at 9:12 PM, Garret Pick pic...@whistle.com wrote:

 Hello,

 I'm having problems getting cassandra to start with the configuration
 listed above.

 Yum wants to install 3.2.4-2.el6 of the JNA along with several other
 packages including java-1.7.0-openjdk

 The documentation states that a JNA version earlier that 3.2.7 should not
 be used, so the jar file should be downloaded and installed directly into
 C*'s lib directory per


 http://www.datastax.com/documentation/cassandra/2.0/cassandra/install/installJnaTar.html

 From /var/log/cassandra/system.log

 all I see is

  INFO [main] 2015-02-25 20:06:10,202 CassandraDaemon.java (line 191)
 Classpath:
 /etc/cassandra/conf:/usr/share/cassandra/lib/antlr-3.2.jar:/usr/share/cassandra/lib/apache-cassandra-2.0.12.jar:/usr/share/cassandra/lib/apache-cassandra-clientutil-2.0.12.jar:/usr/share/cassandra/lib/apache-cassandra-thrift-2.0.12.jar:/usr/share/cassandra/lib/commons-cli-1.1.jar:/usr/share/cassandra/lib/commons-codec-1.2.jar:/usr/share/cassandra/lib/commons-lang3-3.1.jar:/usr/share/cassandra/lib/compress-lzf-0.8.4.jar:/usr/share/cassandra/lib/concurrentlinkedhashmap-lru-1.3.jar:/usr/share/cassandra/lib/disruptor-3.0.1.jar:/usr/share/cassandra/lib/guava-15.0.jar:/usr/share/cassandra/lib/high-scale-lib-1.1.2.jar:/usr/share/cassandra/lib/jackson-core-asl-1.9.2.jar:/usr/share/cassandra/lib/jackson-mapper-asl-1.9.2.jar:/usr/share/cassandra/lib/jamm-0.2.5.jar:/usr/share/cassandra/lib/jbcrypt-0.3m.jar:/usr/share/cassandra/lib/jline-1.0.jar:/usr/share/cassandra/lib/jna.jar:/usr/share/cassandra/lib/json-simple-1.1.jar:/usr/share/cassandra/lib/libthrift-0.9.1.jar:/usr/share/cassandra/lib/log4j-1.2.16.jar:/usr/share/cassandra/lib/lz4-1.2.0.jar:/usr/share/cassandra/lib/metrics-core-2.2.0.jar:/usr/share/cassandra/lib/netty-3.6.6.Final.jar:/usr/share/cassandra/lib/reporter-config-2.1.0.jar:/usr/share/cassandra/lib/servlet-api-2.5-20081211.jar:/usr/share/cassandra/lib/slf4j-api-1.7.2.jar:/usr/share/cassandra/lib/slf4j-log4j12-1.7.2.jar:/usr/share/cassandra/lib/snakeyaml-1.11.jar:/usr/share/cassandra/lib/snappy-java-1.0.5.jar:/usr/share/cassandra/lib/snaptree-0.1.jar:/usr/share/cassandra/lib/stress.jar:/usr/share/cassandra/lib/super-csv-2.1.0.jar:/usr/share/cassandra/lib/thrift-server-0.3.7.jar:/usr/share/cassandra/lib/jamm-0.2.5.jar

 and it never actually starts

 Note that JNA is in the classpath above and is when I remove it, cassandra
 starts successfully.

 I tried installing the DSE package and it looks like it wants to install
 the older 3.2.4 JNA as a dependency so there seems to be a discrepancy in
 documentation

 Per


 http://www.datastax.com/documentation/datastax_enterprise/4.6/datastax_enterprise/install/installRHELdse.html

 Note: JNA (Java Native Access) is automatically installed.

 thanks for any help,
 Garret


-- 


--

Re: Setting up JNA on CentOS 6.6. with cassandra20-2.0.12 and Oracle Java 1.7.0_75

[Carlos Rolo]

Also I always install JNA from the JNA page.

I did the installation for this blog post in CentOS 6.5:
http://www.pythian.com/blog/from-0-to-cassandra-an-exhaustive-approach-to-installing-cassandra/

Regards,

Carlos Juzarte Rolo
Cassandra Consultant

Pythian - Love your data

rolo@pythian | Twitter: cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo
http://linkedin.com/in/carlosjuzarterolo*
Tel: 1649
www.pythian.com

On Wed, Feb 25, 2015 at 9:53 PM, Garret Pick pic...@whistle.com wrote:

 Hi,

 On this page


 http://www.datastax.com/documentation/cassandra/2.0/cassandra/install/installJnaRHEL.html

 it says

 Cassandra requires JNA 3.2.7 or later. Some Yum repositories may provide
 earlier versions

 and at the bottom

 If you can't install using Yum or it provides a version of the JNA
 earlier than 3.2.7, install as described in Installing the JNA from the JAR
 file.

 Which version of OS and Cassandra are you running?

 thanks,
 Garret




 On Wed, Feb 25, 2015 at 10:46 AM, J. Ryan Earl o...@jryanearl.us wrote:
 
  We've been using jna-3.2.4-2.el6.x86_64 with the Sun/Oracle JDK for
 probably 2-years now, and it works just fine.  Where are you seeing 3.2.7
 required at?  I searched the pages you link and that string isn't even in
 there.
 
  Regardless, I assure you the newest jna that ships in the EL6 repo works
 without issues.
 
  On Wed, Feb 25, 2015 at 2:12 PM, Garret Pick pic...@whistle.com wrote:
 
  Hello,
 
  I'm having problems getting cassandra to start with the configuration
 listed above.
 
  Yum wants to install 3.2.4-2.el6 of the JNA along with several other
 packages including java-1.7.0-openjdk
 
  The documentation states that a JNA version earlier that 3.2.7 should
 not be used, so the jar file should be downloaded and installed directly
 into C*'s lib directory per
 
 
 http://www.datastax.com/documentation/cassandra/2.0/cassandra/install/installJnaTar.html
 
  From /var/log/cassandra/system.log
 
  all I see is
 
   INFO [main] 2015-02-25 20:06:10,202 CassandraDaemon.java (line 191)
 Classpath:
 /etc/cassandra/conf:/usr/share/cassandra/lib/antlr-3.2.jar:/usr/share/cassandra/lib/apache-cassandra-2.0.12.jar:/usr/share/cassandra/lib/apache-cassandra-clientutil-2.0.12.jar:/usr/share/cassandra/lib/apache-cassandra-thrift-2.0.12.jar:/usr/share/cassandra/lib/commons-cli-1.1.jar:/usr/share/cassandra/lib/commons-codec-1.2.jar:/usr/share/cassandra/lib/commons-lang3-3.1.jar:/usr/share/cassandra/lib/compress-lzf-0.8.4.jar:/usr/share/cassandra/lib/concurrentlinkedhashmap-lru-1.3.jar:/usr/share/cassandra/lib/disruptor-3.0.1.jar:/usr/share/cassandra/lib/guava-15.0.jar:/usr/share/cassandra/lib/high-scale-lib-1.1.2.jar:/usr/share/cassandra/lib/jackson-core-asl-1.9.2.jar:/usr/share/cassandra/lib/jackson-mapper-asl-1.9.2.jar:/usr/share/cassandra/lib/jamm-0.2.5.jar:/usr/share/cassandra/lib/jbcrypt-0.3m.jar:/usr/share/cassandra/lib/jline-1.0.jar:/usr/share/cassandra/lib/jna.jar:/usr/share/cassandra/lib/json-simple-1.1.jar:/usr/share/cassandra/lib/libthrift-0.9.1.jar:/usr/share/cassandra/lib/log4j-1.2.16.jar:/usr/share/cassandra/lib/lz4-1.2.0.jar:/usr/share/cassandra/lib/metrics-core-2.2.0.jar:/usr/share/cassandra/lib/netty-3.6.6.Final.jar:/usr/share/cassandra/lib/reporter-config-2.1.0.jar:/usr/share/cassandra/lib/servlet-api-2.5-20081211.jar:/usr/share/cassandra/lib/slf4j-api-1.7.2.jar:/usr/share/cassandra/lib/slf4j-log4j12-1.7.2.jar:/usr/share/cassandra/lib/snakeyaml-1.11.jar:/usr/share/cassandra/lib/snappy-java-1.0.5.jar:/usr/share/cassandra/lib/snaptree-0.1.jar:/usr/share/cassandra/lib/stress.jar:/usr/share/cassandra/lib/super-csv-2.1.0.jar:/usr/share/cassandra/lib/thrift-server-0.3.7.jar:/usr/share/cassandra/lib/jamm-0.2.5.jar
 
  and it never actually starts
 
  Note that JNA is in the classpath above and is when I remove it,
 cassandra starts successfully.
 
  I tried installing the DSE package and it looks like it wants to
 install the older 3.2.4 JNA as a dependency so there seems to be a
 discrepancy in documentation
 
  Per
 
 
 http://www.datastax.com/documentation/datastax_enterprise/4.6/datastax_enterprise/install/installRHELdse.html
 
  Note: JNA (Java Native Access) is automatically installed.
 
  thanks for any help,
  Garret
 
 


-- 


--

Re: Possible problem with disk latency

[Carlos Rolo]

Your latency doesn't seem that high that can cause that problem. I suspect
more of a problem with the Cassandra version (2.1.3) than that with the
hard drives. I didn't look deep into the information provided but for your
reference, the only time I had serious (leading to OOM and all sort of
weird behavior) my hard drives where near 70ms latency.

Regards,

Carlos Juzarte Rolo
Cassandra Consultant

Pythian - Love your data

rolo@pythian | Twitter: cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo
http://linkedin.com/in/carlosjuzarterolo*
Tel: 1649
www.pythian.com

On Wed, Feb 25, 2015 at 11:19 AM, Ja Sam ptrstp...@gmail.com wrote:

 Hi,
 I write some question before about my problems with C* cluster. All my
 environment is described here:
 https://www.mail-archive.com/user@cassandra.apache.org/msg40982.html
 To sum up I have thousands SSTables in one DC and much much less in
 second. I write only to first DC.

 Anyway after reading a lot of post/mails/google I start to think that the
 only reason of above is disk problems.

 My OpsCenter with some stats is following:
 https://drive.google.com/file/d/0B4N_AbBPGGwLR21CZk9OV1kxVDA/view

 My iostats are like this:
 https://drive.google.com/file/d/0B4N_AbBPGGwLTTZEeG1SYkF0cXc/view
 (dm-XX are C* drives. dm-11 is for commitlog)

 If You could be so kind and validate above and give me an answer is my
 disk are real problems or not? And give me a tip what should I do with
 above cluster? Maybe I have misconfiguration?

 Regards
 Piotrek


-- 


--

Re: AMI to use to launch a cluster with OpsCenter on AWS

[Carlos Rolo]

Regarding AWS the only thing I normally do (besides the normal
installation, etc) is setting up the firewall zones so the ports needed for
Cassandra are open.

You can follow this guide:
https://razvantudorica.com/02/create-a-cassandra-cluster-with-opscenter-on-amazon-ec2/a

Regards,

Carlos Juzarte Rolo
Cassandra Consultant

Pythian - Love your data

rolo@pythian | Twitter: cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo
http://linkedin.com/in/carlosjuzarterolo*
Tel: 1649
www.pythian.com

On Sat, Feb 21, 2015 at 4:48 AM, Clint Kelly clint.ke...@gmail.com wrote:

 BTW I was able to use this script:

 https://github.com/joaquincasares/cassandralauncher

 to get a cluster up and running pretty easily on AWS.  Cheers to the
 author for this.

 Still curious for answers to my questions above, but not as urgent.

 Best regards,
 Clint


 On Fri, Feb 20, 2015 at 5:36 PM, Clint Kelly clint.ke...@gmail.com
 wrote:

 Hi all,

 I am trying to follow the instructions here for installing DSE 4.6 on AWS:


 http://www.datastax.com/documentation/datastax_enterprise/4.6/datastax_enterprise/install/installAMIOpsc.html

 I was successful creating a single-node instance running OpsCenter, which
 I intended to bootstrap creating a larger cluster running Cassandra and
 Spark.

 During my first attempt, however, OpsCenter reported problems talking to
 agents in the new cluster I was creating.  I ssh'ed into one of the new
 instances that I created with OpsCenter and saw that this was the problem:

 

 DataStax AMI for DataStax Enterprise

 and DataStax Community

 AMI version 2.4



 

 DataStax AMI 2.5 released 02.25.2014

 http://goo.gl/g1RRd7


 This AMI (version 2.4) will be left

 available, but no longer updated.

 



 These notices occurred during the startup of this instance:

 [ERROR] 02/21/15-00:53:01 sudo chown -R cassandra:cassandra
 /mnt/cassandra:

 [WARN] Permissions not set correctly. Please run manually:

 [WARN] sudo chown -hR cassandra:cassandra /mnt/cassandra

 [WARN] sudo service dse restart


 It looks like by default, the OpsCenter GUI selects an out-of-date AMI
 (ami-4c32ba7c)
 when you click on Create Cluster and attempt to create a brand-new
 cluster on EC2.

 What is the recommended image to use here?  I found a version 2.5.1 of
 the autoclustering AMI (
 http://thecloudmarket.com/image/ami-ada2b6c4--datastax-auto-clustering-ami-2-5-1-hvm).
 Is that correct?  Or should I be using one of the regular AMIs listed at
 http://www.datastax.com/documentation/datastax_enterprise/4.6/datastax_enterprise/install/installAMIOpsc.html
 ?  Or just a standard ubuntu image?

 FWIW, I tried just using one of the AMIs listed on the DSE 4.6 page
 (ami-32f7c977), and I still see the Waiting for the agent to start
 message, although if I log in, things look like they have kind of worked:

 Cluster started with these options:

 None


 Raiding complete.


 Waiting for nodetool...

 The cluster is now in it's finalization phase. This should only take a
 moment...


 Note: You can also use CTRL+C to view the logs if desired:

 AMI log: ~/datastax_ami/ami.log

 Cassandra log: /var/log/cassandra/system.log



 Datacenter: us-west-2

 =

 Status=Up/Down

 |/ State=Normal/Leaving/Joining/Moving

 --  Address  Load   Tokens  Owns (effective)  Host ID
   Rack

 UN  10.28.24.19  51.33 KB   256 100.0%
  ce0d365a-d58b-4700-b861-9f30af400476  2a



 Opscenter: http://ec2-54-71-102-180.us-west-2.compute.amazonaws.com:/

 Please wait 60 seconds if this is the cluster's first start...



 Tools:

 Run: datastax_tools

 Demos:

 Run: datastax_demos

 Support:

 Run: datastax_support



 

 DataStax AMI for DataStax Enterprise

 and DataStax Community

 AMI version 2.5

 DataStax Community version 2.1.3-1


 



 These notices occurred during the startup of this instance:

 [ERROR] 02/21/15-01:18:55 sudo chown opscenter-agent:opscenter-agent
 /var/lib/datastax-agent/conf:

 [ERROR] 02/21/15-01:19:04 sudo chown -R opscenter-agent:opscenter-agent
 /var/log/datastax-agent:

 [ERROR] 02/21/15-01:19:04 sudo chown -R opscenter-agent:opscenter-agent
 /mnt/datastax-agent:


 I would appreciate any help...  I assume what I'm trying to do here is
 pretty common.





-- 


--

Re: C* 2.1.2 invokes oom-killer

[Carlos Rolo]

So compaction doesn't seem to be your problem (You can check with nodetool
compactionstats just to be sure).

How much is your write latency on your column families? I had OOM related
to this before, and there was a tipping point around 70ms.

-- 


--

Re: C* 2.1.2 invokes oom-killer

[Carlos Rolo]

Do you have trickle_fsync enabled? Try to enable that and see if it solves
your problem, since you are getting out of non-heap memory.

Another question, is always the same nodes that die? Or is 2 out of 4 that
die?

Regards,

Carlos Juzarte Rolo
Cassandra Consultant

Pythian - Love your data

rolo@pythian | Twitter: cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo
http://linkedin.com/in/carlosjuzarterolo*
Tel: 1649
www.pythian.com

On Thu, Feb 19, 2015 at 10:49 AM, Michał Łowicki mlowi...@gmail.com wrote:



 On Thu, Feb 19, 2015 at 10:41 AM, Carlos Rolo r...@pythian.com wrote:

 So compaction doesn't seem to be your problem (You can check with
 nodetool compactionstats just to be sure).


 pending tasks: 0



 How much is your write latency on your column families? I had OOM related
 to this before, and there was a tipping point around 70ms.


 Write request latency is below 0.05 ms/op (avg). Checked with OpsCenter.



 --






 --
 BR,
 Michał Łowicki


-- 


--

Re: C* 2.1.2 invokes oom-killer

[Carlos Rolo]

Then you are probably hitting a bug... Trying to find out in Jira. The bad
news is the fix is only to be released on 2.1.4. Once I find it out I will
post it here.

Regards,

Carlos Juzarte Rolo
Cassandra Consultant

Pythian - Love your data

rolo@pythian | Twitter: cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo
http://linkedin.com/in/carlosjuzarterolo*
Tel: 1649
www.pythian.com

On Thu, Feb 19, 2015 at 12:16 PM, Michał Łowicki mlowi...@gmail.com wrote:

 |trickle_fsync| has been enabled for long time in our settings (just
 noticed):

 trickle_fsync: true

 trickle_fsync_interval_in_kb: 10240

 On Thu, Feb 19, 2015 at 12:12 PM, Michał Łowicki mlowi...@gmail.com
 wrote:



 On Thu, Feb 19, 2015 at 11:02 AM, Carlos Rolo r...@pythian.com wrote:

 Do you have trickle_fsync enabled? Try to enable that and see if it
 solves your problem, since you are getting out of non-heap memory.

 Another question, is always the same nodes that die? Or is 2 out of 4
 that die?


 Always the same nodes. Upgraded to 2.1.3 two hours ago so we'll monitor
 if maybe issue has been fixed there. If not will try to enable
 |tricke_fsync|



 Regards,

 Carlos Juzarte Rolo
 Cassandra Consultant

 Pythian - Love your data

 rolo@pythian | Twitter: cjrolo | Linkedin: 
 *linkedin.com/in/carlosjuzarterolo
 http://linkedin.com/in/carlosjuzarterolo*
 Tel: 1649
 www.pythian.com

 On Thu, Feb 19, 2015 at 10:49 AM, Michał Łowicki mlowi...@gmail.com
 wrote:



 On Thu, Feb 19, 2015 at 10:41 AM, Carlos Rolo r...@pythian.com wrote:

 So compaction doesn't seem to be your problem (You can check with
 nodetool compactionstats just to be sure).


 pending tasks: 0



 How much is your write latency on your column families? I had OOM
 related to this before, and there was a tipping point around 70ms.


 Write request latency is below 0.05 ms/op (avg). Checked with OpsCenter.



 --






 --
 BR,
 Michał Łowicki



 --






 --
 BR,
 Michał Łowicki




 --
 BR,
 Michał Łowicki


-- 


--

Re: can't delete tmp file

[Carlos Rolo]

You should upgrade to 2.1.3 for sure.

Check the changelog here:
https://git1-us-west.apache.org/repos/asf?p=cassandra.git;a=blob_plain;f=CHANGES.txt;hb=refs/tags/cassandra-2.1.3


Regards,

Carlos Juzarte Rolo
Cassandra Consultant

Pythian - Love your data

rolo@pythian | Twitter: cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo
http://linkedin.com/in/carlosjuzarterolo*
Tel: 1649
www.pythian.com

On Thu, Feb 19, 2015 at 1:44 PM, 曹志富 cao.zh...@gmail.com wrote:

 Thanks you Roland

 --
 曹志富
 手机：18611121927
 邮箱：caozf.zh...@gmail.com
 微博：http://weibo.com/boliza/

 2015-02-19 20:32 GMT+08:00 Roland Etzenhammer r.etzenham...@t-online.de:

 Hi,

 try 2.1.3 - with 2.1.2 this is normal. From the changelog:

 * Make sure we don't add tmplink files to the compaction strategy
 (CASSANDRA-8580)
 * Remove tmplink files for offline compactions (CASSANDRA-8321)

 In most cases they are safe to delete, I did this when the node was down.

 Cheers,
 Roland




-- 


--

Re: run cassandra on a small instance

[Carlos Rolo]

What I normally do is install plain CentOS (Not any AMI build for
Cassandra) and I don't use them for production! I run them for testing,
fire drills and some cassandra-stress benchmarks. I will look if I had more
than 5h Cassandra uptime. I can even put one up now and do the test and get
the results back to you.

Regards,

Carlos Juzarte Rolo
Cassandra Consultant

Pythian - Love your data

rolo@pythian | Twitter: cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo
http://linkedin.com/in/carlosjuzarterolo*
Tel: 1649
www.pythian.com

On Thu, Feb 19, 2015 at 6:41 PM, Tim Dunphy bluethu...@gmail.com wrote:

 I have Cassandra instances running on VMs with smaller RAM (1GB even) and
 I don't go OOM when testing them. Although I use them in AWS and other
 providers, never tried Digital Ocean.
 Does Cassandra just fails after some time running or it is failing on
 some specific read/write?


 Hi  Carlos,

 Ok, that's really interesting. So I have to ask, did you have to do
 anything special to get Cassandra to run on those 1GB AWS instances? I'd
 love to do the same. I even tried there as well and failed due to lack of
 memory to run it.

 And there is no specific reason other than lack of memory that I can tell
 for it to fail. And it doesn's seem to matter what data I use either.
 Because even if I remove the data directory with rm -rf, the phenomenon is
 the same. It'll run for a while, usually about 5 hours and then just crash
 with the word 'killed' as the last line of output.

 Thanks
 Tim


 On Thu, Feb 19, 2015 at 3:40 AM, Carlos Rolo r...@pythian.com wrote:

 I have Cassandra instances running on VMs with smaller RAM (1GB even) and
 I don't go OOM when testing them. Although I use them in AWS and other
 providers, never tried Digital Ocean.

 Does Cassandra just fails after some time running or it is failing on
 some specific read/write?

 Regards,

 Carlos Juzarte Rolo
 Cassandra Consultant

 Pythian - Love your data

 rolo@pythian | Twitter: cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo
 http://linkedin.com/in/carlosjuzarterolo*
 Tel: 1649
 www.pythian.com

 On Thu, Feb 19, 2015 at 7:16 AM, Tim Dunphy bluethu...@gmail.com wrote:

 Hey guys,

 After the upgrade to 2.1.3, and after almost exactly 5 hours running
 cassandra did indeed crash again on the 2GB ram VM.

 This is how the memory on the VM looked after the crash:

 [root@web2:~] #free -m
  total   used   free sharedbuffers cached
 Mem:  2002   1227774  8 45386
 -/+ buffers/cache:794   1207
 Swap:0  0  0


 And that's with this set in the cassandra-env.sh file:

 MAX_HEAP_SIZE=800M
 HEAP_NEWSIZE=200M

 So I'm thinking now, do I just have to abandon this idea I have of
 running Cassandra on a 2GB instance? Or is this something we can all agree
 can be done? And if so, how can we do that? :)

 Thanks
 Tim

 On Wed, Feb 18, 2015 at 8:39 PM, Jason Kushmaul | WDA 
 jason.kushm...@wda.com wrote:

 I asked this previously when a similar message came through, with a
 similar response.



 planetcassandra seems to have it “right”, in that stable=2.0,
 development=2.1, whereas the apache site says stable is 2.1.

 “Right” in they assume latest minor version is development.  Why not
 have the apache site do the same?  That’s just my lowly non-contributing
 opinion though.



 *Jason  *



 *From:* Andrew [mailto:redmu...@gmail.com]
 *Sent:* Wednesday, February 18, 2015 8:26 PM
 *To:* Robert Coli; user@cassandra.apache.org
 *Subject:* Re: run cassandra on a small instance



 Robert,



 Let me know if I’m off base about this—but I feel like I see a lot of
 posts that are like this (i.e., use this arbitrary version, not this other
 arbitrary version).  Why are releases going out if they’re “broken”?  This
 seems like a very confusing way for new (and existing) users to approach
 versions...



 Andrew



 On February 18, 2015 at 5:16:27 PM, Robert Coli (rc...@eventbrite.com)
 wrote:

 On Wed, Feb 18, 2015 at 5:09 PM, Tim Dunphy bluethu...@gmail.com
 wrote:

 I'm attempting to run Cassandra 2.1.2 on a smallish 2.GB ram instance
 over at Digital Ocean. It's a CentOS 7 host.



 2.1.2 is IMO broken and should not be used for any purpose.



 Use 2.1.1 or 2.1.3.




 https://engineering.eventbrite.com/what-version-of-cassandra-should-i-run/



 =Rob






 --
 GPG me!!

 gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B



 --






 --
 GPG me!!

 gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B



-- 


--

Re: Many pending compactions

[Carlos Rolo]

Hi 100% in agreement with Roland,

2.1.x series is a pain! I would never recommend the current 2.1.x series
for production.

Clocks is a pain, and check your connectivity! Also check tpstats to see if
your threadpools are being overrun.

Regards,

Carlos Juzarte Rolo
Cassandra Consultant

Pythian - Love your data

rolo@pythian | Twitter: cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo
http://linkedin.com/in/carlosjuzarterolo*
Tel: 1649
www.pythian.com

On Mon, Feb 16, 2015 at 8:12 PM, Roland Etzenhammer 
r.etzenham...@t-online.de wrote:

 Hi,

 1) Actual Cassandra 2.1.3, it was upgraded from 2.1.0 (suggested by Al
 Tobey from DataStax)
 7) minimal reads (usually none, sometimes few)

 those two points keep me repeating an anwser I got. First where did you
 get 2.1.3 from? Maybe I missed it, I will have a look. But if it is 2.1.2
 whis is the latest released version, that version has many bugs - most of
 them I got kicked by while testing 2.1.2. I got many problems with
 compactions not beeing triggred on column families not beeing read,
 compactions and repairs not beeing completed.  See

 https://www.mail-archive.com/search?l=user@cassandra.
 apache.orgq=subject:%22Re%3A+Compaction+failing+to+trigger%
 22o=newestf=1
 https://www.mail-archive.com/user%40cassandra.apache.org/msg40768.html

 Apart from that, how are those both datacenters connected? Maybe there is
 a bottleneck.

 Also do you have ntp up and running on all nodes to keep all clocks in
 thight sync?

 Note: I'm no expert (yet) - just sharing my 2 cents.

 Cheers,
 Roland


-- 


--

Re: best supported spark connector for Cassandra

[Carlos Rolo]

Not for sure ;)

If you need Cassandra support I can forward you to someone to talk to at
Pythian.

Regards,

Regards,

Carlos Juzarte Rolo
Cassandra Consultant

Pythian - Love your data

rolo@pythian | Twitter: cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo
http://linkedin.com/in/carlosjuzarterolo*
Tel: 1649
www.pythian.com

On Fri, Feb 13, 2015 at 3:05 PM, Marcelo Valle (BLOOMBERG/ LONDON) 
mvallemil...@bloomberg.net wrote:

 Actually, I am not the one looking for support, but I thank you a lot
 anyway.
 But from your message I guess the answer is yes, Datastax is not the only
 Cassandra vendor offering support and changing official Cassandra source at
 this moment, is this right?

 From: user@cassandra.apache.org
 Subject: Re: best supported spark connector for Cassandra

 Of course, Stratio Deep and Stratio Cassandra are licensed  Apache 2.0.

 Regarding the Cassandra support, I can introduce you to someone in Stratio
 that can help you.

 2015-02-12 15:05 GMT+01:00 Marcelo Valle (BLOOMBERG/ LONDON) 
 mvallemil...@bloomberg.net:

 Thanks for the hint Gaspar.
 Do you know if Stratio Deep / Stratio Cassandra are also licensed Apache
 2.0?

 I had interest in knowing more about Stratio when I was working on a
 start up. Now, on a blueship, it seems one of the hardest obstacles to use
 Cassandra in a project is the need of an area supporting it, and it seems
 people are specially concerned about how many vendors an open source
 solution has to provide support.

 This seems to be kind of an advantage of HBase, as there are many vendors
 supporting it, but I wonder if Stratio can be considered an alternative to
 Datastax reggarding Cassandra support?

 It's not my call here to decide anything, but as part of the community it
 helps to have this business scenario clear. I could say Cassandra could be
 the best fit technical solution for some projects but sometimes
 non-technical factors are in the game, like this need for having more than
 one vendor available...


 From: gmu...@stratio.com
 Subject: Re: best supported spark connector for Cassandra

 My suggestion is to use Java or Scala instead of Python. For Java/Scala
 both the Datastax and Stratio drivers are valid and similar options. As far
 as I know they both take care about data locality and are not based on the
 Hadoop interface. The advantage of Stratio Deep is that allows you to
 integrate Spark not only with Cassandra but with MongoDB, Elasticsearch,
 Aerospike and others as well.
 Stratio has a forked Cassandra for including some additional features
 such as Lucene based secondary indexes. So Stratio driver works fine with
 the Apache Cassandra and also with their fork.

 You can find some examples of using Deep here:
 https://github.com/Stratio/deep-examples  Please if you need some help
 with Stratio Deep do not hesitate to contact us.


 2015-02-11 17:18 GMT+01:00 shahab shahab.mok...@gmail.com:

 I am using Calliope cassandra-spark connector(
 http://tuplejump.github.io/calliope/), which is quite handy and easy to
 use!
 The only problem is that it is a bit outdates , works with Spark 1.1.0,
 hopefully new version comes soon.

 best,
 /Shahab

 On Wed, Feb 11, 2015 at 2:51 PM, Marcelo Valle (BLOOMBERG/ LONDON) 
 mvallemil...@bloomberg.net wrote:

 I just finished a scala course, nice exercise to check what I learned :D

 Thanks for the answer!

 From: user@cassandra.apache.org
 Subject: Re: best supported spark connector for Cassandra

 Start looking at the Spark/Cassandra connector here (in Scala):
 https://github.com/datastax/spark-cassandra-connector/tree/master/spark-cassandra-connector/src/main/scala/com/datastax/spark/connector

 Data locality is provided by this method:
 https://github.com/datastax/spark-cassandra-connector/blob/master/spark-cassandra-connector/src/main/scala/com/datastax/spark/connector/rdd/CassandraRDD.scala#L329-L336

 Start digging from this all the way down the code.

 As for Stratio Deep, I can't tell how the did the integration with
 Spark. Take some time to dig down their code to understand the logic.



 On Wed, Feb 11, 2015 at 2:25 PM, Marcelo Valle (BLOOMBERG/ LONDON) 
 mvallemil...@bloomberg.net wrote:

 Taking the opportunity Spark was being discussed in another thread, I
 decided to start a new one as I have interest in using Spark + Cassandra 
 in
 the feature.

 About 3 years ago, Spark was not an existing option and we tried to
 use hadoop to process Cassandra data. My experience was horrible and we
 reached the conclusion it was faster to develop an internal tool than
 insist on Hadoop _for our specific case_.

 How I can see Spark is starting to be known as a better hadoop and
 it seems market is going this way now. I can also see I have many more
 options to decide how to integrate Cassandra using the Spark RDD concept
 than using the ColumnFamilyInputFormat.

 I have found this java driver made by Datastax:
 https://github.com/datastax/spark-cassandra-connector

 I also have found python 

Re: Adding new node - OPSCenter problems

[Carlos Rolo]

Hello,

What is the output of nodetool status? All nodes should appear, otherwise
there is some configuration error.

Regards,

Carlos Juzarte Rolo
Cassandra Consultant

Pythian - Love your data

rolo@pythian | Twitter: cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo
http://linkedin.com/in/carlosjuzarterolo*
Tel: 1649
www.pythian.com

On Wed, Feb 11, 2015 at 3:46 PM, Batranut Bogdan batra...@yahoo.com wrote:

 Hello all,

 I have added new 3 nodes to existing cluster. I must point out that I have
 copied the cassandra yaml file, from an existing node and just changed
 listen_addres per instructions here: Adding nodes to an existing cluster
 | DataStax Cassandra 2.0 Documentation
 http://www.datastax.com/documentation/cassandra/2.0/cassandra/operations/ops_add_node_to_cluster_t.html






 Adding nodes to an existing cluster | DataStax Cassandra 2.0 Documentation
 http://www.datastax.com/documentation/cassandra/2.0/cassandra/operations/ops_add_node_to_cluster_t.html
 Steps to add nodes when using virtual nodes.
 View on www.datastax.com
 http://www.datastax.com/documentation/cassandra/2.0/cassandra/operations/ops_add_node_to_cluster_t.html
 Preview by Yahoo


 Installed datastax agents but in OpsCenter I see the new nodes in a
 different, empty Cluster. Also opscenter does not see the datacenter for
 the new nodes. They are all in the same datacenter, and the name of the
 Cluster is the same for all 9 nodes.
 Any ideeas?


-- 


--

Re: Two problems with Cassandra

[Carlos Rolo]

Hello Pavel,

What is the size of the Cluster (# of nodes)? And you need to iterate over
the full 1TB every time you do the update? Or just parts of it?

IMO information is short to make any kind of assessment of the problem you
are having.

I can suggest to try a 2.0.x (or 2.1.1) release to see if you get the same
problem.

Regards,

Carlos Juzarte Rolo
Cassandra Consultant

Pythian - Love your data

rolo@pythian | Twitter: cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo
http://linkedin.com/in/carlosjuzarterolo*
Tel: 1649
www.pythian.com

On Wed, Feb 11, 2015 at 11:22 AM, Pavel Velikhov pavel.velik...@gmail.com
wrote:

 Hi,

   I’m using Cassandra to store NLP data, the dataset is not that huge
 (about 1TB), but I need to iterate over it quite frequently, updating the
 full dataset (each record, but not necessarily each column).

   I’ve run into two problems (I’m using the latest Cassandra):

   1. I was trying to copy from one Cassandra cluster to another via a
 python driver, however the driver confused the two instances
   2. While trying to update the full dataset with a simple transformation
 (again via python driver), single node and clustered Cassandra run out of
 memory no matter what settings I try, even I put a lot of sleeps into the
 mix. However simpler transformations (updating just one column, specially
 when there is a lot of processing overhead) work just fine.

 I’m really concerned about #2, since we’re moving all heavy processing to
 a Spark cluster and will expand it, and I would expect much heavier traffic
 to/from Cassandra. Any hints, war stories, etc. very appreciated!

 Thank you,
 Pavel Velikhov

-- 


--