Re: apache-cassandra 2.2.8 rpm

2018-06-05 Thread Carlos Rolo
upgrade to the 2.2.12. [image: Pythian] *Carlos Rolo* | Open Source Consultant | [image: LinkedIn] <https://www.linkedin.com/in/carlosjuzarterolo/> *m* +351 918 918 100 r...@pythian.com *www.pythian.com* <https://www.google.com/url?q=https%3A%2F%2Fwww.pythian.co

Re: cassandra repair takes ages

2018-04-22 Thread Carlos Rolo
g a repair > throughout all the nodes. > > Are there any more downsides to run nodetool repair -pr simultaneously on > each node, besides the cpu and mem overload? > > Also if someone can clarify about the safety of an incremental repair. > > > thanks, > > nuno > --

Re: cassandra repair takes ages

2018-04-20 Thread Carlos Rolo
Changing the datadrives to SSD would help to speed up the repairs. Also don't run 3 node, RF2. That makes Quorum = All. Regards, Carlos Juzarte Rolo Cassandra Consultant / Datastax Certified Architect / Cassandra MVP Pythian - Love your data rolo@pythian | Twitter: @cjrolo | Skype: cjr2k3 | Li

Re: Latest version and Features

2018-04-12 Thread Carlos Rolo
/in/carlosjuzarterolo>* Mobile: +351 918 918 100 www.pythian.com On Thu, Apr 12, 2018 at 5:02 AM, Michael Shuler wrote: > On 04/11/2018 06:12 PM, Carlos Rolo wrote: > > > > I blogged about this decision recently > > here: https://blog.pythian.com/what-cassandra-version-shou

Re: Latest version and Features

2018-04-11 Thread Carlos Rolo
If you are on 3.1.0 I would move forward to 3.11.2. I blogged about this decision recently here: https://blog.pythian.com/what-cassandra-version-should-i-use-2018/ Regards, Carlos Juzarte Rolo Cassandra Consultant / Datastax Certified Architect / Cassandra MVP Pythian - Love your data rolo@pyt

Re: Is this SSTable restore merging scenario possible ?

2018-03-21 Thread Carlos Rolo
As said before, as long as you rename the UUIDs to match you should be good. The Production "win out" depends on the timestamps. In Cassandra last write wins, so as long as, for the same row, the production timestamps are more recent than the secondary cluster, the production data would "win over"

Re: Is there any limit in the number of partitions that a table can have

2018-03-07 Thread Carlos Rolo
eja > > On Wed, Mar 7, 2018 at 5:20 PM, Jeff Jirsa wrote: > >> >> >> On Wed, Mar 7, 2018 at 7:13 AM, Carlos Rolo wrote: >> >>> Hi Jeff, >>> >>> Could you expand: "Tables without clustering keys are often deceptively >>> expen

Re: Is there any limit in the number of partitions that a table can have

2018-03-07 Thread Carlos Rolo
Hi Jeff, Could you expand: "Tables without clustering keys are often deceptively expensive to compact, as a lot of work (relative to the other cell boundaries) happens on partition boundaries." This is something I didn't know and highly interesting to know more about! -- Carlos Ro

Re: [External] Re: Whch version is the best version to run now?

2018-03-06 Thread Carlos Rolo
Hello, Our 5 cents. Either 3.0.16 or 3.11.x We are really happy with the way 3.11.1/2 is behaving. We still have a lot of really well behaving Clusters in 2.1/2.2 latest. Regards, Carlos Juzarte Rolo Cassandra Consultant / Datastax Certified Architect / Cassandra MVP Pythian - Love your data

Re: Cassandra Summit 2019 / Cassandra Summit 2018

2018-02-27 Thread Carlos Rolo
Hello all, I'm interested planning/organizing a small kinda of NGCC in Lisbon, Portugal in late May early June. Just waiting for the venue to confirm possible dates. Would be a 1day event kinda last year, is this something people would be interested? I can push a google form for accessing the int

Re: What happens if multiple processes send create table if not exist statement to cassandra?

2018-01-27 Thread Carlos Rolo
Don't do that. Worst case you might get different schemas in flight and no agreement on your cluster. If you are already doing that, check "nodetool describecluster" after you do that. Like Jeff said, it is likely to cause problems. Regards, Carlos Juzarte Rolo Cassandra Consultant / Datastax C

RE: Meltdown/Spectre Linux patch - Performance impact on Cassandra?

2018-01-13 Thread Carlos Rolo
We are seeing almost no impact on Azure (+1 or 2%). Non-patched OS. On AWS we don't have any solid data yet. On 13 Jan 2018 09:46, "Steinmaurer, Thomas" < thomas.steinmau...@dynatrace.com> wrote: Hello Ben, thanks for the notice. Similar here + others reporting as well: https://blog.appoptics

Re: Stable Cassandra 3.x version for production

2017-11-07 Thread Carlos Rolo
DSE is already pushing 3.11 in 5.1 If you're going into 3.x either 3.11.1 or 3.0.15. I would recommend 3.11.1 to take advantage of a couple of features (ex: Slow query log) that you don't have on 3.0. If you're stable on 2.x and you don't need any new feature, you can leave it there safely (altho

Re: Schema Mismatch Issue in Production

2017-10-12 Thread Carlos Rolo
Which version are you running? I got stuck in a similar situation (With a lot more nodes) and the only way to make it good was to stop the whole cluster, start nodes 1 by 1. Regards, Carlos Juzarte Rolo Cassandra Consultant / Datastax Certified Architect / Cassandra MVP Pythian - Love your dat

Re: Materialized views stability

2017-10-02 Thread Carlos Rolo
I've been dealing with MV extensively, and I second Blake. MVs are not suitable for production. Unless you're ready for the pain (The out of sync is a major pain point), I would not go that way. Regards, Carlos Juzarte Rolo Cassandra Consultant / Datastax Certified Architect / Cassandra MVP Pyth

Re: Reaper 0.7 is released!

2017-09-27 Thread Carlos Rolo
Thanks a lot for the release! On 27 Sep 2017 10:21 pm, "Jon Haddad" wrote: > We’ve discussed it, and it’s in GitHub, but we haven’t scheduled it yet. > For now we’re trying to make it easier to use, cut down on it’s query > count, and ensure what’s there is solid. The team (Mick, Alex, Anthony)

Re: Self-healing data integrity?

2017-09-14 Thread Carlos Rolo
Wouldn't be easier for 1) The CRC to be checked by the sender, and don't send if it doesn't match? 2) And once the stream ends, you could compare the 2 CRCs to see if something got weird during transfer? Also you could implement this in 2 pieces instead of reviewing the streaming architecture as

Re: Manual repair not showing in the log.

2017-09-07 Thread Carlos Rolo
Can you check if you have any validation compaction running in nodetool compactionstats? On 7 Sep 2017 7:56 pm, "Mark Furlong" wrote: I have started a repair and I received the message ‘Starting repair command #1, repairing 25301 ranges for keyspace x (parallelism=PARALLEL, full=true). When I lo

Re: truncate table in C* 3.11.0

2017-09-07 Thread Carlos Rolo
If you waited less than 60s, no warning/error was issued. Do the following: * Check if all nodes are up (truncate fails if not) * Check if you got a snapshot generated (unless you have auto_snapshot disabled) * Check if you have still the sstables in the directories (you shouldn't) If it didn't h

Re: Cassandra Setup Question

2017-08-23 Thread Carlos Rolo
Use networktopologystrategy as replication strategy and make sure you have dc1: 3 and dc2: 3. This way you have 3 replicas in each DC. On 23 Aug 2017 12:53, "Jonathan Baynes" wrote: > Hi Community, > > > > Quick question regarding Replication Factor. > > > > In my Production Environment I cur

Re: Adding a new node with the double of disk space

2017-08-18 Thread Carlos Rolo
gt; wrote: > >> Are you saying if a node had double the hardware capacity in every way it >> would be a bad idea to up num_tokens? I thought that was the whole idea of >> that setting though? >> >> On Thu, Aug 17, 2017 at 9:52 AM, Carlos Rolo wrote: >> >&g

Re: Adding a new node with the double of disk space

2017-08-17 Thread Carlos Rolo
No. If you would double all the hardware on that node vs the others would still be a bad idea. Keep the cluster uniform vnodes wise. Regards, Carlos Juzarte Rolo Cassandra Consultant / Datastax Certified Architect / Cassandra MVP Pythian - Love your data rolo@pythian | Twitter: @cjrolo | Skype

Re: c* updates not getting reflected.

2017-07-11 Thread Carlos Rolo
What consistency are you using on those queries? On 11 Jul 2017 19:09, "techpyaasa ." wrote: > Hi, > > We have a table with following schema: > > CREATE TABLE ks1.cf1 ( pid bigint, cid bigint, resp_json text, status int, > PRIMARY KEY (pid, cid) ) WITH CLUSTERING ORDER BY (cid ASC) with LCS > co

Re: Reaper v0.6.1 released

2017-06-15 Thread Carlos Rolo
Great! Thanks a lot! Regards, Carlos Juzarte Rolo Cassandra Consultant / Datastax Certified Architect / Cassandra MVP Pythian - Love your data rolo@pythian | Twitter: @cjrolo | Skype: cjr2k3 | Linkedin: *linkedin.com/in/carlosjuzarterolo * Mobile: +351

Re: Stable version apache cassandra 3.X /3.0.X

2017-05-31 Thread Carlos Rolo
On sync in Jon. Only go 3.0.x if you REALLY need something from there (ex: MV) even then, be carefull. 3.x wait for 3.11.x. 3.10 if you REALLY need something from there right now. Latest 2.2.x or 2.1.x if you are just doing baseline Cassandra and need the stability. Regards, Carlos Juzarte Rol

Re: Slowness in C* cluster after implementing multiple network interface configuration.

2017-05-24 Thread Carlos Rolo
It might be a bug. Cassandra, AFAIK, scans those files for changes and updates the topology (So you don't need a restart if you change the files). It might be the case that the absence of the file, is still noticed by Cassandra even if it is not really used. I can do a small test to confirm, if so

Re: Cassandra Schema version mismatch

2017-05-06 Thread Carlos Rolo
Sometimes "resetlocalschema" (happened to me, didn't check why) will not work, and you need to stop the offending nodes and bring them back one by one.. That solved the issue. In the cases I've seen this happened the clusters in question where either: a) A couple of nodes down (cloud provider cau

Re: Cassandra Schema version mismatch

2017-05-05 Thread Carlos Rolo
Are you changing the schema in a dynamic fashion? If you get problems (network, gc pauses, etc) during the schema changes it might lead to that. Regards, Carlos Juzarte Rolo Cassandra Consultant / Datastax Certified Architect / Cassandra MVP Pythian - Love your data rolo@pythian | Twitter: @cjr

Re: cassandra OOM

2017-04-25 Thread Carlos Rolo
To add some contribution to this thread, we have seen both cases. CMS easily outperforming G1 for the same Heapsize and the inverse too. On the same cluster different workloads (datacenter based) we have both collectors because of performance based on the workload. It would be good to colect this

Re: Drop tables takes too long

2017-04-20 Thread Carlos Rolo
You have 4800 Tables in total? That is a lot of tables, plus MVs? or MVs are already considered in the 60*80 account? I would recommend to reduce the table number. Other thing is that you need to check your log file for GC Pauses, and how long those pauses take. You also might need to increase th

Re: WriteTimeoutException with LWT after few milliseconds

2017-04-12 Thread Carlos Rolo
You can try to use TRACING to debug the situation, but for a LWT to fail so fast, the most probable cause is what you stated: "It is possible that there are concurrent inserts on the same PK - actually thats the reason why I use LWTs." AKA, someone inserted first. Regards, Carlos Juzarte Rolo Cas

Re: too many compactions pending and compaction is slow on few tables

2017-04-07 Thread Carlos Rolo
Is not a good idea to do LCS on spinning. Change to STCS, and reduce the compactors to 2 (if you have more than 2). Check if that helps. On Apr 7, 2017 20:18, "Matija Gobec" wrote: > It does as the "new" data, even if the values are the same, has new write > time timestamp. > Spinning disks are

Re: Node always dieing

2017-04-06 Thread Carlos Rolo
i3 are having those issues more than the other instances it seems. Not the first report I heard about. Regards, Carlos Juzarte Rolo Cassandra Consultant / Datastax Certified Architect / Cassandra MVP Pythian - Love your data rolo@pythian | Twitter: @cjrolo | Skype: cjr2k3 | Linkedin: *linkedin.

Re: Node always dieing

2017-04-06 Thread Carlos Rolo
There was some issue with the i3 instances and Cassandra. Did you had this cluster running always on i3? On Apr 6, 2017 13:06, "Cogumelos Maravilha" wrote: > Limit Soft Limit Hard Limit > Units > Max cpu time unlimitedunlimited > seconds > M

Re: Archive node

2017-03-06 Thread Carlos Rolo
I would not suggest to do that, because the new "Archive" node would be a new DC that you would need to build (Operational wise). You could also snapshot the old one once it finishes and use SSTableloader to push it into your Development DC. This way you have isolation from Production. Plus no ope

Re: Read exceptions after upgrading to 3.0.10

2017-02-24 Thread Carlos Rolo
By any chances are you using the PHP/C++ driver? -- --

Re: Global TTL vs Insert TTL

2017-02-01 Thread Carlos Rolo
if it can see than the maxTimestamp of an SSTable is older >> than the table Global TTL, the SSTable can be entirely dropped during >> compaction >> >> Using dynamic TTL at runtime, since Cassandra doesn't how and cannot >> track each individual TTL value, the previ

Re: Global TTL vs Insert TTL

2017-02-01 Thread Carlos Rolo
To reinforce Alain statement: "I would say that the unsafe part is more about using C* 3.9" this is key. You would be better on 3.0.x unless you need features on the 3.x series. Regards, Carlos Juzarte Rolo Cassandra Consultant / Datastax Certified Architect / Cassandra MVP Pythian - Love your

Re: Has anyone deployed a production cluster with less than 6 nodes per DC?

2016-12-26 Thread Carlos Rolo
It depends on a lot of factors. What causes the cluster to get crazy? I/O, Network, CPU? I manage clusters of all sizes (even 3 nodes per DC) but it all depends on usage and configuration. Regards, Carlos Regards, Carlos Juzarte Rolo Cassandra Consultant / Datastax Certified Architect / Cassa

Re: Join_ring=false Use Cases

2016-12-20 Thread Carlos Rolo
Beware the Java Driver limitations around whitelisting IPs. Works fine in Python. Regards, Carlos Juzarte Rolo Cassandra Consultant / Datastax Certified Architect / Cassandra MVP Pythian - Love your data rolo@pythian | Twitter: @cjrolo | Skype: cjr2k3 | Linkedin: *linkedin.com/in/carlosjuzar

Re: Failure when setting up cassandra in cluster

2016-08-22 Thread Carlos Rolo
If Ryan answer doesn't help, post Cassandra version. There is a bug with cql and some python version that would lead to that error. Also, please post "nodetool status". Regards, Carlos Juzarte Rolo Cassandra Consultant / Datastax Certified Architect / Cassandra MVP Pythian - Love your data rol

Re: Support/Consulting companies

2016-08-22 Thread Carlos Rolo
https://www.pythian.com/ Regards, Carlos Juzarte Rolo Cassandra Consultant / Datastax Certified Architect / Cassandra MVP Pythian - Love your data rolo@pythian | Twitter: @cjrolo | Skype: cjr2k3 | Linkedin: *linkedin.com/in/carlosjuzarterolo * Mobile: +

Re: IF EXISTS checks on all nodes?

2016-05-12 Thread Carlos Rolo
Hello, As far as I know, lightweight transactions only apply to a single partition, so in your case it will only execute on the nodes responsible for that partition. And as a consequence, those nodes will all be in the same state when the transaction ends (If it would apply). Please refer to this

Re: COPY TO export fails with

2016-05-10 Thread Carlos Rolo
Hello, That is a lot of data to do an "COPY TO. If you want a fast way to export, and you're fine with Java, you can use Cassandra SSTableReader classes to read the sstables directly. Spark also works. Regards, Carlos Juzarte Rolo Cassandra Consultant / Datastax Certified Architect / Cassandra

Re: Changing snitch from PropertyFile to Gossip

2016-04-25 Thread Carlos Rolo
pecific reason? > > C*heers, > --- > Alain Rodriguez - al...@thelastpickle.com > France > > The Last Pickle - Apache Cassandra Consulting > http://www.thelastpickle.com > > 2016-04-24 15:44 GMT+02:00 Carlos Rolo : > >> As long as the topology doesn&

Re: Changing snitch from PropertyFile to Gossip

2016-04-24 Thread Carlos Rolo
As long as the topology doesn't change, yes. Repair once you finish. Em 24/04/2016 13:23, "AJ" escreveu: > Is it possible to do this without down time i.e. run in mixed mode while > doing a rolling upgrade? -- --

Re: Most stable version?

2016-04-22 Thread Carlos Rolo
will be a stable train for 3? > (or that 3.x will be bumped to 4.0 when stable)? > > We're on 2.2.5 and haven't seen any major problems with it. > > -J > > > > Sent via iPhone > > On Apr 22, 2016, at 03:34, Carlos Rolo wrote: > > If you need SASI, y

Re: Most stable version?

2016-04-22 Thread Carlos Rolo
; > Currently I use 2.2.5, but is the latest 2.1.13 recommended for production > use? > > Regards, > Satoshi > > > On Mon, Apr 18, 2016 at 11:45 PM, Carlos Rolo wrote: > >> Sorry to resurrect this now, but I don't consider anything after 3.0.x >> s

Re: Most stable version?

2016-04-18 Thread Carlos Rolo
Sorry to resurrect this now, but I don't consider anything after 3.0.x stable. I wrote a blog post about this to be clear: https://www.pythian.com/blog/cassandra-version-production/ Use it and pick a version based on your needs. Regards, Carlos Juzarte Rolo Cassandra Consultant / Datastax Certi

Re: Do I have to use repair -inc with the option -par forcely?

2016-02-16 Thread Carlos Rolo
+1 on what Alain said, but I do think if you are high enough on a 2.1.x (will look later) version you don't need to follow the documentation. It is outdated. Run a full repair, the you can start incremental repairs since the SSTables will have the metadata on them about the last repair. Wait some

Re: Cassandra eats all cpu cores, high load average

2016-02-12 Thread Carlos Rolo
if you have internode_compression: all, try disabling it. Also I would move to STCS if you have a lot of tombstones. If they get pilled in higher levels you have to wait until those higher levels get compacted before you get them out. For G1 your heap is too small. Bump that to 16GB (or at least 1

Re: Latest stable release

2016-02-08 Thread Carlos Rolo
I honestly go with 2.1.13 unless you need the features on 2.2.x. I would not recommend 3.x for now (unless you need the features). Regards, Carlos Juzarte Rolo Cassandra Consultant / Datastax Certified Architect / Cassandra MVP Pythian - Love your data rolo@pythian | Twitter: @cjrolo | Linked

Re: Cassandra is consuming a lot of disk space

2016-01-13 Thread Carlos Rolo
You can check if the snapshot exists in the snapshot folder. Repairs stream sstables over, than can temporary increase disk space. But I think Carlos Alonso might be correct. Running compactions might be the issue. Regards, Carlos Juzarte Rolo Cassandra Consultant Pythian - Love your data rolo@

Re: Unable to start one Cassandra node: OutOfMemoryError

2015-12-10 Thread Carlos Rolo
Dealt with that recently, and the only solution that made it work was to increase heap sizes. Regards, Carlos Juzarte Rolo Cassandra Consultant Pythian - Love your data rolo@pythian | Twitter: @cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo * M

Re: scylladb

2015-11-11 Thread Carlos Rolo
/in/carlosjuzarterolo>* Mobile: +351 91 891 81 00 | Tel: +1 613 565 8696 x1649 www.pythian.com On Wed, Nov 11, 2015 at 4:46 PM, Dani Traphagen wrote: > Killer, @cjrolo. Will you update via this thread? > > On Wed, Nov 11, 2015 at 7:57 AM, Carlos Rolo wrote: > >> Not yet, b

Re: scylladb

2015-11-11 Thread Carlos Rolo
Not yet, but not far from doing it. No rain here yet! :) On a more serious tone, should be done before end of the Month. -- --

Re: scylladb

2015-11-05 Thread Carlos Rolo
891 81 00 | Tel: +1 613 565 8696 x1649 www.pythian.com On Thu, Nov 5, 2015 at 12:07 PM, Dani Traphagen wrote: > As of two days ago, they say they've got it @cjrolo. > > https://github.com/scylladb/scylla/wiki/RELEASE-Scylla-0.11-Beta > > > On Thursday, November 5, 2015, Ca

Re: scylladb

2015-11-05 Thread Carlos Rolo
I will not try until multi-DC is implemented. More than an month has passed since I looked for it, so it could possibly be in place, if so I may take some time to test it. Regards, Carlos Juzarte Rolo Cassandra Consultant Pythian - Love your data rolo@pythian | Twitter: @cjrolo | Linkedin: *lin

Re: any update about CASSANDRA-10420

2015-10-16 Thread Carlos Rolo
Can you provide more information? The description is generic, driver version and a test case that the devs can use to reproduce the bug would be optimal. I'm not involved in the development of cassandra, but that bug description doesn't seem complete. Maybe it is a driver bug and not a Cassandra

Re: Removed node is not completely removed

2015-10-14 Thread Carlos Rolo
Check system.peers table to see if the IP is still there. If so edit the table and remove the offending IP. You are probably running into this: https://issues.apache.org/jira/browse/CASSANDRA-6053 Regards, Carlos Juzarte Rolo Cassandra Consultant Pythian - Love your data rolo@pythian | Twitter

Re: howto do sql query like in a relational database

2015-07-22 Thread Carlos Rolo
Hello Anton, You need to look into Datastax Entreprise (DSE) Offering. It integrates Solr search which allows you to do searches like the one you mention. There are also some opensource projects doing this kind of integration, so its up to you. And as Oded mentioned Cassandra really shines on key

Re: Seed gossip version error

2015-07-21 Thread Carlos Rolo
That error should only occur when you have a mismatch between the Seed version and the new node version. Are you sure all your nodes are running in the same version? Regards, Carlos Juzarte Rolo Cassandra Consultant Pythian - Love your data rolo@pythian | Twitter: cjrolo | Linkedin: *linkedin.c

Re: cassandra repair error

2015-07-16 Thread Carlos Rolo
Hello, It seems that is a problem with Cassandra trying to delete a directory that is not empty. First time seeing this error. Are your harddrives with enough space and working correctly? Regards, Carlos Juzarte Rolo Cassandra Consultant Pythian - Love your data rolo@pythian | Twitter: cjrolo

Re: Wrong peers

2015-07-06 Thread Carlos Rolo
There is a bug in Jira related to this, it is not a driver issue, is a Cassandra issue. It is solved on 2.0.14 I think. I will post the ticket once I find it. Regards, Carlos Juzarte Rolo Cassandra Consultant Pythian - Love your data rolo@pythian | Twitter: cjrolo | Linkedin: *linkedin.com/in/c

Re: [MASSMAIL]Re: Error while adding a new node.

2015-07-02 Thread Carlos Rolo
PM, Marcos Ortiz wrote: > The recommended version to use is 2.1.5 because, like you Carlos said, > 2.1.6 and 2.1.7 are very new to consider them like > stable. > > On 02/07/15 08:55, Carlos Rolo wrote: > > Indeed you should upgrade to 2.1.7. > > And then report if yo

Re: Error while adding a new node.

2015-07-02 Thread Carlos Rolo
Indeed you should upgrade to 2.1.7. And then report if you are still facing problems. Versions up to 2.1.5 (in the 2.1.x series) are not considered stable. Regards, Carlos Juzarte Rolo Cassandra Consultant Pythian - Love your data rolo@pythian | Twitter: cjrolo | Linkedin: *linkedin.com/in/car

Re: compaction occurring very frequently

2015-06-18 Thread Carlos Rolo
Hi, 2.1.x does have compaction issues. Upgrading to 2.1.6 should help you. You can also search this mailing list for some recommendations for that specific problem since it is a recurrent issue. Regards, Carlos Juzarte Rolo Cassandra Consultant Pythian - Love your data rolo@pythian | Twitter:

Re: Replication in one datacetner only

2015-06-16 Thread Carlos Rolo
Yes, set the replication factor of the keyspace to only use the datacenter you want. More information here: http://docs.datastax.com/en/cql/3.1/cql/cql_reference/create_keyspace_r.html Regards, Carlos Juzarte Rolo Cassandra Consultant Pythian - Love your data rolo@pythian | Twitter: cjrolo | L

Re: Missing data

2015-06-15 Thread Carlos Rolo
Hi Jean, The problem of that Warning is that you are reading too many tombstones per request. If you do have Tombstones without doing DELETE it because you probably TTL'ed the data when inserting (By mistake? Or did you set default_time_to_live in your table?). You can use nodetool cfstats to see

Re: Question about "nodetool status ..." output

2015-06-12 Thread Carlos Rolo
Your data model also contributes to the balance (or lack of) of the cluster. If you have a really bad data partitioning Cassandra will not do any magic. Regarding that cluster, I would decommission the x.52 node and add it again with the correct configuration. After the bootstrap, run a cleanup. I

Re: Lucene index plugin for Apache Cassandra

2015-06-12 Thread Carlos Rolo
Seems like an interesting tool! What operational recommendations would you make to users of this tool (Extra hardware capacity, extra metrics to monitor, etc)? Regards, Carlos Juzarte Rolo Cassandra Consultant Pythian - Love your data rolo@pythian | Twitter: cjrolo | Linkedin: *linkedin.com/in

Re: Hundreds of sstables after every Repair

2015-06-09 Thread Carlos Rolo
Hello, Do you have your clocks synced across your cluster? Are you using NTP and have it properly configured? Sometimes clock out of sync can trigger weird behaviour. Regards, Carlos Juzarte Rolo Cassandra Consultant Pythian - Love your data rolo@pythian | Twitter: cjrolo | Linkedin: *linkedi

Re: cassandra installation

2015-06-09 Thread Carlos Rolo
(Some self-promotion here) You can follow this blog to help you out: http://www.pythian.com/blog/from-0-to-cassandra-an-exhaustive-approach-to-installing-cassandra/ Regards, Carlos Juzarte Rolo Cassandra Consultant Pythian - Love your data rolo@pythian | Twitter: cjrolo | Linkedin: *linkedin.c

Re: Ghost compaction process

2015-06-08 Thread Carlos Rolo
HI, Is it 2.0.14 or 2.1.4? If you are on 2.1.4 I would recommend an upgrade to 2.1.5 regardless of that issue. >From the data you provide it is difficult to access what is the issue. If you are running with RF=2 you can always add another node and kill that one if that is the only node that shows

Re: Reading too many tombstones

2015-06-04 Thread Carlos Rolo
The TTL data will only be removed after the gc_grace_seconds. So your data with 30 days TTL will be still in Cassandra for 10 days more (40 in total). Is your data being there for more than that? Otherwise it is expected behaviour and probably you should do something on your data model to avoid sca

Re: How to interpret some GC logs

2015-06-03 Thread Carlos Rolo
GC Logs are a weird science. I use a couple of resources to get through them. Regarding your question my 1.8.0_40 always have the first the "->". I "greped" through 2h of logs, and on a test environment. I use the following set of options: -XX:+PrintGCDetails" -XX:+PrintGCDateStamps" -XX:+PrintHe

Re: Hbase vs Cassandra

2015-05-29 Thread Carlos Rolo
>From 2013 both can still be relevant: http://www.pythian.com/blog/watch-hbase-vs-cassandra/ Regards, Carlos Juzarte Rolo Cassandra Consultant Pythian - Love your data rolo@pythian | Twitter: cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo * Mobi

Re: Multiple cassandra instances per physical node

2015-05-21 Thread Carlos Rolo
Hi, I also advice against multiple instances on the same hardware. If you have really big boxes why not virtualize? Other option is experiment with CCM. Although there are some limitations with CCM (ex: JNA is disabled) If you follow up on this I would to hear how it went. Em 21/05/2015 19:33, "

Re: New node got stuck joining the cluster after a while

2015-04-28 Thread Carlos Rolo
Hi, The 2.1.x series is not recommeded for use, especially the first versions. I would downgrade to 2.0.14 or if must stay on 2.1 upgrade your cluster to 2.1.4 or the imminent release of 2.1.5. This mailing list as a few tips how to deal with the 2.1.x releases, but the best way is indeed a downg

Re: minimum bandwidth requirement between two Geo Redundant sites of Cassandra database

2015-04-28 Thread Carlos Rolo
Hi, I would not recommend anything below 1Gbps for the bandwidth. Latency try to have it as low as you can. Regards, Carlos Juzarte Rolo Cassandra Consultant Pythian - Love your data rolo@pythian | Twitter: cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo

Re: What is 'Read Reuqests' on OpsCenter exaclty?

2015-04-24 Thread Carlos Rolo
workload and usage >> patterns. >> >> *Avg* - The average of values recorded during a time interval. >> >> A future version of OpsC will include tooltips with these descriptions >> for better clarity. >> On Apr 23, 2015 6:30 AM, "Carlos Rolo" w

Re: What is 'Read Reuqests' on OpsCenter exaclty?

2015-04-23 Thread Carlos Rolo
Probably it takes in account the read repair, plus a read that have consistency != 1 will produce reads on other machines (which are taken in account). I don't know the internals of opscenter but I would assume that this is the case. If you want to test it further, disable read_repair, and make al

Re: LCS Strategy, compaction pending tasks keep increasing

2015-04-21 Thread Carlos Rolo
Are you on version 2.1.x? Regards, Carlos Juzarte Rolo Cassandra Consultant Pythian - Love your data rolo@pythian | Twitter: cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo * Mobile: +31 6 159 61 814 | Tel: +1 613 565 8696 x1649 www.pythian.com O

Re: Adding nodes to existing cluster

2015-04-20 Thread Carlos Rolo
o you see any reason to bootstrap them one by one if we're not using > rack awareness? > > > On Mon, Apr 20, 2015 at 2:49 PM, Carlos Rolo wrote: > > Start one node at a time. Wait 2 minutes before starting each node. > > > > > > How much data and nodes you

Re: Adding nodes to existing cluster

2015-04-20 Thread Carlos Rolo
Start one node at a time. Wait 2 minutes before starting each node. How much data and nodes you have already? Depending on that, the streaming of data can stress on the resources you have. I would recommend to start one and monitor, if things are ok, add another one. And so on. Regards, Carlos

Re: Best practice: Multiple clusters vs multiple tables in a single cluster?

2015-04-02 Thread Carlos Rolo
Adding a new keyspace should be perfectly fine. Unless you have completely distinct workloads for the different keyspaces. Even so you can balanced some stuff at keyspace/table level. But I would go with a new keyspace not with a new cluster given the small size you say you have. Regards, Carlos

Re: Replication to second data center with different number of nodes

2015-03-29 Thread Carlos Rolo
Sharing my experience here. 1) Never had any issues with different size DCs. If the hardware is the same, keep the # to 256. 2) In most of the cases I keep the 256 vnodes and no performance problems (when they are triggered, the cause is not the vnodes #) Regards, Carlos Juzarte Rolo Cassandra C

Re: Upgrade from 2.0.9 to 2.1.3

2015-03-06 Thread Carlos Rolo
I would not recommend an upgrade to 2.1.x for now. Do you have any specific reason to upgrade? For upgrading from 2.0.9 you can just do a direct upgrade. Regards, Carlos Juzarte Rolo Cassandra Consultant Pythian - Love your data rolo@pythian | Twitter: cjrolo | Linkedin: *linkedin.com/in/carlo

Re: Write timeout under load but Read is fine

2015-03-06 Thread Carlos Rolo
What is the consistency level you are using? Are you using Thrift or CQL? Are you using SSDs? Check if compactions are running when you get the timeouts. Regards, Carlos Juzarte Rolo Cassandra Consultant Pythian - Love your data rolo@pythian | Twitter: cjrolo | Linkedin: *linkedin.com/in/carlos

Re: Setting up JNA on CentOS 6.6. with cassandra20-2.0.12 and Oracle Java 1.7.0_75

2015-02-25 Thread Carlos Rolo
Also I always install JNA from the JNA page. I did the installation for this blog post in CentOS 6.5: http://www.pythian.com/blog/from-0-to-cassandra-an-exhaustive-approach-to-installing-cassandra/ Regards, Carlos Juzarte Rolo Cassandra Consultant Pythian - Love your data rolo@pythian | Twitte

Re: Setting up JNA on CentOS 6.6. with cassandra20-2.0.12 and Oracle Java 1.7.0_75

2015-02-25 Thread Carlos Rolo
Hello, I always install JNA into the lib directory of java itself Since I normally have java in /opt/java I put the JNA into /opt/java/lib. ~$ grep JNA /var/log/cassandra/system.log INFO HH:MM:SS JNA mlockall successful Regards, Carlos Juzarte Rolo Cassandra Consultant Pythian - Love your d

Re: Unexplained query slowness

2015-02-25 Thread Carlos Rolo
You can use query tracing to check what is happening. Also you fire jconsole/JavaVisualVM and push out some metrics like the 99th read Beans for that column family. A simpler check is using cfstats and look for weird numbers (high number sstables, if you are deleting check how much tombstones per s

Re: Possible problem with disk latency

2015-02-25 Thread Carlos Rolo
Your latency doesn't seem that high that can cause that problem. I suspect more of a problem with the Cassandra version (2.1.3) than that with the hard drives. I didn't look deep into the information provided but for your reference, the only time I had serious (leading to OOM and all sort of weird

Re: AMI to use to launch a cluster with OpsCenter on AWS

2015-02-23 Thread Carlos Rolo
Regarding AWS the only thing I normally do (besides the normal installation, etc) is setting up the firewall zones so the ports needed for Cassandra are open. You can follow this guide: https://razvantudorica.com/02/create-a-cassandra-cluster-with-opscenter-on-amazon-ec2/a Regards, Carlos Juzart

Re: run cassandra on a small instance

2015-02-19 Thread Carlos Rolo
un for a while, usually about 5 hours and then just crash > with the word 'killed' as the last line of output. > > Thanks > Tim > > > On Thu, Feb 19, 2015 at 3:40 AM, Carlos Rolo wrote: > >> I have Cassandra instances running on VMs with smaller RAM (1GB eve

Re: C* 2.1.2 invokes oom-killer

2015-02-19 Thread Carlos Rolo
nc_interval_in_kb: 10240 > > On Thu, Feb 19, 2015 at 12:12 PM, Michał Łowicki > wrote: > >> >> >> On Thu, Feb 19, 2015 at 11:02 AM, Carlos Rolo wrote: >> >>> Do you have trickle_fsync enabled? Try to enable that and see if it >>> solves your p

Re: can't delete tmp file

2015-02-19 Thread Carlos Rolo
You should upgrade to 2.1.3 for sure. Check the changelog here: https://git1-us-west.apache.org/repos/asf?p=cassandra.git;a=blob_plain;f=CHANGES.txt;hb=refs/tags/cassandra-2.1.3 Regards, Carlos Juzarte Rolo Cassandra Consultant Pythian - Love your data rolo@pythian | Twitter: cjrolo | Linkedi

Re: C* 2.1.2 invokes oom-killer

2015-02-19 Thread Carlos Rolo
rolo@pythian | Twitter: cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo <http://linkedin.com/in/carlosjuzarterolo>* Tel: 1649 www.pythian.com On Thu, Feb 19, 2015 at 10:49 AM, Michał Łowicki wrote: > > > On Thu, Feb 19, 2015 at 10:41 AM, Carlos Rolo wrote: > >> So co

Re: C* 2.1.2 invokes oom-killer

2015-02-19 Thread Carlos Rolo
So compaction doesn't seem to be your problem (You can check with nodetool compactionstats just to be sure). How much is your write latency on your column families? I had OOM related to this before, and there was a tipping point around 70ms. -- --

Re: C* 2.1.2 invokes oom-killer

2015-02-19 Thread Carlos Rolo
Can you check how many SSTables you have? It is more or less a know fact that 2.1.2 has lots of problems with compaction so a upgrade can solve it. But a high number of SSTables can confirm that indeed compaction is your problem not something else. Regards, Carlos Juzarte Rolo Cassandra Consultan

  1   2   >