Re: SSTableMetadata Util

verify >>>> -rwxr-xr-x. 1 root root2045 Jul 25 06:12 sstableutil >>>> -rwxr-xr-x. 1 root root2042 Jul 25 06:12 sstableupgrade >>>> -rwxr-xr-x. 1 root root2042 Jul 25 06:12 sstablescrub >>>> -rwxr-xr-x. 1 root root2034 Jul 25 06:12 sstableloader >>>> >>>> >>>> If this utility is no longer available how can i get sstable metadata >>>> like repaired_at, Estimated droppable tombstones >>>> >>>> >>>> Thanks >>>> Pranay >>>> >>> -- Regards, Igor Zubchenok CTO at Multi Brains LLC Founder of taxistartup.com saytaxi.com chauffy.com Skype: igor.zubchenok

High load, low IO wait, moderate CPU usage

Hi there, I have a Cassandra cluster running on Kubernetes. This cluster has 8 running instances with 8Gb of memory and 5 CPU cores. I can see a high load avg in multiple instances, but no IO wait and moderate CPU usage. Do you know how I can solve this issue? Best, Igor

Re: nodetool repair -pr

it is noted that I should run it at ALL data centers. Looking for a qualified answer. On Fri, 8 Jun 2018 at 18:08 Igor Zubchenok wrote: > I want to repair all nodes at all data centers. > > Example: > DC1 > nodeA > nodeB > nodeC > DC2 > node D > node E &g

Re: nodetool repair -pr

s option to repair only a local > data center." > On Jun 8, 2018, 10:42 AM -0400, user@cassandra.apache.org, wrote: > > > *nodetool repair -pr* > > -- Regards, Igor Zubchenok CTO at Multi Brains LLC Founder of taxistartup.com saytaxi.com chauffy.com Skype: igor.zubchenok

nodetool repair -pr

Hi! I want to repair all nodes in all datacenters. Should I run *nodetool repair -pr* at all nodes of a SINGLE datacenter or at all nodes of ALL datacenters? -- Regards, Igor Zubchenok CTO at Multi Brains LLC Founder of taxistartup.com saytaxi.com chauffy.com Skype: igor.zubchenok

Re: Cassandra 3.11 is compacting forever

a compation >> >> On Fri, Sep 1, 2017 at 3:09 PM, Igor Leão <igor.l...@ubee.in> wrote: >> >>> Some generic errors: >>> >>> *[aladdin@ip-172-16-1-10 cassandra]$ tail cassandra.log | grep -i error* >>> *[aladdin@ip-172-16-1-10 cassandra]$ tail

Re: Cassandra 3.11 is compacting forever

Some generic errors: *[aladdin@ip-172-16-1-10 cassandra]$ tail cassandra.log | grep -i error* *[aladdin@ip-172-16-1-10 cassandra]$ tail cassandra.log | grep -i excep* *[aladdin@ip-172-16-1-10 cassandra]$ tail cassandra.log | grep -i fail* *[aladdin@ip-172-16-1-10 cassandra]$ tail debug.log | grep

Re: Cassandra 3.11 is compacting forever

Hey Kurt, Thanks for your reply. Soon as the whole cluster was upgraded (using existing nodes) it worked pretty well. After a while, the high cpu usage/ pending compactions was back affecting all cluster. It's still an open problem. 2017-08-21 20:24 GMT-03:00 kurt greaves :

Cassandra 3.11 is compacting forever

running `Cassandra 3.9` and one node running `Cassandra 3.11` without any further issues. Have you guys faced this problem before? Thanks in advance! Igor

Re: Upgrading from Cassandra 3.9 to Cassandra 3.11

tru...@gmail.com>: > Hi Igor, > Have a look into that document. ;-) > https://github.com/apache/cassandra/blob/cassandra-3.11/NEWS.txt > Thanks, > Petrus Silva > > On Fri, Aug 4, 2017 at 10:15 AM, Igor Leão <igor.l...@ubee.in> wrote: > >> Hi there, >>

Upgrading from Cassandra 3.9 to Cassandra 3.11

Hi there, Is it possible to add a Cassandra 3.11 node into a Cassandra 3.9 cluster? I wish I could upgrade this cluster without uninstalling and installing Cassandra on existing nodes, just by adding and removing nodes. Thanks in advance!

Re: Replication issue with Multi DC setup in cassandra

gt;>> ideally. >>> >>> On May 16, 2017, at 3:19 PM, suraj pasuparthy < >>> suraj.pasupar...@gmail.com> wrote: >>> >>> elp me fig >>> >>> >>> >> >> >> -- >> Suraj Pasuparthy >

Re: Node containing all data of the cluster

Thank you Varun and DuyHai! 2017-05-10 20:57 GMT-03:00 Varun Gupta <var...@uber.com>: > Hi Igor, > > You can setup cluster with configuration as below. > > Replication: DC1: 3 and DC2: 1. > > If you are using datastax java driver, then use dcaware load balancing >

Node containing all data of the cluster

Hey everyone, Imagine a have Cassandra cluster with 4 nodes. Is it possible to have a separate node which would not receive requests but would by in sync with the rest of the cluster? Ideally this super node would have all data of the cluster. I want to take a snapshot of this node from time to

Re: Logging queries

problematic query. > > copy system_traces.events TO 'traces_dump.csv'; > > Also do make sure you dont set trace probability to a high number if > working on a production database as it can adversely impact performance. > > Regards, > > On Sun, Feb 19, 2017 at 1:28 AM, Igor Leão <

Re: Logging queries

Hi Bhuvan, Thanks a lot! Any idea if something can be done for C* 2.X? Best, Igor 2017-02-18 16:41 GMT-03:00 Bhuvan Rawal <bhu1ra...@gmail.com>: > Hi Igor, > > If you are using java driver, you can log slow queries on client side > using QueryLogger. > https://docs.data

Logging queries

Hi there, I'm wondering how to log queries from Cassandra. These queries can be either slow queries or all queries. The only constraint is that I should do this on server side. I tried using `nodetool settraceprobability`, which writes all queries to the keyspace `system_traces`. When I try to

RE: cassandra boot is stuck in hint compaction.

mich...@pbandjelly.org Sent: Sunday, May 25, 2014 8:06 PM To: user@cassandra.apache.org Subject: Re: cassandra boot is stuck in hint compaction. On 05/25/2014 04:12 AM, Igor Shprukh wrote: ​hi guys, we have a 6 node cluster, consisting of 5 linux machines and a windows one. Mixed linux/windows

cassandra boot is stuck in hint compaction.

?hi guys, we have a 6 node cluster, consisting of 5 linux machines and a windows one. after a hard shutdown of the windows machine, the node is stuck on hints compaction for more than half an hour and cassandra won't start. must say that it is a strong

Re: Reduce Cassandra GC

If you are talking about 1.2.x then I also have memory problems on the idle cluster: java memory constantly slow grows up to limit, then spend long time for GC. I never seen such behaviour for 1.0.x and 1.1.x, where on idle cluster java memory stay on the same value. On 06/07/2013 05:19 PM,

Re: High performance disk io

Hello Christopher, BTW, are you talking about 99th percentiles on client side, or about percentiles from cassandra histograms for CF on cassandra side? Thanks! On 05/22/2013 05:41 PM, Christopher Wirt wrote: Hi Igor, Yea same here, 15ms for 99^th percentile is our max. Currently getting

Re: High performance disk io

Hello What level of read performance do you expect? We have limit 15 ms for 99 percentile with average read latency near 0.9ms. For some CF 99 percentile actually equals to 2ms, for other - to 10ms, this depends on the data volume you read in each query. Tuning read performance involved

Re: High performance disk io

On 05/22/2013 05:41 PM, Christopher Wirt wrote: Hi Igor, Yea same here, 15ms for 99^th percentile is our max. Currently getting one or two ms for most CF. It goes up at peak times which is what we want to avoid. Our 99 percentile also goes up at peak times but stay at acceptable level

Re: How to add new DC to cluster when GossipingPropertyFileSnitch is used

I see no reason to restart all nodes. You can continue to use seed from first DC - seed used for loading ring configuration(locations, token ranges, etc), not data. On 05/17/2013 10:34 AM, Sergey Naumov wrote: If I understand you correctly, GossipingPropertyFileSnitch is useful for

Re: How to add new DC to cluster when GossipingPropertyFileSnitch is used

should have identical seeds lists. 2. At least at one DC nodes MUST have in its seed lists seeds from all other DCs. 2013/5/17 Igor i...@4friends.od.ua mailto:i...@4friends.od.ua I see no reason to restart all nodes. You can continue to use seed from first DC - seed used for loading ring

Re: SSTable size versus read performance

My 5 cents: I'd check blockdev --getra for data drives - too high values for readahead (default to 256 for debian) can hurt read performance. On 05/16/2013 05:14 PM, Keith Wright wrote: Hi all, I currently have 2 clusters, one running on 1.1.10 using CQL2 and one running on 1.2.4 using

Re: SSTable size versus read performance

the chunk size to 8 kb From: Igor i...@4friends.od.ua mailto:i...@4friends.od.ua Reply-To: user@cassandra.apache.org mailto:user@cassandra.apache.org user@cassandra.apache.org mailto:user@cassandra.apache.org Date: Thursday, May 16, 2013 1:55 PM To: user

1.2.3 and 1.2.4 memory usage growth on idle cluster

Hello Does anybody seen memory problems on idle cluster? I have 8-node ring with cassandra 1.2.3 which never been used and stay idle for several weeks. Yesterday when I decided to upgrade it to 1.2.4 I found lot of messages like INFO 11:10:56,273 GC for ParNew: 1039 ms for 1 collections,

Re: High disk I/O during reads

You can try to disable readahead on cassandra data disk. Jon Scarborough j...@fifth-aeon.net написал(а): Checked tpstats, there are very few dropped messages. Checked histograms. Mostly nothing surprising. The vast majority of rows are small, and most reads only access one or two SSTables.

please explain read path when key not in database

Hello! Explain please, how this work when I request for key which is not in database * The closest node (as determined by proximity sorting as described above) will be sent a command to perform an actual data read (i.e., return data to the co-ordinating node). * As required by

Re: Q on schema migratins

Cassandra Developer New Zealand @aaronmorton http://www.thelastpickle.com On 23/02/2013, at 5:17 AM, Igor i...@4friends.od.ua mailto:i...@4friends.od.ua wrote: Hello Cassandra 1.0.7 Some time ago we used secondary index on one of CF. Due to performance reasons we dropped this secondary index

how to read only from local DC without LOCAL_QUORUM?

Hello! We have 1.0.7 multi-DC cassandra setup with strict time limits for read (15ms). We use RF=1 per DC and reads with CL=ONE. Data in datacenters are in sync, but we have next problem: when application looks for key which is not yet in database, coordinator wait for digests from remote

Re: Amazingly bad compaction performance

Hello Too much GC? Check JVM heap settings and real usage. On 06/27/2012 01:37 AM, Dustin Wenz wrote: We occasionally see fairly poor compaction performance on random nodes in our 7-node cluster, and I have no idea why. This is one example from the log: [CompactionExecutor:45]

Re: nodetool repair requirement

On 05/13/2012 07:18 PM, Thanh Ha wrote: Hi All, Do I have to do maintenance nodetool repair on CFs that do not have deletions? Probably you should (depending how you do reads), if your nodes for some reasons have different data (like connectivity problems, node down, etc). I only perform

Re: repair strange behavior

the temporary extra volume comes from? Le 21 avr. 2012 20:43, Igor i...@4friends.od.ua mailto:i...@4friends.od.ua a écrit : Hi I can't understand the repair behavior in my case. I have 12 nodes ring (all 1.0.7): 10.254.237.2LA ADS-LA-1Up Normal 50.92 GB

repair strange behavior

Hi I can't understand the repair behavior in my case. I have 12 nodes ring (all 1.0.7): 10.254.237.2LA ADS-LA-1Up Normal 50.92 GB 0.00% 0 10.254.238.2TX TX-24-RACK Up Normal 33.29 GB 0.00% 1 10.254.236.2VA ADS-VA-1

Re: size tiered compaction - improvement

For my use case it would be nice to have per CF TTL (to protect myself from application bug and from storage leak due to missed TTL), but seems you can't avoid tombstones even in this case and if you change CF TTL during runtime. On 04/18/2012 03:06 PM, Viktor Jevdokimov wrote: Our use case

Re: size tiered compaction - improvement

Thank you Jonatathan, I missed this point about converting TTL data to tombstones first. When you say: You absolutely can. That's what the user defined part is: you give it the exact list of sstables you want compacted. does it mean that I can use list (not just one) of sstables as

Re: Repair Process Taking too long

Hi! What is the difference between 'repair' and '-pr repair'? Simple repair touch all token ranges (for all nodes) and -pr touch only range for which given node responsible? On 04/12/2012 05:59 PM, Sylvain Lebresne wrote: On Thu, Apr 12, 2012 at 4:06 PM, Frank Ngbuzzt...@gmail.com wrote:

Re: size tiered compaction - improvement

it shoould be wiped at any compaction on table A. Or I missed something? On 04/14/2012 11:27 AM, Radim Kolar wrote: Dne 4.4.2012 6:52, Igor napsal(a): Here is small python script I run once per day. You have to adjust size and/or age limits in the 'if' operator. Also I use mx4j interface

Re: need of regular nodetool repair

it would be pretty rare for that to occur. At least that is how I interpret this. 2012/4/11 Igor i...@4friends.od.ua mailto:i...@4friends.od.ua BTW, I heard that we don't need to run repair if all your data have TTL, all HH works, and you never delete your data. On 04/11/2012 11:34 AM

Re: need of regular nodetool repair

On 04/11/2012 12:04 PM, ruslan usifov wrote: HH - this is hinted handoff? Yes 2012/4/11 Igor i...@4friends.od.ua mailto:i...@4friends.od.ua On 04/11/2012 11:49 AM, R. Verlangen wrote: Not everything, just HH :) I hope this works for me for the next reasons: I have quite large

Re: Repair Process Taking too long

Hi You can check with nodetool which part of repair process is slow - network streams or verify compactions. use nodetool netstats or compactionstats. On 04/10/2012 05:16 PM, Frank Ng wrote: Hello, I am on Cassandra 1.0.7. My repair processes are taking over 30 hours to complete. Is it

Re: Repair Process Taking too long

this? On Tue, Apr 10, 2012 at 10:42 AM, Igor i...@4friends.od.ua mailto:i...@4friends.od.ua wrote: Hi You can check with nodetool which part of repair process is slow - network streams or verify compactions. use nodetool netstats or compactionstats. On 04/10/2012 05:16 PM

Re: Repair Process Taking too long

of ranges? zgrep -E (Performing streaming repair|out of sync) On Tue, Apr 10, 2012 at 9:45 AM, Igor i...@4friends.od.ua mailto:i...@4friends.od.ua wrote: On 04/10/2012 07:16 PM, Frank Ng wrote: Short answer - yes. But you are asking wrong question. I think both processes

Re: size tiered compaction - improvement

if you know for sure that you will free lot of space compacting some old table, then you can call UserdefinedCompaction for this table(you can do this from cron). There is also a ticket in jira with discussion on per-sstable expierd column and tombstones counters.   -Original

Re: size tiered compaction - improvement

The first is keyspace name, second is sstable name (like transaction-hc-1024-Data.db   -Original Message- From: Radim Kolar h...@filez.com To: user@cassandra.apache.org Sent: Wed, 04 Apr 2012 3:14 Subject: Re: size tiered compaction - improvement Dne 3.4.2012 23:04,

Re: size tiered compaction - improvement

Here is small python script I run once per day. You have to adjust size and/or age limits in the 'if' operator. Also I use mx4j interface for jmx calls. #!/usr/bin/env python import sys,os,glob,time,urllib2 CASSANDRA_DATA='/spool1/cassandra/data' DONTTOUCH=('system',) now = time.time() def

Re: Advice on architecture

On 03/28/2012 02:04 PM, Radim Kolar wrote: RAID0 would help me use more efficiently the total disk space available at each node, but tests have shown that under write load it behaves much worse than using separate data dirs, one per disk. there are different strategies how RAID0 splits reads,

Re: repair broke TTL based expiration

Hello Datasize should decrease during minor compactions. Check logs for compactions results.   -Original Message- From: Radim Kolar h...@filez.com To: user@cassandra.apache.org Sent: Mon, 19 Mar 2012 12:16 Subject: repair broke TTL based expiration I suspect that running cluster

Re: repair broke TTL based expiration

You can try to play with comaction thresholds - looks like your data wait too long before sizetiered compaction start to merge old large sstables. I have the same scenario as you (no deletes, all data with TTL) and I use script which call userdefinedcompaction on these old sstables.

Re: Node joining / unknown

Maybe it wait for verification compaction on other node?   -Original Message- From: R. Verlangen ro...@us2.nl To: user@cassandra.apache.org Sent: Wed, 07 Mar 2012 22:15 Subject: Re: Node joining / unknown At this moment the node has joined the ring (after a restart: tried that before,

Re: Node joining / unknown

just run nodetool compactionstat on other nodes. -Original Message- From: R. Verlangen ro...@us2.nl To: user@cassandra.apache.org Sent: Wed, 07 Mar 2012 23:09 Subject: Re: Node joining / unknown @Brandon: Thank you for the information. I'll do that next time. @Igor: Any ways to find

rolling restart after gc_grace change

Hi! On the http://wiki.apache.org/cassandra/Operations#Dealing_with_the_consequences_of_nodetool_repair_not_running_within_GCGraceSeconds you can read: To minimize the amount of forgotten deletes, first increase GCGraceSeconds across the cluster (rolling restart required) Rolling restart

improving cassandra-vs-mongodb-vs-couchdb-vs-redis

that a Cassandra expert can improve that generic description) Thanks, Igor

Re: improving cassandra-vs-mongodb-vs-couchdb-vs-redis

to paper over 6 technologies and compare them with a few bullet points is really doing the world an injustice. On Tue, Dec 27, 2011 at 5:01 PM, Igor Lino icampi...@gmx.de wrote: Hi! I was trying to get an understanding of the real strengths of Cassandra against other

repair waiting for something

Hi, 10 nodes cassandra 1.0.3, several DC. weekly nodetool repair stuck for unusual long time for node 10.254.237.2. output log on this node: INFO 11:19:42,045 Starting repair command #1, repairing 5 ranges. INFO 11:19:42,053 [repair #040aae00-28a1-11e1--e378018944ff] new session: will

Re: One ColumnFamily places data on only 3 out of 4 nodes

Do you use randompartitiner? What nodetool getendpoints show for several random keys? -Original Message- From: Bart Swedrowski b...@timedout.org To: user@cassandra.apache.org Sent: Wed, 14 Dec 2011 12:56 Subject: Re: One ColumnFamily places data on only 3 out of 4 nodes Anyone? On 12

Re: One ColumnFamily places data on only 3 out of 4 nodes

No idea, try to check logs for errors, and increase verbosity level on that node. -Original Message- From: Bart Swedrowski b...@timedout.org To: user@cassandra.apache.org Sent: Wed, 14 Dec 2011 16:45 Subject: Re: One ColumnFamily places data on only 3 out of 4 nodes On 14 December

plan to switch fro SimpleStrategy to NetworkTopologyStrategy

Hi, This is my first post, so first of all - thanks to Cassandra authors and community for their excellent job! Now to my question... I need a plan for transition from SimpleStrategy to NetworkSopologyStrategy (as I have to add two servers from remote datacenter with RTT up to 120ms to my