Re: One thread pool per repair in nodetool tpstats

2017-02-21 Thread Matija Gobec
They appear for each repair run and disappear when repair run finishes. On Tue, Feb 21, 2017 at 11:14 AM, Vincent Rischmann wrote: > Hi, > > I upgraded to Cassandra 2.2.8 and noticed something weird in nodetool > tpstats: > > Pool NameActive Pending

Re: Logging queries

2017-02-18 Thread Matija Gobec
Hi Igor, Your best bet is to wait for our next release of diagnostics for 2.x branch. We are planning it for next week. Best, Matija On Sat, Feb 18, 2017 at 8:58 PM, Igor Leão wrote: > Hi Bhuvan, > Thanks a lot! > >

Re: Is it possible to replay hints after running nodetool drain?

2016-09-02 Thread Matija Gobec
Hi Jerome, The node being drained stops listening to requests but the other nodes being coordinators for given requests will store hints for that downed node for a configured period of time (max_hint_window_in_ms is 3 hours by default). If the downed node is back online in this time window it

Re: How to get information of each read/write request?

2016-08-30 Thread Matija Gobec
Hi Jun, If you are looking to track each request zipking is your best bet. The last pickle has a blog about tracing using zipkin. Regarding the stats you see in nodetool did you check the

Re: Throughout of hints delivery

2016-09-17 Thread Matija Gobec
Hi, You should first figure out why you have so many hints and then think about throughput of hints delivery. Hints are generated for dead nodes and in a healthy cluster are not present. Are all your nodes alive and running? What is the issue of inter DC connectivity? Matija -- *Matija Gobec

Re: Introducing Cassandra 3.7 LTS

2016-10-19 Thread Matija Gobec
Hi Ben, Thanks for this awesome contribution. I'm eager to give it a try and test it out. Best, Matija On Wed, Oct 19, 2016 at 8:55 PM, Ben Bromhead wrote: > Hi All > > I am proud to announce we are making available our production build of > Cassandra 3.7 that we run at

Re: Unsubscribe

2016-10-12 Thread Matija Gobec
Omar, Send an empty email to user-unsubscr...@cassandra.apache.org to unsubscribe. See you On Wed, Oct 12, 2016 at 1:33 PM, Omar Mambelli wrote: > Unsubscribe > > -- > Inviato da iPhone >

Re: unsubscrible

2016-10-12 Thread Matija Gobec
Steven, Send an empty email to user-unsubscr...@cassandra.apache.org to unsubscribe. See you On Wed, Oct 12, 2016 at 8:15 PM, zhao yi wrote: > > > Best regards, > > Steven Zhao > >

Re: Schema Changes

2016-11-15 Thread Matija Gobec
We used cassandra migration tool for schema versioning and schema agreement. Check it out here . Short: When executing schema altering statements use these to wait for schema propagation

Re: FW: Cassandra trigger to send notifications

2016-12-16 Thread Matija Gobec
Hi Oren, I've spent a reasonable time working out triggers and I would say that your best bet is doing this in the app. Just publish a rabbitmq message from the app when you execute a statement. If your goal is to have an audit then try batch writing data to the tables and delta to their audit

Re: quick questions

2016-12-17 Thread Matija Gobec
QUORUM is by documentation: quorum = (sum_of_replication_factors / 2) + 1 Its not fixed value (as 4). On Sat, Dec 17, 2016 at 10:21 PM, Kant Kodali wrote: > I keep hearing that the minimum number of Cassandra nodes required to > achieve Quorum consensus is 4 I wonder why

Re: Join_ring=false Use Cases

2016-12-20 Thread Matija Gobec
There is a talk from cassandra summit 2016 about coordinator nodes by Eric Lubow from SimpleReach. He explains how you can use that join_ring=false. On Tue, Dec 20, 2016 at 10:23 PM, kurt Greaves wrote: > It seems that you're correct in saying that writes don't propagate

Re: All nodes hosting replicas down

2016-12-18 Thread Matija Gobec
If you are reading and none of the replicas is online you will get an exception on the read (tried x replicas but 0 responded) and your read will fail. Writes on the other hand are going to go through only if your write consistency is ANY. If your write consistency is ONE or anything upwards, then

Re: Cqlsh timeout and schema refresh exceptions

2016-12-19 Thread Matija Gobec
There is an exposed API for schema agreement and I would advise you to use that if you can. Look at this JIRA ticket . On Mon, Dec 19, 2016 at 8:46 PM, Vladimir Yudovin wrote: > Regarding schema agreement - try to

Re: Has anyone deployed a production cluster with less than 6 nodes per DC?

2016-12-27 Thread Matija Gobec
There is nothing wrong with general purpose EBS volumes if we are talking about gp2 (SSD backed ones). With bigger volumes you get more IOPs and 3.4TB volume gives you 10.000 IOPs which, in your case, is an overkill (you are probably looking at 1TB). Take a look at TWCS since you are inserting

Re: About Tombstones and TTLs

2016-12-19 Thread Matija Gobec
Hi, gc_grace_seconds is used to maintain data consistency in some failure scenarios. When manually deleting data that action creates tombstones which are kept for that defined period before being compacted. If one of the replica nodes is down while deleting data and it gets back up after the

Re: Help with data modelling (from MySQL to Cassandra)

2017-03-26 Thread Matija Gobec
Have one table hold document metadata (doc_id, title, description, ...) and have another table elements where partition key is doc_id and clustering key is element_id. Only problem here is if you need to query and/or update element just by element_id but I don't know your queries up front. On

Re: Help with data modelling (from MySQL to Cassandra)

2017-03-27 Thread Matija Gobec
>> >> Cheers, >> >> >> On Mon, Mar 27, 2017 at 4:40 AM Zoltan Lorincz <zol...@gmail.com> wrote: >> >>> Querying by (doc_id and element_id ) OR just by (element_id) is fine, >>> but the real question is, will it be efficient to query 100k+

Re: Grouping time series data into blocks of times

2017-03-18 Thread Matija Gobec
If its a sliding 30 min window you will need to implement it and have an in-memory timestamp list but out of order messages will always be a headache. If you are ok with a fixed 30 min window (each 30 min eg 5:00, 5:30, 6:00,..) then just add a time bucket into the partition key and you are done.

Re: ONE has much higher latency than LOCAL_ONE

2017-03-21 Thread Matija Gobec
Are you running a multi DC cluster? If yes do you have application in both data centers/regions ? On Tue, Mar 21, 2017 at 8:07 PM, Shannon Carey wrote: > I am seeing unexpected behavior: consistency level ONE increases read > latency 99th percentile to ~108ms (95th

Re: how to recover a dead node using commit log when memtable is lost

2017-04-05 Thread Matija Gobec
Flushes have nothing to do with data persistence and node failure. Each write is acknowledged only when data has been written to the commit log AND memtable. That solves the issues of node failures and data consistency. When the node boots back up it replays commit log files and you don't loose

Re: Making a Cassandra node cluster unique

2017-04-05 Thread Matija Gobec
You CAN have two separate clusters with same name and configuration. Separation of the clusters is just a matter of defining seed nodes properly. That being said, it doesn't mean you SHOULD have clusters with same name. We usually run same cluster name when testing on test/stage cluster and

Re: autoscaling

2017-03-09 Thread Matija Gobec
Hi, Autoscaling is not possible with a Cassandra cluster. Any topology change triggers series of streaming and data shuffle around the cluster. Scaling the cluster up or down is an operational challenge which is usually planned in production because of the performance impact it can make. Matija

Re: Row cache tuning

2017-03-11 Thread Matija Gobec
Hi, In 99% of use cases Cassandra's row cache is not something you should look into. Leveraging page cache yields good results and if accounted for can provide you with performance increase on read side. I'm not a fan of a default row cache implementation and its invalidation mechanism on updates

Re: Does Java driver v3.1.x degrade cluster connect/close performance?

2017-03-06 Thread Matija Gobec
Interesting question since I never measured connect and close times. Usually this is something you do once the application starts and thats it. Do you plan to misuse it and create a new cluster object and open a new connection for each request? On Mon, Mar 6, 2017 at 7:19 AM, Satoshi Hikida

Re: too many compactions pending and compaction is slow on few tables

2017-04-07 Thread Matija Gobec
It does as the "new" data, even if the values are the same, has new write time timestamp. Spinning disks are hard to run LCS on. Do you maybe have some kind of non stripe raid in place? On Fri, Apr 7, 2017 at 8:46 PM, Giri P wrote: > Does LCS try compacting already compacted

Re: EC2 instance recommendations

2017-05-23 Thread Matija Gobec
We are running on I3s since they came out. NVMe SSDs are really fast and I managed to push them to 75k IOPs. As Bhuvan mentioned the i3 storage is ephemeral. If you can work around it and plan for failure recovery you are good to go. I ran Cassandra on m4s before and had no problems with EBS

Re: EC2 instance recommendations

2017-05-23 Thread Matija Gobec
inding that to get a reasonable amount of IOPS (gp2) > out of EBS at a reasonable rate, it gets more expensive than an I3. > > > > *From: *Jonathan Haddad <j...@jonhaddad.com> > *Date: *Tuesday, May 23, 2017 at 9:42 AM > *To: *"Gopal, Dhruva" <dhruva.go...@aspect.

Re: Working With Prepared Statements

2017-08-29 Thread Matija Gobec
Do you have any concrete questions re prepared statements? They are faster to execute since the statement is already parsed and in C* and you just pass the parameters. No additional statement processing is needed. Matija On Tue, Aug 29, 2017 at 12:33 PM, Shalom Sagges

Re: Working With Prepared Statements

2017-08-29 Thread Matija Gobec
facebook.com/LivePersonInc> We Create Meaningful Connections > > > > On Tue, Aug 29, 2017 at 2:18 PM, Matija Gobec <matija0...@gmail.com> > wrote: > >> Do you have any concrete questions re prepared statements? >> >> They are faster to execute since the stat

Re: SSH remote access,permissions issue

2018-02-07 Thread Matija Gobec
Hi Milenko, There is a high chance that JMX is listening on the instance IP instead of locahost which is default when you run nodetool command. Try executing nodetool with -h nodeip or check netstat as Nicolas mentioned. If you run nodetool status on one of the existing nodes can you see that new

Re: Upgrade from 1.2.x to 2.0.x, upgradesstables has doubled the size on disk?

2018-01-01 Thread Matija Gobec
Dan, What partitioner are you using and did you just swap out the binary? Going from 70GB to 200GB+ is extremely odd in any scenario. Maybe Carlos Rolo has an idea about this issue. He did a ton of 1.2 cluster upgrades. As for the tombstones, its the stat for the last five minutes. You could

Re: NVMe SSD benchmarking with Cassandra

2018-01-17 Thread Matija Gobec
Justin, NVMe drives have their own IO queueing mechanism and there is a huge performance difference vs the linux queue. Next to properly configured file system and scheduler try setting "scsi_mod.use_blk_mq=1" in grub cmdline. If you are looking for a BFQ scheduler, its probably a module so you

Re: Cassandra vs MySQL

2018-03-12 Thread Matija Gobec
Hi Oliver, Few years back I had a similar problem where there was a lot of data in MySQL and it was starting to choke. I migrated data to Cassandra, ran benchmarks and blew MySQL out of the water with a small 3 node C* cluster. If you have a use case for Cassandra the answer is yes, but keep in