CPU high load

Hi, I am running 6 nodes vnode cluster with DSE 4.8.1, and since few weeks ago, all of the cluster nodes are hitting avg. 15-20 cpu load. These nodes are running on VMs(VMware vSphere) that have 8vcpu (1core/socket)-16 vRAM.(JVM options : -Xms8G -Xmx8G -Xmn800M) At first I thought this is

Re: Is my cluster normal?

Hi Jonathan, Here is the result: ubuntu@ip-172-31-44-250:~$ iostat -dmx 2 10 Linux 3.13.0-74-generic (ip-172-31-44-250) 07/12/2016 _x86_64_ (4 CPU) Device: rrqm/s wrqm/s r/s w/srMB/swMB/s avgrq-sz avgqu-sz await r_await w_await svctm %util xvda 0.01

Re: Is my cluster normal?

While I'm surprised you don't have any dropped message I have to point the finger against the following tpstats line: Native-Transport-Requests 128 128 347508531 2 15891862 where the the first '128' are the active reuests and the second '128' are the pending ones.

Re: NoHostAvailableException coming up on our server

I am using 3.0.0 version over apache-cassandra-3.3 On Tue, Jul 12, 2016 at 2:37 PM Riccardo Ferrari wrote: > What driver version are you using? > > You can look at the LoggingRetryPolicy to have more meaningful messages in > your logs. > > best, > > On Tue, Jul 12, 2016 at

Re: NoHostAvailableException coming up on our server

What driver version are you using? You can look at the LoggingRetryPolicy to have more meaningful messages in your logs. best, On Tue, Jul 12, 2016 at 9:02 PM, Abhinav Solan wrote: > Thanks, Johnny > Actually, they were running .. it went through a series of read and

Re: Questions on LCS behaviour after big BulkLoad cluster bootstrap

Garo, When your loading data in using LCS in a bulk fashion like this, there are a few things you should do. You can disable STCS in L0 (https://issues.apache.org/jira/browse/CASSANDRA-6621 ) with the JVM flag

Re: (C)* stable version after 3.5

Hey Varun - yeah, I would stick to 3.0.x for stability reasons unless there is a feature in the later releases you want. You should be able to upgrade directly to 3.0 from 2.1 without going via 2.2 > On 12 Jul 2016, at 15:01, Alain RODRIGUEZ wrote: > > Hi, > > The only

Re: Is my cluster normal?

When you have high system load it means your CPU is waiting for *something*, and in my experience it's usually slow disk. A disk connected over network has been a culprit for me many times. On Tue, Jul 12, 2016 at 12:33 PM Jonathan Haddad wrote: > Can do you do: > > iostat

Re: Is my cluster normal?

Can do you do: iostat -dmx 2 10 On Tue, Jul 12, 2016 at 11:20 AM Yuan Fang wrote: > Hi Jeff, > > The read being low is because we do not have much read operations right > now. > > The heap is only 4GB. > > MAX_HEAP_SIZE=4GB > > On Thu, Jul 7, 2016 at 7:17 PM, Jeff

Re: NoHostAvailableException coming up on our server

Thanks, Johnny Actually, they were running .. it went through a series of read and writes .. and recovered after the error. Is there any settings I can specify in preparing the Session at java client driver level, here are my current settings - PoolingOptions poolingOptions = new PoolingOptions()

Re: NoHostAvailableException coming up on our server

Abhinav - your getting that as the driver isn’t finding any hosts up for your query. You probably need to check if all the nodes in your cluster are running. See: http://docs.datastax.com/en/drivers/java/3.0/com/datastax/driver/core/exceptions/NoHostAvailableException.html Johnny > On 12 Jul

Re: Is my cluster normal?

Hi Jeff, The read being low is because we do not have much read operations right now. The heap is only 4GB. MAX_HEAP_SIZE=4GB On Thu, Jul 7, 2016 at 7:17 PM, Jeff Jirsa wrote: > EBS iops scale with volume size. > > > > A 600G EBS volume only guarantees 1800 iops –

Re: Is my cluster normal?

Hi Jonathan, The IOs are like below. I am not sure why one node always has a much bigger KB_read/s than other nodes. It seems not good. == avg-cpu: %user %nice %system %iowait %steal %idle 54.78 24.489.350.960.08 10.35 Device:tps

NoHostAvailableException coming up on our server

Hi Everyone, I am getting this error on our server, it comes and goes seems the connection drops a comes back after a while - Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: :9042

Re: (C)* stable version after 3.5

You may want to read more about Cassandra release process, find: http://www.planetcassandra.org/blog/cassandra-2-2-3-0-and-beyond/ On Tue, Jul 12, 2016 at 4:01 PM, Alain RODRIGUEZ wrote: > Hi, > > The only "fix" release after 3.5 is 3.7. Yet hard to say if it is more >

Re: (C)* stable version after 3.5

Hi, The only "fix" release after 3.5 is 3.7. Yet hard to say if it is more stable, we can hope so. For Tic-Toc releases (on 3.X) Odd numbers are fix releases. Even numbers are feature releases. Not sure why you want something above 3.5, but take care, those versions are really recent, and less

Re: how to start a embed cassandra instance?

If you're looking something similar to cassandra-unit with Apache 2 licence, there is a module in Achilles project that provides the same thing: https://github.com/doanduyhai/Achilles/wiki/CQL-embedded-cassandra-server On Tue, Jul 12, 2016 at 12:56 PM, Peddi, Praveen wrote: >

Re: Are updates on map columns causing tombstones?

I did a short write-up on this. http://www.sestevez.com/on-cassandra-collections-updates-and-tombstones/ all the best, Sebastián On Jul 11, 2016 4:22 PM, "Alain RODRIGUEZ" wrote: > Hi Jan, > > >> when I replace the content of a map-valued column (when I replace the >>

Re: how to start a embed cassandra instance?

We do something similar by starting CassandraDaemon class directly (you would need to provide a yaml file though). You can start and stop CassandraDaemon class from your unit test (typically @BeforeClass). Praveen On Jul 12, 2016, at 3:30 AM, Stone Fang

Re: Can't get performance for scanning the column that is not row key.

Wow. I have cassandra 1.2 and about fifty nodes. This Cassandra version is no longer supported, 2.0 isn't either and 2.1 is reaching end of life... You could use materialised views, SASI indexes or this kind of new features. Also CQL performances are better in recent versions etc. You don't

Re: How to print out the metrics information, like compaction and garbage collection?

Hi Jun, In case you did not solved your issue yet: Metrics are exposed through JMX. Any jmx tool will allow you to read those metrics. For a more automated and interesting usage (creating dashboards etc), you can use a metric reporter as described for graphite there:

Re: Can you enable Thrift on a node and still connect remotely to it using CQL/port 9042?

My understanding is this should work. As this is a small issue from a month ago, please shout out if you are still facing this issue. I'll try to help. C*heers, --- Alain Rodriguez - al...@thelastpickle.com France The Last Pickle - Apache Cassandra Consulting

(C)* stable version after 3.5

Hi all users, Currently we are using cassandra-2.1.13 but we want to upgrade *2.1.13 to 3.x* in production. Could anyone please tell me which is the most stable *cassandra version after 3.5*. Thanking You!! Regards, Varun Barala

Re: Help regarding cassandra node networking..

Hi, A few comments as I read: > I'm using node0 as seeder It is recommended to use at least 2 nodes per DC as seeds, for fault tolerance and good gossip propagation. Also I think you are a bit confused about what a seed node is and is not. See:

how to start a embed cassandra instance?

Hi, how to start a embed cassandra instance?so we can do a unit test on local,dont need to start a cassandra server. https://github.com/jsevellec/cassandra-unit this project is good,but the license is not suitable. how do you achieve this? thanks in advance stone

Re: How I can ignore clustering key order and read latests writes first?

Thanks for response. Note: I'm not good in English (I'm not native and I'm learning); Please forgive me if it words aren't easy to understand ;) I didn't satisfied for some reasons: - as you mentioned, it don't (logically) override the last value! and it will cause lot of side effects (from