Re: Is my cluster normal?

Those numbers, as I suspected, line up pretty well with your AWS configuration and network latencies within AWS. It is clear that this is a WRITE ONLY test. You might want to do a mixed (e.g. 50% read, 50% write) test for sanity. Note that the test will populate the data BEFORE it begins doing the

Re: Is my cluster normal?

EBS iops scale with volume size. A 600G EBS volume only guarantees 1800 iops – if you’re exhausting those on writes, you’re going to suffer on reads. You have a 16G server, and probably a good chunk of that allocated to heap. Consequently, you have almost no page cache, so your reads are

Re: [RELEASE] Apache Cassandra 3.0.8 released

Sorry, I totally missed that. Uploading now. On Thu, Jul 7, 2016 at 4:51 AM, horschi wrote: > Same for 2.2.7. > > On Thu, Jul 7, 2016 at 10:49 AM, Julien Anguenot > wrote: > >> Hey, >> >> The Debian packages do not seem to have been published. Normal?

Re: Is my cluster normal?

What's your CPU looking like? If it's low, check your IO with iostat or dstat. I know some people have used Ebs and say it's fine but ive been burned too many times. On Thu, Jul 7, 2016 at 6:12 PM Yuan Fang wrote: > Hi Riccardo, > > Very low IO-wait. About 0.3%. > No

Re: How to get current value of commitlog_segment_size_in_mb?

commitlog_segment_size_in_mb is configed in cassandra.yaml.dont think that it wolud be stored in Cassandra system table. the following is the introduction of Cassandra System table. https://docs.datastax.com/en/cql/3.1/cql/cql_using/use_query_system_c.html On Fri, Jul 8, 2016 at 4:23 AM,

Re: Is my cluster normal?

Hi Riccardo, Very low IO-wait. About 0.3%. No stolen CPU. It is a casssandra only instance. I did not see any dropped messages. ubuntu@cassandra1:/mnt/data$ nodetool tpstats Pool NameActive Pending Completed Blocked All time blocked MutationStage

Re: Is my cluster normal?

Hi Yuan, You machine instance is 4 vcpus that is 4 threads (not cores!!!), aside from any Cassandra specific discussion a system load of 10 on a 4 threads machine is way too much in my opinion. If that is the running average system load I would look deeper into system details. Is that IO wait? Is

Re: Is my cluster normal?

Thanks Ben! For the post, it seems they got a little better but similar result than i did. Good to know it. I am not sure if a little fine tuning of heap memory will help or not. On Thu, Jul 7, 2016 at 2:58 PM, Ben Slater wrote: > Hi Yuan, > > You might find this

Re: Is my cluster normal?

Hi Ryan, The version of cassandra is 3.0.6 and java version "1.8.0_91" Yuan On Thu, Jul 7, 2016 at 3:11 PM, Ryan Svihla wrote: > what version of cassandra and java? > > Regards, > > Ryan Svihla > > On Jul 7, 2016, at 4:51 PM, Yuan Fang wrote: > >

Re: Is my cluster normal?

what version of cassandra and java? Regards, Ryan Svihla > On Jul 7, 2016, at 4:51 PM, Yuan Fang wrote: > > Yes, here is my stress test result: > Results: > op rate : 12200 [WRITE:12200] > partition rate: 12200 [WRITE:12200] > row rate

unsubscribe

Re: Is my cluster normal?

Hi Yuan, You might find this blog post a useful comparison: https://www.instaclustr.com/blog/2016/01/07/multi-data-center-apache-spark-and-apache-cassandra-benchmark/ Although the focus is on Spark and Cassandra and multi-DC there are also some single DC benchmarks of m4.xl clusters plus some

Re: Is my cluster normal?

Yes, here is my stress test result: Results: op rate : 12200 [WRITE:12200] partition rate: 12200 [WRITE:12200] row rate : 12200 [WRITE:12200] latency mean : 16.4 [WRITE:16.4] latency median: 7.1 [WRITE:7.1] latency 95th

Re: Is my cluster normal?

Lots of variables you're leaving out. Depends on write size, if you're using logged batch or not, what consistency level, what RF, if the writes come in bursts, etc, etc. However, that's all sort of moot for determining "normal" really you need a baseline as all those variables end up

Re: Is my cluster normal?

yes, it is about 8k writes per node. On Thu, Jul 7, 2016 at 2:18 PM, daemeon reiydelle wrote: > Are you saying 7k writes per node? or 30k writes per node? > > > *...* > > > > *Daemeon C.M. ReiydelleUSA (+1) 415.501.0198 > <%28%2B1%29%20415.501.0198>London (+44) (0) 20

Re: Is my cluster normal?

Are you saying 7k writes per node? or 30k writes per node? *...* *Daemeon C.M. ReiydelleUSA (+1) 415.501.0198London (+44) (0) 20 8144 9872* On Thu, Jul 7, 2016 at 2:05 PM, Yuan Fang wrote: > writes 30k/second is the main thing. > > > On Thu, Jul 7, 2016 at 1:51

Re: Is my cluster normal?

writes 30k/second is the main thing. On Thu, Jul 7, 2016 at 1:51 PM, daemeon reiydelle wrote: > Assuming you meant 100k, that likely for something with 16mb of storage > (probably way small) where the data is more that 64k hence will not fit > into the row cache. > > >

Re: Is my cluster normal?

Assuming you meant 100k, that likely for something with 16mb of storage (probably way small) where the data is more that 64k hence will not fit into the row cache. *...* *Daemeon C.M. ReiydelleUSA (+1) 415.501.0198London (+44) (0) 20 8144 9872* On Thu, Jul 7, 2016 at 1:25 PM, Yuan Fang

Is my cluster normal?

I have a cluster of 4 m4.xlarge nodes(4 cpus and 16 gb memory and 600GB ssd EBS). I can reach a cluster wide write requests of 30k/second and read request about 100/second. The cluster OS load constantly above 10. Are those normal? Thanks! Best, Yuan

How to get current value of commitlog_segment_size_in_mb?

Hi, In my project I need to read current value for "commitlog_segment_size_in_mb", I am looking for CQL query to do this. Any idea if this information gets stored in any of the Cassandra system table? Thanks, Jaydeep

Re: Debugging high tail read latencies (internal timeout)

Hmm. Would you mind looking at your network interface (appropriate netstat commands). if I am right you will be seeing packet errors, drops, retries, packet out of window receives, etc. What you may be missing is that you reported zero DROPPED latency. Not mean LATENCY. Check your netstats. ANY

Re: Debugging high tail read latencies (internal timeout)

Hi Nimi, My suspicions would probably lie somewhere between GC and large partitions. The first tool would probably be a trace but if you experience full client timeouts from dropped messages you may find it hard to find the issue. You can try running the trace with cqlsh's timeouts cranked all

Divergence from Cassandra partition_count and partition keys count

Hello, Im trying to use spark with cassandra and it was oddly generating several spark jobs because spark follow the guidelines generated by partitions_count and mean_partition_size. The problem is that I have a very small table (300MB) with only 16 distinct partition keys running on a single C*

Re: DTCS SSTable count issue

48 sstables isn’t unreasonable in a DTCS table. It will continue to grow over time, but ideally data will expire as it nears your 90 day TTL and those tables should start dropping away as they age. 3.0.7 introduces an alternative to DTCS you may find easier to use called TWCS. It will

Blog post on Cassandra's inner workings and performance - feedback welcome (URL fixed)

Hi all, I'm currently in the process of understanding the inner workings of Cassandra with regards to network and local storage mechanisms and operations. In order to do so, I've written a blog post about it which is now in a "first final" version. Any feedback, especially corrections regarding

DTCS SSTable count issue

Hi everyone, This is my first question, apologize may I do something wrong. I have a small Cassandra cluster build upon 3 nodes. Originally born as 2.0.X cluster was upgraded to 2.0.15 then 2.1.13 and finally to 3.0.4 recently 3.0.6. Ubuntu is the OS. There are few tables that have

Re: Blog post on Cassandra's inner workings and performance - feedback welcome

Awfully sorry, here is the correct URI to the blog post: http://journeymonitor.com:4000/tutorials/2016/02/29/cassandra-inner-workings-and-how-this-relates-to-performance/ Regards, Manuel On Thu, Jul 7, 2016 at 3:11 PM, Manuel Kiessling wrote: > Hi all, > > I'm currently

Re: Blog post on Cassandra's inner workings and performance - feedback welcome

http://localhost:4000/tutorials/2016/02/29/cassandra-inner-workings-and-how-this-relates-to-performance/ This is not a valid address. On Thu, Jul 7, 2016 at 9:11 AM, Manuel Kiessling wrote: > Hi all, > > I'm currently in the process of understanding the inner workings of >

Re: Blog post on Cassandra's inner workings and performance - feedback welcome

Hi Manuel, Nice initiative, but I think you posted a link to your local machine. Best regards, Steven Choo ste...@humanswitch.io www.humanswitch.io www.iqnomy.com Office: HumanSwitch Bijsterveldenlaan 5 5045 ZZ Tilburg Netherlands 0031-(0)13-303 11 60 On Thu, Jul 7, 2016 at 3:11 PM,

Blog post on Cassandra's inner workings and performance - feedback welcome

Hi all, I'm currently in the process of understanding the inner workings of Cassandra with regards to network and local storage mechanisms and operations. In order to do so, I've written a blog post about it which is now in a "first final" version. Any feedback, especially corrections regarding

Re: whats the default .yaml file that cassandra-stress uses

I added concurrent_writes: 48 to the cqlstress-example.yaml, which is recommended as No. of cores * 8. But no improvement is made when trying 100 writes test. Since cassandra.yaml specifies concurrent_writes, but cqlstress-example.yaml doesn't. BTW, the following warning was shown as the

Re: [RELEASE] Apache Cassandra 3.0.8 released

Same for 2.2.7. On Thu, Jul 7, 2016 at 10:49 AM, Julien Anguenot wrote: > Hey, > > The Debian packages do not seem to have been published. Normal? > > Thank you. > >J. > > On Jul 6, 2016, at 4:20 PM, Jake Luciani wrote: > > The Cassandra team is

Re: [RELEASE] Apache Cassandra 3.0.8 released

Hey, The Debian packages do not seem to have been published. Normal? Thank you. J. > On Jul 6, 2016, at 4:20 PM, Jake Luciani wrote: > > The Cassandra team is pleased to announce the release of Apache Cassandra > version 3.0.8. > > Apache Cassandra is a fully

Re: whats the default .yaml file that cassandra-stress uses

there is a simple config file for testing."cqlstress-example.yaml" under the tool folder. you can customize this file to achieve your test stone. On Thu, Jul 7, 2016 at 4:28 PM, Daiyue Weng wrote: > Hi, I am wondering what's the default .yaml file that cassandra-stress >

whats the default .yaml file that cassandra-stress uses

Hi, I am wondering what's the default .yaml file that cassandra-stress uses when testing writes and reads when command 'profile=' is not specified. Is it the cassandra.yaml? Does it affect the performance of cassandra-stress test by modifying/tuning it? ps.I am running cassandra instances on