Re: Reaper repair seems to "hang"

2017-01-03 Thread Bhuvan Rawal
Hi Daniel, Looks like yours is a different case. If you're running incremental repair for the first time it make take long time esp. if table is large. And repair may seem to stuck even when things are working. You can try nodetool compactionstats when repair appears stuck, you'll find a

Re: Trying to find cause of exception

2017-01-03 Thread Ney, Richard
Johnny, Would these WARN cause the read issues I’m seeing WARN [GossipTasks:1] 2017-01-03 03:27:48,926 Gossiper.java:752 - Gossip stage has 7 pending tasks; skipping status check (no nodes will be marked down) WARN [ScheduledTasks:1] 2017-01-03 03:27:48,997 MonitoringTask.java:150 - 1

`nodetool verify` outcome check

2017-01-03 Thread Jens Rantil
Hi, We've had a discussion internally to start to run `nodetool verify` periodically to test for bitrot. Does anyone know how I could check if the verification failed or succeeded from, say, a script? Is there an error exit code or some output I could grep for? Thanks, Jens -- Jens Rantil

Re: Why does Cassandra recommends Oracle JVM instead of OpenJDK?

2017-01-03 Thread Kai Wang
Back in the day, HotSpot was recommended because OpenJDK had some stability and performance issues. But in 2015 or maybe 2014 I heard in a presentation (don't remember by whom) that OpenJDK is pretty on par with HotSpot for C*. But I guess the documentation was never properly updated. On Tue,

Re: Growing Hints

2017-01-03 Thread Anshu Vajpayee
Anyone aware about issue ? Hints are still growing although gossip and repair was successfull. Gossip is flowing without any issue as all nodes status is UN. Hints are growing and there is timed out message in log hinted handoff delivery. Hints are not truncating after defined time period.

Re: Why does Cassandra recommends Oracle JVM instead of OpenJDK?

2017-01-03 Thread Eric Evans
On Mon, Jan 2, 2017 at 2:26 PM, Edward Capriolo wrote: > Lets be clear: > What I am saying is avoiding being loose with the word "free" > > https://en.wikipedia.org/wiki/Free_software_license > > Many things with the JVM are free too. Most importantly it is free to use. > >

Re: Reaper repair seems to "hang"

2017-01-03 Thread Bhuvan Rawal
Hi Daniel, We faced a similar issue during repair with reaper. We ran repair with more repair threads than number of cassandra nodes. But on and off repair was getting stuck and we had to do rolling restart of cluster or wait for lock time to expire (~1hr). We had a look at the stuck repair,

RE: Growing Hints

2017-01-03 Thread SEAN_R_DURITY
Version number may help. Sean Durity From: Anshu Vajpayee [mailto:anshu.vajpa...@gmail.com] Sent: Tuesday, January 03, 2017 10:09 AM To: user@cassandra.apache.org Subject: Re: Growing Hints Anyone aware about issue ? Hints are still growing although gossip and repair was successfull. Gossip

RE: Query on Cassandra clusters

2017-01-03 Thread SEAN_R_DURITY
A couple thoughts (for after you up/downgrade to one version for all nodes): - 16 GB of total RAM on a node is a minimum I would use; 32 would be much better - With a lower amount of memory, I think would keep memtables on-heap in order to keep a tighter rein on how much they

Dropped Mutation Messages in two DCs at different sites

2017-01-03 Thread Benyi Wang
I need to batch load a lot of data everyday into a keyspace across two DCs, one DC is at west coast and the other is at east coast. I assume that the network delay between two DCs at different sites will cause a lot of dropped mutation messages if I write too fast in LOCAL DC using LOCAL_QUORUM.

Re: Cassandra cluster performance

2017-01-03 Thread Branislav Janosik -T (bjanosik - AAP3 INC at Cisco)
Hi, Our column family definition is "CREATE TABLE onem2m.cse(" + "name TEXT PRIMARY KEY," + "resourceId TEXT," + ")"; "CREATE TABLE IF NOT EXISTS onem2m.AeIdToResourceIdMapping(" + "cseBaseCseId TEXT," + "aeId TEXT," + "resourceId TEXT," +

Re: Reaper repair seems to "hang"

2017-01-03 Thread Daniel Kleviansky
Hi Bhuvan, Thank you so very much for your detailed reply. Just to ensure everyone is across the same information, and responses are not duplicated across two different forums, I thought I'd share with the mailing list that I've created a GitHub issue at:

RE: Trying to find cause of exception

2017-01-03 Thread Amit Singh F
Hi Rcichard, From below msgs, mutations drops are clear. 1. Can you check Nodetool tpstats output , dropped MUTATION value. If it is on high side. Quickly schedule repair on that node to prevent any data loss. 2. Also please check with your N/W team ,is there any fluctuation or

RE: Growing Hints

2017-01-03 Thread Anshu Vajpayee
Cassandra verison 2.1.13 On Jan 4, 2017 12:34 AM, wrote: > Version number may help. > > > > Sean Durity > > > > *From:* Anshu Vajpayee [mailto:anshu.vajpa...@gmail.com] > *Sent:* Tuesday, January 03, 2017 10:09 AM > *To:* user@cassandra.apache.org > *Subject:* Re:

Re: Cassandra cluster performance

2017-01-03 Thread Branislav Janosik -T (bjanosik - AAP3 INC at Cisco)
Hi, No we are not using async writes. From: kurt Greaves Reply-To: "user@cassandra.apache.org" Date: Friday, December 23, 2016 at 12:17 AM To: "user@cassandra.apache.org" Subject: Re: Cassandra cluster performance

Re: Incremental repair for the first time

2017-01-03 Thread Kathiresan S
Thank you! We are planning to upgrade to 3.0.10 for this issue. >From the NEWS txt file ( https://github.com/apache/cassandra/blob/trunk/NEWS.txt), it looks like there is no need for sstableupgrade when we upgrade from 3.0.4 to 3.0.10 (i.e. Just installing 3.0.10 Cassandra would suffice and it