Re: Cassandra DTests deadlocks on random test

2017-10-22 Thread Lutaya Shafiq Holmes
;> Sergey >> >> From: Michael Shuler >> Sent: Friday, October 20, 2017 3:28 PM >> To: dev@cassandra.apache.org >> Subject: Re: Cassandra DTests deadlocks on random test >> >>> On 10/20/2017 02:30 AM, Sergey wrote: >>> >>> I believ

Re: Cassandra DTests deadlocks on random test

2017-10-21 Thread Jeff Jirsa
20, 2017, at 10:17 PM, Sergey <cassandra.bu...@gmail.com> wrote: > > What environment are you guys using to run dtests? > > Best regards, > Sergey > > From: Michael Shuler > Sent: Friday, October 20, 2017 3:28 PM > To: dev@cassandra.apache.org > Subject: Re: Ca

RE: Cassandra DTests deadlocks on random test

2017-10-20 Thread Sergey
What environment are you guys using to run dtests? Best regards, Sergey From: Michael Shuler Sent: Friday, October 20, 2017 3:28 PM To: dev@cassandra.apache.org Subject: Re: Cassandra DTests deadlocks on random test On 10/20/2017 02:30 AM, Sergey wrote: > > I believe the probl

Re: Cassandra DTests deadlocks on random test

2017-10-20 Thread Michael Shuler
On 10/20/2017 02:30 AM, Sergey wrote: > > I believe the problem can be not related to RAM space, since today I > run into the same issue on 200GB RAM machine. Awesome! You've found a bug with either cassandra-dtest or cassandra. Welcome to Apache Cassandra Test Engineering. :) -- Kind regards,

RE: Cassandra DTests deadlocks on random test

2017-10-20 Thread Sergey
@cassandra.apache.org Subject: Re: Cassandra DTests deadlocks on random test 7.6G RAM may be a little bit too small, we've seen similar random hangs in the past on non-resource-intensive tests on m3.large. It doesn't appear you are skipping resource-intensive tests. Our standard dtest instance type has been an m3

Re: Cassandra DTests deadlocks on random test

2017-10-19 Thread Murukesh Mohanan
I have had this problem too on similarly spec'd OpenStack instances (but I'm reasonably certain I didn't include the resource-intensive tests). My solution was to run the dtests in small batches (say 5-10 each), with a timeout (say 1.2x the max for 5 tests from a good run). Kill the test if

Cassandra DTests deadlocks on random test

2017-10-19 Thread Sergey La
Hi! I have created the patch for the Cassandra version 3.0.14 and trying to test it using the cassandra dtests. Problem is - dtests deadlocks at some random tests, time and again - on unpatched 3.0.14 version of Cassandra. What I have done. I cloned the cassandra repository (origin