Re: Flakey Dtests

[Michael Kjellman]

do you know why this is the case? shouldn’t -all test...all?

> On Nov 27, 2017, at 7:39 PM, Michael Shuler  wrote:
> 
> The `test-cdc` target is not a dependent of `test-all`, so it was set up
> as a separate job in Jenkins:
> https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-3.11-test-cdc/
> https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-trunk-test-cdc/
> 
> -- 
> Michael
> 
>> On 11/27/2017 03:45 PM, Michael Kjellman wrote:
>> Hey Jay:
>> 
>> Thanks!! I just took a quick look at the JIRA and noticed that there is a 
>> “test-cdc” ant target? So, does that mean CDC get’s no testing with ant 
>> test? Do you know any of the history around this?
>> 
>>> On Nov 27, 2017, at 9:44 AM, Jay Zhuang  
>>> wrote:
>>> 
>>> I fixed one CDC uTest, please 
>>> review:https://issues.apache.org/jira/browse/CASSANDRA-14066
>>> 
>>> 
>>>   On Friday, November 17, 2017 6:34 AM, Josh McKenzie 
>>>  wrote:
>>> 
>>> 
 
 Do we have any volunteers to fix the broken Materialized Views and CDC
 DTests?
>>> 
>>> I'll try to take a look at the CDC tests next week; looks like one of the
>>> base unit tests is failing as well.
>>> 
>>> On Fri, Nov 17, 2017 at 12:09 AM, Michael Kjellman <
>>> mkjell...@internalcircle.com> wrote:
>>> 
 Quick update re: dtests and off-heap memtables:
 
 I’ve filed CASSANDRA-14056 (Many dtests fail with ConfigurationException:
 offheap_objects are not available in 3.0 when OFFHEAP_MEMTABLES=“true”)
 
 Looks like we’re gonna need to do some work to test this configuration and
 right now it’s pretty broken...
 
 Do we have any volunteers to fix the broken Materialized Views and CDC
 DTests?
 
 best,
 kjellman
 
 
> On Nov 15, 2017, at 5:59 PM, Michael Kjellman <
 mkjell...@internalcircle.com> wrote:
> 
> yes - true- some are flaky, but almost all of the ones i filed fail 100%
 () of the time. i look forward to triaging just the remaining flaky ones
 (hopefully - without powers combined - by the end of this month!!)
> 
> appreciate everyone’s help - no matter how small... i already personally
 did a few “fun” random-python-class-is-missing-return-after-method stuff.
> 
> we’ve wanted this for a while and now is our time to actually execute
 and make good on our previous dev list promises.
> 
> best,
> kjellman
> 
>> On Nov 15, 2017, at 5:45 PM, Jeff Jirsa  wrote:
>> 
>> In lieu of a weekly wrap-up, here's a pre-Thanksgiving call for help.
>> 
>> If you haven't been paying attention to JIRA, you likely didn't notice
 that
>> Josh went through and triage/categorized a bunch of issues by adding
>> components, and Michael took the time to open a bunch of JIRAs for
 failing
>> tests.
>> 
>> How many is a bunch? Something like 35 or so just for tests currently
>> failing on trunk.  If you're a regular contributor, you already know
 that
>> dtests are flakey - it'd be great if a few of us can go through and fix
 a
>> few. Even incremental improvements are improvements. Here's an easy
 search
>> to find them:
>> 
>> https://issues.apache.org/jira/secure/IssueNavigator.
 jspa?reset=true=project+%3D+CASSANDRA+AND+
 component+%3D+Testing+ORDER+BY+updated+DESC%2C+priority+
 DESC%2C+created+ASC=hide
>> 
>> If you're a new contributor, fixing tests is often a good way to learn a
>> new part of the codebase. Many of these are dtests, which live in a
>> different repo ( https://github.com/apache/cassandra-dtest ) and are in
>> python, but have no fear, the repo has instructions for setting up and
>> running dtests(
>> https://github.com/apache/cassandra-dtest/blob/master/INSTALL.md )
>> 
>> Normal contribution workflow applies: self-assign the ticket if you
 want to
>> work on it, click on 'start progress' to indicate that you're working on
>> it, mark it 'patch available' when you've uploaded code to be reviewed
 (in
>> a github branch, or as a standalone patch file attached to the JIRA). If
>> you have questions, feel free to email the dev list (that's what it's
 here
>> for).
>> 
>> Many thanks will be given,
>> - Jeff
 
 
>>> 
>> 
>> 
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
>> For additional commands, e-mail: dev-h...@cassandra.apache.org
>> 
> 
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
> 

Re: Flakey Dtests

[Michael Shuler]

The `test-cdc` target is not a dependent of `test-all`, so it was set up
as a separate job in Jenkins:
https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-3.11-test-cdc/
https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-trunk-test-cdc/

-- 
Michael

On 11/27/2017 03:45 PM, Michael Kjellman wrote:
> Hey Jay:
> 
> Thanks!! I just took a quick look at the JIRA and noticed that there is a 
> “test-cdc” ant target? So, does that mean CDC get’s no testing with ant test? 
> Do you know any of the history around this?
> 
>> On Nov 27, 2017, at 9:44 AM, Jay Zhuang  wrote:
>>
>> I fixed one CDC uTest, please 
>> review:https://issues.apache.org/jira/browse/CASSANDRA-14066
>>
>>
>>On Friday, November 17, 2017 6:34 AM, Josh McKenzie 
>>  wrote:
>>
>>
>>>
>>> Do we have any volunteers to fix the broken Materialized Views and CDC
>>> DTests?
>>
>> I'll try to take a look at the CDC tests next week; looks like one of the
>> base unit tests is failing as well.
>>
>> On Fri, Nov 17, 2017 at 12:09 AM, Michael Kjellman <
>> mkjell...@internalcircle.com> wrote:
>>
>>> Quick update re: dtests and off-heap memtables:
>>>
>>> I’ve filed CASSANDRA-14056 (Many dtests fail with ConfigurationException:
>>> offheap_objects are not available in 3.0 when OFFHEAP_MEMTABLES=“true”)
>>>
>>> Looks like we’re gonna need to do some work to test this configuration and
>>> right now it’s pretty broken...
>>>
>>> Do we have any volunteers to fix the broken Materialized Views and CDC
>>> DTests?
>>>
>>> best,
>>> kjellman
>>>
>>>
 On Nov 15, 2017, at 5:59 PM, Michael Kjellman <
>>> mkjell...@internalcircle.com> wrote:

 yes - true- some are flaky, but almost all of the ones i filed fail 100%
>>> () of the time. i look forward to triaging just the remaining flaky ones
>>> (hopefully - without powers combined - by the end of this month!!)

 appreciate everyone’s help - no matter how small... i already personally
>>> did a few “fun” random-python-class-is-missing-return-after-method stuff.

 we’ve wanted this for a while and now is our time to actually execute
>>> and make good on our previous dev list promises.

 best,
 kjellman

> On Nov 15, 2017, at 5:45 PM, Jeff Jirsa  wrote:
>
> In lieu of a weekly wrap-up, here's a pre-Thanksgiving call for help.
>
> If you haven't been paying attention to JIRA, you likely didn't notice
>>> that
> Josh went through and triage/categorized a bunch of issues by adding
> components, and Michael took the time to open a bunch of JIRAs for
>>> failing
> tests.
>
> How many is a bunch? Something like 35 or so just for tests currently
> failing on trunk.  If you're a regular contributor, you already know
>>> that
> dtests are flakey - it'd be great if a few of us can go through and fix
>>> a
> few. Even incremental improvements are improvements. Here's an easy
>>> search
> to find them:
>
> https://issues.apache.org/jira/secure/IssueNavigator.
>>> jspa?reset=true=project+%3D+CASSANDRA+AND+
>>> component+%3D+Testing+ORDER+BY+updated+DESC%2C+priority+
>>> DESC%2C+created+ASC=hide
>
> If you're a new contributor, fixing tests is often a good way to learn a
> new part of the codebase. Many of these are dtests, which live in a
> different repo ( https://github.com/apache/cassandra-dtest ) and are in
> python, but have no fear, the repo has instructions for setting up and
> running dtests(
> https://github.com/apache/cassandra-dtest/blob/master/INSTALL.md )
>
> Normal contribution workflow applies: self-assign the ticket if you
>>> want to
> work on it, click on 'start progress' to indicate that you're working on
> it, mark it 'patch available' when you've uploaded code to be reviewed
>>> (in
> a github branch, or as a standalone patch file attached to the JIRA). If
> you have questions, feel free to email the dev list (that's what it's
>>> here
> for).
>
> Many thanks will be given,
> - Jeff
>>>
>>>
>>
> 
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
> 


-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

Re: Flakey Dtests

[Josh McKenzie]

>
> I fixed one CDC uTest, please review:https://issues.apache.
> org/jira/browse/CASSANDRA-14066

Thanks for taking that Jay! I ended up buried on things last week and am
OOO this week so won't be able to get to these until next week at the
earliest.

Also, it just came up that we somehow neglected to merge the PR for the
12148 dtest changes on master, so I'm sure that's playing in to the
failures there (failed hand-off from one author to another on the PR led to
this unfortunately).

On Mon, Nov 27, 2017 at 5:55 PM, Jaydeep Chovatia <
chovatia.jayd...@gmail.com> wrote:

> This is useful info, Thanks!
>
> Jaydeep
>
> On Mon, Nov 27, 2017 at 2:43 PM, Michael Kjellman <
> mkjell...@internalcircle.com> wrote:
>
> > Complicated question unfortunately — and something we’re actively working
> > on improving:
> >
> > Cassci is no longer being offered/run by Datastax and so we've need to
> > come up with a new solution, and what that ultimately is is still a WIP —
> > it’s loss was very huge obviously and a testament to the awesome resource
> > and effort that was put into providing it to the community for all those
> > years.
> >
> >  - Short Term/Current: Tests (both dtests and unit tests) are being run
> > via the ASF Jenkins (https://builds.apache.org) - but that solution
> isn’t
> > hugely helpful as it’s resource constrained.
> >  - Short-Medium Term: we hope to get a fully baked CircleCI solution to
> > get reliable fast test runs.
> >  - Long Term: Actively being discussed but I’m optimistic that we can get
> > something awesome for the project with some stable combination of
> CircleCI
> > + ASF Jenkins, and once we do I’m sure this will change any long term
> plans.
> >
> > For Unit Tests (a.k.a the Java ones in tree - https://github.com/apache/
> > cassandra/tree/trunk/test/unit/org/apache/cassandra):
> > Take a look at https://builds.apache.org/view/A-D/view/Cassandra/job/
> > Cassandra-trunk-test/… looks like the last successful job to finish was
> > #389. (https://builds.apache.org/view/A-D/view/Cassandra/job/
> > Cassandra-trunk-test/389/testReport/). There are currently a total of 6
> > tests  (all from CompressedInputStreamTest) failing on trunk via ASF
> > Jenkins. These specific test failures are environmental. The only *unit*
> > test on trunk that I currently know to be flaky is
> > org.apache.cassandra.cql3.ViewTest. testRegularColumnTimestampUpdates
> > (tracked as https://issues.apache.org/jira/browse/CASSANDRA-14054)
> >
> > For Distributed Tests (DTests) (a.k.a the Python ones -
> > https://github.com/apache/cassandra-dtest):
> > The situation is a great deal more complicated due to the length of time
> > and number of resources executing all of the dtests take (and executing
> the
> > tests across the various configurations)...
> >
> > There are 4 dtest jobs on ASF Jenkins for trunk:
> > https://builds.apache.org/view/A-D/view/Cassandra/job/
> > Cassandra-trunk-dtest/
> > https://builds.apache.org/view/A-D/view/Cassandra/job/
> > Cassandra-trunk-dtest-large/
> > https://builds.apache.org/view/A-D/view/Cassandra/job/
> > Cassandra-trunk-dtest-novnode/
> > https://builds.apache.org/view/A-D/view/Cassandra/job/
> > Cassandra-trunk-dtest-offheap/
> >
> > It looks like you’ll need to go back to run #353 (
> > https://builds.apache.org/view/A-D/view/Cassandra/job/
> > Cassandra-trunk-dtest/353/testReport/) to see the test results as the
> > last 2 jobs that were triggered failed to execute. Depending on the
> > environment variables set tests are executed or skipped — so you’ll see
> > different tests being run on the no-vnode job/off-heap job/regular dtest
> > job (or some tests might be run multiple times)
> >
> >
> > More recently we’ve been woking on getting CircleCI running. Some sample
> > runs from my personal fork can be seen at https://circleci.com/gh/
> > mkjellman/cassandra/tree/trunk_circle. I’m personally using a paid
> > account to get more CircleCI resources (with 100 containers we can
> actually
> > build the project, run all of the unit tests, and run all of the dtests
> in
> > roughly 28 minutes!). I’m actively working to determine out exactly can
> > (and cannot) be executed reliably, routinely, and easily by anyone with
> > just a simple free CircleCI account.
> >
> > I’m also working on getting scheduled CircleCI daily runs setup against
> > trunk/3.0 — more on both of those when we’ve got that story fully baked..
> > Hope this answers your question! There are quite a few dtests currently
> > failing and as Jeff mentioned I’ve created JIRAs for a lot of them
> already
> > so any help (no matter how trivial or annoying it might be or seem) to
> get
> > everything green again.
> >
> > best,
> > kjellman
> >
> >
> > On Nov 27, 2017, at 1:54 PM, Jaydeep Chovatia <
> chovatia.jayd...@gmail.com<
> > mailto:chovatia.jayd...@gmail.com>> wrote:
> >
> > Is there a way to check which tests are failing in trunk currently?
> > Previously this URL  was 

Re: Flakey Dtests

[Jaydeep Chovatia]

This is useful info, Thanks!

Jaydeep

On Mon, Nov 27, 2017 at 2:43 PM, Michael Kjellman <
mkjell...@internalcircle.com> wrote:

> Complicated question unfortunately — and something we’re actively working
> on improving:
>
> Cassci is no longer being offered/run by Datastax and so we've need to
> come up with a new solution, and what that ultimately is is still a WIP —
> it’s loss was very huge obviously and a testament to the awesome resource
> and effort that was put into providing it to the community for all those
> years.
>
>  - Short Term/Current: Tests (both dtests and unit tests) are being run
> via the ASF Jenkins (https://builds.apache.org) - but that solution isn’t
> hugely helpful as it’s resource constrained.
>  - Short-Medium Term: we hope to get a fully baked CircleCI solution to
> get reliable fast test runs.
>  - Long Term: Actively being discussed but I’m optimistic that we can get
> something awesome for the project with some stable combination of CircleCI
> + ASF Jenkins, and once we do I’m sure this will change any long term plans.
>
> For Unit Tests (a.k.a the Java ones in tree - https://github.com/apache/
> cassandra/tree/trunk/test/unit/org/apache/cassandra):
> Take a look at https://builds.apache.org/view/A-D/view/Cassandra/job/
> Cassandra-trunk-test/… looks like the last successful job to finish was
> #389. (https://builds.apache.org/view/A-D/view/Cassandra/job/
> Cassandra-trunk-test/389/testReport/). There are currently a total of 6
> tests  (all from CompressedInputStreamTest) failing on trunk via ASF
> Jenkins. These specific test failures are environmental. The only *unit*
> test on trunk that I currently know to be flaky is
> org.apache.cassandra.cql3.ViewTest. testRegularColumnTimestampUpdates
> (tracked as https://issues.apache.org/jira/browse/CASSANDRA-14054)
>
> For Distributed Tests (DTests) (a.k.a the Python ones -
> https://github.com/apache/cassandra-dtest):
> The situation is a great deal more complicated due to the length of time
> and number of resources executing all of the dtests take (and executing the
> tests across the various configurations)...
>
> There are 4 dtest jobs on ASF Jenkins for trunk:
> https://builds.apache.org/view/A-D/view/Cassandra/job/
> Cassandra-trunk-dtest/
> https://builds.apache.org/view/A-D/view/Cassandra/job/
> Cassandra-trunk-dtest-large/
> https://builds.apache.org/view/A-D/view/Cassandra/job/
> Cassandra-trunk-dtest-novnode/
> https://builds.apache.org/view/A-D/view/Cassandra/job/
> Cassandra-trunk-dtest-offheap/
>
> It looks like you’ll need to go back to run #353 (
> https://builds.apache.org/view/A-D/view/Cassandra/job/
> Cassandra-trunk-dtest/353/testReport/) to see the test results as the
> last 2 jobs that were triggered failed to execute. Depending on the
> environment variables set tests are executed or skipped — so you’ll see
> different tests being run on the no-vnode job/off-heap job/regular dtest
> job (or some tests might be run multiple times)
>
>
> More recently we’ve been woking on getting CircleCI running. Some sample
> runs from my personal fork can be seen at https://circleci.com/gh/
> mkjellman/cassandra/tree/trunk_circle. I’m personally using a paid
> account to get more CircleCI resources (with 100 containers we can actually
> build the project, run all of the unit tests, and run all of the dtests in
> roughly 28 minutes!). I’m actively working to determine out exactly can
> (and cannot) be executed reliably, routinely, and easily by anyone with
> just a simple free CircleCI account.
>
> I’m also working on getting scheduled CircleCI daily runs setup against
> trunk/3.0 — more on both of those when we’ve got that story fully baked..
> Hope this answers your question! There are quite a few dtests currently
> failing and as Jeff mentioned I’ve created JIRAs for a lot of them already
> so any help (no matter how trivial or annoying it might be or seem) to get
> everything green again.
>
> best,
> kjellman
>
>
> On Nov 27, 2017, at 1:54 PM, Jaydeep Chovatia  mailto:chovatia.jayd...@gmail.com>> wrote:
>
> Is there a way to check which tests are failing in trunk currently?
> Previously this URL  was giving such results
> but is no longer working.
>
> Jaydeep
>
> On Wed, Nov 15, 2017 at 5:44 PM, Jeff Jirsa  a...@gmail.com>> wrote:
>
> In lieu of a weekly wrap-up, here's a pre-Thanksgiving call for help.
>
> If you haven't been paying attention to JIRA, you likely didn't notice that
> Josh went through and triage/categorized a bunch of issues by adding
> components, and Michael took the time to open a bunch of JIRAs for failing
> tests.
>
> How many is a bunch? Something like 35 or so just for tests currently
> failing on trunk.  If you're a regular contributor, you already know that
> dtests are flakey - it'd be great if a few of us can go through and fix a
> few. Even incremental improvements are improvements. Here's an easy 

Re: Flakey Dtests

[Michael Kjellman]

and just to make it super clear how awesome this is: currently the dtests when 
executed via ASF Jenkins (if they actually run successfully at all) take 
roughly 15+ hours to execute. Being able to run *everything* reliably and 
stably in 28 minutes is obviously many many orders of magnitude better.

best,
kjellman

On Nov 27, 2017, at 2:43 PM, Michael Kjellman 
> wrote:

(with 100 containers we can actually build the project, run all of the unit 
tests, and run all of the dtests in roughly 28 minutes!).

Re: Flakey Dtests

[Michael Kjellman]

Complicated question unfortunately — and something we’re actively working on 
improving:

Cassci is no longer being offered/run by Datastax and so we've need to come up 
with a new solution, and what that ultimately is is still a WIP — it’s loss was 
very huge obviously and a testament to the awesome resource and effort that was 
put into providing it to the community for all those years.

 - Short Term/Current: Tests (both dtests and unit tests) are being run via the 
ASF Jenkins (https://builds.apache.org) - but that solution isn’t hugely 
helpful as it’s resource constrained.
 - Short-Medium Term: we hope to get a fully baked CircleCI solution to get 
reliable fast test runs.
 - Long Term: Actively being discussed but I’m optimistic that we can get 
something awesome for the project with some stable combination of CircleCI + 
ASF Jenkins, and once we do I’m sure this will change any long term plans.

For Unit Tests (a.k.a the Java ones in tree - 
https://github.com/apache/cassandra/tree/trunk/test/unit/org/apache/cassandra):
Take a look at 
https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-trunk-test/… 
looks like the last successful job to finish was #389. 
(https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-trunk-test/389/testReport/).
 There are currently a total of 6 tests  (all from CompressedInputStreamTest) 
failing on trunk via ASF Jenkins. These specific test failures are 
environmental. The only *unit* test on trunk that I currently know to be flaky 
is org.apache.cassandra.cql3.ViewTest. testRegularColumnTimestampUpdates 
(tracked as https://issues.apache.org/jira/browse/CASSANDRA-14054)

For Distributed Tests (DTests) (a.k.a the Python ones - 
https://github.com/apache/cassandra-dtest):
The situation is a great deal more complicated due to the length of time and 
number of resources executing all of the dtests take (and executing the tests 
across the various configurations)...

There are 4 dtest jobs on ASF Jenkins for trunk:
https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-trunk-dtest/
https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-trunk-dtest-large/
https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-trunk-dtest-novnode/
https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-trunk-dtest-offheap/

It looks like you’ll need to go back to run #353 
(https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-trunk-dtest/353/testReport/)
 to see the test results as the last 2 jobs that were triggered failed to 
execute. Depending on the environment variables set tests are executed or 
skipped — so you’ll see different tests being run on the no-vnode job/off-heap 
job/regular dtest job (or some tests might be run multiple times)


More recently we’ve been woking on getting CircleCI running. Some sample runs 
from my personal fork can be seen at 
https://circleci.com/gh/mkjellman/cassandra/tree/trunk_circle. I’m personally 
using a paid account to get more CircleCI resources (with 100 containers we can 
actually build the project, run all of the unit tests, and run all of the 
dtests in roughly 28 minutes!). I’m actively working to determine out exactly 
can (and cannot) be executed reliably, routinely, and easily by anyone with 
just a simple free CircleCI account.

I’m also working on getting scheduled CircleCI daily runs setup against 
trunk/3.0 — more on both of those when we’ve got that story fully baked.. Hope 
this answers your question! There are quite a few dtests currently failing and 
as Jeff mentioned I’ve created JIRAs for a lot of them already so any help (no 
matter how trivial or annoying it might be or seem) to get everything green 
again.

best,
kjellman


On Nov 27, 2017, at 1:54 PM, Jaydeep Chovatia 
> wrote:

Is there a way to check which tests are failing in trunk currently?
Previously this URL  was giving such results
but is no longer working.

Jaydeep

On Wed, Nov 15, 2017 at 5:44 PM, Jeff Jirsa 
> wrote:

In lieu of a weekly wrap-up, here's a pre-Thanksgiving call for help.

If you haven't been paying attention to JIRA, you likely didn't notice that
Josh went through and triage/categorized a bunch of issues by adding
components, and Michael took the time to open a bunch of JIRAs for failing
tests.

How many is a bunch? Something like 35 or so just for tests currently
failing on trunk.  If you're a regular contributor, you already know that
dtests are flakey - it'd be great if a few of us can go through and fix a
few. Even incremental improvements are improvements. Here's an easy search
to find them:

https://issues.apache.org/jira/secure/IssueNavigator.
jspa?reset=true=project+%3D+CASSANDRA+AND+
component+%3D+Testing+ORDER+BY+updated+DESC%2C+priority+
DESC%2C+created+ASC=hide

If you're a new contributor, fixing tests is often a good way to learn a
new 

Re: Flakey Dtests

[Jaydeep Chovatia]

Is there a way to check which tests are failing in trunk currently?
Previously this URL  was giving such results
but is no longer working.

Jaydeep

On Wed, Nov 15, 2017 at 5:44 PM, Jeff Jirsa  wrote:

> In lieu of a weekly wrap-up, here's a pre-Thanksgiving call for help.
>
> If you haven't been paying attention to JIRA, you likely didn't notice that
> Josh went through and triage/categorized a bunch of issues by adding
> components, and Michael took the time to open a bunch of JIRAs for failing
> tests.
>
> How many is a bunch? Something like 35 or so just for tests currently
> failing on trunk.  If you're a regular contributor, you already know that
> dtests are flakey - it'd be great if a few of us can go through and fix a
> few. Even incremental improvements are improvements. Here's an easy search
> to find them:
>
> https://issues.apache.org/jira/secure/IssueNavigator.
> jspa?reset=true=project+%3D+CASSANDRA+AND+
> component+%3D+Testing+ORDER+BY+updated+DESC%2C+priority+
> DESC%2C+created+ASC=hide
>
> If you're a new contributor, fixing tests is often a good way to learn a
> new part of the codebase. Many of these are dtests, which live in a
> different repo ( https://github.com/apache/cassandra-dtest ) and are in
> python, but have no fear, the repo has instructions for setting up and
> running dtests(
> https://github.com/apache/cassandra-dtest/blob/master/INSTALL.md )
>
> Normal contribution workflow applies: self-assign the ticket if you want to
> work on it, click on 'start progress' to indicate that you're working on
> it, mark it 'patch available' when you've uploaded code to be reviewed (in
> a github branch, or as a standalone patch file attached to the JIRA). If
> you have questions, feel free to email the dev list (that's what it's here
> for).
>
> Many thanks will be given,
> - Jeff
>

Re: Flakey Dtests

[Michael Kjellman]

Hey Jay:

Thanks!! I just took a quick look at the JIRA and noticed that there is a 
“test-cdc” ant target? So, does that mean CDC get’s no testing with ant test? 
Do you know any of the history around this?

> On Nov 27, 2017, at 9:44 AM, Jay Zhuang  wrote:
> 
> I fixed one CDC uTest, please 
> review:https://issues.apache.org/jira/browse/CASSANDRA-14066
> 
> 
>On Friday, November 17, 2017 6:34 AM, Josh McKenzie  
> wrote:
> 
> 
>> 
>> Do we have any volunteers to fix the broken Materialized Views and CDC
>> DTests?
> 
> I'll try to take a look at the CDC tests next week; looks like one of the
> base unit tests is failing as well.
> 
> On Fri, Nov 17, 2017 at 12:09 AM, Michael Kjellman <
> mkjell...@internalcircle.com> wrote:
> 
>> Quick update re: dtests and off-heap memtables:
>> 
>> I’ve filed CASSANDRA-14056 (Many dtests fail with ConfigurationException:
>> offheap_objects are not available in 3.0 when OFFHEAP_MEMTABLES=“true”)
>> 
>> Looks like we’re gonna need to do some work to test this configuration and
>> right now it’s pretty broken...
>> 
>> Do we have any volunteers to fix the broken Materialized Views and CDC
>> DTests?
>> 
>> best,
>> kjellman
>> 
>> 
>>> On Nov 15, 2017, at 5:59 PM, Michael Kjellman <
>> mkjell...@internalcircle.com> wrote:
>>> 
>>> yes - true- some are flaky, but almost all of the ones i filed fail 100%
>> () of the time. i look forward to triaging just the remaining flaky ones
>> (hopefully - without powers combined - by the end of this month!!)
>>> 
>>> appreciate everyone’s help - no matter how small... i already personally
>> did a few “fun” random-python-class-is-missing-return-after-method stuff.
>>> 
>>> we’ve wanted this for a while and now is our time to actually execute
>> and make good on our previous dev list promises.
>>> 
>>> best,
>>> kjellman
>>> 
 On Nov 15, 2017, at 5:45 PM, Jeff Jirsa  wrote:
 
 In lieu of a weekly wrap-up, here's a pre-Thanksgiving call for help.
 
 If you haven't been paying attention to JIRA, you likely didn't notice
>> that
 Josh went through and triage/categorized a bunch of issues by adding
 components, and Michael took the time to open a bunch of JIRAs for
>> failing
 tests.
 
 How many is a bunch? Something like 35 or so just for tests currently
 failing on trunk.  If you're a regular contributor, you already know
>> that
 dtests are flakey - it'd be great if a few of us can go through and fix
>> a
 few. Even incremental improvements are improvements. Here's an easy
>> search
 to find them:
 
 https://issues.apache.org/jira/secure/IssueNavigator.
>> jspa?reset=true=project+%3D+CASSANDRA+AND+
>> component+%3D+Testing+ORDER+BY+updated+DESC%2C+priority+
>> DESC%2C+created+ASC=hide
 
 If you're a new contributor, fixing tests is often a good way to learn a
 new part of the codebase. Many of these are dtests, which live in a
 different repo ( https://github.com/apache/cassandra-dtest ) and are in
 python, but have no fear, the repo has instructions for setting up and
 running dtests(
 https://github.com/apache/cassandra-dtest/blob/master/INSTALL.md )
 
 Normal contribution workflow applies: self-assign the ticket if you
>> want to
 work on it, click on 'start progress' to indicate that you're working on
 it, mark it 'patch available' when you've uploaded code to be reviewed
>> (in
 a github branch, or as a standalone patch file attached to the JIRA). If
 you have questions, feel free to email the dev list (that's what it's
>> here
 for).
 
 Many thanks will be given,
 - Jeff
>> 
>> 
> 

Re: Flakey Dtests

[Jay Zhuang]

I fixed one CDC uTest, please 
review:https://issues.apache.org/jira/browse/CASSANDRA-14066


On Friday, November 17, 2017 6:34 AM, Josh McKenzie  
wrote:
 

 >
> Do we have any volunteers to fix the broken Materialized Views and CDC
> DTests?

I'll try to take a look at the CDC tests next week; looks like one of the
base unit tests is failing as well.

On Fri, Nov 17, 2017 at 12:09 AM, Michael Kjellman <
mkjell...@internalcircle.com> wrote:

> Quick update re: dtests and off-heap memtables:
>
> I’ve filed CASSANDRA-14056 (Many dtests fail with ConfigurationException:
> offheap_objects are not available in 3.0 when OFFHEAP_MEMTABLES=“true”)
>
> Looks like we’re gonna need to do some work to test this configuration and
> right now it’s pretty broken...
>
> Do we have any volunteers to fix the broken Materialized Views and CDC
> DTests?
>
> best,
> kjellman
>
>
> > On Nov 15, 2017, at 5:59 PM, Michael Kjellman <
> mkjell...@internalcircle.com> wrote:
> >
> > yes - true- some are flaky, but almost all of the ones i filed fail 100%
> () of the time. i look forward to triaging just the remaining flaky ones
> (hopefully - without powers combined - by the end of this month!!)
> >
> > appreciate everyone’s help - no matter how small... i already personally
> did a few “fun” random-python-class-is-missing-return-after-method stuff.
> >
> > we’ve wanted this for a while and now is our time to actually execute
> and make good on our previous dev list promises.
> >
> > best,
> > kjellman
> >
> >> On Nov 15, 2017, at 5:45 PM, Jeff Jirsa  wrote:
> >>
> >> In lieu of a weekly wrap-up, here's a pre-Thanksgiving call for help.
> >>
> >> If you haven't been paying attention to JIRA, you likely didn't notice
> that
> >> Josh went through and triage/categorized a bunch of issues by adding
> >> components, and Michael took the time to open a bunch of JIRAs for
> failing
> >> tests.
> >>
> >> How many is a bunch? Something like 35 or so just for tests currently
> >> failing on trunk.  If you're a regular contributor, you already know
> that
> >> dtests are flakey - it'd be great if a few of us can go through and fix
> a
> >> few. Even incremental improvements are improvements. Here's an easy
> search
> >> to find them:
> >>
> >> https://issues.apache.org/jira/secure/IssueNavigator.
> jspa?reset=true=project+%3D+CASSANDRA+AND+
> component+%3D+Testing+ORDER+BY+updated+DESC%2C+priority+
> DESC%2C+created+ASC=hide
> >>
> >> If you're a new contributor, fixing tests is often a good way to learn a
> >> new part of the codebase. Many of these are dtests, which live in a
> >> different repo ( https://github.com/apache/cassandra-dtest ) and are in
> >> python, but have no fear, the repo has instructions for setting up and
> >> running dtests(
> >> https://github.com/apache/cassandra-dtest/blob/master/INSTALL.md )
> >>
> >> Normal contribution workflow applies: self-assign the ticket if you
> want to
> >> work on it, click on 'start progress' to indicate that you're working on
> >> it, mark it 'patch available' when you've uploaded code to be reviewed
> (in
> >> a github branch, or as a standalone patch file attached to the JIRA). If
> >> you have questions, feel free to email the dev list (that's what it's
> here
> >> for).
> >>
> >> Many thanks will be given,
> >> - Jeff
>
>

   

Re: CCM dependency in dtests

[Michael Kjellman]

thanks for driving this Stefan this is definitely an issue that I recently 
saw too trying to get all the dtests passing. having logic you need to fix in 3 
repos isn’t ideal at all. 

> On Nov 27, 2017, at 4:05 AM, Stefan Podkowinski  wrote:
> 
> Just wanted to bring a recent discussion about how to use ccm from
> dtests to your attention:
> https://github.com/apache/cassandra-dtest/pull/13
> 
> Basically the idea is to not depend on a released ccm artifact, but to
> use a dedicated git branch in the ccm repo instead for executing dtests.
> Motivation and details can be found in the PR, please feel free to comment.
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
> 

-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

CCM dependency in dtests

[Stefan Podkowinski]

Just wanted to bring a recent discussion about how to use ccm from
dtests to your attention:
https://github.com/apache/cassandra-dtest/pull/13

Basically the idea is to not depend on a released ccm artifact, but to
use a dedicated git branch in the ccm repo instead for executing dtests.
Motivation and details can be found in the PR, please feel free to comment.

-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org