This is useful info, Thanks! Jaydeep
On Mon, Nov 27, 2017 at 2:43 PM, Michael Kjellman < mkjell...@internalcircle.com> wrote: > Complicated question unfortunately — and something we’re actively working > on improving: > > Cassci is no longer being offered/run by Datastax and so we've need to > come up with a new solution, and what that ultimately is is still a WIP — > it’s loss was very huge obviously and a testament to the awesome resource > and effort that was put into providing it to the community for all those > years. > > - Short Term/Current: Tests (both dtests and unit tests) are being run > via the ASF Jenkins (https://builds.apache.org) - but that solution isn’t > hugely helpful as it’s resource constrained. > - Short-Medium Term: we hope to get a fully baked CircleCI solution to > get reliable fast test runs. > - Long Term: Actively being discussed but I’m optimistic that we can get > something awesome for the project with some stable combination of CircleCI > + ASF Jenkins, and once we do I’m sure this will change any long term plans. > > For Unit Tests (a.k.a the Java ones in tree - https://github.com/apache/ > cassandra/tree/trunk/test/unit/org/apache/cassandra): > Take a look at https://builds.apache.org/view/A-D/view/Cassandra/job/ > Cassandra-trunk-test/… looks like the last successful job to finish was > #389. (https://builds.apache.org/view/A-D/view/Cassandra/job/ > Cassandra-trunk-test/389/testReport/). There are currently a total of 6 > tests (all from CompressedInputStreamTest) failing on trunk via ASF > Jenkins. These specific test failures are environmental. The only *unit* > test on trunk that I currently know to be flaky is > org.apache.cassandra.cql3.ViewTest. testRegularColumnTimestampUpdates > (tracked as https://issues.apache.org/jira/browse/CASSANDRA-14054) > > For Distributed Tests (DTests) (a.k.a the Python ones - > https://github.com/apache/cassandra-dtest): > The situation is a great deal more complicated due to the length of time > and number of resources executing all of the dtests take (and executing the > tests across the various configurations)... > > There are 4 dtest jobs on ASF Jenkins for trunk: > https://builds.apache.org/view/A-D/view/Cassandra/job/ > Cassandra-trunk-dtest/ > https://builds.apache.org/view/A-D/view/Cassandra/job/ > Cassandra-trunk-dtest-large/ > https://builds.apache.org/view/A-D/view/Cassandra/job/ > Cassandra-trunk-dtest-novnode/ > https://builds.apache.org/view/A-D/view/Cassandra/job/ > Cassandra-trunk-dtest-offheap/ > > It looks like you’ll need to go back to run #353 ( > https://builds.apache.org/view/A-D/view/Cassandra/job/ > Cassandra-trunk-dtest/353/testReport/) to see the test results as the > last 2 jobs that were triggered failed to execute. Depending on the > environment variables set tests are executed or skipped — so you’ll see > different tests being run on the no-vnode job/off-heap job/regular dtest > job (or some tests might be run multiple times) > > > More recently we’ve been woking on getting CircleCI running. Some sample > runs from my personal fork can be seen at https://circleci.com/gh/ > mkjellman/cassandra/tree/trunk_circle. I’m personally using a paid > account to get more CircleCI resources (with 100 containers we can actually > build the project, run all of the unit tests, and run all of the dtests in > roughly 28 minutes!). I’m actively working to determine out exactly can > (and cannot) be executed reliably, routinely, and easily by anyone with > just a simple free CircleCI account. > > I’m also working on getting scheduled CircleCI daily runs setup against > trunk/3.0 — more on both of those when we’ve got that story fully baked.. > Hope this answers your question! There are quite a few dtests currently > failing and as Jeff mentioned I’ve created JIRAs for a lot of them already > so any help (no matter how trivial or annoying it might be or seem) to get > everything green again. > > best, > kjellman > > > On Nov 27, 2017, at 1:54 PM, Jaydeep Chovatia <chovatia.jayd...@gmail.com< > mailto:chovatia.jayd...@gmail.com>> wrote: > > Is there a way to check which tests are failing in trunk currently? > Previously this URL <http://cassci.datastax.com/> was giving such results > but is no longer working. > > Jaydeep > > On Wed, Nov 15, 2017 at 5:44 PM, Jeff Jirsa <jji...@gmail.com<mailto:jjirs > a...@gmail.com>> wrote: > > In lieu of a weekly wrap-up, here's a pre-Thanksgiving call for help. > > If you haven't been paying attention to JIRA, you likely didn't notice that > Josh went through and triage/categorized a bunch of issues by adding > components, and Michael took the time to open a bunch of JIRAs for failing > tests. > > How many is a bunch? Something like 35 or so just for tests currently > failing on trunk. If you're a regular contributor, you already know that > dtests are flakey - it'd be great if a few of us can go through and fix a > few. Even incremental improvements are improvements. Here's an easy search > to find them: > > https://issues.apache.org/jira/secure/IssueNavigator. > jspa?reset=true&jqlQuery=project+%3D+CASSANDRA+AND+ > component+%3D+Testing+ORDER+BY+updated+DESC%2C+priority+ > DESC%2C+created+ASC&mode=hide > > If you're a new contributor, fixing tests is often a good way to learn a > new part of the codebase. Many of these are dtests, which live in a > different repo ( https://github.com/apache/cassandra-dtest ) and are in > python, but have no fear, the repo has instructions for setting up and > running dtests( > https://github.com/apache/cassandra-dtest/blob/master/INSTALL.md ) > > Normal contribution workflow applies: self-assign the ticket if you want to > work on it, click on 'start progress' to indicate that you're working on > it, mark it 'patch available' when you've uploaded code to be reviewed (in > a github branch, or as a standalone patch file attached to the JIRA). If > you have questions, feel free to email the dev list (that's what it's here > for). > > Many thanks will be given, > - Jeff > > >
