Should also add I do have plans to release both plugins just haven't done the final touches to make them useable by anyone else yet
On Friday, July 24, 2015 at 9:48:54 AM UTC+9:30, David van Laatum wrote: > > https://github.com/davidvanlaatum/UnitTestDB is my code, I use it along > with another of my plugins > https://github.com/davidvanlaatum/DistributedTests. We use them together > to run the nearly 20,000 phpunit tests across a cluster of servers to take > the total runtime down from several hours to under an hour. > > The problem we had with the existing jenkins stuff was that it was hard to > split up the unit tests using the matrix plugin into equal runtime groups > and if one unit test fatal errored a bunch of unit tests just disappeared > from the list and sometimes no one noticed for quite some time. > > This lead me to write these two plugins. The distributed one runs a master > job that sets up a copy of the code does any db updates via liquibase etc > then runs a php script to generate a json file containing a list of tests, > sets up a apache virtual host for the unit tests to talk to, then starts a > configurable number of slave jobs that run the checkout/setup and start > grabbing the next group of tests off the stack and runs them. If there is > history around runtime of a group it sorts them such that groups that take > the longest run first. When each slave finishes it copies the junit.xml > files back to the master jobs workspace and runs the normal junit plugins > to add the results to jenkins then the UnitTestDB is called and adds info > to the db from the junit plugins data. > > The db tracks how long unit tests have been failing and who possibly > caused the failure with the option for people to claim or say it wasn't > them. One key difference is that if it doesn't see a unit test in a > particular run it doesn't remove the failure. This fixes the problem of a > unit test starts failing then another change introduces a fatal error that > causes the unit test not to run, the fatal error gets fixed and the > original failure now shows up as started happening in a new build. On > the UnitTestDB page where people claim etc there is also an option to say a > particular unit test no longer exists. > > I also have another jenkins job that runs several times a day to email > people that are currently blamed for a failure along with any unclaimed > failures. > > I have been considering bypassing the normal jenkins junit plugins and > send directly to the db as storing 20,000 unit tests per build in jenkins > makes it use a lot of memory and very sluggish. > > On Friday, July 24, 2015 at 12:30:42 AM UTC+9:30, Kanstantsin Shautsou > wrote: >> >> Hi, could you describe detailed how do you use it/architecture? >> >> On Wednesday, July 22, 2015 at 2:42:58 PM UTC+3, David van Laatum wrote: >>> >>> Sure. I am using it to store info about our nearly 20,000 unit tests >>> (per branch) in a db across the distributed cluster that runs them. Has >>> worked mostly well for ages but recently started failing on hibernate >>> trying to invoke the validation stuff and failing because it didn't have >>> everything it needed >>> >>> On Wednesday, July 22, 2015 at 8:41:51 PM UTC+9:30, Ullrich Hafner wrote: >>>> >>>> Are you using these plug-ins in a new plug-in? It would be interesting >>>> to hear about your experience… >>>> >>>> The plug-in has no active maintainer anymore, interested in taking over? >>>> >>>> Ulli >>>> >>>> Am 22.07.2015 um 02:33 schrieb David van Laatum <[email protected] >>>> >: >>>> >>>> I created https://github.com/jenkinsci/database-plugin/pull/1 last >>>> year and would now like to make some other changes. >>>> >>>> Can I have commit access to the database-plugin and the mysql and >>>> postgresql drivers? >>>> https://github.com/jenkinsci/database-postgresql-plugin >>>> https://github.com/jenkinsci/database-mysql-plugin >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Jenkins Developers" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/jenkinsci-dev/01a1f6a9-d0c0-4146-8f42-f5c5197bca62%40googlegroups.com >>>> >>>> <https://groups.google.com/d/msgid/jenkinsci-dev/01a1f6a9-d0c0-4146-8f42-f5c5197bca62%40googlegroups.com?utm_medium=email&utm_source=footer> >>>> . >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>>> >>>> -- You received this message because you are subscribed to the Google Groups "Jenkins Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/1235c91f-e7a4-4260-be41-1cd789fe306a%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
