We can definitely have a job that will take a commit as a parameter that is run simply on demand rather than via any sort of trigger or timer. I have had a job for this actually at various points but I don't know if the ones I have right now are pointed at the correct repositories.
For the time being, to start impelementing this since there seems to be no objection, I have set up a job to run on each commit. If it looks like it's working good after the next commit I will change AsterixDB to use 'mvn test' instead of 'mvn verify' for automatic verification. -Ian On Wed, Jan 13, 2016 at 4:27 PM, Till Westmann <[email protected]> wrote: > I think that to find out quickly which change introduces the issue, it > would be nice if there was an easy way to have Jenkins run the “long suite” > for a specific commit. That way we could identify the problematic commit in > the standardized environment and without relying on developer setup. > > > On 13 Jan 2016, at 16:20, Mike Carey wrote: > > Agreed. I'd vote for making sure that the broken daily test report be >> accompanied by a list of the changes that occurred since the last >> successful run - and that it be the responsibility of all folks with >> changes on that list to gang up and ensure that the problem is identified >> and resolved in a timely fashion. We should probably spell out what that >> looks like, but it seems eminently reasonable.... >> >> On 1/13/16 4:03 PM, Chris Hillery wrote: >> >>> +1 for Ian's initial proposal. This kind of pre-commit validation is >>> always >>> a balancing act between being comprehensive and being timely, and I think >>> sticking with the distinction between "mvn test" and "mvn verify" is a >>> clean and simple way to divide the tests. >>> >>> Given that pushing the Gerrit head to ASF is manual anyway, it wouldn't >>> be >>> TOO much headache to also enact Ildar's suggestion - basically, whoever >>> does the ASF push would need to wait to ensure each commit passes the >>> corresponding daily tests. However, IMHO at least this is probably not >>> necessary. If a bug is uncovered in the daily tests, we'll need to push >>> an >>> additional commit to fix it, and that additional commit (plus the >>> original >>> buggy commit) will still end up on ASF anyway. And I worry a bit about >>> anything that would cause the ASF Github mirror to stay out of sync with >>> Gerrit for very long. >>> >>> Ceej >>> aka Chris Hillery >>> >>> On Wed, Jan 13, 2016 at 3:42 PM, Ildar Absalyamov < >>> [email protected]> wrote: >>> >>> It would be really nice to break Jenkins test build into two stage >>>> process: ‘mvn test’ on each patch set submission and ‘mvn verify’ after >>>> the >>>> patch received +2 and got merged into Gerrit, BUT before it got merged >>>> to >>>> ASF repo. Although I am not sure what should be done in a case when the >>>> latter build does not pass, since it’s already merged into Gerrit and we >>>> cannot revert that. >>>> >>>> On Jan 13, 2016, at 15:34, Ian Maxon <[email protected]> wrote: >>>>> >>>>> Hi all, >>>>> I'd like to field the idea of changing the criteria we use to allow >>>>> >>>> Jenkins >>>> >>>>> to give +1 verified on patch sets. Right now, it runs 'mvn verify', >>>>> which >>>>> runs darn near about every test we have. While this was somewhat OK in >>>>> >>>> the >>>> >>>>> past, I think now, and going forward, it takes a long time, and it will >>>>> only take longer as we make the integration tests more thorough. >>>>> >>>>> Additionally, I'd like to expand and re-enable the automated >>>>> >>>> vagrant-based >>>> >>>>> cluster integration tests, and unfortunately there isn't a simple way >>>>> to >>>>> make these work inside a docker container. I can get it to work by >>>>> hand, >>>>> but to make it automated I would have to fork or submit a patch to the >>>>> Jenkins Docker cloud plugin, and it might not be a small or >>>>> particularly >>>>> clean/easy patch either. >>>>> >>>>> I'd like to change the goal that's run in the automatic tests on every >>>>> commit to 'mvn test' rather than 'mvn verify'. This will still run all >>>>> of >>>>> the ExecutionTest(s), and the optimizer tests, and so on. It will stop >>>>> >>>> the >>>> >>>>> recovery tests and asterix installer tests from being run every time, >>>>> however. >>>>> >>>>> However these tests are still certainly valuable, so I'd run them on a >>>>> timer, perhaps daily, so that we would still catch bugs that these >>>>> tests >>>>> reveal rather promptly. Due to the fact we only commit a single digit >>>>> number of times to master each day generally, the granularity of which >>>>> commit will have introduced a bug will hopefully be rather low. >>>>> >>>>> Thoughts/comments? >>>>> >>>>> Thanks, >>>>> -Ian >>>>> >>>> Best regards, >>>> Ildar >>>> >>>> >>>>
