We're using 'mvn test' now so all the patches should complete in about 10 minutes rather than closer to an hour. The integration test seems to be working fine, I'll check to make sure it's triggered on the next push to master.
On Thu, Jan 14, 2016 at 3:07 PM, Ian Maxon <[email protected]> wrote: > We can definitely have a job that will take a commit as a parameter that > is run simply on demand rather than via any sort of trigger or timer. I > have had a job for this actually at various points but I don't know if the > ones I have right now are pointed at the correct repositories. > > > For the time being, to start impelementing this since there seems to be no > objection, I have set up a job to run on each commit. If it looks like it's > working good after the next commit I will change AsterixDB to use 'mvn > test' instead of 'mvn verify' for automatic verification. > > -Ian > > On Wed, Jan 13, 2016 at 4:27 PM, Till Westmann <[email protected]> wrote: > >> I think that to find out quickly which change introduces the issue, it >> would be nice if there was an easy way to have Jenkins run the “long suite” >> for a specific commit. That way we could identify the problematic commit in >> the standardized environment and without relying on developer setup. >> >> >> On 13 Jan 2016, at 16:20, Mike Carey wrote: >> >> Agreed. I'd vote for making sure that the broken daily test report be >>> accompanied by a list of the changes that occurred since the last >>> successful run - and that it be the responsibility of all folks with >>> changes on that list to gang up and ensure that the problem is identified >>> and resolved in a timely fashion. We should probably spell out what that >>> looks like, but it seems eminently reasonable.... >>> >>> On 1/13/16 4:03 PM, Chris Hillery wrote: >>> >>>> +1 for Ian's initial proposal. This kind of pre-commit validation is >>>> always >>>> a balancing act between being comprehensive and being timely, and I >>>> think >>>> sticking with the distinction between "mvn test" and "mvn verify" is a >>>> clean and simple way to divide the tests. >>>> >>>> Given that pushing the Gerrit head to ASF is manual anyway, it wouldn't >>>> be >>>> TOO much headache to also enact Ildar's suggestion - basically, whoever >>>> does the ASF push would need to wait to ensure each commit passes the >>>> corresponding daily tests. However, IMHO at least this is probably not >>>> necessary. If a bug is uncovered in the daily tests, we'll need to push >>>> an >>>> additional commit to fix it, and that additional commit (plus the >>>> original >>>> buggy commit) will still end up on ASF anyway. And I worry a bit about >>>> anything that would cause the ASF Github mirror to stay out of sync with >>>> Gerrit for very long. >>>> >>>> Ceej >>>> aka Chris Hillery >>>> >>>> On Wed, Jan 13, 2016 at 3:42 PM, Ildar Absalyamov < >>>> [email protected]> wrote: >>>> >>>> It would be really nice to break Jenkins test build into two stage >>>>> process: ‘mvn test’ on each patch set submission and ‘mvn verify’ >>>>> after the >>>>> patch received +2 and got merged into Gerrit, BUT before it got merged >>>>> to >>>>> ASF repo. Although I am not sure what should be done in a case when the >>>>> latter build does not pass, since it’s already merged into Gerrit and >>>>> we >>>>> cannot revert that. >>>>> >>>>> On Jan 13, 2016, at 15:34, Ian Maxon <[email protected]> wrote: >>>>>> >>>>>> Hi all, >>>>>> I'd like to field the idea of changing the criteria we use to allow >>>>>> >>>>> Jenkins >>>>> >>>>>> to give +1 verified on patch sets. Right now, it runs 'mvn verify', >>>>>> which >>>>>> runs darn near about every test we have. While this was somewhat OK in >>>>>> >>>>> the >>>>> >>>>>> past, I think now, and going forward, it takes a long time, and it >>>>>> will >>>>>> only take longer as we make the integration tests more thorough. >>>>>> >>>>>> Additionally, I'd like to expand and re-enable the automated >>>>>> >>>>> vagrant-based >>>>> >>>>>> cluster integration tests, and unfortunately there isn't a simple way >>>>>> to >>>>>> make these work inside a docker container. I can get it to work by >>>>>> hand, >>>>>> but to make it automated I would have to fork or submit a patch to the >>>>>> Jenkins Docker cloud plugin, and it might not be a small or >>>>>> particularly >>>>>> clean/easy patch either. >>>>>> >>>>>> I'd like to change the goal that's run in the automatic tests on every >>>>>> commit to 'mvn test' rather than 'mvn verify'. This will still run >>>>>> all of >>>>>> the ExecutionTest(s), and the optimizer tests, and so on. It will stop >>>>>> >>>>> the >>>>> >>>>>> recovery tests and asterix installer tests from being run every time, >>>>>> however. >>>>>> >>>>>> However these tests are still certainly valuable, so I'd run them on a >>>>>> timer, perhaps daily, so that we would still catch bugs that these >>>>>> tests >>>>>> reveal rather promptly. Due to the fact we only commit a single digit >>>>>> number of times to master each day generally, the granularity of which >>>>>> commit will have introduced a bug will hopefully be rather low. >>>>>> >>>>>> Thoughts/comments? >>>>>> >>>>>> Thanks, >>>>>> -Ian >>>>>> >>>>> Best regards, >>>>> Ildar >>>>> >>>>> >>>>> >
