Hello everyone, First of all I'd like to thank that Anna is willing to invest some efforts on making our CI system better, and sorry for my delayed answer ( however I do still hope it helps regardless the stage of the ongoing efforts )
I'd like to share the following thoughts here: - Although it would make sense to eliminate the four ( right now totally equal ) CI cycles and create only one, but regardless some "static noise" it doesn't cause any serious issues for our current commit flow. - Creating a precommit hook sounds like a great idea, I would encourage the community to move on that path. - However: According to my humblest opinion the biggest problem with the current CI is that it doesn't execute the so called " 3rd party tests" ( which is generally our DB integration test layer), and thus it provides only a limited safety belt for us ( and we've seen quite a few regression on this front in the past one year ). Although we do have solution for running those tests manually from command line, it's quite difficult to setup/test those things from a single desktop, thus cause serious difficulties in validating some changeset before commit. - Still an issue we have on this front is our build system/scripts/mechanism, which again could slow down the Dev+commit flow. Although we've started efforts on this front, the final solution was not fully delivered yet. As a conclusion of the elements above: Anna! Would you mind first focusing on the build scripts and the "3rd party" CI automation instead of eliminating the obsolete stuff? IMHO that would be a much better usage of your efforts and would provide a much bigger impact for the community. With my kindest regards, Attila On Mar 28, 2017 2:44 PM, "Erzsebet Szilagyi" <liz.szila...@cloudera.com> wrote: > Great ideas! > > I agree with Bogi and Szabolcs on the redundant test jobs. > > Would this pre-commit hook launch the same process as the current > post-commit hook, or would this do something different? > I think in the first case we could rework the post-commit check into the > pre-commit hook, in the latter I'm curious about what exactly this check > would add. > In general I support the idea: we have seen a number of problems that could > have been avoided, so this shall be a very useful change! > > Thank you, > Liz > > On Fri, Mar 24, 2017 at 9:39 AM, Szabolcs Vasas <va...@cloudera.com> > wrote: > > > Hi Anna, > > > > Removing the redundant test execution jobs sounds great, I think you can > go > > ahead with that. > > > > Regarding the pre-commit hook: what would be the purpose of it exactly? > > Would it execute the unit tests before the patch is committed? > > > > Regards, > > Szabolcs > > > > On Thu, Mar 23, 2017 at 4:03 PM, Anna Szonyi <szo...@cloudera.com> > wrote: > > > > > Hi All, > > > > > > I would like to make the following changes to the Sqoop CI system: > > > Disable the SCM polling for the Sqoop-hadoop23 Sqoop-hadoop20 and > > > Sqoop-hadoop100 jobs (and later delete the jobs themselves), > > > as the current trunk version of sqoop no longer contains these > profiles, > > so > > > these runs are redundant. > > > > > > I would also like to propose the creation of a pre-commit hook for > Sqoop > > > (like the existing one for Sqoop2). > > > > > > Please let me know if you have any objections. > > > > > > Thanks, > > > Anna > > > > > > > > > > > -- > > Szabolcs Vasas > > Software Engineer > > <http://www.cloudera.com> > > >
