hey everyone! just for visibility, after some lengthy conversations w/some PMC members (mostly sean and josh) about the location of the jenkins job builder temples being in a private, databricks repo, we've decided to move them in to the main apache spark repo.
https://docs.openstack.org/infra/jenkins-job-builder/ On Tue, Oct 9, 2018 at 10:22 PM Sean Owen <sro...@gmail.com> wrote: > Some responses inline -- this discussion can do to dev@ though. > > dev@ added. > On Tue, Oct 9, 2018 at 3:28 PM shane knapp <skn...@berkeley.edu> wrote: > > JBB templates in spark repo: > > * code path is currently undecided, maybe build/? i honestly don't have > any strong opinions > > How about a subfolder of dev/? that's where many items like the > release scripts and build style checkers live. > > works for me. > > * this stuff will only in the master branch > > Sure, it'll get versioned with branches anyway, but only the master > branch will matter. > > > * the current JJB templates include release and packaging jobs, which > aren't run via jenkins anymore. this means we can remove the job builder > configs for these two, as well as the encrypted secrets. > > Sure, if it's not used, remove it. I suppose the concerns below lessen > if none of the jobs in question here create release artifacts. > > exactly. > > * the JJB templates are able to be run by anyone w/jenkins login access > without the need to commit changes to the repo. this means there's a > non-zero potential for bad actors to change the build configs. since we > will only be managing test and compile jobs through this, the chances for > Real Bad Stuff[tm] is minimized. i will also have a local server, not on > the jenkins network, run a nightly cron job that grabs the latest configs > from github and syncs them to jenkins. > > You mean anyone with access to amplab Jenkins? I think this is an > acceptable risk, especially as it's never been an issue to date. The > worst case is deleting or sabotaging CI jobs, right? not great, but, > nobody would be able to commit code or anything. That's the worst > case, right? > > re: access to jenkins -- correct. re: worst case, deleting a job -- correct (but a cron sync of the jobs from the tip of master will repair and damage done by nefarious folks). re: committing code -- correct > This might be a good time to ask whether we want to use a different CI > system. I don't see a need to. I don't see any problem that's surfaced > by publishing the configs as part of the project. If riselab is OK > continuing to subsidize the build infra, and it continues to work for > the project, it seems fine. As long as the PMC is meaningfully in > control of it, I can't see any issue. > i have confirmation from RISELab's PI (ion stoica) that we are committed to continuing to host the apache spark build system here. if that situation changes (which i cannot foresee), it will of course be brought up w/the community and the PMC immediately. i would also like some heads up from the PMC if the situation changes on their end. ;) btw, work will not begin on this until next week. once i get a jira and PR opened, i'll respond to this thread w/those links. shane -- Shane Knapp UC Berkeley EECS Research / RISELab Staff Technical Lead https://rise.cs.berkeley.edu
