i would have to agree, this is definitely an issue. I'm currently using Amazon EFS as the storage backend for my Jenkins master, who doesn't have any executors. The flyweight executors checking out repositories is definitely a concern of mine. EFS will automatically scale so storage isn't a huge issue for me, but duplication and clone times are. (for larger repos)
On Wednesday, August 10, 2016 at 2:31:45 PM UTC-4, Kevin Phillips wrote: > > In my opinion, even more problematic than possibly running unsafe code in > a flyweight executor on the master is the fact that these flyweight > pipeline threads perform a full checkout of the target repository - at > least when using Git as the SCM. So if one has a relatively large code > repository containing product code - say several gigabytes - and they want > to store their Jenkinsfile with the code so they can leverage the > MultiBranch options in Jenkins Pipeline, they end up with many copies of > their entire codebase being checked out on the master, possibly causing the > master to run out of disk space. Add to this the fact that these > "temporary" workspaces created on the master by the flyweight tasks are not > automatically deleted when not in use and there is no way to purge them > from the Jenkins dashboard and you are just asking for problems. This isn't > a case of whether this will become a problem as much as it is an issue of > when it will become a problem. > > In my opinion, Jenkinsfile's stored in SCM should be checked out on an > agent as a typical freestyle job would do, and a default "node" allocated > by the executor that launches the script contained therein for the node on > which it is already running on. Then the rest of the Jenkinsfile could be > processed in place without requiring any sort of flyweight executor at all, > and avoiding the issues with having multiple checkouts of the same code > spread across a build farm. Also, I think this would be inline with how > some other more modern continuous integration servers work like travis-ci, > so it's not without precidence. > > On Monday, June 13, 2016 at 6:03:20 PM UTC-3, Eli White wrote: >> >> We follow the Jenkins configuration best practices and have no executors >> on our master node and force everything to run on our agents. >> >> We are starting to work with pipeline jobs and are worried that bad >> Jenkinsfiles could cause problems on our master. >> How can we force the flyweight jobs to run on a designated machine >> *other* than master? >> >> How do other people handle this? >> > -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/d6b297d9-0a17-42ea-a608-45f324b33caf%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
