>From my experiences with this, I sympathise with your frustration. In our situation (not sure if this maps with yours) the ideal sequence of events would be:
1. Project is created 2. Branches are indexed, but not built 3. Polling (or hooks) now enabled 4..? Scheduled indexing now auto-builds new branches if selected. Hooks will trigger builds The main thing is that on creation of a multibranch project referencing a repository with a significant number of old branches, everything doesn't get built all at once - like yourself, we found this would happily bring a Jenkins instance to its knees quite effectively. Once the initial creation and first branch index is complete, however, existing functionality is fine. Tony On Wed, Apr 15, 2020 at 2:11 PM Adam Gabryś <[email protected]> wrote: > Hello, > > we provide CI for many teams in a big company. Unfortunately, we have a > huge problem with server loads after the spin up process, after restarts > and jobs modification. Multibranch jobs execute branch indexing on every > single change and directly after the job creation. It is great, because > people see what is available. On the other side, when people don't delete > old branches, we waste a lot of resources on building old stuff. In our > case it completely blocks servers for days. We tried to use build > strategies, but none of them solved it. The closest solution was > SkipInitialBuildOnFirstBranchIndexing > <https://github.com/jenkinsci/basic-branch-build-strategies-plugin/blob/master/src/main/java/jenkins/branch/buildstrategies/basic/SkipInitialBuildOnFirstBranchIndexing.java>, > but unfortunately it: > > - blocks all new branches and PRs builds executed by webhooks (it > always skip the first build no matter how it is triggered) > - won’t solve the problem when the branch indexing is executed for a > second time (again, everything is build) > > For now we use the patch visible in this PR > <https://github.com/jenkinsci/branch-api-plugin/pull/186> (I'm aware that > it won't be merged - it is just a workaround, I skipped tests). I wanted to > ask - do you have any ideas on how to solve this properly? We are open to > implement the proposed solution 🙂 > > For now, I have two ideas to make it configurable: > > 1. add an option to disable automatic builds on branch indexing > 2. extend BranchBuildStrategy > > <https://github.com/jenkinsci/branch-api-plugin/blob/master/src/main/java/jenkins/branch/BranchBuildStrategy.java> > class > and pass information about the trigger cause > > Both solutions don't cover the workaround solution, because they will only > allow the skipping of all builds caused by branch indexing (we will lose > the ability to execute builds of branches/PRs for which webhooks have not > been received - for example, lost due to network problems). > > > Disable Automatic build on branch indexing > > The first solution requires to add a new property to the multibranch > project and simply skip the build if causeFactory instanceof > IndexingCauseFactory. I don't know how to add this configurable property, > but I can get this knowledge 😉 > > This approach does not change the API, just adds a new parameter. > > Extend BranchBuildStrategy API > > This approach is backward compatible, but introduces new methods to API > (in a class which is extended by many plugins). > > How could we implement it: > > - add a new BranchBuildStrategy#isAutomaticBuild method which takes > one more parameter (the trigger cause - indexing vs. webhook): > > public boolean isAutomaticBuild(@NonNull TriggerCause cause, > @NonNull SCMSource source, > @NonNull SCMHead head, > @NonNull SCMRevision currRevision, > @CheckForNull SCMRevision lastBuiltRevision, > @CheckForNull SCMRevision lastSeenRevision, > @NonNull TaskListener listener) { > // by default delegate to the version without the cause > return isAutomaticBuild(source, head, currRevision, lastBuiltRevision, > lastSeenRevision, listener); > } > > > - add a new BranchBuildStrategy#automaticBuild which will be executed > by the MultiBranchProject: > > public final boolean automaticBuild(@NonNull TriggerCause cause, > @NonNull SCMSource source, > @NonNull SCMHead head, > @NonNull SCMRevision currRevision, > @CheckForNull SCMRevision > lastBuiltRevision, > @CheckForNull SCMRevision > lastSeenRevision, > @NonNull TaskListener listener) { > if (Util.isOverridden(BranchBuildStrategy.class, getClass(), > "isAutomaticBuild", TriggerCause.class, > SCMSource.class, SCMHead.class, SCMRevision.class, > SCMRevision.class, SCMRevision.class, > TaskListener.class)) { > return isAutomaticBuild(cause, source, head, currRevision, > lastBuiltRevision, lastSeenRevision, listener); > } > return automaticBuild(source, head, currRevision, lastBuiltRevision, > lastSeenRevision, listener); > } > > > It falls back to the original automaticBuild method (backward compatible). > > - create a new build strategy (in new or existing plugin) > SkipBuildOnBranchIndexing > > ------------------------------ > > Let me know what do you think. > > > Kind regards > Adam Gabryś > > -- > You received this message because you are subscribed to the Google Groups > "Jenkins Developers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/jenkinsci-dev/751d8c6f-50cd-4d10-bb90-861690f5d1fc%40googlegroups.com > <https://groups.google.com/d/msgid/jenkinsci-dev/751d8c6f-50cd-4d10-bb90-861690f5d1fc%40googlegroups.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "Jenkins Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAEWqh9EW5zQkBB-s0hmPP_9C7fbfHU9o4RW-NjxOQQY5F%2BK8LQ%40mail.gmail.com.
