Hi Tony, Sorry, but I don't understand. You wrote the existing functionality is fine, but the current behavior is as follow:
1. Project is created 2. Branches are indexed, all branches are built ← kills the server 3. Polling (or hooks) now enabled 4. Scheduled indexing now auto-builds new branches if selected. Hooks will trigger builds My workaround changes it to what you described: 1. Project is created 2. Branches are indexed, but not built 3. Polling (or hooks) now enabled 4. Scheduled indexing now auto-builds new branches if selected. Hooks will trigger builds I think such feature is required by many people, so I would like to implement a solution which could be merged to the plugin. Patching every plugin's release is very ineffective and error prone. My workaround was declined (I was expecting that), so I'm looking for a way to implement it in a way which will be approved by the maintainers. Kind regards Adam Gabryś ________________________________ From: [email protected] <[email protected]> on behalf of Tony Noble <[email protected]> Sent: Thursday, April 16, 2020 12:25 PM To: [email protected] <[email protected]> Subject: Re: Prevent building branches/PRs existing before the first branch indexing (Branch API plugin) >From my experiences with this, I sympathise with your frustration. In our >situation (not sure if this maps with yours) the ideal sequence of events >would be: 1. Project is created 2. Branches are indexed, but not built 3. Polling (or hooks) now enabled 4..? Scheduled indexing now auto-builds new branches if selected. Hooks will trigger builds The main thing is that on creation of a multibranch project referencing a repository with a significant number of old branches, everything doesn't get built all at once - like yourself, we found this would happily bring a Jenkins instance to its knees quite effectively. Once the initial creation and first branch index is complete, however, existing functionality is fine. Tony On Wed, Apr 15, 2020 at 2:11 PM Adam Gabryś <[email protected]<mailto:[email protected]>> wrote: Hello, we provide CI for many teams in a big company. Unfortunately, we have a huge problem with server loads after the spin up process, after restarts and jobs modification. Multibranch jobs execute branch indexing on every single change and directly after the job creation. It is great, because people see what is available. On the other side, when people don't delete old branches, we waste a lot of resources on building old stuff. In our case it completely blocks servers for days. We tried to use build strategies, but none of them solved it. The closest solution was SkipInitialBuildOnFirstBranchIndexing<https://github.com/jenkinsci/basic-branch-build-strategies-plugin/blob/master/src/main/java/jenkins/branch/buildstrategies/basic/SkipInitialBuildOnFirstBranchIndexing.java>, but unfortunately it: * blocks all new branches and PRs builds executed by webhooks (it always skip the first build no matter how it is triggered) * won’t solve the problem when the branch indexing is executed for a second time (again, everything is build) For now we use the patch visible in this PR<https://github.com/jenkinsci/branch-api-plugin/pull/186> (I'm aware that it won't be merged - it is just a workaround, I skipped tests). I wanted to ask - do you have any ideas on how to solve this properly? We are open to implement the proposed solution 🙂 For now, I have two ideas to make it configurable: 1. add an option to disable automatic builds on branch indexing 2. extend BranchBuildStrategy<https://github.com/jenkinsci/branch-api-plugin/blob/master/src/main/java/jenkins/branch/BranchBuildStrategy.java> class and pass information about the trigger cause Both solutions don't cover the workaround solution, because they will only allow the skipping of all builds caused by branch indexing (we will lose the ability to execute builds of branches/PRs for which webhooks have not been received - for example, lost due to network problems). Disable Automatic build on branch indexing The first solution requires to add a new property to the multibranch project and simply skip the build if causeFactory instanceof IndexingCauseFactory. I don't know how to add this configurable property, but I can get this knowledge 😉 This approach does not change the API, just adds a new parameter. Extend BranchBuildStrategy API This approach is backward compatible, but introduces new methods to API (in a class which is extended by many plugins). How could we implement it: * add a new BranchBuildStrategy#isAutomaticBuild method which takes one more parameter (the trigger cause - indexing vs. webhook): public boolean isAutomaticBuild(@NonNull TriggerCause cause, @NonNull SCMSource source, @NonNull SCMHead head, @NonNull SCMRevision currRevision, @CheckForNull SCMRevision lastBuiltRevision, @CheckForNull SCMRevision lastSeenRevision, @NonNull TaskListener listener) { // by default delegate to the version without the cause return isAutomaticBuild(source, head, currRevision, lastBuiltRevision, lastSeenRevision, listener); } * add a new BranchBuildStrategy#automaticBuild which will be executed by the MultiBranchProject: public final boolean automaticBuild(@NonNull TriggerCause cause, @NonNull SCMSource source, @NonNull SCMHead head, @NonNull SCMRevision currRevision, @CheckForNull SCMRevision lastBuiltRevision, @CheckForNull SCMRevision lastSeenRevision, @NonNull TaskListener listener) { if (Util.isOverridden(BranchBuildStrategy.class, getClass(), "isAutomaticBuild", TriggerCause.class, SCMSource.class, SCMHead.class, SCMRevision.class, SCMRevision.class, SCMRevision.class, TaskListener.class)) { return isAutomaticBuild(cause, source, head, currRevision, lastBuiltRevision, lastSeenRevision, listener); } return automaticBuild(source, head, currRevision, lastBuiltRevision, lastSeenRevision, listener); } It falls back to the original automaticBuild method (backward compatible). * create a new build strategy (in new or existing plugin) SkipBuildOnBranchIndexing ________________________________ Let me know what do you think. Kind regards Adam Gabryś -- You received this message because you are subscribed to the Google Groups "Jenkins Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]<mailto:[email protected]>. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/751d8c6f-50cd-4d10-bb90-861690f5d1fc%40googlegroups.com<https://groups.google.com/d/msgid/jenkinsci-dev/751d8c6f-50cd-4d10-bb90-861690f5d1fc%40googlegroups.com?utm_medium=email&utm_source=footer>. -- You received this message because you are subscribed to a topic in the Google Groups "Jenkins Developers" group. To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-dev/NJoc_YQMy44/unsubscribe. To unsubscribe from this group and all its topics, send an email to [email protected]<mailto:[email protected]>. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAEWqh9EW5zQkBB-s0hmPP_9C7fbfHU9o4RW-NjxOQQY5F%2BK8LQ%40mail.gmail.com<https://groups.google.com/d/msgid/jenkinsci-dev/CAEWqh9EW5zQkBB-s0hmPP_9C7fbfHU9o4RW-NjxOQQY5F%2BK8LQ%40mail.gmail.com?utm_medium=email&utm_source=footer>. -- You received this message because you are subscribed to the Google Groups "Jenkins Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/AM0PR0502MB4050657CDC633169F05C404A98D80%40AM0PR0502MB4050.eurprd05.prod.outlook.com.
