How can I add builds to the queue a little at a time?

I am beginning the process of changing to pipelines. We currently use flows to run thousands of jobs(builds). Say we have 6000 builds to run on 100 Windows virtual machines and 6000 builds to run on 100 Linux virtual machines. To reduce available virtual machines with no jobs to run we try to

Re: Is there a maximum number of parallel builds a pipeline job can run?

I need to retry it may be that the jobs never all started Branch: job_0 - (1ms in block) Branch: job_9830 - (3min 16sec in block) Branch: job_9831 - (No timing in block) -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this

Re: Is there a maximum number of parallel builds a pipeline job can run?

So not sure if this is a Jenkins limitation or pipeline. The jobs never exceeded available resources. The garbage collection was stable. So why did it lock-up? It did say I used - 2950 million active threads - 350 threads -- You received this message because you are subscribed to the

Re: Is there a maximum number of parallel builds a pipeline job can run?

So 3 hr 30 min later the pipeline is locked up -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscr...@googlegroups.com. To view this

Re: Is there a maximum number of parallel builds a pipeline job can run?

So I have not run in an ode block I will do more testing thank you. Here is a sample of my code: *#!groovy* *import hudson.model.** *stage('many_jobs') {* *parallel (get_many_jobs(6000))* *}* *def get_many_jobs(num_builds) {* *// The map we'll store the parallel steps in before

Re: Is there a maximum number of parallel builds a pipeline job can run?

So I was not running in a node block I will do more testing thank you. Here is a sample of my code: *#!groovy* *import hudson.model.** *stage('many_jobs') {* *parallel (get_many_jobs(6000))* *}* *def get_many_jobs(num_builds) {* *// The map we'll store the parallel steps in before

Is there a maximum number of parallel builds a pipeline job can run?

I am doing testing of our Jenkins setup. I have a build job that sleeps for 30 sec. I ran a pipeline that runs 12,000 of these in parallel on 100 agents. Best possible time is 1 hr. The builds seem to finish in a reasonable amount of time. So after build queue is emptied and the pipeline

pipline slow vs flow

So I am doing testing on our current system. We use flows now. So I have a flow and a pipeline doing the same task: - 100 executors - 2000 builds - build - echo 'Hello World' flow time = 2 min 33 sec pipeline time = 5 min 11 sec After all the echos are complete the pipeline seems to

Re: pipline slow vs flow

I am not trying to say anything I just found it curious. We are doing testing to determine the optimal setup for Jenkins. We currently use flows calling flows. We believe we have too many builds in the build queue (3500). In the process of testing we saw this behavior and I was hoping for

Re: How can I add builds to the queue a little at a time?

Thank you for your insights. This reinforces my thoughts after reading https://jenkins.io/blog/2016/11/21/gc-tuning/ -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an

parallel remote trigger builds in a pipeline

#!groovy import hudson.model.* stage('Testing') { parallel ( ['windows_remote': {build job: 'remote_trigger_30', parameters: [string(name: 'branch_name', value: branch_name)]}, 'linux_remote': {build job: 'remote_trigger_60', parameters: [string(name: 'branch_name', value:

Re: parallel remote trigger builds in a pipeline

I assume you have "Block until the remote triggered projects finish their builds." = False? - Does it not just set your 'done' variable to true as soon as the remote build starts, since it does not block or wait. - Also with this set to False it does not get the log of the remote

How to properly use parallels in pipelines

*parallel* ( 'windows': { stage('windows tests') { *parallel* ( *stage*('some_tests_1') { *parallel* ( some_tests_1, # is a chunk of tests ) }

many parallel stages

Hello, I was looking for help with a many parallel pipeline. Here is what I am looking to do *parallel*( 'linux': { stage('build_linux') { build job: 'linux_build', parameters:[] build job: 'linux_archive', parameters:[] *parallel*(

Pipeline calling pipeline calling freestyle job

I have: booleanParam(name: 'force_build', value: force_build.toBoolean()) This works Pipeline with parameter (force_build=True) -> freestsyle job with parameter (force_build=False) The value of force_build in the freestyle job is now True This does not work. Pipeline job with a

Re: Custom pipeline jobs

Anyone have any ideas how I can create dynamic pipeline scripts -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscr...@googlegroups.com.

Custom pipeline jobs

Is it possible to pass groovy scripts to a pipeline job I want to be able to configure custom groovy scripts based on options selected by my users. I could have a massive script with many "if" clauses. It would be nicer to create a groovy script based on selected options. this api would then be

Re: Custom pipeline jobs

Jenn, We do not test with a CI strategy. We have a limited number things that we do with some parallelization. 1. clone or merge 2. in parallel a. build product for (window (32, 64), linux (32, 64)) b. run many different types of tests c. generate report on tests 3. generate