[JIRA] [workflow-plugin] (JENKINS-34201) Pipeline plugin can't handle large numbers of parallel build jobs
Title: Message Title Jesse Glick updated an issue Jenkins / JENKINS-34201 Pipeline plugin can't handle large numbers of parallel build jobs Change By: Jesse Glick Labels: pipeline Add Comment This message was sent by Atlassian JIRA (v6.4.2#64017-sha1:e244265) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [workflow-plugin] (JENKINS-34201) Pipeline plugin can't handle large numbers of parallel build jobs
Title: Message Title Jesse Glick commented on JENKINS-34201 Re: Pipeline plugin can't handle large numbers of parallel build jobs Why are you running build inside node? That makes no sense—just wasting an executor slot for no reason. Part of the issue could be the insufficiently unique build parameters. If you schedule a build when a queue item for that job already exists with a given set of parameters, the attempt to reschedule will simply be ignored. As of JENKINS-28063 that should not cause a hang, though. Reproducible from scratch somehow? Add Comment This message was sent by Atlassian JIRA (v6.4.2#64017-sha1:e244265) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [workflow-plugin] (JENKINS-34201) Pipeline plugin can't handle large numbers of parallel build jobs
Title: Message Title
Tom De Vylder updated an issue
Jenkins / JENKINS-34201
Pipeline plugin can't handle large numbers of parallel build jobs
Change By:
Tom De Vylder
Consider following snippet:{code:java}stage name: 'foo', concurrency: 10foo = [:]foo['failFast'] = truefor (int i = 0; i < 25; i++) {for (int j = 0; j < 4; j++) {foo["branch${i}-${j}"] = {node {build job: 'job1', parameters: [[ $class: 'StringParameterValue', name: 'foo', value: 'f' ],[ $class: 'StringParameterValue', name: 'bar', value: 'b' ],[ $class: 'StringParameterValue', name: 'baz', value: 'z' ]], quietPeriod: 0}node {build job: 'job2', parameters: [[ $class: 'StringParameterValue', name: 'foo', value: 'f' ],[ $class: 'StringParameterValue', name: 'bar', value: 'b' ],[ $class: 'StringParameterValue', name: 'baz', value: 'z' ]], quietPeriod: 0}}}parallel foo}{code}It starts to build the requested jobs just fine.INFO: job1 #125 main build action completed: SUCCESINFO: job1 #127 main build action completed: SUCCESINFO: job1 #126 main build action completed: SUCCESINFO: job2 #124 main build action completed: SUCCES...However when all of the jobs are done it seems Pipeline can't seem to merge all those results and is just stuck. After 24h it's still hanging, seemingly waiting for a parallel job to finish.Now when I remove the outermost for loop things run just fine.{code:java}for (int j = 0; j < 4; j++) {foo["branch${j}"] = {...}}{code} Removing the inner for loop and increasing the outer loop to 100 results in foo[] being too big for Jenkins to handle. Same happens without said for loops obviously, which lead me to start using them.{code:java}for (int i = 0; i < 100; j++) {foo["branch${i}"] = {...}} {code} There's probably a better way to handle this.Any pointers how to get there?
Add Comment
[JIRA] [workflow-plugin] (JENKINS-34201) Pipeline plugin can't handle large numbers of parallel build jobs
Title: Message Title
Tom De Vylder updated an issue
Jenkins / JENKINS-34201
Pipeline plugin can't handle large numbers of parallel build jobs
Change By:
Tom De Vylder
Consider following snippet:{code:java}stage name: 'foo', concurrency: 10foo = [:]foo['failFast'] = truefor (int i = 0; i < 25; i++) {for (int j = 0; j < 4; j++) {foo["branch${i}-${j}"] = {node {build job: 'job1', parameters: [[ $class: 'StringParameterValue', name: 'foo', value: 'f' ],[ $class: 'StringParameterValue', name: 'bar', value: 'b' ],[ $class: 'StringParameterValue', name: 'baz', value: 'z' ]], quietPeriod: 0}node {build job: 'job2', parameters: [[ $class: 'StringParameterValue', name: 'foo', value: 'f' ],[ $class: 'StringParameterValue', name: 'bar', value: 'b' ],[ $class: 'StringParameterValue', name: 'baz', value: 'z' ]], quietPeriod: 0}}}parallel foo}{code}It starts to build the requested jobs just fine.INFO: job1 #125 main build action completed: SUCCESINFO: job1 #127 main build action completed: SUCCESINFO: job1 #126 main build action completed: SUCCESINFO: job2 #124 main build action completed: SUCCES...However when all of the jobs are done it seems Pipeline can't seem to merge all those results and is just stuck. After 24h it's still hanging, seemingly waiting for a parallel job to finish.Now when I remove the outermost for loop things run just fine.{code:java}for (int j = 0; j < 4; j++) {foo["branch${j}"] = {...}}{code} Removing the inner for loop and increasing the outer loop to 100 results in foo[] being too big for Jenkins to handle. Same happens without said for loops obviously, which lead me to start using them.{code:java}for (int i = 0; i < 100; j++) {foo["branch${i}"] = {...}}There's probably a better way to handle this.Any pointers how to get there?
Add Comment
[JIRA] [workflow-plugin] (JENKINS-34201) Pipeline plugin can't handle large numbers of parallel build jobs
Title: Message Title
Tom De Vylder created an issue
Jenkins / JENKINS-34201
Pipeline plugin can't handle large numbers of parallel build jobs
Issue Type:
Bug
Assignee:
Jesse Glick
Components:
workflow-plugin
Created:
2016/Apr/13 12:54 PM
Environment:
Jenkins 1.656 on RHEL7 with Pipeline plugin 2.0
Labels:
pipeline
Priority:
Minor
Reporter:
Tom De Vylder
Consider following snippet:
stage name: 'foo', concurrency: 10
foo = [:]
foo['failFast'] = true
for (int i = 0; i < 25; i++) {
for (int j = 0; j < 4; j++) {
foo["branch${i}-${j}"] = {
node {
build job: 'job1', parameters: [
[ $class: 'StringParameterValue', name: 'foo', value: 'f' ],
[ $class: 'StringParameterValue', name: 'bar', value: 'b' ],
[ $class: 'StringParameterValue', name: 'baz', value: 'z' ]
], quietPeriod: 0
}
n
