I think you must be right, my solution was to stop caring where the flow
runs, and focus on an alternative method of retrieving the report/log data.
I used the Build Flow Toolbox which lets you copy files with the flow:
job = build('downstream job')
toolbox.copyFiles(job.workspace, build.workspace)
//Unrelated but also useful:
out.println job.build.log
On Sunday, March 8, 2015 at 9:03:30 AM UTC-7, stefan thomasson wrote:
>
> Hi,
> If i understood your description, i am wondering if this Isn't the
> expected behaviour for buildflow as it is a "fly weight" job and therefore
> do not consume any executors?
>
> 5 mar 2015 kl. 23:39 skrev Ty Satrang <[email protected] <javascript:>>:
>
> I am not sure how you want to receive that, but here is a gist of the
> config that will reproduce the issue.
> Flow: https://gist.github.com/tsatrangtw/73a391aa75b45cb38235
> Downsteam: https://gist.github.com/tsatrangtw/4c20c9a85ce63081c160
>
>
> On Thursday, March 5, 2015 at 2:15:47 PM UTC-8, rginga wrote:
>>
>> Can you provide a simple example of your main buildflow job?
>>
>>
>>
>> *From:* [email protected] [mailto:[email protected]] *On
>> Behalf Of *Ty Satrang
>> *Sent:* Thursday, March 05, 2015 5:12 PM
>> *To:* [email protected]
>> *Subject:* Re: [Build flow plugin] Cannot load balance multiple workflows
>>
>>
>>
>> Thank you, I did see this copy pasta in another thread, and I did try
>> changing the scheduling strategy with a plug-in. It seems that this
>> particular problem is somewhat exclusive to the Build Flow plugin. It
>> doesn't just queue jobs up, as you can see in the image link, it runs the
>> flow jobs all at once on a single node.
>>
>>
>> On Thursday, March 5, 2015 at 11:46:37 AM UTC-8, rginga wrote:
>>
>> Maybe I misunderstand your problem. Let’s keep it simple.
>>
>>
>>
>> The reason the multiple runs of your build flow job run on the same node
>> is because, when selecting a node, it tries to run it one the same node it
>> previously ran on (#2 in the Strategy below). If your nodes had just 1
>> executor defined, then any consecutive runs would run on different nodes.
>> But with 8 executors, it just runs it there.
>>
>>
>>
>> *Scheduling strategy*
>>
>> Some slaves are faster, while others are slow. Some slaves are closer
>> (network wise) to a master, others are far away. So doing a good build
>> distribution is a challenge. Currently, Jenkins employs the following
>> strategy:
>>
>> 1. If a project is configured to stick to one computer, that's always
>> honored.
>> 2. Jenkins tries to build a project on the same computer that it was
>> previously built.
>> 3. Jenkins tries to move long builds to slaves, because the amount of
>> network interaction between a master and a slave tends to be logarithmic
>> to
>> the duration of a build (IOW, even if project A takes twice as long to
>> build as project B, it won't require double network transfer.) So this
>> strategy reduces the network overhead.
>>
>> If you have interesting ideas (or better yet, implementations), please
>> let me know.
>>
>>
>>
>> *From:* [email protected] [mailto:[email protected]] *On
>> Behalf Of *Ty Satrang
>> *Sent:* Thursday, March 05, 2015 1:52 PM
>> *To:* [email protected]
>> *Subject:* Re: [Build flow plugin] Cannot load balance multiple workflows
>>
>>
>>
>> "if you have a label assigned to many slaves, each with one executor, it
>> will do what you want"
>> My slaves do all share the same label and I tried to "Restrict where
>> this project can be run" and I get the same behavior.
>>
>> "One way to do what you want is to have each “archive” whatever files
>> you want which puts them onto the Jenkins server and then use “Copy
>> artifacts from another project”"
>>
>> I suppose this could work, but does it copy the files to the workspace or
>> does it just archive them? I am using a plugin to publish reports from the
>> workspace.
>>
>>
>>
>>
>> On Thursday, March 5, 2015 at 8:12:59 AM UTC-8, rginga wrote:
>>
>> Its strategy is to try to use the same node it last built on. If you have
>> available executors, it will use one. if you have a label assigned to many
>> slaves, each with one executor, it will do what you want.
>>
>>
>>
>> One way to do what you want is to have each “archive” whatever files you
>> want which puts them onto the Jenkins server and then use “Copy artifacts
>> from another project”
>>
>>
>>
>> *From:* [email protected] [mailto:[email protected]] *On
>> Behalf Of *Ty Satrang
>> *Sent:* Thursday, March 05, 2015 3:50 AM
>> *To:* [email protected]
>> *Subject:* [Build flow plugin] Cannot load balance multiple workflows
>>
>>
>>
>> http://i.imgur.com/VEZ1Erc.png
>>
>>
>>
>> For some reason my flow jobs like to all run on a single node instead of
>> choosing an idle node.
>>
>>
>>
>> This is a problem for me because I am using the Node and Label parameter
>> plugin
>> <https://wiki.jenkins-ci.org/display/JENKINS/NodeLabel+Parameter+Plugin> to
>> run the jobs on the same VM as the flow job so I can copy the reports and
>> logs to the parent workspace for publishing.
>>
>>
>>
>> I tried using the Least Load plugin
>> <http://wiki.jenkins-ci.org/display/JENKINS/Least+Load+Plugin>, but I
>> get the same behavior.
>>
>>
>>
>> I also tried the Throttle Concurrent Builds Plug-in
>> <http://wiki.jenkins-ci.org/display/JENKINS/Throttle+Concurrent+Builds+Plugin>
>> to
>> restrict the jobs per node to 1, to no effect.
>>
>>
>>
>> I am just realizing this now but, maybe it would be easier to get the
>> files to the parent workspace another way and let the flow jobs all run on
>> whatever single node they want. (maybe?
>> https://github.com/dnozay/build-flow-toolbox-plugin )
>>
>>
>>
>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Jenkins Users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected].
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/jenkinsci-users/3d1d0150-3125-4b54-a846-b0a6687022cf%40googlegroups.com
>>
>> <https://groups.google.com/d/msgid/jenkinsci-users/3d1d0150-3125-4b54-a846-b0a6687022cf%40googlegroups.com?utm_medium=email&utm_source=footer>
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Jenkins Users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected].
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/jenkinsci-users/0a9cb8a5-d3ef-4f02-8506-0e4b0a986493%40googlegroups.com
>>
>> <https://groups.google.com/d/msgid/jenkinsci-users/0a9cb8a5-d3ef-4f02-8506-0e4b0a986493%40googlegroups.com?utm_medium=email&utm_source=footer>
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Jenkins Users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected].
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/jenkinsci-users/e787a0c5-7c45-4100-af52-05269ce704d6%40googlegroups.com
>>
>> <https://groups.google.com/d/msgid/jenkinsci-users/e787a0c5-7c45-4100-af52-05269ce704d6%40googlegroups.com?utm_medium=email&utm_source=footer>
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Jenkins Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected] <javascript:>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jenkinsci-users/aea6321b-8ab0-4311-b405-3dda4f533458%40googlegroups.com
>
> <https://groups.google.com/d/msgid/jenkinsci-users/aea6321b-8ab0-4311-b405-3dda4f533458%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
> For more options, visit https://groups.google.com/d/optout.
>
>
--
You received this message because you are subscribed to the Google Groups
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/jenkinsci-users/5ab0ebd1-cb3a-4045-82b6-30b403bdb6f6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
