Hi,
I feel like this is certainly a situation someone has run into before but I
haven't been able to think up a non-trivial solution for my use case:
- Lets say I have two pipelines, P1 and P2, that do stuff and both need
the same two executors/slaves/nodes A and B
- P1 enters the queue first and takes both A and B and starts doing it's
stuff in parallel (using parallel)
- Then P2 enters the queue while A and B are being used, still, by P1; A
and B are still online but they're unavailable for P2 to use (you'd see it
waiting for the executors to be available in the console, for example)
- However, B fails and has the node taken offline on failure
- P1 completes with A eventually passing and B failed and took the node
offline. Perhaps, depending on timing, P2 started using A when it passed.
- P2 is now waiting forever until node B is brought online again
How can one check in the pipeline P2 that node B is offline and just break
out?
If the node was offline at the start of P2 then it's easy to check and
exclude it. However, if B is online when P2 enters the queue and sets up
the parallel runs of A and B and sits and waits for them to be available
and one of them goes offline then how does the pipeline get notified it's
offline and move on to do whatever? Doesn't seem to happen automatically
and I can't figure out how to check inside a node block that itself is
offline (B checking if it itself is offline).
Here's a simple pipeline groovy script I made to help me figure out the
issue:
// Branches for parallel node runs
def branches = [:]
// Nodes. In real setup this would only contain nodes that are online at
the time the pipeline runs
def node_names = ["A", "B"]
// Short sleep time of 15 seconds. Later, it'll get reduced to 5 if the
node name is B
def sleep_time = 15
// Loop through the nodes and create the data in the branch list to run in
parallel on at the end
node_names.each { node_name ->
println node_name
branches["node_" + node_name] = {
// Doing something like this doesn't do anything
//if (!isNodeOnline(node_name)) {
// println "node name " + node_name + " is offline, returning"
// return
//}
node(node_name) {
// If the node that is being looked at is B then set the sleep
time to 5 so that it runs
// a shorter time than A. Later, it's hardcoded to fail B and
take it offline. This way
// A stays in the queue running and B is done and offline.
def temp_sleep_time = sleep_time
if (node_name == "B") {
temp_sleep_time = 5
}
timestamps {
stage("pre-build") {
println "Prebuilding " + node_name + "!"
sleep time: temp_sleep_time, unit: 'SECONDS'
println "Done with pre-build!"
}
stage("build") {
println "Building " + node_name + "!"
sleep time: temp_sleep_time, unit: 'SECONDS'
println "Done with build!"
}
stage("post-build") {
println "Post building " + node_name + "!"
if (node_name == "B") {
println "Taking node offline and failing build!"
takeNodeOffline("Derp", "B")
currentBuild.result = "FAILED"
done = true
return
}
sleep time: temp_sleep_time, unit: 'SECONDS'
println "Done with post-build!"
}
}
}
}
}
parallel branches
Is this possible? Am I missing something obvious?
--
You received this message because you are subscribed to the Google Groups
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/jenkinsci-users/33694c88-9c60-404f-878b-839672db538a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
