bryancall opened a new pull request, #472:
URL: https://github.com/apache/trafficserver-ci/pull/472
Adds `options { timeout(time: 24, unit: 'HOURS') }` to
`jenkins/branch/branch_build.pipeline`, so a hung branch build aborts itself
after a day.
## Why
`branch_build` starts the child jobs (in_tree, the OS builds, autest, docs
and so on) and waits for each to report back. When a child's completion never
reaches the parent, as happens across a Jenkins controller restart, the parent
stays "in progress" forever. `branch_build` runs one build at a time, so every
later build for that branch sits in the queue with "Build #N is already in
progress", and the branch gets no post-merge CI until someone notices and
aborts it by hand. The pipeline has no timeout anywhere, so nothing stops it.
The two builds that were stuck until today:
| Build | Started (UTC) | Stuck for | Blocked |
|---|---|---|---|
| [master/branch_build
#1403](https://ci.trafficserver.apache.org/job/master/job/branch_build/1403/) |
2026-06-29 13:54 | 93 days (2,239 h) | every master branch build since 6/29 |
| [9.2.x/branch_build
#192](https://ci.trafficserver.apache.org/job/9.2.x/job/branch_build/192/) |
2026-07-27 22:06 | 65 days (1,559 h) | every 9.2.x branch build since 7/28 |
Both were aborted by hand on 9/30. Their consoles show the pipeline being
resumed with "Ready to run at …" after each controller restart (7/27, 8/3,
8/14, 8/20, 9/17), with nothing after the last one. The children had long
finished (for example [master/in_tree
#1403](https://ci.trafficserver.apache.org/job/master/job/in_tree/1403/) and
[master/cmake
#944](https://ci.trafficserver.apache.org/job/master/job/cmake/944/) succeeded
on 6/29), but the parent never registered it.
It is not a one-off. Earlier runs that hung until they were aborted or
failed:
| Build | Ran for |
|---|---|
| [master
#1401](https://ci.trafficserver.apache.org/job/master/job/branch_build/1401/) |
52 days |
| [master
#1400](https://ci.trafficserver.apache.org/job/master/job/branch_build/1400/) |
48 days |
| [10.0.x
#105](https://ci.trafficserver.apache.org/job/10.0.x/job/branch_build/105/) |
30 days |
| [10.2.x
#1](https://ci.trafficserver.apache.org/job/10.2.x/job/branch_build/1/) | 21
days |
| [10.1.x
#6](https://ci.trafficserver.apache.org/job/10.1.x/job/branch_build/6/) | 5
days |
## Why 24 hours
Runs that finish normally take 2 to 8 hours, for example [master
#503](https://ci.trafficserver.apache.org/job/master/job/branch_build/503/) (2
h) and [9.2.x
#163](https://ci.trafficserver.apache.org/job/9.2.x/job/branch_build/163/) (3
h). A few finished runs took longer, [9.2.x
#190](https://ci.trafficserver.apache.org/job/9.2.x/job/branch_build/190/) (26
h) and [10.2.x
#2](https://ci.trafficserver.apache.org/job/10.2.x/job/branch_build/2/) (69 h),
probably from waiting for a `branch` executor, which a pipeline timeout also
counts. 24 hours leaves room for a slow day and still caps a hang at a day
instead of months. It is one number, easy to raise.
## Testing
The Jenkins declarative linter on ci.trafficserver.apache.org
(`pipeline-model-converter/validate`) accepts the changed file, as it does the
original.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
