Hi,
We are trying to use the Kubernetes Plugin to run tests in a different
container - but *almost* every time we try and execute a command in the
second container, the build fails immediately. I say "almost" because the
build always succeeds after restarting the Jenkins master (or possibly
after it's been idle for a long time).
Here's the Jenkinsfile we're using to test:
def label = "k8s-test-${UUID.randomUUID().toString()}"
podTemplate(
cloud: 'kubernetes-test.k8s.local',
namespace: 'mynamespace',
label: label,
yaml: """
apiVersion: v1
kind: Pod
spec:
containers:
- name: busybox
image: busybox
command: ['cat']
tty: true
"""
) {
node (label) {
stage ('Get Agent Info') {
sh "java -version"
}
stage ('Run tests') {
container('busybox') {
sh "echo foo"
sh "netstat -tln"
sh "df -h"
}
}
stage ('Grab Logs') {
containerLog('jnlp')
}
}
}
This fails at the "echo foo" step. The error reported by the pipeline is:
Task okhttp3.RealCall$AsyncCall@b5277fa rejected from
java.util.concurrent.ThreadPoolExecutor@6c383ec7[Terminated, pool size = 0,
active threads = 0, queued tasks = 0, completed tasks = 3]
What we know so far:
* Both containers are successfully deployed to the same pod, as expected.
* If we don't try and run the shell commands on the 'busybox' container,
the build completes successfully.
* If Jenkins is restarted, the build passes. But subsequent builds fail.
* The Agent logs contain similar exceptions whether the builds pass or
fail, so it's hard to pinpoint a cause.
The ThreadPool exception may imply that *something* has run out of
Executors, but it's not clear what, or how to rectify the situation.
Has anyone seen anything like this? Any ideas how to get to the bottom of
it? As per the plugin's README, I've created Jenkins log recorders for
org.csanchez.jenkins.plugins.kubernetes and okhttp3, but there are no
obvious problems reported in the logs.
Software Versions:
Jenkins: v2.150.1
Kubernetes Plugin: 1.13.7
Kubernetes: 1.10.11 (same behaviour observed on 1.8.7)
All suggestions appreciated!
Thanks,
Simon.
--
You received this message because you are subscribed to the Google Groups
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/jenkinsci-users/141b526e-f00b-434d-9f43-ec97fdb23df7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
