Hello everybody,
this question is about implementing an option to run a build pod (using the kubernetes plugin from Carlos Sanchez) as a different user. @Carlos: Great work, we love the ability to define our own pod templates! *Software versions:* - Jenkins version: 2.89.2 - Kubernetes Plugin version: 1.1.2 - Durable Task Plugin Version: 1.17 *First, let's assume the following scenario:* One JNLP container and 2 containers with different images in a podTemplate. - jnlp container - image jenkins/jnlp-slave:alpine - running as user 'jenkins' with uid 10000 (ten thousand) - node container - image node:9.3-alpine - running as root user with uid 0 - gradle container - image gradle:4.4-alpine - running as user 'gradle' with uid 1000 (one thousand). *podTemplate from Jenkinsfile:* podTemplate(label: nodeLabel, containers: [ containerTemplate(name: 'node', image: 'node:9.3-alpine', ttyEnabled: true, command: 'cat'), containerTemplate(name: 'gradle', image: 'gradle:4.4-alpine', ttyEnabled: true, command: 'cat') ]) Switching into container 'gradle' is not possible due to permission denied errors (seems to be caused by the durable task plugin, but I am not sure about that). Root cause seems to be that the jnlp container runs with uid 10000 (yes, ten thousand - the 'old' image jenkins/jnlp-slave:2.62 ran with uid 1000) and therefore the working directory has file/directory permissions that prevent other users except uid 10000 and root (uid 0) from accessing the working directory. I also added a pod yaml file (see attachment pod-permission-denied.yaml) that shows the "problem": 1. You can run that pod (kubectl apply -f pod-permission-denied.yaml) and 2. start a shell in the different containers (kubectl exec -ti -c [jnlp|node|gradle] permission-denied sh) to 3. run some commands like ‘id’ or ‘whoami’ within a container to show you the different user-ids the containers are running with. See attachments - Jenkinsfile-permission-denied.groovy (contains Jenkinsfile to reproduce the “problem”) - Jenkinsfile-permission-denied_output.txt (contains the stdout of the Jenkins build job) - pod-permission-denied.yaml (simplified k8s pod declaration) *Suggested solution:* Add options to set '*runAsUser*' and '*fsGroup*' at podTemplate level to be able to get rid of permission problems. 'runAsUser' and 'fsGroup' stand for the user id (uid) and match the options from podSecurityContext at pod level in kubernetes pod specification (see https://kubernetes.io/docs/tasks/configure-pod-container/security-context/) *suggested p**odTemplate from Jenkinsfile:* podTemplate(label: nodeLabel, *runAsUser: 0, fsGroup: 0*, containers: [ containerTemplate(name: 'node', image: 'node:9.3-alpine', ttyEnabled: true, command: 'cat'), containerTemplate(name: 'gradle', image: 'gradle:4.4-alpine', ttyEnabled: true, command: 'cat') ]) The pod ‘run-as-root’ (see attachment pod-run-as-root.yaml) is a minimal modified version of the pod ‘permission-denied’ (see above) with a different pod security context: securityContext: runAsUser: 0 fsGroup: 0 1. You can run that pod (kubectl apply -f pod-run-as-root.yaml) and 2. start a shell in the different containers (kubectl exec -ti -c [jnlp|node|gradle] run-as-root sh) to 3. run some commands like ‘id’ or ‘whoami’ within a container to show you that *each container is running as root now*. Therefore no permission denied problems *should* occur if this will be implemented in the kubernetes plugin. See attachments - pod-run-as-root.yaml (simplified k8s pod declaration with podSecurityContext set to root user) *What do you think? Could this be a proper way to “override” the the container specific default users?* -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/e06a9c7b-79bd-43cc-9428-498432f0fcfd%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Jenkinsfile-permission-denied.groovy
Description: Binary data
[Pipeline] echo
Unique node label e39d941e-7631-4c84-938d-112651c26dd6
[Pipeline] podTemplate
[Pipeline] {
[Pipeline] node
Still waiting to schedule task
Waiting for next available executor
Running on jenkins-slave-d5qxs-2hj4f in
/home/jenkins/workspace/selftests/access-denied-sample
[Pipeline] {
[Pipeline] stage
[Pipeline] { (simulate checkout scm)
[Pipeline] echo
no container specified, thus this code runs in container jnlp with uid 10000
(ten thousand)
[Pipeline] sh
[access-denied-sample] Running shell script
+ whoami
jenkins
+ id
uid=10000(jenkins) gid=10000(jenkins) groups=10000(jenkins)
+ ls -las
total 8
4 drwxr-xr-x 2 jenkins jenkins 4096 Jan 9 10:31 .
4 drwxr-xr-x 4 jenkins jenkins 4096 Jan 9 10:31 ..
+ echo jnlp
+ ls -las
total 12
4 drwxr-xr-x 2 jenkins jenkins 4096 Jan 9 10:31 .
4 drwxr-xr-x 4 jenkins jenkins 4096 Jan 9 10:31 ..
4 -rw-r--r-- 1 jenkins jenkins 5 Jan 9 10:31 jnlp.txt
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (node)
[Pipeline] container
[Pipeline] {
[Pipeline] echo
This code runs as root user (uid 0)
[Pipeline] sh
[access-denied-sample] Running shell script
+ whoami
root
+ id
uid=0(root) gid=0(root)
groups=0(root),1(bin),2(daemon),3(sys),4(adm),6(disk),10(wheel),11(floppy),20(dialout),26(tape),27(video)
+ ls -las
total 12
4 drwxr-xr-x 2 10000 10000 4096 Jan 9 10:31 .
4 drwxr-xr-x 4 10000 10000 4096 Jan 9 10:31 ..
4 -rw-r--r-- 1 10000 10000 5 Jan 9 10:31 jnlp.txt
+ echo node
+ ls -las
total 16
4 drwxr-xr-x 2 10000 10000 4096 Jan 9 10:31 .
4 drwxr-xr-x 4 10000 10000 4096 Jan 9 10:31 ..
4 -rw-r--r-- 1 10000 10000 5 Jan 9 10:31 jnlp.txt
4 -rw-r--r-- 1 root root 5 Jan 9 10:31 node.txt
+ cat jnlp.txt
jnlp
+ cat node.txt
node
[Pipeline] }
[Pipeline] // container
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (gradle)
[Pipeline] container
[Pipeline] {
[Pipeline] echo
This code runs as user gradle (uid 1000)
[Pipeline] sh
[access-denied-sample] Running shell script
sh: can't create
/home/jenkins/workspace/selftests/access-denied-sample@tmp/durable-92458b77/jenkins-log.txt:
Permission denied
sh: can't create
/home/jenkins/workspace/selftests/access-denied-sample@tmp/durable-92458b77/jenkins-result.txt:
Permission denied
touch:
/home/jenkins/workspace/selftests/access-denied-sample@tmp/durable-92458b77/jenkins-log.txt:
Permission denied
touch:
/home/jenkins/workspace/selftests/access-denied-sample@tmp/durable-92458b77/jenkins-log.txt:
Permission denied
touch:
/home/jenkins/workspace/selftests/access-denied-sample@tmp/durable-92458b77/jenkins-log.txt:
Permission denied
touch:
/home/jenkins/workspace/selftests/access-denied-sample@tmp/durable-92458b77/jenkins-log.txt:
Permission denied
touch:
/home/jenkins/workspace/selftests/access-denied-sample@tmp/durable-92458b77/jenkins-log.txt:
Permission denied
touch:
/home/jenkins/workspace/selftests/access-denied-sample@tmp/durable-92458b77/jenkins-log.txt:
Permission denied
touch:
/home/jenkins/workspace/selftests/access-denied-sample@tmp/durable-92458b77/jenkins-log.txt:
Permission denied
[Pipeline] }
[Pipeline] // container
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] }
[Pipeline] // podTemplate
[Pipeline] End of Pipeline
ERROR: script returned exit code -2
Finished: FAILURE
pod-permission-denied.yaml
Description: Binary data
pod-run-as-root.yaml
Description: Binary data
