[
https://issues.apache.org/jira/browse/SUBMARINE-1229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
cdmikechen updated SUBMARINE-1229:
----------------------------------
Description:
Minio initContainer pod always shows that it cannot end correctly. I've seen
the log in the pod and found the following errors:
{code}
mc: <ERROR> Unable to save new mc config. mkdir /.mc: permission denied.
mc: <ERROR> Unable to save new mc config. mkdir /.mc: permission denied.
mc: <ERROR> Unable to save new mc config. mkdir /.mc: permission denied.
mc: <ERROR> Unable to save new mc config. mkdir /.mc: permission denied.
mc: <ERROR> Unable to save new mc config. mkdir /.mc: permission denied.
mc: <ERROR> Unable to save new mc config. mkdir /.mc: permission denied.
mc: <ERROR> Unable to save new mc config. mkdir /.mc: permission denied.
mc: <ERROR> Unable to save new mc config. mkdir /.mc: permission denied.
mc: <ERROR> Unable to save new mc config. mkdir /.mc: permission denied.
mc: <ERROR> Unable to save new mc config. mkdir /.mc: permission denied.
mc: <ERROR> Unable to save new mc config. mkdir /.mc: permission denied.
mc: <ERROR> Unable to save new mc config. mkdir /.mc: permission denied.
mc: <ERROR> Unable to save new mc config. mkdir /.mc: permission denied.
{code}
I checked resource yaml and found that openshift add a *runasuser* in
deployment:
{code}
initContainers:
- resources: {}
terminationMessagePath: /dev/termination-log
name: submarine-server-initcontainer
command:
- /bin/bash
- '-c'
- >-
cnt=0; while ! /bin/bash -c 'mc config host add minio
http://submarine-minio-service:9000 submarine_minio submarine_minio'
2>&1; do sleep 15; ((cnt=cnt+1)); if [ $cnt -eq 80 ];then echo 'ERROR:
wait too long for minio pod'; exit 1; fi; done; if /bin/bash -c 'mc ls
minio/submarine' >/dev/null 2>&1; then echo 'Bucket minio/submarine
already exists, skipping creation.'; else /bin/bash -c 'mc mb
minio/submarine'; fi;
securityContext:
capabilities:
drop:
- KILL
- MKNOD
- SETGID
- SETUID
runAsUser: 1000790000
imagePullPolicy: Always
volumeMounts:
- name: submarine-server-token-bnj4d
readOnly: true
mountPath: /var/run/secrets/kubernetes.io/serviceaccount
terminationMessagePolicy: File
image:
'syharbor.sypesco.com:5000/platform/minio/mc:RELEASE.2021-12-29T06-52-55Z'
{code}
Openshift will add *runAsUser* with a custom uid under *securityContext* when
we didn't add *RunAsAny* to serviceaccount.
was:
Minio initContainer pod always shows that it cannot end correctly. I've seen
the log in the pod and found the following errors:
{code}
mc: <ERROR> Unable to save new mc config. mkdir /.mc: permission denied.
mc: <ERROR> Unable to save new mc config. mkdir /.mc: permission denied.
mc: <ERROR> Unable to save new mc config. mkdir /.mc: permission denied.
mc: <ERROR> Unable to save new mc config. mkdir /.mc: permission denied.
mc: <ERROR> Unable to save new mc config. mkdir /.mc: permission denied.
mc: <ERROR> Unable to save new mc config. mkdir /.mc: permission denied.
mc: <ERROR> Unable to save new mc config. mkdir /.mc: permission denied.
mc: <ERROR> Unable to save new mc config. mkdir /.mc: permission denied.
mc: <ERROR> Unable to save new mc config. mkdir /.mc: permission denied.
mc: <ERROR> Unable to save new mc config. mkdir /.mc: permission denied.
mc: <ERROR> Unable to save new mc config. mkdir /.mc: permission denied.
mc: <ERROR> Unable to save new mc config. mkdir /.mc: permission denied.
mc: <ERROR> Unable to save new mc config. mkdir /.mc: permission denied.
{code}
I checked resource yaml and found that openshift add a *runasuser* in
deployment:
{code}
initContainers:
- resources: {}
terminationMessagePath: /dev/termination-log
name: submarine-server-initcontainer
command:
- /bin/bash
- '-c'
- >-
cnt=0; while ! /bin/bash -c 'mc config host add minio
http://submarine-minio-service:9000 submarine_minio submarine_minio'
2>&1; do sleep 15; ((cnt=cnt+1)); if [ $cnt -eq 80 ];then echo 'ERROR:
wait too long for minio pod'; exit 1; fi; done; if /bin/bash -c 'mc ls
minio/submarine' >/dev/null 2>&1; then echo 'Bucket minio/submarine
already exists, skipping creation.'; else /bin/bash -c 'mc mb
minio/submarine'; fi;
securityContext:
capabilities:
drop:
- KILL
- MKNOD
- SETGID
- SETUID
runAsUser: 1000790000
imagePullPolicy: Always
volumeMounts:
- name: submarine-server-token-bnj4d
readOnly: true
mountPath: /var/run/secrets/kubernetes.io/serviceaccount
terminationMessagePolicy: File
image:
'syharbor.sypesco.com:5000/platform/minio/mc:RELEASE.2021-12-29T06-52-55Z'
{code}
> Minio initContainer pod (mlflow and server) not working properly in openshift
> -----------------------------------------------------------------------------
>
> Key: SUBMARINE-1229
> URL: https://issues.apache.org/jira/browse/SUBMARINE-1229
> Project: Apache Submarine
> Issue Type: Bug
> Components: Cloud-native Deployment
> Reporter: cdmikechen
> Priority: Major
> Fix For: 0.7.0
>
>
> Minio initContainer pod always shows that it cannot end correctly. I've seen
> the log in the pod and found the following errors:
> {code}
> mc: <ERROR> Unable to save new mc config. mkdir /.mc: permission denied.
> mc: <ERROR> Unable to save new mc config. mkdir /.mc: permission denied.
> mc: <ERROR> Unable to save new mc config. mkdir /.mc: permission denied.
> mc: <ERROR> Unable to save new mc config. mkdir /.mc: permission denied.
> mc: <ERROR> Unable to save new mc config. mkdir /.mc: permission denied.
> mc: <ERROR> Unable to save new mc config. mkdir /.mc: permission denied.
> mc: <ERROR> Unable to save new mc config. mkdir /.mc: permission denied.
> mc: <ERROR> Unable to save new mc config. mkdir /.mc: permission denied.
> mc: <ERROR> Unable to save new mc config. mkdir /.mc: permission denied.
> mc: <ERROR> Unable to save new mc config. mkdir /.mc: permission denied.
> mc: <ERROR> Unable to save new mc config. mkdir /.mc: permission denied.
> mc: <ERROR> Unable to save new mc config. mkdir /.mc: permission denied.
> mc: <ERROR> Unable to save new mc config. mkdir /.mc: permission denied.
> {code}
> I checked resource yaml and found that openshift add a *runasuser* in
> deployment:
> {code}
> initContainers:
> - resources: {}
> terminationMessagePath: /dev/termination-log
> name: submarine-server-initcontainer
> command:
> - /bin/bash
> - '-c'
> - >-
> cnt=0; while ! /bin/bash -c 'mc config host add minio
> http://submarine-minio-service:9000 submarine_minio submarine_minio'
> 2>&1; do sleep 15; ((cnt=cnt+1)); if [ $cnt -eq 80 ];then echo 'ERROR:
> wait too long for minio pod'; exit 1; fi; done; if /bin/bash -c 'mc ls
> minio/submarine' >/dev/null 2>&1; then echo 'Bucket minio/submarine
> already exists, skipping creation.'; else /bin/bash -c 'mc mb
> minio/submarine'; fi;
> securityContext:
> capabilities:
> drop:
> - KILL
> - MKNOD
> - SETGID
> - SETUID
> runAsUser: 1000790000
> imagePullPolicy: Always
> volumeMounts:
> - name: submarine-server-token-bnj4d
> readOnly: true
> mountPath: /var/run/secrets/kubernetes.io/serviceaccount
> terminationMessagePolicy: File
> image:
> 'syharbor.sypesco.com:5000/platform/minio/mc:RELEASE.2021-12-29T06-52-55Z'
> {code}
> Openshift will add *runAsUser* with a custom uid under *securityContext* when
> we didn't add *RunAsAny* to serviceaccount.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@submarine.apache.org
For additional commands, e-mail: dev-h...@submarine.apache.org
