Public
Hi Shawn,
In addition to the below issue with user constraints, I tried mapping volumes
as you suggested and copy config content (had to make it OpenShift friendly). I
think it doesn't like volumeMoutn inside initContainers part. The volume is
shown as mounted on OpenShift. The error I get now is:
Unable to mount volumes for pod
"nifi-1-wgx69_eai-platform-d00(db6366cd-4e97-11ea-a55f-0a95d6eb42fb)": timeout
expired waiting for volumes to attach or mount for pod
"eai-platform-d00"/"nifi-1-wgx69". list of unmounted volumes=[nificonf-mount
apache-nifi-token-6rn6l]. list of unattached volumes=[nificonf-mount
apache-nifi-token-6rn6l]
My yml is below. If you can spot obvious issues in this yml, then please let me
know,
apiVersion: v1
kind: Template
metadata:
creationTimestamp: null
name: nifi
annotations:
description: Template for Apache NiFi flows
tags: app,nifi
iconClass: icon-apache
parameters:
- description: Namespace
name: NAMESPACE
required: true
- description: Namespace which contains team images
name: DOCKER_TEAM_NAMESPACE
required: true
- description: Hostname and port of the docker registry to pull from
name: DOCKER_PARENT_REGISTRY
required: true
- description: Group ID of the container user
name: SUPPLEMENTAL_GROUP
required: true
- description: Fully qualified hostname for http service route
name: HOSTNAME_HTTP
required: true
- description: Container memory allocation
name: MEM_LIMIT
required: true
- description: Container memory allocation
name: MEM_REQUEST
required: true
- name: APPLICATION_NAME
description: "The name for the application."
value: "nifi"
required: true
- name: CPU_REQUEST
description: "Minimum amount of CPU that the container may consume. Unit (m)
has to be included."
value: "600m"
required: true
- name: CPU_LIMIT
description: "Maximum amount of CPU that the container may consume. Unit (m)
has to be included."
value: "1000m"
required: true
- name: MEMORY_REQUEST
description: "Minimum amount of memory that the container may consume. Unit
(Mi, Gi) has to be included."
value: "900Mi"
required: true
- name: MEMORY_LIMIT
description: "Maximum amount of memory that the container may consume. Unit
(Mi, Gi) has to be included."
value: "1300Mi"
required: true
- name: NIFI_JAVA_XMS
description: "Startup heap to be requested by the JVM."
value: "-Xms512m"
required: true
- name: NIFI_JAVA_XMX
description: "Maximum heap to be requested by the JVM."
value: "-Xms512m"
required: true
- name: NIFI_TIMEZONE
description: "Timezone to be used by the JVM."
value: "-Duser.timezone=Europe/London"
required: true
- name: APPDATA_BASE_PATH
description: Mount path for appdata
required: true
labels:
template: nifi
message: |
Apache nifi should be available shortly at: http://${HOSTNAME_HTTP}
objects:
- kind: Service
apiVersion: v1
spec:
ports:
- name: web
port: 8080
targetPort: 8080
selector:
deploymentConfig: ${APPLICATION_NAME}
metadata:
name: ${APPLICATION_NAME}
labels:
app: ${APPLICATION_NAME}
annotations:
description: The web server's http port.
- kind: Route
apiVersion: v1
id: "${APPLICATION_NAME}-http"
metadata:
name: ${APPLICATION_NAME}
labels:
app: ${APPLICATION_NAME}
annotations:
description: Route for application's http service.
spec:
host: ${HOSTNAME_HTTP}
to:
name: ${APPLICATION_NAME}
- kind: ImageStream
apiVersion: v1
metadata:
name: ${APPLICATION_NAME}
labels:
app: ${APPLICATION_NAME}
- kind: DeploymentConfig
apiVersion: v1
metadata:
creationTimestamp: null
name: ${APPLICATION_NAME}
labels:
app: platform
serviceunit: ${APPLICATION_NAME}
spec:
replicas: 1
selector:
name: ${APPLICATION_NAME}
strategy:
resources: {}
rollingParams:
intervalSeconds: 1
maxSurge: 25%
maxUnavailable: 25%
timeoutSeconds: 600
updatePeriodSeconds: 1
type: Rolling
template:
metadata:
creationTimestamp: null
name: ${APPLICATION_NAME}
labels:
name: ${APPLICATION_NAME}
spec:
initContainers:
- name: initconf
image:
${DOCKER_PARENT_REGISTRY}/${DOCKER_TEAM_NAMESPACE}/platform-nifi:latest
volumeMounts:
- mountPath: "/opt/nifi/nifi-current/conf"
name: nificonf-mount
command:
- sh
- '-c'
- '\cp /opt/nifi/nifi-current/conf/* /opt/nifi/nifi-current/conf/'
containers:
- env:
- name: "NIFI_JAVA_XMS"
value: ${NIFI_JAVA_XMS}
- name: "NIFI_JAVA_XMX"
value: ${NIFI_JAVA_XMX}
- name: "NIFI_TIMEZONE"
value: ${NIFI_TIMEZONE}
image:
${DOCKER_PARENT_REGISTRY}/${DOCKER_TEAM_NAMESPACE}/platform-nifi:latest
imagePullPolicy: Always
name: ${APPLICATION_NAME}
ports:
- name: http
containerPort: 8080
protocol: TCP
resources:
limits:
cpu: ${CPU_LIMIT}
memory: ${MEMORY_LIMIT}
requests:
cpu: ${CPU_REQUEST}
memory: ${MEMORY_REQUEST}
volumeMounts:
- mountPath: "/opt/nifi/nifi-current/conf"
name: nificonf-mount
securityContext:
supplementalGroups:
- ${SUPPLEMENTAL_GROUP}
serviceAccount: apache-nifi
serviceAccountName: apache-nifi
terminationGracePeriodSeconds: 30
volumes:
- name: nificonf-mount
persistentVolumeClaim:
claimName: nificonf-claim
triggers: {}
- apiVersion: v1
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: nificonf-claim
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 1Gi
selector:
matchLabels:
function: ${NAMESPACE}-nificonf
Thanks
Natalia Fill
Analyst Software Developer
-----Original Message-----
From: Fill, Natalia
Sent: 13 February 2020 16:09
To: [email protected]; Endre Kovacs
Cc: Ali, Rizwan
Subject: RE: Running Nifi on OpenShift
Public
Hi Shawn,
First I tried modifying securityContect first and the familiar error is
appeared. I remember trying to run as user 1000 a few days ago and had error
similar to below. OpenShift has restrictions on this value.
Error creating: pods "nifi-4-" is forbidden: unable to validate against
any security context constraint: [fsGroup: Invalid value: []int64{1000}: 1000
is not an allowed group
spec.containers[0].securityContext.securityContext.runAsUser: Invalid value:
1000: must be in the ranges: [1000470000, 1000479999]]
So if Nifi has to run as user 1000 and OpenShift only allows range [1000470000,
1000479999] then the issue is not resolvable in the current image.
Let me know if you have other views on it.
Thanks
Natalia Fill
Analyst Software Developer
-----Original Message-----
From: Fill, Natalia [mailto:[email protected]]
Sent: 13 February 2020 14:32
To: [email protected]; Endre Kovacs
Cc: Ali, Rizwan
Subject: RE: Running Nifi on OpenShift
Public
Hi Shawn,
Thank you for your message. I will add your suggested configs and try it out
today. It certainly has new content not present in my yml so hopefully it will
resolve the issue.
Thanks
Natalia Fill
Analyst Software Developer
-----Original Message-----
From: Shawn Weeks [mailto:[email protected]]
Sent: 13 February 2020 14:26
To: [email protected]; Endre Kovacs
Cc: Ali, Rizwan
Subject: Re: Running Nifi on OpenShift
Your attachment didn't make it through but here are a couple of things to note.
First of all if you try and put the ./conf directory in a volume you'll have to
run a init container to copy the initial contents to the volume. Kubernetes
unlike Docker does not replicate from the container.
Here is how I did that and I'm generally available on Slack if you want quicker
answers.
initContainers:
- name: init-nifi-conf
image: apache/nifi:latest
volumeMounts:
- mountPath: "/opt/nifi/nifi-current/new-conf"
name: nifi-conf-claim
command:
- sh
- '-c'
- '\cp /opt/nifi/nifi-current/conf/*
/opt/nifi/nifi-current/new-conf/'
The other thing you'll want to include is this to set the user and group id to
1000 which is what the apache image container expects since your not running as
root.
securityContext:
runAsUser: 1000
runAsGroup: 1000
fsGroup: 1000
Here is my complete yaml.
apiVersion: v1
kind: Service
metadata:
name: nifi-service
namespace: nifi
spec:
clusterIP: None
selector:
app: nifi
ports:
- protocol: TCP
port: 8080
type: ClusterIP
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: nifi-ingress
namespace: nifi
spec:
rules:
- host: nifi.dev.example.com
http:
paths:
- backend:
serviceName: nifi-service
servicePort: 8080
tls:
- hosts:
- nifi.dev.example.com
secretName: nifi-ssl-cert
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: nifi-workload
namespace: nifi
spec:
replicas: 3
podManagementPolicy: Parallel
updateStrategy:
type: RollingUpdate
serviceName: nifi-service
selector:
matchLabels:
app: nifi
template:
metadata:
labels:
app: nifi
spec:
nodeSelector:
node-role.nifi: "true"
securityContext:
runAsUser: 1000
runAsGroup: 1000
fsGroup: 1000
initContainers:
- name: init-nifi-conf
image: apache/nifi:latest
volumeMounts:
- mountPath: "/opt/nifi/nifi-current/new-conf"
name: nifi-conf-claim
command:
- sh
- '-c'
- '\cp /opt/nifi/nifi-current/conf/*
/opt/nifi/nifi-current/new-conf/'
containers:
- image: apache/nifi:latest
imagePullPolicy: Always
name: nifi
ports:
- containerPort: 8080
- containerPort: 10000
volumeMounts:
- mountPath: "/opt/nifi/nifi-current/conf"
name: nifi-conf-claim
- mountPath: "/opt/nifi/nifi-current/database_repository"
name: nifi-db-claim
- mountPath: "/opt/nifi/nifi-current/flowfile_repository"
name: nifi-flow-claim
- mountPath: "/opt/nifi/nifi-current/content_repository"
name: nifi-content-claim
- mountPath: "/opt/nifi/nifi-current/provenance_repository"
name: nifi-prov-claim
- mountPath: "/opt/nifi/nifi-current/state"
name: nifi-state-claim
- mountPath: "/opt/nifi/nifi-current/logs"
name: nifi-logs-claim
env:
- name: MY_POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: NIFI_CLUSTER_IS_NODE
value: "true"
- name: NIFI_ZK_CONNECT_STRING
value:
"zookeeper-0.zookeeper-headless.nifi:2181,zookeeper-1.zookeeper-headless.nifi:2181,zookeeper-2.zookeeper-headless.nifi:2181"
- name: NIFI_CLUSTER_NODE_PROTOCOL_PORT
value: "11443"
- name: "NIFI_ELECTION_MAX_CANDIDATES"
value: "3"
- name: "NIFI_JVM_HEAP_INIT"
value: "64g"
- name: "NIFI_JVM_HEAP_MAX"
value: "64g"
- name: "NIFI_WEB_HTTP_HOST"
value: "$(MY_POD_NAME).nifi-service"
- name: NIFI_CLUSTER_ADDRESS
value: "$(MY_POD_NAME).nifi-service"
- name: NIFI_REMOTE_INPUT_HOST
value: "$(MY_POD_NAME).nifi-service"
volumeClaimTemplates:
- metadata:
name: nifi-conf-claim
spec:
accessModes: ["ReadWriteOnce"]
resources:
requests:
storage: 10Gi
- metadata:
name: nifi-db-claim
spec:
accessModes: ["ReadWriteOnce"]
resources:
requests:
storage: 10Gi
- metadata:
name: nifi-flow-claim
spec:
accessModes: ["ReadWriteOnce"]
resources:
requests:
storage: 10Gi
- metadata:
name: nifi-content-claim
spec:
accessModes: ["ReadWriteOnce"]
resources:
requests:
storage: 10Gi
- metadata:
name: nifi-prov-claim
spec:
accessModes: ["ReadWriteOnce"]
resources:
requests:
storage: 10Gi
- metadata:
name: nifi-state-claim
spec:
accessModes: ["ReadWriteOnce"]
resources:
requests:
storage: 10Gi
- metadata:
name: nifi-logs-claim
spec:
accessModes: ["ReadWriteOnce"]
resources:
requests:
storage: 10Gi
On 2/13/20, 3:50 AM, "Fill, Natalia" <[email protected]> wrote:
Public
Hi Shawn,
We have internal Jenkins deployment process, which eventually comes down to
running yml configs on OpenShift.
I attached two yml files. One version with storage mounted and one without.
The one with storage mounted expects nifi properties file, which I think
should come from image. So there is something wrong about this set up. I would
expect it to use default properties and don't which ones to give it. See my
point 4 in original email below.
The one without persistent storage mounted comes up with permission error:
/opt/nifi/nifi-current/conf/sedXGg2lo: Permission denied. See original email
for full story about this.
I had few goes on trying to resolve it as per my original story below.
I have read somewhere that the issue could be due to the fact that Nifi
image tries to run as root but OpenShift doesn't allow it by default. Not sure
if this is still true for the latest 1.11.1 version of docker image.
If you can suggest what is wrong with these yml files or may be some
settings need to change on OpenShift admin side it hopefully will help to
resolve the issue.
Thank you
Natalia Fill
Analyst Software Developer
-----Original Message-----
From: Shawn Weeks [mailto:[email protected]]
Sent: 12 February 2020 21:16
To: [email protected]; Endre Kovacs
Cc: Ali, Rizwan
Subject: Re: Running Nifi on OpenShift
I recognize that running NiFi on Kubernetes isn't quite as easy as starting
it in Docker but it's also not that hard if you've worked with Kubernetes a
bit. More than likely the issue is in your Kubernetes Yaml that you used to
deploy NiFi with. This is separate than nifi.properties and would have been the
config file you used in the command "kubectl apply -f nifi.yaml" or are you
trying to deploy with Helm?
Thanks
Shawn
On 2/12/20, 2:26 PM, "Fill, Natalia" <[email protected]> wrote:
Public
Hi Endre,
I certainly agree with the bare metal option. The reason I have a
specific request for OpenShift is the requirement to adhere to organisational
architectural road map.
I cannot agree more that it is not a single person task. I was working
on it for few days with OpenShift administrator (on CC list) helping me out.
Your links certainly give an impression that this task is not for faint
hearted.
Best regards,
Natalia
-----Original Message-----
From: Endre Kovacs [mailto:[email protected]]
Sent: 12 February 2020 19:43
To: [email protected]
Subject: Re: Running Nifi on OpenShift
Hi,
If to make NiFi work on K8S is a beast, then to make it work on
Openshift, is a category-5 Kaiju [1][2].
This is definitely not a few days task for a single person.
Why not run NiFi just in docker (docker-compose)? Or on bare metal?
Best regards,
Endre
[1] https://en.wikipedia.org/wiki/Kaiju
[2] https://en.wikipedia.org/wiki/Pacific_Rim_(film)
Sent with ProtonMail Secure Email.
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Wednesday, February 12, 2020 8:14 PM, Fill, Natalia
<[email protected]> wrote:
> Public
>
> Hi,
> I am trying to run Nifi pod on OpenShift for several days now and
unfortunately unsuccessfully.
>
> The error that I am getting persistently is replacing target file
> /opt/nifi/nifi-current/conf/nifi.properties
> sed: couldn't open temporary file
> /opt/nifi/nifi-current/conf/sedXGg2lo: Permission denied
>
> I have tried several things to resolve the issue:
> My images are downloaded from https://hub.docker.com/r/apache/nifi
>
> 1. First I run 1.10.0 image which resulted in error above
>
>
>
> 2. Upgraded to 1.11.1 image, the error still persist
>
> 3. Tried wrapping the above images in my own image with following
> modifications to docker file (used various paths to chmod opt/
> opt/nifi), still the same error
>
> FROM xxxRegistry/apache-nifi:1.11.1
> USER root
> RUN chmod -R 777 /opt
> USER 1000
>
> 4. Mounted volume opt/nifi, but this resulted in nifi properties file
> not being found, so removed volume as it overwrites Nifi paths
>
> 5. Involved OpenShift administrators to create privileged account for
> nifi and altered my yml to use that account (SUPPLEMENTAL_GROUP is
> what all our pods run under and sn_nif was created specially to
> resolve this case)
>
> securityContext:
> supplementalGroups:
>
> - ${SUPPLEMENTAL_GROUP}
> serviceAccount: sn-nif
> serviceAccountName: sn-nif
>
>
> 6. Removed securityContext to ensure serviceAccount is used
>
>
>
> Can someone please suggest how to resolve this issue. Otherwise I
will have to give up on Nifi as I don't have any more time on this project to
spend on Nifi config.
>
> Thank you
>
> Natalia
>
> Natalia Fill
> Analyst Software Developer
> Legal and General Investment Management One Coleman Street, London,
> EC2R 5AA
> 020 3124 3430
> www.lgim.com
> This e-mail (and any attachments) may contain privileged and/or
confidential information. If you are not the intended recipient please do not
disclose, copy, distribute, disseminate or take any action in reliance on it.
If you have received this message in error please reply and tell us and then
delete it. Should you wish to communicate with us by e-mail we cannot guarantee
the security of any data outside our own computer systems.
>
> Any information contained in this message may be subject to
applicable terms and conditions and must not be construed as giving investment
advice within or outside the United Kingdom or Republic of Ireland.
>
> Telephone Conversations may be recorded for your protection and to
> ensure quality of service
>
> Legal & General Investment Management Limited (no 2091894), LGIM Real
> Assets (Operator) Limited (no 05522016), LGIM (International) Limited
> (no 7716001) Legal & General Unit Trust Managers (no 1009418), GO ETF
> Solutions LLP (OC329482) and LGIM Corporate Director Limited (no
> 7105051) are authorised and regulated by the Financial Conduct
> Authority. All are registered in England & Wales with a registered
> office at One Coleman Street, London, EC2R 5AA
>
> Legal & General Assurance (Pensions Management) Limited (no 1006112)
is authorised by the Prudential Regulation Authority and regulated by the
Financial Conduct Authority and the Prudential Regulation Authority. It is
registered in England & Wales with a registered office at One Coleman Street,
London, EC2R 5AA.
>
> Legal & General Property Limited (no 2091897) is authorised and
regulated by the Financial Conduct Authority for insurance mediation
activities. It is registered in England & Wales with a registered office at One
Coleman Street, London, EC2R 5AA.
>
> LGIM Managers (Europe) Limited is authorised and regulated by the
Central Bank of Ireland (C173733). It is registered in the Republic of Ireland
(no 609677) with a registered office at 33/34 Sir John Rogerson's Quay, Dublin
2, D02 XK09.
>
> Legal & General Group PLC, Registered Office One Coleman Street,
London, EC2R 5AA.
>
> Registered in England no: 1417162
>
> **** This email has come from the internet and has been scanned for
> all viruses and potentially offensive content by Messagelabs on
behalf
> of Legal & General ****
________________________________________________________________________
*** This email has come from the internet and has been scanned for all
viruses and potentially offensive content by Messagelabs on behalf of Legal &
General. Please report unwanted spam email to [email protected] ***
Please consider the environment before printing this email.
________________________________________________________________________
**** This email has come from the internet and has been scanned for all
viruses and potentially offensive content by Messagelabs on behalf of Legal &
General ****
________________________________________________________________________
*** This email has come from the internet and has been scanned for all
viruses and potentially offensive content by Messagelabs on behalf of Legal &
General. Please report unwanted spam email to [email protected] ***
Please consider the environment before printing this email.
________________________________________________________________________
**** This email has come from the internet and has been scanned for all
viruses and potentially offensive content by Messagelabs on behalf of Legal &
General ****
________________________________________________________________________
*** This email has come from the internet and has been scanned for all viruses
and potentially offensive content by Messagelabs on behalf of Legal & General.
Please report unwanted spam email to [email protected] ***
Please consider the environment before printing this email.
________________________________________________________________________
**** This email has come from the internet and has been scanned for all viruses
and potentially offensive content by Messagelabs on behalf of Legal & General
****
________________________________________________________________________
*** This email has come from the internet and has been scanned for all viruses
and potentially offensive content by Messagelabs on behalf of Legal & General.
Please report unwanted spam email to [email protected] ***
Please consider the environment before printing this email.
________________________________________________________________________
**** This email has come from the internet and has been scanned for all viruses
and potentially offensive content by Messagelabs on behalf of Legal & General
****
