Public Thanks Jon, I will pass your suggestions to our OpenShift administrator. Hopefully there is no internal rules that will restrict us implementing it. Thank you
Natalia Fill Analyst Software Developer -----Original Message----- From: Jon Logan [mailto:[email protected]] Sent: 13 February 2020 16:50 To: [email protected] Cc: Ali, Rizwan; Endre Kovacs Subject: Re: Running Nifi on OpenShift I think this describes what you would need to do. https://cookbook.openshift.org/users-and-role-based-access-control/how-can-i-enable-an-image-to-run-as-a-set-user-id.html On Thu, Feb 13, 2020 at 11:38 AM Jon Logan <[email protected]> wrote: > That's a OpenShift security feature so that your user IDs are more > unique, and have less access between containers. I would suggest > trying to alter your range of user IDs on your cluster if you don't > want to modify the image. > > On Thu, Feb 13, 2020 at 11:09 AM Fill, Natalia <[email protected]> > wrote: > >> Public >> >> Hi Shawn, >> First I tried modifying securityContect first and the familiar error >> is appeared. I remember trying to run as user 1000 a few days ago and >> had error similar to below. OpenShift has restrictions on this value. >> >> Error creating: pods "nifi-4-" is forbidden: unable to >> validate against any security context constraint: [fsGroup: Invalid value: >> []int64{1000}: 1000 is not an allowed group >> spec.containers[0].securityContext.securityContext.runAsUser: Invalid >> value: 1000: must be in the ranges: [1000470000, 1000479999]] >> >> So if Nifi has to run as user 1000 and OpenShift only allows range >> [1000470000, 1000479999] then the issue is not resolvable in the >> current image. >> Let me know if you have other views on it. >> >> Thanks >> >> Natalia Fill >> Analyst Software Developer >> >> -----Original Message----- >> From: Fill, Natalia [mailto:[email protected]] >> Sent: 13 February 2020 14:32 >> To: [email protected]; Endre Kovacs >> Cc: Ali, Rizwan >> Subject: RE: Running Nifi on OpenShift >> >> Public >> >> Hi Shawn, >> >> Thank you for your message. I will add your suggested configs and try >> it out today. It certainly has new content not present in my yml so >> hopefully it will resolve the issue. >> >> Thanks >> >> Natalia Fill >> Analyst Software Developer >> >> -----Original Message----- >> From: Shawn Weeks [mailto:[email protected]] >> Sent: 13 February 2020 14:26 >> To: [email protected]; Endre Kovacs >> Cc: Ali, Rizwan >> Subject: Re: Running Nifi on OpenShift >> >> Your attachment didn't make it through but here are a couple of >> things to note. First of all if you try and put the ./conf directory >> in a volume you'll have to run a init container to copy the initial >> contents to the volume. Kubernetes unlike Docker does not replicate from the >> container. >> >> Here is how I did that and I'm generally available on Slack if you >> want quicker answers. >> >> initContainers: >> - name: init-nifi-conf >> image: apache/nifi:latest >> volumeMounts: >> - mountPath: "/opt/nifi/nifi-current/new-conf" >> name: nifi-conf-claim >> command: >> - sh >> - '-c' >> - '\cp /opt/nifi/nifi-current/conf/* >> /opt/nifi/nifi-current/new-conf/' >> >> The other thing you'll want to include is this to set the user and >> group id to 1000 which is what the apache image container expects >> since your not running as root. >> >> securityContext: >> runAsUser: 1000 >> runAsGroup: 1000 >> fsGroup: 1000 >> >> Here is my complete yaml. >> >> apiVersion: v1 >> kind: Service >> metadata: >> name: nifi-service >> namespace: nifi >> spec: >> clusterIP: None >> selector: >> app: nifi >> ports: >> - protocol: TCP >> port: 8080 >> type: ClusterIP >> --- >> apiVersion: networking.k8s.io/v1beta1 >> kind: Ingress >> metadata: >> name: nifi-ingress >> namespace: nifi >> spec: >> rules: >> - host: nifi.dev.example.com >> http: >> paths: >> - backend: >> serviceName: nifi-service >> servicePort: 8080 >> tls: >> - hosts: >> - nifi.dev.example.com >> secretName: nifi-ssl-cert >> --- >> apiVersion: apps/v1 >> kind: StatefulSet >> metadata: >> name: nifi-workload >> namespace: nifi >> spec: >> replicas: 3 >> podManagementPolicy: Parallel >> updateStrategy: >> type: RollingUpdate >> serviceName: nifi-service >> selector: >> matchLabels: >> app: nifi >> template: >> metadata: >> labels: >> app: nifi >> spec: >> nodeSelector: >> node-role.nifi: "true" >> securityContext: >> runAsUser: 1000 >> runAsGroup: 1000 >> fsGroup: 1000 >> initContainers: >> - name: init-nifi-conf >> image: apache/nifi:latest >> volumeMounts: >> - mountPath: "/opt/nifi/nifi-current/new-conf" >> name: nifi-conf-claim >> command: >> - sh >> - '-c' >> - '\cp /opt/nifi/nifi-current/conf/* >> /opt/nifi/nifi-current/new-conf/' >> containers: >> - image: apache/nifi:latest >> imagePullPolicy: Always >> name: nifi >> ports: >> - containerPort: 8080 >> - containerPort: 10000 >> volumeMounts: >> - mountPath: "/opt/nifi/nifi-current/conf" >> name: nifi-conf-claim >> - mountPath: "/opt/nifi/nifi-current/database_repository" >> name: nifi-db-claim >> - mountPath: "/opt/nifi/nifi-current/flowfile_repository" >> name: nifi-flow-claim >> - mountPath: "/opt/nifi/nifi-current/content_repository" >> name: nifi-content-claim >> - mountPath: "/opt/nifi/nifi-current/provenance_repository" >> name: nifi-prov-claim >> - mountPath: "/opt/nifi/nifi-current/state" >> name: nifi-state-claim >> - mountPath: "/opt/nifi/nifi-current/logs" >> name: nifi-logs-claim >> env: >> - name: MY_POD_NAME >> valueFrom: >> fieldRef: >> fieldPath: metadata.name >> - name: NIFI_CLUSTER_IS_NODE >> value: "true" >> - name: NIFI_ZK_CONNECT_STRING >> value: >> "zookeeper-0.zookeeper-headless.nifi:2181,zookeeper-1.zookeeper-headless.nifi:2181,zookeeper-2.zookeeper-headless.nifi:2181" >> - name: NIFI_CLUSTER_NODE_PROTOCOL_PORT >> value: "11443" >> - name: "NIFI_ELECTION_MAX_CANDIDATES" >> value: "3" >> - name: "NIFI_JVM_HEAP_INIT" >> value: "64g" >> - name: "NIFI_JVM_HEAP_MAX" >> value: "64g" >> - name: "NIFI_WEB_HTTP_HOST" >> value: "$(MY_POD_NAME).nifi-service" >> - name: NIFI_CLUSTER_ADDRESS >> value: "$(MY_POD_NAME).nifi-service" >> - name: NIFI_REMOTE_INPUT_HOST >> value: "$(MY_POD_NAME).nifi-service" >> volumeClaimTemplates: >> - metadata: >> name: nifi-conf-claim >> spec: >> accessModes: ["ReadWriteOnce"] >> resources: >> requests: >> storage: 10Gi >> - metadata: >> name: nifi-db-claim >> spec: >> accessModes: ["ReadWriteOnce"] >> resources: >> requests: >> storage: 10Gi >> - metadata: >> name: nifi-flow-claim >> spec: >> accessModes: ["ReadWriteOnce"] >> resources: >> requests: >> storage: 10Gi >> - metadata: >> name: nifi-content-claim >> spec: >> accessModes: ["ReadWriteOnce"] >> resources: >> requests: >> storage: 10Gi >> - metadata: >> name: nifi-prov-claim >> spec: >> accessModes: ["ReadWriteOnce"] >> resources: >> requests: >> storage: 10Gi >> - metadata: >> name: nifi-state-claim >> spec: >> accessModes: ["ReadWriteOnce"] >> resources: >> requests: >> storage: 10Gi >> - metadata: >> name: nifi-logs-claim >> spec: >> accessModes: ["ReadWriteOnce"] >> resources: >> requests: >> storage: 10Gi >> >> On 2/13/20, 3:50 AM, "Fill, Natalia" <[email protected]> wrote: >> >> Public >> >> Hi Shawn, >> >> We have internal Jenkins deployment process, which eventually >> comes down to running yml configs on OpenShift. >> I attached two yml files. One version with storage mounted and >> one without. >> The one with storage mounted expects nifi properties file, which >> I think should come from image. So there is something wrong about >> this set up. I would expect it to use default properties and don't >> which ones to give it. See my point 4 in original email below. >> The one without persistent storage mounted comes up with >> permission >> error: /opt/nifi/nifi-current/conf/sedXGg2lo: Permission denied. See >> original email for full story about this. >> I had few goes on trying to resolve it as per my original story below. >> I have read somewhere that the issue could be due to the fact >> that Nifi image tries to run as root but OpenShift doesn't allow it by >> default. >> Not sure if this is still true for the latest 1.11.1 version of >> docker image. >> If you can suggest what is wrong with these yml files or may be >> some settings need to change on OpenShift admin side it hopefully >> will help to resolve the issue. >> >> Thank you >> >> Natalia Fill >> Analyst Software Developer >> >> -----Original Message----- >> From: Shawn Weeks [mailto:[email protected]] >> Sent: 12 February 2020 21:16 >> To: [email protected]; Endre Kovacs >> Cc: Ali, Rizwan >> Subject: Re: Running Nifi on OpenShift >> >> I recognize that running NiFi on Kubernetes isn't quite as easy >> as starting it in Docker but it's also not that hard if you've worked >> with Kubernetes a bit. More than likely the issue is in your >> Kubernetes Yaml that you used to deploy NiFi with. This is separate >> than nifi.properties and would have been the config file you used in >> the command "kubectl apply -f nifi.yaml" or are you trying to deploy with >> Helm? >> >> Thanks >> Shawn >> >> On 2/12/20, 2:26 PM, "Fill, Natalia" <[email protected]> wrote: >> >> Public >> >> Hi Endre, >> >> I certainly agree with the bare metal option. The reason I >> have a specific request for OpenShift is the requirement to adhere to >> organisational architectural road map. >> I cannot agree more that it is not a single person task. I >> was working on it for few days with OpenShift administrator (on CC >> list) helping me out. >> Your links certainly give an impression that this task is not >> for faint hearted. >> >> Best regards, >> >> Natalia >> >> -----Original Message----- >> From: Endre Kovacs [mailto:[email protected]] >> Sent: 12 February 2020 19:43 >> To: [email protected] >> Subject: Re: Running Nifi on OpenShift >> >> Hi, >> >> If to make NiFi work on K8S is a beast, then to make it work >> on Openshift, is a category-5 Kaiju [1][2]. >> >> This is definitely not a few days task for a single person. >> >> Why not run NiFi just in docker (docker-compose)? Or on bare >> metal? >> >> Best regards, >> Endre >> >> >> [1] https://en.wikipedia.org/wiki/Kaiju >> [2] https://en.wikipedia.org/wiki/Pacific_Rim_(film) >> >> Sent with ProtonMail Secure Email. >> >> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ >> On Wednesday, February 12, 2020 8:14 PM, Fill, Natalia < >> [email protected]> wrote: >> >> > Public >> > >> > Hi, >> > I am trying to run Nifi pod on OpenShift for several days >> now and unfortunately unsuccessfully. >> > >> > The error that I am getting persistently is replacing >> target file >> > /opt/nifi/nifi-current/conf/nifi.properties >> > sed: couldn't open temporary file >> > /opt/nifi/nifi-current/conf/sedXGg2lo: Permission denied >> > >> > I have tried several things to resolve the issue: >> > My images are downloaded from >> https://hub.docker.com/r/apache/nifi >> > >> > 1. First I run 1.10.0 image which resulted in error above >> > >> > >> > >> > 2. Upgraded to 1.11.1 image, the error still persist >> > >> > 3. Tried wrapping the above images in my own image with >> following >> > modifications to docker file (used various paths to chmod opt/ >> > opt/nifi), still the same error >> > >> > FROM xxxRegistry/apache-nifi:1.11.1 >> > USER root >> > RUN chmod -R 777 /opt >> > USER 1000 >> > >> > 4. Mounted volume opt/nifi, but this resulted in nifi >> properties file >> > not being found, so removed volume as it overwrites Nifi paths >> > >> > 5. Involved OpenShift administrators to create privileged >> account for >> > nifi and altered my yml to use that account >> (SUPPLEMENTAL_GROUP is >> > what all our pods run under and sn_nif was created specially to >> > resolve this case) >> > >> > securityContext: >> > supplementalGroups: >> > >> > - ${SUPPLEMENTAL_GROUP} >> > serviceAccount: sn-nif >> > serviceAccountName: sn-nif >> > >> > >> > 6. Removed securityContext to ensure serviceAccount is >> used >> > >> > >> > >> > Can someone please suggest how to resolve this issue. >> Otherwise I will have to give up on Nifi as I don't have any more >> time on this project to spend on Nifi config. >> > >> > Thank you >> > >> > Natalia >> > >> > Natalia Fill >> > Analyst Software Developer >> > Legal and General Investment Management One Coleman Street, >> London, >> > EC2R 5AA >> > 020 3124 3430 >> > www.lgim.com >> > This e-mail (and any attachments) may contain privileged >> and/or confidential information. If you are not the intended >> recipient please do not disclose, copy, distribute, disseminate or >> take any action in reliance on it. If you have received this message >> in error please reply and tell us and then delete it. Should you wish >> to communicate with us by e-mail we cannot guarantee the security of any >> data outside our own computer systems. >> > >> > Any information contained in this message may be subject to >> applicable terms and conditions and must not be construed as giving >> investment advice within or outside the United Kingdom or Republic of >> Ireland. >> > >> > Telephone Conversations may be recorded for your protection >> and to >> > ensure quality of service >> > >> > Legal & General Investment Management Limited (no 2091894), >> LGIM Real >> > Assets (Operator) Limited (no 05522016), LGIM >> (International) Limited >> > (no 7716001) Legal & General Unit Trust Managers (no >> 1009418), GO ETF >> > Solutions LLP (OC329482) and LGIM Corporate Director >> Limited (no >> > 7105051) are authorised and regulated by the Financial Conduct >> > Authority. All are registered in England & Wales with a >> registered >> > office at One Coleman Street, London, EC2R 5AA >> > >> > Legal & General Assurance (Pensions Management) Limited (no >> 1006112) is authorised by the Prudential Regulation Authority and >> regulated by the Financial Conduct Authority and the Prudential Regulation >> Authority. >> It is registered in England & Wales with a registered office at One >> Coleman Street, London, EC2R 5AA. >> > >> > Legal & General Property Limited (no 2091897) is authorised >> and regulated by the Financial Conduct Authority for insurance >> mediation activities. It is registered in England & Wales with a >> registered office at One Coleman Street, London, EC2R 5AA. >> > >> > LGIM Managers (Europe) Limited is authorised and regulated >> by the Central Bank of Ireland (C173733). It is registered in the >> Republic of Ireland (no 609677) with a registered office at 33/34 Sir >> John Rogerson's Quay, Dublin 2, D02 XK09. >> > >> > Legal & General Group PLC, Registered Office One Coleman >> Street, London, EC2R 5AA. >> > >> > Registered in England no: 1417162 >> > >> > **** This email has come from the internet and has been >> scanned for >> > all viruses and potentially offensive content by >> Messagelabs on behalf >> > of Legal & General **** >> >> >> >> ________________________________________________________________________ >> *** This email has come from the internet and has been >> scanned for all viruses and potentially offensive content by >> Messagelabs on behalf of Legal & General. Please report unwanted spam >> email to [email protected] *** >> >> Please consider the environment before printing this email. >> >> >> ________________________________________________________________________ >> **** This email has come from the internet and has been >> scanned for all viruses and potentially offensive content by >> Messagelabs on behalf of Legal & General **** >> >> >> >> >> ________________________________________________________________________ >> *** This email has come from the internet and has been scanned >> for all viruses and potentially offensive content by Messagelabs on >> behalf of Legal & General. Please report unwanted spam email to >> [email protected] >> *** >> >> Please consider the environment before printing this email. >> >> >> ________________________________________________________________________ >> **** This email has come from the internet and has been scanned >> for all viruses and potentially offensive content by Messagelabs on >> behalf of Legal & General **** >> >> >> >> _____________________________________________________________________ >> ___ >> *** This email has come from the internet and has been scanned for >> all viruses and potentially offensive content by Messagelabs on >> behalf of Legal & General. Please report unwanted spam email to >> [email protected] *** >> >> Please consider the environment before printing this email. >> >> _____________________________________________________________________ >> ___ >> **** This email has come from the internet and has been scanned for >> all viruses and potentially offensive content by Messagelabs on >> behalf of Legal & General **** >> >> _____________________________________________________________________ >> ___ >> *** This email has come from the internet and has been scanned for >> all viruses and potentially offensive content by Messagelabs on >> behalf of Legal & General. Please report unwanted spam email to >> [email protected] *** >> >> Please consider the environment before printing this email. >> >> _____________________________________________________________________ >> ___ >> **** This email has come from the internet and has been scanned for >> all viruses and potentially offensive content by Messagelabs on >> behalf of Legal & General **** >> > ________________________________________________________________________ *** This email has come from the internet and has been scanned for all viruses and potentially offensive content by Messagelabs on behalf of Legal & General. Please report unwanted spam email to [email protected] *** Please consider the environment before printing this email. ________________________________________________________________________ **** This email has come from the internet and has been scanned for all viruses and potentially offensive content by Messagelabs on behalf of Legal & General ****
