Hi Jarek, hi Jed, Thank you both for your inputs. Following your advice Jarek, I opened PR #27420. To get my hands dirty I also implemented option 2 and opened PR #27419. I’ll be sure to go to the Slack channel for any new discussions Jed, and in the meantime, I’ll follow up on the PRs themselves.
Have a nice evening, cheers, Jules -- Jules Triomphe Data, Analytics & AI Engineer Data, Analytics & AI Swisscom +41 79 382 04 07 Swisscom (Schweiz) AG EPFL Innovation Park, Building F, 3rd Floor 1015 Lausanne Ready for the future? https://swisscom.ch/ai<https://swisscom.ch/ai/> From: Jed Cunningham <[email protected]> Date: Friday, 28 October 2022 at 23:58 To: [email protected] <[email protected]> Subject: Re: [DISCUSSION][Helm Chart] webserver_config.py in non-webserver pods Hey Jules, You could also likely just add an empty webserver conf file to your image so the various components don't try and generate it. Or, just add your complete webserver conf into the image - it doesn't have to come from the chart. Another good place for these type of conversations is in #helm-chart-official<https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapache-airflow.slack.com%2Farchives%2FC027H098M1C&data=05%7C01%7CJules.Triomphe%40swisscom.com%7Cb0b5deac58a44f128ad408dab92f8e57%7C364e5b87c1c7420d9beec35d19b557a1%7C0%7C0%7C638025911146650366%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=kf8q1Gxrz2ubzh%2B4g9OFl6A9YkXmGmUYqwA%2B2FhLwp8%3D&reserved=0>, if you aren't ready to PR something or want to weigh different options. Jed On Fri, Oct 28, 2022 at 8:19 AM Jarek Potiuk <[email protected]<mailto:[email protected]>> wrote: Glad you want to contribute! This is a great approach when you see something that you miss! I personally think it's best to go for 1) - this makes the environments between various components more consistent and might prevent other problems - I think there is a silen, never documented assumption we have that "AIRFLOW_HOME" is writeable for all airflow commands. Others might have different preferences though. But a suggestion: rather than opening discussion in the devlist (it has rather low-volume, important discussions and conclusions) use Github Discussions or even better - propose a PR and relevant people can discuss it there. Discussing the approach while seeing the code is always better and in this case the change is small enough that just `creating -> discussions -> changing after` approach. For small changes like that, the latter is usually faster than making extensive discussion upfront before attempting a PR. And not everyone in devlist is interested in such nitty-gritty-implementation-details of Helm Chart :) J. On Fri, Oct 28, 2022 at 1:40 PM <[email protected]<mailto:[email protected]>> wrote: > > Hi everyone, > > > > First timer here. > > We have readOnlyRootFilesystem enforced in our Kubernetes cluster, but we do > not want to set webserverConfig in values.yaml to be able to have proper > linting and autocompletion. Unfortunately, the chart is not adapted to this > use-case because the generated config file (if webserverConfig is set) is > mounted only in the webserver pod, and all other pods try to generate it when > they start (including init containers and sidecars). Of course, as the > filesystem is read-only, this fails, and the pods can’t start. > > > > I would like to create a PR to fix this issue. There would be 2 obvious ways: > > Add the extraVolumeMounts to init and sidecar containers (for scheduler, > triggerer, webserver and workers), so that users can mount a writable > directory to airflowHome, where the webserver_config.py file can be created > Add an option to specify a webserverConfigConfigmapName that would be mounted > to all containers at airflow_webserver_config_path and supersede any > specified webserverConfig. > > A third way would be to add logic to parse the securityContext and add a > specific empty volume to host the webserver config but recent changes > introduced or to be introduced via #25985, #25530 and #24588 makes this > cumbersome and error-prone. > > > > I would like to pursue option 2 to favor simplicity of use and to remain in > line with past decisions not to add volumes that only the main container > should have to init and sidecar containers. Does the community prefer any > other implementation (option 1, 3 or any other that I have not listed) or > have any suggestions ? > > > > Cheers, > > -- > > Jules Triomphe > > Data, Analytics & AI Engineer > Data, Analytics & AI > Swisscom > > +41 79 382 04 07 > > Swisscom (Schweiz) AG > EPFL Innovation Park, Building F, 3rd Floor > 1015 Lausanne > > Ready for the future? https://swisscom.ch/ai > >
