jscheffl commented on PR #61065: URL: https://github.com/apache/airflow/pull/61065#issuecomment-3801346916
Not sure if I am really getting a fan of this proposal. Yes I understand you feel like Jinja is horrible to maintain. I think it is not. On the other hand the real large file tree with scattered pieces of parts of the deployment also make it complex to get an overview about results. And changes need to be made in multiple files if features are developed. So rather you change complexity if Jinja with a huge file tree. What is missing in the proposal is the test-ability. Also I think it is cool to use Kustomize for specific features - but if you extract all features then cross dependencies again will introduce complexity, like if Feature A + B is needed then config X need to be made as cross product. Mainly the proposal you made took the worker which is the most complex beast in template that I found in whole deployment. This for sure need to be cleaned and made simpler. And yes, a log of conditions then also can be removed. I'd propose rather getting into a pareto principle of 80/20 to say that most common core complexity is handled in Helm and therefore like today we have also the complexity covered with tests. And all 20% sugar that is adding a lot of complexity can be Kustomize. We can debate where the border of 80/20 is (or if it is 70/30) - but stiching a large fileset in Kustomize on top of a very static template just changes to the other extreme in my view. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
