Miretpl commented on PR #61065: URL: https://github.com/apache/airflow/pull/61065#issuecomment-3832076517
DISCLAIMER: I never used Kustomize, so everything that I've written can be, well, not worth much. I checked the whole proporsal and I have mixed feelings about it (even though this generated code looks like it was done specifically for the thesis - simplify the chart - instead of looking at the architecture/maintainability/development perspective now and in the future - from my perspective it is important as I'm not quite sure if the human implementation with knowledge, expirence and understanding of Kustomize limitations and bottlenecks would follow the same logic). I have mixed feelings (mainly), because of three things: 1. Reading really complex kustomization will not be easy to follow, e.g. in cases where we change something in container 2 (what is that?) to some value and in the next patch, we change volume mount 3 (what is that?) to something different. I think (if there is no more readable way of doing that), reading Helm Chart itself without Kustomize is more readable and easier to follow 2. Whole testability. I agree that testing particular components, e.g. git-sync, would be easier, but we would need to test all possible combinations between particular additional components too. It is not an issue where there is a small number of them, but with time, as Airflow is growing, this part of the project will grow too and be more and more complex (I see this as a similar case to testing compatibility between microservices - it is easy when there is e.g. 3, but if there is 8, it is getting much more complex). 3. Usage itself by users - now, if someone wants to deploy Airflow on K8S, it is rather simple - create a custom values.yaml, change a couple of things, use the package from artifacthub, and you are good to go. With Kustomize (if we were to go with everything there), it becomes more complex as it requires more setup to prepare and maintain --- Personally, when Kustomize was proposed on the devlist and removing the complexity itself, I started to think in what direction Helm Chart should go. One of the first things which came to my mind was that maybe Helm Chart should be only Airflow specific. I mean by that that postgres, pgbouncer, statsd, etc., should not be part of the chart itself (and they could be moved to Kustomize). I agree with Jens (for now, at least) that some combination, probably, is the best way to go, but on the other hand, I need to dig deeper into how Kustomize works (basically get some experience with it), as I'm not sure about it yet. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
