FloChehab commented on pull request #12010: URL: https://github.com/apache/airflow/pull/12010#issuecomment-757358852
> We run a set of integration tests for Kubernetes for each PR. To do it, we run a KIND cluster on Github Action, and in it, we deploy Airflow using the Helm Chart. After deployment, we perform a few simple operations, e.g. run DAG to check if everything works. After implementation, we perform a few simple operations, e.g. run DAG to check if everything works. To be able to test the Helm Chart with more confidence, I want to work on system tests that deploy infrastructure with Terraform and then install Airflow with Helm Chart. And then it does some basic operations. This feels quite ambitious and I personally thought the CI with a KIND cluster would be sufficient (for testing the integration with "external" services, you could start them separately in the same cluster) and relatively time & cost effective. To be honest, at the moment I am not sure I would give a hand on such a subject in the coming months ; I am trying to do bit less infrastructure related work these days :) (but I'll keep contributing on the helm chart). > Some changes would be useful in order for this change to be accepted. First of all, we should delete references to Google solutions, but the ability to configure your own annotations sounds interesting. > For example, if you want to use Workflow Identity then you need to annotate Service Account, so we don't want to support just one annotation, but we've added generic support for any annotation. > See: > #11131 > #11387 > > Before this change was posted to the community, the change looked like this: >  > It's not something we want to accept, so we had to rewrite it. Thanks for the info! I'll look into this in the coming weeks. > Additional facilities, if needed, can be deployed independently of Helm Chart. I was thinking about this, and I feel it's pretty annoying and harder to do / maintain in a production setting; unless maybe if we go for some subchart mechanism (which I am not familiar with) or something else? I link when related things are bounded together; that's why I added for example the ability to create configmaps & secrets (for airflow) from within this chart: https://github.com/apache/airflow/pull/12164. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
