GitHub user hterik added a comment to the discussion: Add a "`DockerExecutor`" inside `CeleryExecutor`
The `KubernetesExecutor` is great if you already have Kubernetes. On smaller on-prem deployments where you don't have AKS/GKE/EKS to run it for you, it's a huge task to run a cluster. For more traditionally managed servers (Proxmox, Ansible, Chef, Systemd, etc), the `CeleryExecutor` can be a more pragmatic starting point. Hence also why I'm curious how this is intended to be solved with the `EdgeExecutor`, it also exists for situations where you don't have K8s. I think it would benefit greatly by having a more integrated self-update or dryrun-with-new-libraries mechanism. Compared with other systems, they tend to have a some story around these use-cases, for example Dagster that has `celery_docker_executor`. GitHub link: https://github.com/apache/airflow/discussions/58687#discussioncomment-15083525 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
