Hello everyone, This thread is opened due to open issue Migrate Celery/Dask/Kubernetes Executor docs to providers <https://github.com/apache/airflow/issues/33916>
*Background:* We had a discussion about extracting Celery, Kubernetes, Dask executors from core to providers (discussion thread <https://lists.apache.org/thread/kwwhz62lddygodpgr3fk4b9tthtld9do>, vote thread <https://lists.apache.org/thread/7gyw7ty9vm0pokjxq7y3b1zw6mrlxfm8>) One of the things we voted on was: Also, resulting from the discussion we will keep documentation for > available executors in Airflow (so they will still be considered as THE > executors available and will be discoverable in the same way as today). *The problem:* Airflow Core and Providers do not share the same release cycle nor cadance. This means that if we add new capabilities to executors or fix an issue which requires both code and doc update - the code will be delivered before/ahead of documentation. Both cases are not good. *My proposal:*Now that Celery, Kubernetes, Dask executors are in providers. The section of core-concept <https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/index.html> should contain only HIGH level information about the executors. It should not contain information about executors internals or how to address common problems. This information should be in the provider docs. The high level info should be short and on a level that is relevant for all executors which means it's not likely to have many changes over time. The core-concept should have links/refers for deep dive read to the provider docs. This is very similar to what we do with Notifiers <https://airflow.apache.org/docs/apache-airflow-providers/core-extensions/notifications.html> core contains high level information and a list of notifers that are linked to provider docs. WDYT? Also, resulting from the discussion we will keep documentation for available executors in Airflow (so they will still be considered as THE executors available and will be discoverable in the same way as today).Moe K8S and Celery Executors (and related) to respective providers?
