o-nikolas commented on code in PR #48528:
URL: https://github.com/apache/airflow/pull/48528#discussion_r2021448704
##########
airflow-core/docs/administration-and-deployment/production-deployment.rst:
##########
@@ -56,11 +56,8 @@ Once that is done, you can run -
Multi-Node Cluster
==================
-Airflow uses
:class:`~airflow.executors.sequential_executor.SequentialExecutor` by default.
However, by its
-nature, the user is limited to executing at most one task at a time.
``Sequential Executor`` also pauses
-the scheduler when it runs a task, hence it is not recommended in a production
setup. You should use the
-:class:`~airflow.executors.local_executor.LocalExecutor` for a single machine.
-For a multi-node setup, you should use the :doc:`Kubernetes executor
<apache-airflow-providers-cncf-kubernetes:kubernetes_executor>` or
+Airflow uses :class:`~airflow.executors.local_executor.LocalExecutor` by
default. For a multi-node setup,
+you should use the :doc:`Kubernetes executor
<apache-airflow-providers-cncf-kubernetes:kubernetes_executor>` or
the :doc:`Celery executor <apache-airflow-providers-celery:celery_executor>`.
Review Comment:
We have many more executors since this was written, I wonder if we should
update that. Out of scope for this PR though, just noting it as I thought of it!
##########
airflow-core/docs/core-concepts/executor/index.rst:
##########
@@ -46,14 +46,17 @@ If you want to check which executor is currently set, you
can use the ``airflow
.. code-block:: bash
$ airflow config get-value core executor
- SequentialExecutor
+ LocalExecutor
Executor Types
--------------
-There are two types of executors - those that run tasks *locally* (inside the
``scheduler`` process), and those that run their tasks *remotely* (usually via
a pool of *workers*). Airflow comes configured with the ``SequentialExecutor``
by default, which is a local executor, and the simplest option for execution.
However, the ``SequentialExecutor`` is not suitable for production since it
does not allow for parallel task running and due to that, some Airflow features
(e.g. running sensors) will not work properly. You should instead use the
``LocalExecutor`` for small, single-machine production installations, or one of
the remote executors for a multi-machine/cloud installation.
+There is only one type of executor that runs tasks *locally* (inside the
``scheduler`` process),
+and those that run their tasks *remotely* (usually via a pool of *workers*).
Airflow comes configured with the ``LocalExecutor`` by default, which is a
local executor, and the simplest option for execution.
Review Comment:
There is only one "core" executor that does. But anyone can write an
executor using the public interface and many of those could be local. I don't
see the reason to specifically call out only one here, I think it's more
misleading than it is helpful?
```suggestion
There are two types of executors - those that run tasks *locally* (inside
the ``scheduler`` process),
and those that run their tasks *remotely* (usually via a pool of *workers*).
Airflow comes configured with the ``LocalExecutor`` by default, which is a
local executor, and the simplest option for execution.
```
##########
airflow-core/src/airflow/cli/commands/standalone_command.py:
##########
@@ -174,12 +174,8 @@ def calculate_env(self):
# Make sure we're using a local executor flavour
executor_class, _ = ExecutorLoader.import_default_executor_cls()
if not executor_class.is_local:
- if "sqlite" in conf.get("database", "sql_alchemy_conn"):
- self.print_output("standalone", "Forcing executor to
SequentialExecutor")
- env["AIRFLOW__CORE__EXECUTOR"] =
executor_constants.SEQUENTIAL_EXECUTOR
- else:
- self.print_output("standalone", "Forcing executor to
LocalExecutor")
- env["AIRFLOW__CORE__EXECUTOR"] =
executor_constants.LOCAL_EXECUTOR
+ self.print_output("standalone", "Forcing executor to
LocalExecutor")
+ env["AIRFLOW__CORE__EXECUTOR"] = executor_constants.LOCAL_EXECUTOR
Review Comment:
Don't we want to still check for sqlite and disable parallelism if so?
##########
airflow-core/docs/howto/set-up-database.rst:
##########
@@ -72,8 +72,8 @@ Setting up a SQLite Database
----------------------------
SQLite database can be used to run Airflow for development purpose as it does
not require any database server
-(the database is stored in a local file). There are many limitations of using
the SQLite database (for example
Review Comment:
We should provide guidance here for LocalExecutor, parallelism must be set
to 1 right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
