This is an automated email from the ASF dual-hosted git repository.
vatsrahul1001 pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/main by this push:
new ee2e15e5e51 Update description on What is Airflow (#70547)
ee2e15e5e51 is described below
commit ee2e15e5e5108756925ab3cc76dfa4efe8552e2d
Author: Elad Kalif <[email protected]>
AuthorDate: Tue Aug 4 10:57:25 2026 +0300
Update description on What is Airflow (#70547)
* Update description on What is Airflow
* change link to registry
* Update airflow-core/docs/index.rst
Co-authored-by: Przemysław Mirowski
<[email protected]>
* fix
---------
Co-authored-by: Rahul Vats <[email protected]>
Co-authored-by: Przemysław Mirowski
<[email protected]>
---
README.md | 2 ++
airflow-core/docs/index.rst | 6 ++++--
2 files changed, 6 insertions(+), 2 deletions(-)
diff --git a/README.md b/README.md
index 520deae0c95..bdc600e5e50 100644
--- a/README.md
+++ b/README.md
@@ -84,6 +84,8 @@ Use Airflow to author workflows (Dags) that orchestrate
tasks. The Airflow sched
Airflow works best with workflows that are mostly static and slowly changing.
When the Dag structure is similar from one run to the next, it clarifies the
unit of work and continuity. Other similar projects include
[Luigi](https://github.com/spotify/luigi), [Oozie](https://oozie.apache.org/)
and [Azkaban](https://azkaban.github.io/).
+Beyond traditional data pipelines, Airflow is widely used to orchestrate
machine learning workflows — training, retraining, evaluation, and deployment —
and increasingly to orchestrate agentic and LLM-based workloads, coordinating
the steps of an AI pipeline (data prep, tool calls, model invocation,
evaluation) rather than acting as the agent itself. This isn't a new direction:
teams have run ML and AI workloads on Airflow for years, and the ecosystem of
providers supporting these use ca [...]
+
Airflow is commonly used to process data, but has the opinion that tasks
should ideally be idempotent (i.e., results of the task will be the same, and
will not create duplicated data in a destination system), and should not pass
large quantities of data from one task to the next (though tasks can pass
metadata using Airflow's [XCom
feature](https://airflow.apache.org/docs/apache-airflow/stable/concepts/xcoms.html)).
For high-volume, data-intensive tasks, a best practice is to delegate to [...]
Airflow is not a streaming solution, but it is often used to process real-time
data, pulling data off streams in batches.
diff --git a/airflow-core/docs/index.rst b/airflow-core/docs/index.rst
index 4280171e42d..879ccb50e08 100644
--- a/airflow-core/docs/index.rst
+++ b/airflow-core/docs/index.rst
@@ -19,8 +19,10 @@ What is Airflow®?
=========================================
`Apache Airflow® <https://github.com/apache/airflow>`_ is an open-source
platform for developing, scheduling,
-and monitoring batch-oriented workflows. Airflow's extensible Python framework
enables you to build workflows
-connecting with virtually any technology. A web-based UI helps you visualize,
manage, and debug your workflows.
+and monitoring workflows, such as traditional time based or event-triggered
batch-oriented data pipelines, machine learning, model training,
+and agentic or LLM-based workloads. Airflow's extensible Python framework
enables you to build workflows
+connecting with virtually any technology, with a growing set of providers for
orchestrating AI and agentic
+tools alongside the rest of your pipeline. A web-based UI helps you visualize,
manage, and debug your workflows.
You can run Airflow in a variety of configurations — from a single process on
your laptop to a distributed system
capable of handling massive workloads.
