ashb commented on code in PR #659:
URL: https://github.com/apache/airflow-site/pull/659#discussion_r973070541


##########
landing-pages/site/content/en/blog/airflow-2.4.0/index.md:
##########
@@ -0,0 +1,140 @@
+---
+title: "Apache Airflow 2.4.0: Data
+linkTitle: "Apache Airflow 2.4.0"
+author: "Ash Berlin-Taylor"
+github: "ashberlin"
+linkedin: "ashberlin-taylor"
+description: "We're proud to announce that Apache Airflow 2.4.0 has been 
released."
+tags: [Release]
+date: "2022-09-19"
+---
+
+Apache Airflow
+Apache Airflow 2.4.0 contains over 650 "user-facing" commits in this release 
(excluding commits to providers or chart) and over 870 in total since 2.3.0 and 
includes 50 new features, 99 improvements, 85 bug fixes, and several doc 
changes.
+
+**Details**:
+
+📦 PyPI: https://pypi.org/project/apache-airflow/2.4.0/ \
+📚 Docs: https://airflow.apache.org/docs/apache-airflow/2.4.0/ \
+🛠️ Release Notes: 
https://airflow.apache.org/docs/apache-airflow/2.4.0/release_notes.html \
+🐳 Docker Image: docker pull apache/airflow:2.4.0 \
+🚏 Constraints: https://github.com/apache/airflow/tree/constraints-2.4.0
+
+
+## Data-aware scheduling (AIP-48)
+
+This one is big. Airflow now has the ability to schedule DAGs based on other 
tasks updating datasets.
+
+What does this mean, exactly? This is a great new feature and lets DAG authors 
create smaller, more self-contained DAGs, that can chain together into a larger 
data-based workflow. If you are currently using `ExternalTaskSensor` or 
`TriggerDagRunOperator` you should take a look at datasets -- in most cases you 
can replace them with something that will speed up the scheduling!
+
+But enough talking, lets have a short example. First lets create the task that 
produces the dataset:
+
+```python
+from airflow import Dataset
+
+
+dataset = Dataset(uri='my-dataset')
+
+with DAG(dag_id='producer', ...)
+    @task(outlets=[dataset])
+    def my_task():
+        ...
+```
+
+And then we can tell Airflow to schedule a DAG whenever this Dataset changes:
+
+```python
+
+from airflow import Dataset
+
+
+dataset = Dataset(uri='my-dataset')
+
+with DAG(dag_id='dataset-consumer', schedule=[dataset]):
+    ...
+```
+
+With these two dags, the instant that the `my_task` finishes Airflow will 
create the DAG run for the `dataset-consumer` workflow.
+
+(If you have the produce and consumer in different files you do not need to 
use the same Dataset object, two `Dataset()`s created with the same URI are 
equal.)
+
+We know that what exists right now won't fit all use cases that people might 
wish for datasets, and in the coming minor releases (2.5, 2.6, etc.) we will 
expand and improve upon this foundation
+
+Datasets represent the abstract concept of a dataset, and (for now) do not 
have any direct read or write capability - in this release we are adding the 
foundational feature that we will build upon in future - and it's part of our 
goal to have smaller releases to get new features in your hands sooner!
+
+For more info on Datasets please see the [docs on Data-aware 
scheduling][data-aware-scheduling], including how datasets are identified 
(URIs), how you can depend on multiple datasets, and how to think about what is 
a Dataset (hint: don't include "date partitions" in a dataset, it's higher 
level than that.)
+
+[data-aware-scheduling]: 
https://airflow.apache.org/docs/apache-airflow/stable/concepts/datasets.html
+
+## More improvments to Dynamic Task Mapping (AIP-42)
+
+You asked, we listened. Dynamic task mapping now includes support for 
`expand_kwargs` (to assign multiple parameters to a non-TaskFlow operator), 
`zip` (to combine multiple things without cross-product) and `map` (to 
transform the parameters on just before the task is run).

Review Comment:
   It's a mathematical expression that means different things in different 
context but here it means
   
   (2,3) x (4,5,6) = (2,4), (2,5), (2,6), (3,4), (3,5), (3,6)
   
   We already use this term in our docs.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to