This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch gh-pages
in repository https://gitbox.apache.org/repos/asf/airflow-site.git
The following commit(s) were added to refs/heads/gh-pages by this push:
new 0ad7876f3 Deploying to gh-pages from @
a36b2ae5114504ff447560a4172c16c49a777e46 🚀
0ad7876f3 is described below
commit 0ad7876f3f0c00644123f69d228250fca88d0d43
Author: jedcunningham <[email protected]>
AuthorDate: Sun May 1 00:22:52 2022 +0000
Deploying to gh-pages from @ a36b2ae5114504ff447560a4172c16c49a777e46 🚀
---
_gen/indexes/en/blog-posts.json | 2 +-
blog/airflow-1.10.10/index.html | 4 +-
blog/airflow-1.10.12/index.html | 4 +-
blog/airflow-1.10.8-1.10.9/index.html | 4 +-
blog/airflow-2.2.0/index.html | 43 ++++-
blog/airflow-2.3.0/grid-view.png | Bin 0 -> 1136078 bytes
blog/{airflow-2.2.0 => airflow-2.3.0}/index.html | 212 +++++++++++++++------
blog/airflow-survey-2020/index.html | 4 +-
blog/airflow-survey/index.html | 4 +-
blog/airflow-two-point-oh-is-here/index.html | 39 +++-
blog/airflow_summit_2021/index.html | 4 +-
blog/announcing-new-website/index.html | 4 +-
blog/apache-airflow-for-newcomers/index.html | 4 +-
.../index.html | 4 +-
.../index.html | 4 +-
.../index.html | 4 +-
.../index.html | 4 +-
.../index.html | 39 +++-
blog/index.html | 35 ++++
blog/index.xml | 150 +++++++++++++++
.../index.html | 4 +-
blog/tags/release/index.html | 37 +++-
blog/tags/release/index.xml | 152 ++++++++++++++-
index.html | 28 +--
index.xml | 150 +++++++++++++++
search/index.html | 4 +-
sitemap.xml | 97 +++++-----
tags/index.html | 2 +-
tags/index.xml | 2 +-
use-cases/adobe/index.html | 4 +-
use-cases/adyen/index.html | 4 +-
use-cases/big-fish-games/index.html | 4 +-
use-cases/dish/index.html | 4 +-
use-cases/experity/index.html | 4 +-
use-cases/onefootball/index.html | 4 +-
use-cases/plarium-krasnodar/index.html | 4 +-
use-cases/seniorlink/index.html | 4 +-
use-cases/sift/index.html | 4 +-
38 files changed, 903 insertions(+), 177 deletions(-)
diff --git a/_gen/indexes/en/blog-posts.json b/_gen/indexes/en/blog-posts.json
index e3ab2f9af..a0eee9788 100644
--- a/_gen/indexes/en/blog-posts.json
+++ b/_gen/indexes/en/blog-posts.json
@@ -1 +1 @@
-[{"content":"## The story behind the Airflow Breeze tool\nInitially, we
started contributing to this fantastic open-source project [Apache Airflow]
with a team of three which then grew to five. When we kicked it off a year ago,
I realized pretty soon where the biggest bottlenecks and areas for improvement
in terms of productivity were. Even with the help of our client, who provided
us with a “homegrown” development environment it took us literally days to set
it up and learn some basics. [...]
\ No newline at end of file
+[{"content":"## The story behind the Airflow Breeze tool\nInitially, we
started contributing to this fantastic open-source project [Apache Airflow]
with a team of three which then grew to five. When we kicked it off a year ago,
I realized pretty soon where the biggest bottlenecks and areas for improvement
in terms of productivity were. Even with the help of our client, who provided
us with a “homegrown” development environment it took us literally days to set
it up and learn some basics. [...]
\ No newline at end of file
diff --git a/blog/airflow-1.10.10/index.html b/blog/airflow-1.10.10/index.html
index 16c0d5595..013459e0d 100644
--- a/blog/airflow-1.10.10/index.html
+++ b/blog/airflow-1.10.10/index.html
@@ -36,13 +36,13 @@
<meta property="og:image" content="/images/feature-image.png" />
<meta property="article:published_time" content="2020-04-09T00:00:00+00:00" />
-<meta property="article:modified_time" content="2022-04-30T23:44:13+01:00"
/><meta property="og:site_name" content="Apache Airflow" />
+<meta property="article:modified_time" content="2022-05-01T01:14:31+01:00"
/><meta property="og:site_name" content="Apache Airflow" />
<meta itemprop="name" content="Apache Airflow 1.10.10">
<meta itemprop="description" content="We are happy to present Apache Airflow
1.10.10">
<meta itemprop="datePublished" content="2020-04-09T00:00:00+00:00" />
-<meta itemprop="dateModified" content="2022-04-30T23:44:13+01:00" />
+<meta itemprop="dateModified" content="2022-05-01T01:14:31+01:00" />
<meta itemprop="wordCount" content="1143">
diff --git a/blog/airflow-1.10.12/index.html b/blog/airflow-1.10.12/index.html
index 461785fb9..8ebae419b 100644
--- a/blog/airflow-1.10.12/index.html
+++ b/blog/airflow-1.10.12/index.html
@@ -36,13 +36,13 @@
<meta property="og:image" content="/images/feature-image.png" />
<meta property="article:published_time" content="2020-08-25T00:00:00+00:00" />
-<meta property="article:modified_time" content="2022-04-30T23:44:13+01:00"
/><meta property="og:site_name" content="Apache Airflow" />
+<meta property="article:modified_time" content="2022-05-01T01:14:31+01:00"
/><meta property="og:site_name" content="Apache Airflow" />
<meta itemprop="name" content="Apache Airflow 1.10.12">
<meta itemprop="description" content="We are happy to present Apache Airflow
1.10.12">
<meta itemprop="datePublished" content="2020-08-25T00:00:00+00:00" />
-<meta itemprop="dateModified" content="2022-04-30T23:44:13+01:00" />
+<meta itemprop="dateModified" content="2022-05-01T01:14:31+01:00" />
<meta itemprop="wordCount" content="911">
diff --git a/blog/airflow-1.10.8-1.10.9/index.html
b/blog/airflow-1.10.8-1.10.9/index.html
index 7b86b124e..01c4c2825 100644
--- a/blog/airflow-1.10.8-1.10.9/index.html
+++ b/blog/airflow-1.10.8-1.10.9/index.html
@@ -36,13 +36,13 @@
<meta property="og:image" content="/images/feature-image.png" />
<meta property="article:published_time" content="2020-02-23T00:00:00+00:00" />
-<meta property="article:modified_time" content="2022-04-30T23:44:13+01:00"
/><meta property="og:site_name" content="Apache Airflow" />
+<meta property="article:modified_time" content="2022-05-01T01:14:31+01:00"
/><meta property="og:site_name" content="Apache Airflow" />
<meta itemprop="name" content="Apache Airflow 1.10.8 & 1.10.9">
<meta itemprop="description" content="We are happy to present the new 1.10.8
and 1.10.9 releases of Apache Airflow.">
<meta itemprop="datePublished" content="2020-02-23T00:00:00+00:00" />
-<meta itemprop="dateModified" content="2022-04-30T23:44:13+01:00" />
+<meta itemprop="dateModified" content="2022-05-01T01:14:31+01:00" />
<meta itemprop="wordCount" content="437">
diff --git a/blog/airflow-2.2.0/index.html b/blog/airflow-2.2.0/index.html
index 9034aa5ab..ea4767f8c 100644
--- a/blog/airflow-2.2.0/index.html
+++ b/blog/airflow-2.2.0/index.html
@@ -36,13 +36,13 @@
<meta property="og:image" content="/images/feature-image.png" />
<meta property="article:published_time" content="2021-10-11T00:00:00+00:00" />
-<meta property="article:modified_time" content="2022-04-30T23:44:13+01:00"
/><meta property="og:site_name" content="Apache Airflow" />
+<meta property="article:modified_time" content="2022-05-01T01:14:31+01:00"
/><meta property="og:site_name" content="Apache Airflow" />
<meta itemprop="name" content="What's new in Apache Airflow 2.2.0">
<meta itemprop="description" content="We're proud to announce that Apache
Airflow 2.2.0 has been released.">
<meta itemprop="datePublished" content="2021-10-11T00:00:00+00:00" />
-<meta itemprop="dateModified" content="2022-04-30T23:44:13+01:00" />
+<meta itemprop="dateModified" content="2022-05-01T01:14:31+01:00" />
<meta itemprop="wordCount" content="658">
@@ -619,10 +619,10 @@
<button class="btn-hollow btn-blue bodytext__medium--cerulean-blue"
>Previous</button>
</a>
- <a >
+ <a href="/blog/airflow-2.3.0/">
-<button class="btn-hollow btn-blue bodytext__medium--cerulean-blue"
disabled>Next</button>
+<button class="btn-hollow btn-blue bodytext__medium--cerulean-blue"
>Next</button>
</a>
</div>
@@ -643,6 +643,41 @@
<div class="list-item list-item--wide">
+<div class="card">
+ <div class="box-event__blogpost">
+ <div class="box-event__blogpost--metadata">
+ <div class="tags-container">
+
+
+ <a class="tag" href="/blog/tags/release/">Release</a>
+
+
+ </div>
+ <span class="bodytext__medium--brownish-grey
box-event__blogpost--date">Sat, Apr 30, 2022</span>
+ </div>
+ <p class="box-event__blogpost--header">
+ <a href="/blog/airflow-2.3.0/">
+ Apache Airflow 2.3.0 is here
+ </a>
+ </p>
+ <p class="box-event__blogpost--author">Ephraim Anierobi</p>
+ <p class="box-event__blogpost--description">We're proud to
announce that Apache Airflow 2.3.0 has been released.</p>
+ <div class="mt-auto">
+ <a href="/blog/airflow-2.3.0/">
+
+
+<button class="btn-hollow btn-blue bodytext__medium--cerulean-blue" >Read
more</button>
+
+ </a>
+ </div>
+ </div>
+</div>
+
+ </div>
+
+ <div class="list-item list-item--wide">
+
+
<div class="card">
<div class="box-event__blogpost">
<div class="box-event__blogpost--metadata">
diff --git a/blog/airflow-2.3.0/grid-view.png b/blog/airflow-2.3.0/grid-view.png
new file mode 100644
index 000000000..59a0cfa4d
Binary files /dev/null and b/blog/airflow-2.3.0/grid-view.png differ
diff --git a/blog/airflow-2.2.0/index.html b/blog/airflow-2.3.0/index.html
similarity index 85%
copy from blog/airflow-2.2.0/index.html
copy to blog/airflow-2.3.0/index.html
index 9034aa5ab..ecfdcd54c 100644
--- a/blog/airflow-2.2.0/index.html
+++ b/blog/airflow-2.3.0/index.html
@@ -29,21 +29,21 @@
<meta name="msapplication-TileImage" content="/favicons/ms-icon-144x144.png">
<meta name="theme-color" content="#ffffff">
-<title>What's new in Apache Airflow 2.2.0 | Apache Airflow</title><meta
property="og:title" content="What's new in Apache Airflow 2.2.0" />
-<meta property="og:description" content="We're proud to announce that
Apache Airflow 2.2.0 has been released." />
+<title>Apache Airflow 2.3.0 is here | Apache Airflow</title><meta
property="og:title" content="Apache Airflow 2.3.0 is here" />
+<meta property="og:description" content="We're proud to announce that
Apache Airflow 2.3.0 has been released." />
<meta property="og:type" content="article" />
-<meta property="og:url" content="/blog/airflow-2.2.0/" />
+<meta property="og:url" content="/blog/airflow-2.3.0/" />
<meta property="og:image" content="/images/feature-image.png" />
-<meta property="article:published_time" content="2021-10-11T00:00:00+00:00" />
-<meta property="article:modified_time" content="2022-04-30T23:44:13+01:00"
/><meta property="og:site_name" content="Apache Airflow" />
-<meta itemprop="name" content="What's new in Apache Airflow 2.2.0">
-<meta itemprop="description" content="We're proud to announce that Apache
Airflow 2.2.0 has been released.">
+<meta property="article:published_time" content="2022-04-30T00:00:00+00:00" />
+<meta property="article:modified_time" content="2022-05-01T01:14:31+01:00"
/><meta property="og:site_name" content="Apache Airflow" />
+<meta itemprop="name" content="Apache Airflow 2.3.0 is here">
+<meta itemprop="description" content="We're proud to announce that Apache
Airflow 2.3.0 has been released.">
-<meta itemprop="datePublished" content="2021-10-11T00:00:00+00:00" />
-<meta itemprop="dateModified" content="2022-04-30T23:44:13+01:00" />
-<meta itemprop="wordCount" content="658">
+<meta itemprop="datePublished" content="2022-04-30T00:00:00+00:00" />
+<meta itemprop="dateModified" content="2022-05-01T01:14:31+01:00" />
+<meta itemprop="wordCount" content="708">
@@ -51,8 +51,8 @@
<meta name="twitter:card" content="summary_large_image"/>
<meta name="twitter:image" content="/images/feature-image.png"/>
-<meta name="twitter:title" content="What's new in Apache Airflow 2.2.0"/>
-<meta name="twitter:description" content="We're proud to announce that
Apache Airflow 2.2.0 has been released."/>
+<meta name="twitter:title" content="Apache Airflow 2.3.0 is here"/>
+<meta name="twitter:description" content="We're proud to announce that
Apache Airflow 2.3.0 has been released."/>
@@ -69,7 +69,7 @@
crossorigin="anonymous"></script>
-<meta name="description" content="We're proud to announce that Apache
Airflow 2.2.0 has been released." />
+<meta name="description" content="We're proud to announce that Apache
Airflow 2.3.0 has been released." />
@@ -446,16 +446,16 @@
</div>
- <span class="bodytext__medium--brownish-grey">Mon, Oct 11, 2021</span>
+ <span class="bodytext__medium--brownish-grey">Sat, Apr 30, 2022</span>
</div>
- <p class="blogpost-content__metadata--title">What's new in Apache
Airflow 2.2.0</p>
+ <p class="blogpost-content__metadata--title">Apache Airflow 2.3.0 is
here</p>
<div class="blogpost-content__metadata--author">
<span class="blogpost-content__metadata--author">
- Jed Cunningham
+ Ephraim Anierobi
</span>
- <a href="https://github.com/jedcunningham/";
class="blogpost-content__metadata--social-media-icon">
+ <a href="https://github.com/ephraimbuddy/";
class="blogpost-content__metadata--social-media-icon">
<svg xmlns="http://www.w3.org/2000/svg"; width="21.737"
height="21.2" viewBox="0 0 21.737 21.2">
<path id="Path_1378" d="M33.971 1181.31a10.87 10.87 0 0 0-3.435
21.182c.543.1.742-.236.742-.524
0-.258-.009-.941-.015-1.848-3.023.657-3.661-1.457-3.661-1.457a2.876 2.876 0 0
0-1.207-1.59c-.987-.674.075-.661.075-.661a2.283 2.283 0 0 1 1.665 1.12 2.314
2.314 0 0 0 3.163.9 2.322 2.322 0 0 1
.69-1.453c-2.413-.274-4.951-1.207-4.951-5.371a4.2 4.2 0 0 1 1.119-2.917 3.908
3.908 0 0 1 .107-2.876s.913-.292 2.989 1.114a10.3 10.3 0 0 1 5.442
0c2.075-1.406 2.986-1.114 2.986-1.114a3.9 3.9 0 0 1 .1 [...]
</svg>
@@ -463,7 +463,7 @@
</a>
- <a href="https://linkedin.com/in/jedidiah-cunningham/";
class="blogpost-content__metadata--social-media-icon">
+ <a href="https://linkedin.com/in/ephraimanierobi/";
class="blogpost-content__metadata--social-media-icon">
<svg xmlns="http://www.w3.org/2000/svg"; width="20" height="20"
viewBox="0 0 20 21">
<g id="Group_1745" data-name="Group 1745" transform="translate(.155
-.055)">
<ellipse id="Ellipse_20" cx="10" cy="10.5" fill="#51504f"
data-name="Ellipse 20" rx="10" ry="10.5" transform="translate(-.155 .055)"/>
@@ -477,85 +477,146 @@
</a>
</div>
- <p class="blogpost-content__metadata--description">We're proud to
announce that Apache Airflow 2.2.0 has been released.</p>
+ <p class="blogpost-content__metadata--description">We're proud to
announce that Apache Airflow 2.3.0 has been released.</p>
</div>
<div class="markdown-content">
-<p>I’m proud to announce that Apache Airflow 2.2.0 has been released. It
contains over 600 commits since 2.1.4 and includes 30 new features, 84
improvements, 85 bug fixes, and many internal and doc changes.</p>
+<p>Apache Airflow 2.3.0 contains over 700 commits since 2.2.0 and includes 50
new features, 99 improvements, 85 bug fixes, and several doc changes.</p>
<p><strong>Details</strong>:</p>
-<p>📦 PyPI: <a href="https://pypi.org/project/apache-airflow/2.2.0/";
target="_blank">https://pypi.org/project/apache-airflow/2.2.0/</a> <br />
-📚 Docs: <a href="https://airflow.apache.org/docs/apache-airflow/2.2.0/";
target="_blank">https://airflow.apache.org/docs/apache-airflow/2.2.0/</a> <br />
-🛠️ Changelog: <a
href="https://airflow.apache.org/docs/apache-airflow/2.2.0/changelog.html";
target="_blank">https://airflow.apache.org/docs/apache-airflow/2.2.0/changelog.html</a>
<br />
-🐳 Docker Image: docker pull apache/airflow:2.2.0 <br />
-🚏 Constraints: <a
href="https://github.com/apache/airflow/tree/constraints-2.2.0";
target="_blank">https://github.com/apache/airflow/tree/constraints-2.2.0</a></p>
+<p>📦 PyPI: <a href="https://pypi.org/project/apache-airflow/2.3.0/";
target="_blank">https://pypi.org/project/apache-airflow/2.3.0/</a> <br />
+📚 Docs: <a href="https://airflow.apache.org/docs/apache-airflow/2.3.0/";
target="_blank">https://airflow.apache.org/docs/apache-airflow/2.3.0/</a> <br />
+🛠️ Release Notes: <a
href="https://airflow.apache.org/docs/apache-airflow/2.3.0/release_notes.html";
target="_blank">https://airflow.apache.org/docs/apache-airflow/2.3.0/release_notes.html</a>
<br />
+🐳 Docker Image: docker pull apache/airflow:2.3.0 <br />
+🚏 Constraints: <a
href="https://github.com/apache/airflow/tree/constraints-2.3.0";
target="_blank">https://github.com/apache/airflow/tree/constraints-2.3.0</a></p>
<p>As the changelog is quite large, the following are some notable new
features that shipped in this release.</p>
-<h2 id="custom-timetables-aip-39">Custom Timetables (AIP-39)</h2>
+<h2 id="dynamic-task-mapping-aip-42">Dynamic Task Mapping(AIP-42)</h2>
-<p>Airflow has historically used cron expressions and timedeltas to represent
when a DAG should run. This worked for a lot of use cases, but not all. For
example, running daily on Monday-Friday, but not on weekends wasn’t
possible.</p>
+<p>There’s now first-class support for dynamic tasks in Airflow. What
this means is that you can generate tasks dynamically at runtime. Much like
using a <code>for</code> loop
+to create a list of tasks, here you can create the same tasks without having
to know the exact number of tasks ahead of time.</p>
-<p>To provide more scheduling flexibility, determining when a DAG should run
is now done with Timetables. Of course, backwards compatibility has been
maintained - cron expressions and timedeltas are still fully supported,
however, timetables are pluggable so you can add your own custom timetable to
fit your needs! For example, you could write a timetable to schedule a
DagRun</p>
+<p>You can have a <code>task</code> generate the list to iterate over, which
is not possible with a <code>for</code> loop.</p>
-<p><code>execution_date</code> has long been confusing to new Airflowers, so
as part of this change a new concept has been added to Airflow to replace it
named <code>data_interval</code>, which is the period of data that a task
should operate on. The following are now available:</p>
+<p>Here is an example:</p>
-<ul>
-<li><code>logical_date</code> (aka <code>execution_date</code>)</li>
-<li><code>data_interval_start</code> (same value as
<code>execution_date</code> for cron)</li>
-<li><code>data_interval_end</code> (aka <code>next_execution_date</code>)</li>
-</ul>
+<pre><code class="language-python">@task
+def make_list():
+ # This can also be from an API call, checking a database, -- almost
anything you like, as long as the
+ # resulting list/dictionary can be stored in the current XCom backend.
+ return [1, 2, {"a": "b"}, "str"]
+
+
+@task
+def consumer(arg):
+ print(list(arg))
+
+
+with DAG(dag_id="dynamic-map", start_date=datetime(2022, 4, 2)) as
dag:
+ consumer.expand(arg=make_list())
+</code></pre>
+
+<p>More information can be found here: <a
href="https://airflow.apache.org/docs/apache-airflow/2.3.0/concepts/dynamic-task-mapping.html";
target="_blank">Dynamic Task Mapping</a></p>
+
+<h2 id="grid-view-replaces-tree-view">Grid View replaces Tree View</h2>
+
+<p>Grid view replaces tree view in Airflow 2.3.0.</p>
+
+<p><strong>Screenshots</strong>:
+<img src="grid-view.png" alt="The new grid view" /></p>
+
+<h2 id="purge-history-from-metadata-database">Purge history from metadata
database</h2>
+
+<p>Airflow 2.3.0 introduces a new <code>airflow db clean</code> command that
can be used to purge old data from the metadata database.</p>
+
+<p>You would want to use this command if you want to reduce the size of the
metadata database.</p>
-<p>If you write your own timetables, keep in mind they should be idempotent
and fast as they are used in the scheduler to create DagRuns.</p>
+<p>More information can be found here: <a
href="https://airflow.apache.org/docs/apache-airflow/2.3.0/usage-cli.html#purge-history-from-metadata-database";
target="_blank">Purge history from metadata database</a></p>
-<p>More information can be found at: <a
href="https://airflow.apache.org/docs/apache-airflow/stable/howto/timetable.html";
target="_blank">Customizing DAG Scheduling with Timetables</a></p>
+<h2 id="localkubernetesexecutor">LocalKubernetesExecutor</h2>
-<h2 id="deferrable-tasks-aip-40">Deferrable Tasks (AIP-40)</h2>
+<p>There is a new executor named LocalKubernetesExecutor. This executor helps
you run some tasks using LocalExecutor and run another set of tasks using the
KubernetesExecutor in the same deployment based on the task’s queue.</p>
-<p>Deferrable tasks allows operators or sensors to defer themselves until a
light-weight async check passes, at which point they can resume executing. Most
importantly, this results in the worker slot, and most notably any resources
used by it, to be returned to Airflow. This allows simple things like
monitoring a job in an external system or watching for an event to be much
cheaper.</p>
+<p>More information can be found here: <a
href="https://airflow.apache.org/docs/apache-airflow/2.3.0/executor/local_kubernetes.html";
target="_blank">LocalKubernetesExecutor</a></p>
-<p>To support this feature, a new component has been added to Airflow, the
triggerer, which is the daemon process that runs the asyncio event loop.</p>
+<h2 id="dagprocessormanager-as-standalone-process-aip-43">DagProcessorManager
as standalone process (AIP-43)</h2>
-<p>Airflow 2.2.0 ships with 2 deferrable sensors,
<code>DateTimeSensorAsync</code> and <code>TimeDeltaSensorAsync</code>, both of
which are drop-in replacements for the existing corresponding sensor.</p>
+<p>As of 2.3.0, you can run the DagProcessorManager as a standalone process.
Because DagProcessorManager runs user code, separating it from the scheduler
process and running it as an independent process in a different host is a good
idea.</p>
-<p>More information can be found at:</p>
+<p>The <code>airflow dag-processor</code> cli command will start a new process
that will run the DagProcessorManager in a separate process. Before you can run
the DagProcessorManager as a standalone process, you need to set the <a
href="https://airflow.apache.org/docs/apache-airflow/stable/configurations-ref.html#standalone_dag_processor";
target="_blank">[scheduler] standalone_dag_processor</a> to
<code>True</code>.</p>
-<p><a
href="https://airflow.apache.org/docs/apache-airflow/stable/concepts/deferring.html";
target="_blank">Deferrable Operators & Triggers</a></p>
+<p>More information can be found here: <a
href="https://airflow.apache.org/docs/apache-airflow/2.3.0/cli-and-env-variables-ref.html#dag-processor";
target="_blank">dag-processor CLI command</a></p>
-<h2 id="custom-task-decorators-and-task-docker">Custom <code>@task</code>
decorators and <code>@task.docker</code></h2>
+<h2 id="json-serialization-for-connections">JSON serialization for
connections</h2>
-<p>Airflow 2.2.0 allows providers to create custom <code>@task</code>
decorators in the TaskFlow interface.</p>
+<p>You can now create connections using the <code>json</code> serialization
format.</p>
-<p>The <code>@task.docker</code> decorator is one such decorator that allows
you to run a function in a docker container. Airflow handles getting the code
into the container and returning xcom - you just worry about your function.
This is particularly useful when you have conflicting dependencies between
Airflow itself and tasks you need to run.</p>
+<pre><code class="language-bash">airflow connections add 'my_prod_db' \
+ --conn-json '{
+ "conn_type": "my-conn-type",
+ "login": "my-login",
+ "password": "my-password",
+ "host": "my-host",
+ "port": 1234,
+ "schema": "my-schema",
+ "extra": {
+ "param1": "val1",
+ "param2": "val2"
+ }
+ }'
+</code></pre>
-<p>More information on creating custom <code>@task</code> decorators can be
found at: <a
href="https://airflow.apache.org/docs/apache-airflow/stable/howto/create-custom-decorator.html";
target="_blank">Creating Custom @task Decorators</a></p>
+<p>You can also use <code>json</code> serialization format when setting the
connection in environment variables.</p>
-<p>More information on the <code>@task.docker</code> decorator can be found
at: <a
href="https://airflow.apache.org/docs/apache-airflow/stable/tutorial_taskflow_api.html#using-the-taskflow-api-with-docker-or-virtual-environments";
target="_blank">Using the Taskflow API with Docker or Virtual
Environments</a></p>
+<p>More information can be found here: <a
href="https://airflow.apache.org/docs/apache-airflow/2.3.0/howto/connection.html";
target="_blank">JSON serialization for connections</a></p>
-<h2 id="validation-of-dag-params">Validation of DAG params</h2>
+<h2 id="airflow-db-downgrade-and-offline-generation-of-sql-scripts">Airflow
<code>db downgrade</code> and Offline generation of SQL scripts</h2>
-<p>You can now apply validation on DAG params by passing a <code>Param</code>
object for each param. The <code>Param</code> object supports the full <a
href="https://json-schema.org/draft/2020-12/json-schema-validation.html";
target="_blank">json-schema validation specifications</a>.</p>
+<p>Airflow 2.3.0 introduced a new command <code>airflow db downgrade</code>
that will downgrade the database to your chosen version.</p>
-<p>Currently this only functions with manually triggered DAGs, but it does set
the stage for future params related functionality.</p>
+<p>You can also generate the downgrade/upgrade SQL scripts for your database
and manually run it against your database or just view the SQL queries that
would be run by the downgrade/upgrade command.</p>
-<p>More information can be found at: <a
href="https://airflow.apache.org/docs/apache-airflow/stable/concepts/params.html";
target="_blank">Params</a></p>
+<p>More information can be found here: <a
href="https://airflow.apache.org/docs/apache-airflow/2.3.0/usage-cli.html#downgrading-airflow";
target="_blank">Airflow <code>db downgrade</code> and Offline generation of
SQL scripts</a></p>
+
+<h2 id="reuse-of-decorated-tasks">Reuse of decorated tasks</h2>
+
+<p>You can now reuse decorated tasks across your dag files. A decorated task
has an <code>override</code> method that allows you to override it’s
arguments.</p>
+
+<p>Here’s an example:</p>
+
+<pre><code class="language-python">@task
+def add_task(x, y):
+ print(f"Task args: x={x}, y={y}")
+ return x + y
+
+
+@dag(start_date=datetime(2022, 1, 1))
+def mydag():
+ start = add_task.override(task_id="start")(1, 2)
+ for i in range(3):
+ start >>
add_task.override(task_id=f"add_start_{i}")(start, i)
+</code></pre>
+
+<p>More information can be found here: <a
href="https://airflow.apache.org/docs/apache-airflow/2.3.0/tutorial_taskflow_api.html#reusing-a-decorated-task";
target="_blank">Reuse of decorated DAGs</a></p>
<h2 id="other-small-features">Other small features</h2>
<p>This isn’t a comprehensive list, but some noteworthy or interesting small
features include:</p>
<ul>
-<li>Testing Connections from the UI - test the credentials for your Connection
actually work</li>
-<li>Duplication Connections from the UI</li>
-<li>DAGs “Next run” info is shown in the UI, including when the run will
actually start</li>
-<li><code>airflow standalone</code> command runs all of the Airflow components
directly without docker - great for local development</li>
+<li>Support different timeout value for dag file parsing</li>
+<li><code>airflow dags reserialize</code> command to reserialize dags</li>
+<li>Events Timetable</li>
+<li>SmoothOperator - Operator that does literally nothing except logging a
YouTube link to
+Sade’s “Smooth Operator”. Enjoy!</li>
</ul>
<h2 id="contributors">Contributors</h2>
-<p>Thanks to everyone who contributed to this release: Andrew Godwin, Ash
Berlin-Taylor, Brent Bovenzi, Elad Kalif, Ephraim Anierobi, James Timmins,
Jarek Potiuk, Jed Cunningham, Josh Fell, Kamil Breguła, Kaxil Naik, Malthe
Borch, Sam Wheating, Sumit Maheshwari, Tzu-ping Chung and many others</p>
+<p>Thanks to everyone who contributed to this release: Ash Berlin-Taylor,
Brent Bovenzi, Daniel Standish, Elad, Ephraim Anierobi, Jarek Potiuk, Jed
Cunningham, Josh Fell, Kamil Breguła, Kanthi, Kaxil Naik, Khalid Mammadov,
Malthe Borch, Ping Zhang, Tzu-ping Chung and many others who keep making
Airflow better for everyone.</p>
</div>
@@ -613,7 +674,7 @@
<div class="pager">
- <a href="/blog/airflow_summit_2021/">
+ <a href="/blog/airflow-2.2.0/">
<button class="btn-hollow btn-blue bodytext__medium--cerulean-blue"
>Previous</button>
@@ -643,6 +704,41 @@
<div class="list-item list-item--wide">
+<div class="card">
+ <div class="box-event__blogpost">
+ <div class="box-event__blogpost--metadata">
+ <div class="tags-container">
+
+
+ <a class="tag" href="/blog/tags/release/">Release</a>
+
+
+ </div>
+ <span class="bodytext__medium--brownish-grey
box-event__blogpost--date">Mon, Oct 11, 2021</span>
+ </div>
+ <p class="box-event__blogpost--header">
+ <a href="/blog/airflow-2.2.0/">
+ What's new in Apache Airflow 2.2.0
+ </a>
+ </p>
+ <p class="box-event__blogpost--author">Jed Cunningham</p>
+ <p class="box-event__blogpost--description">We're proud to
announce that Apache Airflow 2.2.0 has been released.</p>
+ <div class="mt-auto">
+ <a href="/blog/airflow-2.2.0/">
+
+
+<button class="btn-hollow btn-blue bodytext__medium--cerulean-blue" >Read
more</button>
+
+ </a>
+ </div>
+ </div>
+</div>
+
+ </div>
+
+ <div class="list-item list-item--wide">
+
+
<div class="card">
<div class="box-event__blogpost">
<div class="box-event__blogpost--metadata">
@@ -687,7 +783,7 @@
<div class="base-layout--button">
- <a
href=https://github.com/apache/airflow-site/edit/main/landing-pages/site/content/en/blog/airflow-2.2.0/index.md>
+ <a
href=https://github.com/apache/airflow-site/edit/main/landing-pages/site/content/en/blog/airflow-2.3.0/index.md>
diff --git a/blog/airflow-survey-2020/index.html
b/blog/airflow-survey-2020/index.html
index 24f8703a1..1b4a7749c 100644
--- a/blog/airflow-survey-2020/index.html
+++ b/blog/airflow-survey-2020/index.html
@@ -36,13 +36,13 @@
<meta property="og:image" content="/images/feature-image.png" />
<meta property="article:published_time" content="2021-03-09T00:00:00+00:00" />
-<meta property="article:modified_time" content="2022-04-30T23:44:13+01:00"
/><meta property="og:site_name" content="Apache Airflow" />
+<meta property="article:modified_time" content="2022-05-01T01:14:31+01:00"
/><meta property="og:site_name" content="Apache Airflow" />
<meta itemprop="name" content="Airflow Survey 2020">
<meta itemprop="description" content="We observe steady growth in number of
users as well as in an amount of active contributors. So listening and
understanding our community is of high importance.">
<meta itemprop="datePublished" content="2021-03-09T00:00:00+00:00" />
-<meta itemprop="dateModified" content="2022-04-30T23:44:13+01:00" />
+<meta itemprop="dateModified" content="2022-05-01T01:14:31+01:00" />
<meta itemprop="wordCount" content="2106">
diff --git a/blog/airflow-survey/index.html b/blog/airflow-survey/index.html
index 922ba9106..c67b61b75 100644
--- a/blog/airflow-survey/index.html
+++ b/blog/airflow-survey/index.html
@@ -36,13 +36,13 @@
<meta property="og:image" content="/images/feature-image.png" />
<meta property="article:published_time" content="2019-12-11T00:00:00+00:00" />
-<meta property="article:modified_time" content="2022-04-30T23:44:13+01:00"
/><meta property="og:site_name" content="Apache Airflow" />
+<meta property="article:modified_time" content="2022-05-01T01:14:31+01:00"
/><meta property="og:site_name" content="Apache Airflow" />
<meta itemprop="name" content="Airflow Survey 2019">
<meta itemprop="description" content="Receiving and adjusting to our users’
feedback is a must. Let’s see who Airflow users are, how they play with it, and
what they miss.">
<meta itemprop="datePublished" content="2019-12-11T00:00:00+00:00" />
-<meta itemprop="dateModified" content="2022-04-30T23:44:13+01:00" />
+<meta itemprop="dateModified" content="2022-05-01T01:14:31+01:00" />
<meta itemprop="wordCount" content="1775">
diff --git a/blog/airflow-two-point-oh-is-here/index.html
b/blog/airflow-two-point-oh-is-here/index.html
index 480f34953..1ff274ca1 100644
--- a/blog/airflow-two-point-oh-is-here/index.html
+++ b/blog/airflow-two-point-oh-is-here/index.html
@@ -36,13 +36,13 @@
<meta property="og:image" content="/images/feature-image.png" />
<meta property="article:published_time" content="2020-12-17T00:00:00+00:00" />
-<meta property="article:modified_time" content="2022-04-30T23:44:13+01:00"
/><meta property="og:site_name" content="Apache Airflow" />
+<meta property="article:modified_time" content="2022-05-01T01:14:31+01:00"
/><meta property="og:site_name" content="Apache Airflow" />
<meta itemprop="name" content="Apache Airflow 2.0 is here!">
<meta itemprop="description" content="We're proud to announce that Apache
Airflow 2.0.0 has been released.">
<meta itemprop="datePublished" content="2020-12-17T00:00:00+00:00" />
-<meta itemprop="dateModified" content="2022-04-30T23:44:13+01:00" />
+<meta itemprop="dateModified" content="2022-05-01T01:14:31+01:00" />
<meta itemprop="wordCount" content="1165">
@@ -703,6 +703,41 @@ Docs on <a
href="http://airflow.apache.org/docs/apache-airflow-providers/package
<div class="list-item list-item--wide">
+<div class="card">
+ <div class="box-event__blogpost">
+ <div class="box-event__blogpost--metadata">
+ <div class="tags-container">
+
+
+ <a class="tag" href="/blog/tags/release/">Release</a>
+
+
+ </div>
+ <span class="bodytext__medium--brownish-grey
box-event__blogpost--date">Sat, Apr 30, 2022</span>
+ </div>
+ <p class="box-event__blogpost--header">
+ <a href="/blog/airflow-2.3.0/">
+ Apache Airflow 2.3.0 is here
+ </a>
+ </p>
+ <p class="box-event__blogpost--author">Ephraim Anierobi</p>
+ <p class="box-event__blogpost--description">We're proud to
announce that Apache Airflow 2.3.0 has been released.</p>
+ <div class="mt-auto">
+ <a href="/blog/airflow-2.3.0/">
+
+
+<button class="btn-hollow btn-blue bodytext__medium--cerulean-blue" >Read
more</button>
+
+ </a>
+ </div>
+ </div>
+</div>
+
+ </div>
+
+ <div class="list-item list-item--wide">
+
+
<div class="card">
<div class="box-event__blogpost">
<div class="box-event__blogpost--metadata">
diff --git a/blog/airflow_summit_2021/index.html
b/blog/airflow_summit_2021/index.html
index f5853d557..97d1cefcb 100644
--- a/blog/airflow_summit_2021/index.html
+++ b/blog/airflow_summit_2021/index.html
@@ -36,13 +36,13 @@
<meta property="og:image" content="/images/feature-image.png" />
<meta property="article:published_time" content="2021-03-21T00:00:00+00:00" />
-<meta property="article:modified_time" content="2022-04-30T23:44:13+01:00"
/><meta property="og:site_name" content="Apache Airflow" />
+<meta property="article:modified_time" content="2022-05-01T01:14:31+01:00"
/><meta property="og:site_name" content="Apache Airflow" />
<meta itemprop="name" content="Airflow Summit 2021">
<meta itemprop="description" content="We are thrilled about Airflow Summit
2021!">
<meta itemprop="datePublished" content="2021-03-21T00:00:00+00:00" />
-<meta itemprop="dateModified" content="2022-04-30T23:44:13+01:00" />
+<meta itemprop="dateModified" content="2022-05-01T01:14:31+01:00" />
<meta itemprop="wordCount" content="77">
diff --git a/blog/announcing-new-website/index.html
b/blog/announcing-new-website/index.html
index e957741d0..accdf73a5 100644
--- a/blog/announcing-new-website/index.html
+++ b/blog/announcing-new-website/index.html
@@ -36,13 +36,13 @@
<meta property="og:image" content="/images/feature-image.png" />
<meta property="article:published_time" content="2019-12-11T00:00:00+00:00" />
-<meta property="article:modified_time" content="2022-04-30T23:44:13+01:00"
/><meta property="og:site_name" content="Apache Airflow" />
+<meta property="article:modified_time" content="2022-05-01T01:14:31+01:00"
/><meta property="og:site_name" content="Apache Airflow" />
<meta itemprop="name" content="New Airflow website">
<meta itemprop="description" content="We are thrilled about our new website!">
<meta itemprop="datePublished" content="2019-12-11T00:00:00+00:00" />
-<meta itemprop="dateModified" content="2022-04-30T23:44:13+01:00" />
+<meta itemprop="dateModified" content="2022-05-01T01:14:31+01:00" />
<meta itemprop="wordCount" content="282">
diff --git a/blog/apache-airflow-for-newcomers/index.html
b/blog/apache-airflow-for-newcomers/index.html
index a0736940e..841ffe8c2 100644
--- a/blog/apache-airflow-for-newcomers/index.html
+++ b/blog/apache-airflow-for-newcomers/index.html
@@ -37,14 +37,14 @@ Authoring Workflow in Apache Airflow. Airflow makes it easy
to author workflows
<meta property="og:image" content="/images/feature-image.png" />
<meta property="article:published_time" content="2020-08-17T00:00:00+00:00" />
-<meta property="article:modified_time" content="2022-04-30T23:44:13+01:00"
/><meta property="og:site_name" content="Apache Airflow" />
+<meta property="article:modified_time" content="2022-05-01T01:14:31+01:00"
/><meta property="og:site_name" content="Apache Airflow" />
<meta itemprop="name" content="Apache Airflow For Newcomers">
<meta itemprop="description" content="Apache Airflow is a platform to
programmatically author, schedule, and monitor workflows. A workflow is a
sequence of tasks that processes a set of data. You can think of workflow as
the path that describes how tasks go from being undone to done. Scheduling, on
the other hand, is the process of planning, controlling, and optimizing when a
particular task should be done.
Authoring Workflow in Apache Airflow. Airflow makes it easy to author
workflows using python scripts.">
<meta itemprop="datePublished" content="2020-08-17T00:00:00+00:00" />
-<meta itemprop="dateModified" content="2022-04-30T23:44:13+01:00" />
+<meta itemprop="dateModified" content="2022-05-01T01:14:31+01:00" />
<meta itemprop="wordCount" content="1070">
diff --git
a/blog/apache-con-europe-2019-thoughts-and-insights-by-airflow-committers/index.html
b/blog/apache-con-europe-2019-thoughts-and-insights-by-airflow-committers/index.html
index 8416b93b4..ef00d5bc6 100644
---
a/blog/apache-con-europe-2019-thoughts-and-insights-by-airflow-committers/index.html
+++
b/blog/apache-con-europe-2019-thoughts-and-insights-by-airflow-committers/index.html
@@ -36,13 +36,13 @@
<meta property="og:image" content="/images/feature-image.png" />
<meta property="article:published_time" content="2019-11-22T00:00:00+00:00" />
-<meta property="article:modified_time" content="2022-04-30T23:44:13+01:00"
/><meta property="og:site_name" content="Apache Airflow" />
+<meta property="article:modified_time" content="2022-05-01T01:14:31+01:00"
/><meta property="og:site_name" content="Apache Airflow" />
<meta itemprop="name" content="ApacheCon Europe 2019 — Thoughts and Insights
by Airflow Committers">
<meta itemprop="description" content="Here come some thoughts by Airflow
committers and contributors from the ApacheCon Europe 2019. Get to know the ASF
community!">
<meta itemprop="datePublished" content="2019-11-22T00:00:00+00:00" />
-<meta itemprop="dateModified" content="2022-04-30T23:44:13+01:00" />
+<meta itemprop="dateModified" content="2022-05-01T01:14:31+01:00" />
<meta itemprop="wordCount" content="138">
diff --git a/blog/documenting-using-local-development-environments/index.html
b/blog/documenting-using-local-development-environments/index.html
index d9c90698e..38f5f5bce 100644
--- a/blog/documenting-using-local-development-environments/index.html
+++ b/blog/documenting-using-local-development-environments/index.html
@@ -36,13 +36,13 @@
<meta property="og:image" content="/images/feature-image.png" />
<meta property="article:published_time" content="2019-11-22T00:00:00+00:00" />
-<meta property="article:modified_time" content="2022-04-30T23:44:13+01:00"
/><meta property="og:site_name" content="Apache Airflow" />
+<meta property="article:modified_time" content="2022-05-01T01:14:31+01:00"
/><meta property="og:site_name" content="Apache Airflow" />
<meta itemprop="name" content="Documenting using local development
environment">
<meta itemprop="description" content="The story behind documenting local
development environment of Apache Airflow">
<meta itemprop="datePublished" content="2019-11-22T00:00:00+00:00" />
-<meta itemprop="dateModified" content="2022-04-30T23:44:13+01:00" />
+<meta itemprop="dateModified" content="2022-05-01T01:14:31+01:00" />
<meta itemprop="wordCount" content="256">
diff --git
a/blog/experience-in-google-season-of-docs-2019-with-apache-airflow/index.html
b/blog/experience-in-google-season-of-docs-2019-with-apache-airflow/index.html
index 0857c9188..563c707e4 100644
---
a/blog/experience-in-google-season-of-docs-2019-with-apache-airflow/index.html
+++
b/blog/experience-in-google-season-of-docs-2019-with-apache-airflow/index.html
@@ -37,14 +37,14 @@ About Me I have been writing tech articles on medium as
well as my blog for the
<meta property="og:image" content="/images/feature-image.png" />
<meta property="article:published_time" content="2019-12-20T00:00:00+00:00" />
-<meta property="article:modified_time" content="2022-04-30T23:44:13+01:00"
/><meta property="og:site_name" content="Apache Airflow" />
+<meta property="article:modified_time" content="2022-05-01T01:14:31+01:00"
/><meta property="og:site_name" content="Apache Airflow" />
<meta itemprop="name" content="Experience in Google Season of Docs 2019 with
Apache Airflow">
<meta itemprop="description" content="I came across Google Season of Docs
(GSoD) almost by accident, thanks to my extensive HackerNews and Twitter
addiction. I was familiar with the Google Summer of Code but not with this
program. It turns out it was the inaugural phase. I read the details, and the
process felt a lot like GSoC except that this was about documentation.
About Me I have been writing tech articles on medium as well as my blog for
the past 1.">
<meta itemprop="datePublished" content="2019-12-20T00:00:00+00:00" />
-<meta itemprop="dateModified" content="2022-04-30T23:44:13+01:00" />
+<meta itemprop="dateModified" content="2022-05-01T01:14:31+01:00" />
<meta itemprop="wordCount" content="1521">
diff --git a/blog/experience-with-airflow-as-an-outreachy-intern/index.html
b/blog/experience-with-airflow-as-an-outreachy-intern/index.html
index e99031ec4..d98ab7cd1 100644
--- a/blog/experience-with-airflow-as-an-outreachy-intern/index.html
+++ b/blog/experience-with-airflow-as-an-outreachy-intern/index.html
@@ -37,14 +37,14 @@ Contribution Period The first thing I had to do was choose
a project under an or
<meta property="og:image" content="/images/feature-image.png" />
<meta property="article:published_time" content="2020-08-30T00:00:00+00:00" />
-<meta property="article:modified_time" content="2022-04-30T23:44:13+01:00"
/><meta property="og:site_name" content="Apache Airflow" />
+<meta property="article:modified_time" content="2022-05-01T01:14:31+01:00"
/><meta property="og:site_name" content="Apache Airflow" />
<meta itemprop="name" content="Journey with Airflow as an Outreachy Intern">
<meta itemprop="description" content="Outreachy is a program which organises
three months paid internships with FOSS projects for people who are typically
underrepresented in those projects.
Contribution Period The first thing I had to do was choose a project under an
organisation. After going through all the projects I chose “Extending the REST
API of Apache Airflow”, because I had a good idea of what REST API(s) are, so I
thought it would be easier to get started with the contributions.">
<meta itemprop="datePublished" content="2020-08-30T00:00:00+00:00" />
-<meta itemprop="dateModified" content="2022-04-30T23:44:13+01:00" />
+<meta itemprop="dateModified" content="2022-05-01T01:14:31+01:00" />
<meta itemprop="wordCount" content="532">
diff --git a/blog/implementing-stable-api-for-apache-airflow/index.html
b/blog/implementing-stable-api-for-apache-airflow/index.html
index e2402844d..a2bee491a 100644
--- a/blog/implementing-stable-api-for-apache-airflow/index.html
+++ b/blog/implementing-stable-api-for-apache-airflow/index.html
@@ -36,13 +36,13 @@
<meta property="og:image" content="/images/feature-image.png" />
<meta property="article:published_time" content="2020-07-19T00:00:00+00:00" />
-<meta property="article:modified_time" content="2022-04-30T23:44:13+01:00"
/><meta property="og:site_name" content="Apache Airflow" />
+<meta property="article:modified_time" content="2022-05-01T01:14:31+01:00"
/><meta property="og:site_name" content="Apache Airflow" />
<meta itemprop="name" content="Implementing Stable API for Apache Airflow">
<meta itemprop="description" content="An Outreachy intern's progress
report on contributing to Apache Airflow REST API.">
<meta itemprop="datePublished" content="2020-07-19T00:00:00+00:00" />
-<meta itemprop="dateModified" content="2022-04-30T23:44:13+01:00" />
+<meta itemprop="dateModified" content="2022-05-01T01:14:31+01:00" />
<meta itemprop="wordCount" content="703">
@@ -655,6 +655,41 @@ wonderful reviews. I’m grateful.</p>
<div class="list-item list-item--wide">
+<div class="card">
+ <div class="box-event__blogpost">
+ <div class="box-event__blogpost--metadata">
+ <div class="tags-container">
+
+
+ <a class="tag" href="/blog/tags/release/">Release</a>
+
+
+ </div>
+ <span class="bodytext__medium--brownish-grey
box-event__blogpost--date">Sat, Apr 30, 2022</span>
+ </div>
+ <p class="box-event__blogpost--header">
+ <a href="/blog/airflow-2.3.0/">
+ Apache Airflow 2.3.0 is here
+ </a>
+ </p>
+ <p class="box-event__blogpost--author">Ephraim Anierobi</p>
+ <p class="box-event__blogpost--description">We're proud to
announce that Apache Airflow 2.3.0 has been released.</p>
+ <div class="mt-auto">
+ <a href="/blog/airflow-2.3.0/">
+
+
+<button class="btn-hollow btn-blue bodytext__medium--cerulean-blue" >Read
more</button>
+
+ </a>
+ </div>
+ </div>
+</div>
+
+ </div>
+
+ <div class="list-item list-item--wide">
+
+
<div class="card">
<div class="box-event__blogpost">
<div class="box-event__blogpost--metadata">
diff --git a/blog/index.html b/blog/index.html
index 6aaa94f59..cd9a08559 100644
--- a/blog/index.html
+++ b/blog/index.html
@@ -489,6 +489,41 @@
<div class="list-item list-item--wide">
+<div class="card">
+ <div class="box-event__blogpost">
+ <div class="box-event__blogpost--metadata">
+ <div class="tags-container">
+
+
+ <a class="tag" href="/blog/tags/release/">Release</a>
+
+
+ </div>
+ <span class="bodytext__medium--brownish-grey
box-event__blogpost--date">Sat, Apr 30, 2022</span>
+ </div>
+ <p class="box-event__blogpost--header">
+ <a href="/blog/airflow-2.3.0/">
+ Apache Airflow 2.3.0 is here
+ </a>
+ </p>
+ <p class="box-event__blogpost--author">Ephraim Anierobi</p>
+ <p class="box-event__blogpost--description">We're proud to
announce that Apache Airflow 2.3.0 has been released.</p>
+ <div class="mt-auto">
+ <a href="/blog/airflow-2.3.0/">
+
+
+<button class="btn-hollow btn-blue bodytext__medium--cerulean-blue" >Read
more</button>
+
+ </a>
+ </div>
+ </div>
+</div>
+
+ </div>
+
+ <div class="list-item list-item--wide">
+
+
<div class="card">
<div class="box-event__blogpost">
<div class="box-event__blogpost--metadata">
diff --git a/blog/index.xml b/blog/index.xml
index a2f8002ca..4d557d6f7 100644
--- a/blog/index.xml
+++ b/blog/index.xml
@@ -13,6 +13,156 @@
+ <item>
+ <title>Blog: Apache Airflow 2.3.0 is here</title>
+ <link>/blog/airflow-2.3.0/</link>
+ <pubDate>Sat, 30 Apr 2022 00:00:00 +0000</pubDate>
+
+ <guid>/blog/airflow-2.3.0/</guid>
+ <description>
+
+
+
+
+<p>Apache Airflow 2.3.0 contains over 700 commits since 2.2.0 and
includes 50 new features, 99 improvements, 85 bug fixes, and several doc
changes.</p>
+
+<p><strong>Details</strong>:</p>
+
+<p>📦 PyPI: <a
href="https://pypi.org/project/apache-airflow/2.3.0/"
target="_blank">https://pypi.org/project/apache-airflow/2.3.0/</a>
<br />
+📚 Docs: <a
href="https://airflow.apache.org/docs/apache-airflow/2.3.0/"
target="_blank">https://airflow.apache.org/docs/apache-airflow/2.3.0/</a>
<br />
+🛠️ Release Notes: <a
href="https://airflow.apache.org/docs/apache-airflow/2.3.0/release_notes.html"
target="_blank">https://airflow.apache.org/docs/apache-airflow/2.3.0/release_notes.html</a>
<br />
+🐳 Docker Image: docker pull apache/airflow:2.3.0 <br />
+🚏 Constraints: <a
href="https://github.com/apache/airflow/tree/constraints-2.3.0"
target="_blank">https://github.com/apache/airflow/tree/constraints-2.3.0</a></p>
+
+<p>As the changelog is quite large, the following are some notable new
features that shipped in this release.</p>
+
+<h2 id="dynamic-task-mapping-aip-42">Dynamic Task
Mapping(AIP-42)</h2>
+
+<p>There&rsquo;s now first-class support for dynamic tasks in
Airflow. What this means is that you can generate tasks dynamically at runtime.
Much like using a <code>for</code> loop
+to create a list of tasks, here you can create the same tasks without having
to know the exact number of tasks ahead of time.</p>
+
+<p>You can have a <code>task</code> generate the list to
iterate over, which is not possible with a <code>for</code>
loop.</p>
+
+<p>Here is an example:</p>
+
+<pre><code class="language-python">@task
+def make_list():
+ # This can also be from an API call, checking a database, -- almost
anything you like, as long as the
+ # resulting list/dictionary can be stored in the current XCom backend.
+ return [1, 2, {&quot;a&quot;: &quot;b&quot;},
&quot;str&quot;]
+
+
+@task
+def consumer(arg):
+ print(list(arg))
+
+
+with DAG(dag_id=&quot;dynamic-map&quot;, start_date=datetime(2022, 4,
2)) as dag:
+ consumer.expand(arg=make_list())
+</code></pre>
+
+<p>More information can be found here: <a
href="https://airflow.apache.org/docs/apache-airflow/2.3.0/concepts/dynamic-task-mapping.html"
target="_blank">Dynamic Task Mapping</a></p>
+
+<h2 id="grid-view-replaces-tree-view">Grid View replaces Tree
View</h2>
+
+<p>Grid view replaces tree view in Airflow 2.3.0.</p>
+
+<p><strong>Screenshots</strong>:
+<img src="grid-view.png" alt="The new grid view"
/></p>
+
+<h2 id="purge-history-from-metadata-database">Purge history from
metadata database</h2>
+
+<p>Airflow 2.3.0 introduces a new <code>airflow db
clean</code> command that can be used to purge old data from the metadata
database.</p>
+
+<p>You would want to use this command if you want to reduce the size of
the metadata database.</p>
+
+<p>More information can be found here: <a
href="https://airflow.apache.org/docs/apache-airflow/2.3.0/usage-cli.html#purge-history-from-metadata-database"
target="_blank">Purge history from metadata
database</a></p>
+
+<h2
id="localkubernetesexecutor">LocalKubernetesExecutor</h2>
+
+<p>There is a new executor named LocalKubernetesExecutor. This executor
helps you run some tasks using LocalExecutor and run another set of tasks using
the KubernetesExecutor in the same deployment based on the task&rsquo;s
queue.</p>
+
+<p>More information can be found here: <a
href="https://airflow.apache.org/docs/apache-airflow/2.3.0/executor/local_kubernetes.html"
target="_blank">LocalKubernetesExecutor</a></p>
+
+<h2
id="dagprocessormanager-as-standalone-process-aip-43">DagProcessorManager
as standalone process (AIP-43)</h2>
+
+<p>As of 2.3.0, you can run the DagProcessorManager as a standalone
process. Because DagProcessorManager runs user code, separating it from the
scheduler process and running it as an independent process in a different host
is a good idea.</p>
+
+<p>The <code>airflow dag-processor</code> cli command will
start a new process that will run the DagProcessorManager in a separate
process. Before you can run the DagProcessorManager as a standalone process,
you need to set the <a
href="https://airflow.apache.org/docs/apache-airflow/stable/configurations-ref.html#standalone_dag_processor"
target="_blank">[scheduler] standalone_dag_processor</a> to
<code>True</code>.</p>
+
+<p>More information can be found here: <a
href="https://airflow.apache.org/docs/apache-airflow/2.3.0/cli-and-env-variables-ref.html#dag-processor"
target="_blank">dag-processor CLI command</a></p>
+
+<h2 id="json-serialization-for-connections">JSON serialization
for connections</h2>
+
+<p>You can now create connections using the
<code>json</code> serialization format.</p>
+
+<pre><code class="language-bash">airflow connections add
'my_prod_db' \
+ --conn-json '{
+ &quot;conn_type&quot;: &quot;my-conn-type&quot;,
+ &quot;login&quot;: &quot;my-login&quot;,
+ &quot;password&quot;: &quot;my-password&quot;,
+ &quot;host&quot;: &quot;my-host&quot;,
+ &quot;port&quot;: 1234,
+ &quot;schema&quot;: &quot;my-schema&quot;,
+ &quot;extra&quot;: {
+ &quot;param1&quot;: &quot;val1&quot;,
+ &quot;param2&quot;: &quot;val2&quot;
+ }
+ }'
+</code></pre>
+
+<p>You can also use <code>json</code> serialization format
when setting the connection in environment variables.</p>
+
+<p>More information can be found here: <a
href="https://airflow.apache.org/docs/apache-airflow/2.3.0/howto/connection.html"
target="_blank">JSON serialization for
connections</a></p>
+
+<h2
id="airflow-db-downgrade-and-offline-generation-of-sql-scripts">Airflow
<code>db downgrade</code> and Offline generation of SQL
scripts</h2>
+
+<p>Airflow 2.3.0 introduced a new command <code>airflow db
downgrade</code> that will downgrade the database to your chosen
version.</p>
+
+<p>You can also generate the downgrade/upgrade SQL scripts for your
database and manually run it against your database or just view the SQL queries
that would be run by the downgrade/upgrade command.</p>
+
+<p>More information can be found here: <a
href="https://airflow.apache.org/docs/apache-airflow/2.3.0/usage-cli.html#downgrading-airflow"
target="_blank">Airflow <code>db downgrade</code> and
Offline generation of SQL scripts</a></p>
+
+<h2 id="reuse-of-decorated-tasks">Reuse of decorated
tasks</h2>
+
+<p>You can now reuse decorated tasks across your dag files. A decorated
task has an <code>override</code> method that allows you to
override it&rsquo;s arguments.</p>
+
+<p>Here&rsquo;s an example:</p>
+
+<pre><code class="language-python">@task
+def add_task(x, y):
+ print(f&quot;Task args: x={x}, y={y}&quot;)
+ return x + y
+
+
+@dag(start_date=datetime(2022, 1, 1))
+def mydag():
+ start = add_task.override(task_id=&quot;start&quot;)(1, 2)
+ for i in range(3):
+ start &gt;&gt;
add_task.override(task_id=f&quot;add_start_{i}&quot;)(start, i)
+</code></pre>
+
+<p>More information can be found here: <a
href="https://airflow.apache.org/docs/apache-airflow/2.3.0/tutorial_taskflow_api.html#reusing-a-decorated-task"
target="_blank">Reuse of decorated DAGs</a></p>
+
+<h2 id="other-small-features">Other small features</h2>
+
+<p>This isn’t a comprehensive list, but some noteworthy or interesting
small features include:</p>
+
+<ul>
+<li>Support different timeout value for dag file parsing</li>
+<li><code>airflow dags reserialize</code> command to
reserialize dags</li>
+<li>Events Timetable</li>
+<li>SmoothOperator - Operator that does literally nothing except logging
a YouTube link to
+Sade&rsquo;s &ldquo;Smooth Operator&rdquo;. Enjoy!</li>
+</ul>
+
+<h2 id="contributors">Contributors</h2>
+
+<p>Thanks to everyone who contributed to this release: Ash
Berlin-Taylor, Brent Bovenzi, Daniel Standish, Elad, Ephraim Anierobi, Jarek
Potiuk, Jed Cunningham, Josh Fell, Kamil Breguła, Kanthi, Kaxil Naik, Khalid
Mammadov, Malthe Borch, Ping Zhang, Tzu-ping Chung and many others who keep
making Airflow better for everyone.</p>
+
+ </description>
+ </item>
+
<item>
<title>Blog: What's new in Apache Airflow 2.2.0</title>
<link>/blog/airflow-2.2.0/</link>
diff --git a/blog/its-a-breeze-to-develop-apache-airflow/index.html
b/blog/its-a-breeze-to-develop-apache-airflow/index.html
index 33abe4ee3..eff041dcf 100644
--- a/blog/its-a-breeze-to-develop-apache-airflow/index.html
+++ b/blog/its-a-breeze-to-develop-apache-airflow/index.html
@@ -36,13 +36,13 @@
<meta property="og:image" content="/images/feature-image.png" />
<meta property="article:published_time" content="2019-11-22T00:00:00+00:00" />
-<meta property="article:modified_time" content="2022-04-30T23:44:13+01:00"
/><meta property="og:site_name" content="Apache Airflow" />
+<meta property="article:modified_time" content="2022-05-01T01:14:31+01:00"
/><meta property="og:site_name" content="Apache Airflow" />
<meta itemprop="name" content="It's a "Breeze" to develop Apache
Airflow">
<meta itemprop="description" content="A Principal Software Engineer's
journey to developer productivity. Learn how Jarek and his team speeded up and
simplified Airflow development for the community.">
<meta itemprop="datePublished" content="2019-11-22T00:00:00+00:00" />
-<meta itemprop="dateModified" content="2022-04-30T23:44:13+01:00" />
+<meta itemprop="dateModified" content="2022-05-01T01:14:31+01:00" />
<meta itemprop="wordCount" content="124">
diff --git a/blog/tags/release/index.html b/blog/tags/release/index.html
index 56444954f..b491c87fa 100644
--- a/blog/tags/release/index.html
+++ b/blog/tags/release/index.html
@@ -37,7 +37,7 @@
<meta property="og:image" content="/images/feature-image.png" />
-<meta property="og:updated_time" content="2021-10-11T00:00:00+00:00" /><meta
property="og:site_name" content="Apache Airflow" />
+<meta property="og:updated_time" content="2022-04-30T00:00:00+00:00" /><meta
property="og:site_name" content="Apache Airflow" />
<meta itemprop="name" content="release">
<meta itemprop="description" content="Platform created by the community to
programmatically author, schedule and monitor workflows.">
@@ -499,6 +499,41 @@
<div class="list-item list-item--wide">
+<div class="card">
+ <div class="box-event__blogpost">
+ <div class="box-event__blogpost--metadata">
+ <div class="tags-container">
+
+
+ <a class="tag" href="/blog/tags/release/">Release</a>
+
+
+ </div>
+ <span class="bodytext__medium--brownish-grey
box-event__blogpost--date">Sat, Apr 30, 2022</span>
+ </div>
+ <p class="box-event__blogpost--header">
+ <a href="/blog/airflow-2.3.0/">
+ Apache Airflow 2.3.0 is here
+ </a>
+ </p>
+ <p class="box-event__blogpost--author">Ephraim Anierobi</p>
+ <p class="box-event__blogpost--description">We're proud to
announce that Apache Airflow 2.3.0 has been released.</p>
+ <div class="mt-auto">
+ <a href="/blog/airflow-2.3.0/">
+
+
+<button class="btn-hollow btn-blue bodytext__medium--cerulean-blue" >Read
more</button>
+
+ </a>
+ </div>
+ </div>
+</div>
+
+ </div>
+
+ <div class="list-item list-item--wide">
+
+
<div class="card">
<div class="box-event__blogpost">
<div class="box-event__blogpost--metadata">
diff --git a/blog/tags/release/index.xml b/blog/tags/release/index.xml
index a44a3ffcf..eb7c63ebf 100644
--- a/blog/tags/release/index.xml
+++ b/blog/tags/release/index.xml
@@ -4,7 +4,7 @@
<link>/blog/tags/release/</link>
<description>Recent content in release on Apache Airflow</description>
<generator>Hugo -- gohugo.io</generator>
- <lastBuildDate>Mon, 11 Oct 2021 00:00:00 +0000</lastBuildDate>
+ <lastBuildDate>Sat, 30 Apr 2022 00:00:00 +0000</lastBuildDate>
<atom:link href="/blog/tags/release/index.xml" rel="self"
type="application/rss+xml" />
@@ -14,6 +14,156 @@
+ <item>
+ <title>Blog: Apache Airflow 2.3.0 is here</title>
+ <link>/blog/airflow-2.3.0/</link>
+ <pubDate>Sat, 30 Apr 2022 00:00:00 +0000</pubDate>
+
+ <guid>/blog/airflow-2.3.0/</guid>
+ <description>
+
+
+
+
+<p>Apache Airflow 2.3.0 contains over 700 commits since 2.2.0 and
includes 50 new features, 99 improvements, 85 bug fixes, and several doc
changes.</p>
+
+<p><strong>Details</strong>:</p>
+
+<p>📦 PyPI: <a
href="https://pypi.org/project/apache-airflow/2.3.0/"
target="_blank">https://pypi.org/project/apache-airflow/2.3.0/</a>
<br />
+📚 Docs: <a
href="https://airflow.apache.org/docs/apache-airflow/2.3.0/"
target="_blank">https://airflow.apache.org/docs/apache-airflow/2.3.0/</a>
<br />
+🛠️ Release Notes: <a
href="https://airflow.apache.org/docs/apache-airflow/2.3.0/release_notes.html"
target="_blank">https://airflow.apache.org/docs/apache-airflow/2.3.0/release_notes.html</a>
<br />
+🐳 Docker Image: docker pull apache/airflow:2.3.0 <br />
+🚏 Constraints: <a
href="https://github.com/apache/airflow/tree/constraints-2.3.0"
target="_blank">https://github.com/apache/airflow/tree/constraints-2.3.0</a></p>
+
+<p>As the changelog is quite large, the following are some notable new
features that shipped in this release.</p>
+
+<h2 id="dynamic-task-mapping-aip-42">Dynamic Task
Mapping(AIP-42)</h2>
+
+<p>There&rsquo;s now first-class support for dynamic tasks in
Airflow. What this means is that you can generate tasks dynamically at runtime.
Much like using a <code>for</code> loop
+to create a list of tasks, here you can create the same tasks without having
to know the exact number of tasks ahead of time.</p>
+
+<p>You can have a <code>task</code> generate the list to
iterate over, which is not possible with a <code>for</code>
loop.</p>
+
+<p>Here is an example:</p>
+
+<pre><code class="language-python">@task
+def make_list():
+ # This can also be from an API call, checking a database, -- almost
anything you like, as long as the
+ # resulting list/dictionary can be stored in the current XCom backend.
+ return [1, 2, {&quot;a&quot;: &quot;b&quot;},
&quot;str&quot;]
+
+
+@task
+def consumer(arg):
+ print(list(arg))
+
+
+with DAG(dag_id=&quot;dynamic-map&quot;, start_date=datetime(2022, 4,
2)) as dag:
+ consumer.expand(arg=make_list())
+</code></pre>
+
+<p>More information can be found here: <a
href="https://airflow.apache.org/docs/apache-airflow/2.3.0/concepts/dynamic-task-mapping.html"
target="_blank">Dynamic Task Mapping</a></p>
+
+<h2 id="grid-view-replaces-tree-view">Grid View replaces Tree
View</h2>
+
+<p>Grid view replaces tree view in Airflow 2.3.0.</p>
+
+<p><strong>Screenshots</strong>:
+<img src="grid-view.png" alt="The new grid view"
/></p>
+
+<h2 id="purge-history-from-metadata-database">Purge history from
metadata database</h2>
+
+<p>Airflow 2.3.0 introduces a new <code>airflow db
clean</code> command that can be used to purge old data from the metadata
database.</p>
+
+<p>You would want to use this command if you want to reduce the size of
the metadata database.</p>
+
+<p>More information can be found here: <a
href="https://airflow.apache.org/docs/apache-airflow/2.3.0/usage-cli.html#purge-history-from-metadata-database"
target="_blank">Purge history from metadata
database</a></p>
+
+<h2
id="localkubernetesexecutor">LocalKubernetesExecutor</h2>
+
+<p>There is a new executor named LocalKubernetesExecutor. This executor
helps you run some tasks using LocalExecutor and run another set of tasks using
the KubernetesExecutor in the same deployment based on the task&rsquo;s
queue.</p>
+
+<p>More information can be found here: <a
href="https://airflow.apache.org/docs/apache-airflow/2.3.0/executor/local_kubernetes.html"
target="_blank">LocalKubernetesExecutor</a></p>
+
+<h2
id="dagprocessormanager-as-standalone-process-aip-43">DagProcessorManager
as standalone process (AIP-43)</h2>
+
+<p>As of 2.3.0, you can run the DagProcessorManager as a standalone
process. Because DagProcessorManager runs user code, separating it from the
scheduler process and running it as an independent process in a different host
is a good idea.</p>
+
+<p>The <code>airflow dag-processor</code> cli command will
start a new process that will run the DagProcessorManager in a separate
process. Before you can run the DagProcessorManager as a standalone process,
you need to set the <a
href="https://airflow.apache.org/docs/apache-airflow/stable/configurations-ref.html#standalone_dag_processor"
target="_blank">[scheduler] standalone_dag_processor</a> to
<code>True</code>.</p>
+
+<p>More information can be found here: <a
href="https://airflow.apache.org/docs/apache-airflow/2.3.0/cli-and-env-variables-ref.html#dag-processor"
target="_blank">dag-processor CLI command</a></p>
+
+<h2 id="json-serialization-for-connections">JSON serialization
for connections</h2>
+
+<p>You can now create connections using the
<code>json</code> serialization format.</p>
+
+<pre><code class="language-bash">airflow connections add
'my_prod_db' \
+ --conn-json '{
+ &quot;conn_type&quot;: &quot;my-conn-type&quot;,
+ &quot;login&quot;: &quot;my-login&quot;,
+ &quot;password&quot;: &quot;my-password&quot;,
+ &quot;host&quot;: &quot;my-host&quot;,
+ &quot;port&quot;: 1234,
+ &quot;schema&quot;: &quot;my-schema&quot;,
+ &quot;extra&quot;: {
+ &quot;param1&quot;: &quot;val1&quot;,
+ &quot;param2&quot;: &quot;val2&quot;
+ }
+ }'
+</code></pre>
+
+<p>You can also use <code>json</code> serialization format
when setting the connection in environment variables.</p>
+
+<p>More information can be found here: <a
href="https://airflow.apache.org/docs/apache-airflow/2.3.0/howto/connection.html"
target="_blank">JSON serialization for
connections</a></p>
+
+<h2
id="airflow-db-downgrade-and-offline-generation-of-sql-scripts">Airflow
<code>db downgrade</code> and Offline generation of SQL
scripts</h2>
+
+<p>Airflow 2.3.0 introduced a new command <code>airflow db
downgrade</code> that will downgrade the database to your chosen
version.</p>
+
+<p>You can also generate the downgrade/upgrade SQL scripts for your
database and manually run it against your database or just view the SQL queries
that would be run by the downgrade/upgrade command.</p>
+
+<p>More information can be found here: <a
href="https://airflow.apache.org/docs/apache-airflow/2.3.0/usage-cli.html#downgrading-airflow"
target="_blank">Airflow <code>db downgrade</code> and
Offline generation of SQL scripts</a></p>
+
+<h2 id="reuse-of-decorated-tasks">Reuse of decorated
tasks</h2>
+
+<p>You can now reuse decorated tasks across your dag files. A decorated
task has an <code>override</code> method that allows you to
override it&rsquo;s arguments.</p>
+
+<p>Here&rsquo;s an example:</p>
+
+<pre><code class="language-python">@task
+def add_task(x, y):
+ print(f&quot;Task args: x={x}, y={y}&quot;)
+ return x + y
+
+
+@dag(start_date=datetime(2022, 1, 1))
+def mydag():
+ start = add_task.override(task_id=&quot;start&quot;)(1, 2)
+ for i in range(3):
+ start &gt;&gt;
add_task.override(task_id=f&quot;add_start_{i}&quot;)(start, i)
+</code></pre>
+
+<p>More information can be found here: <a
href="https://airflow.apache.org/docs/apache-airflow/2.3.0/tutorial_taskflow_api.html#reusing-a-decorated-task"
target="_blank">Reuse of decorated DAGs</a></p>
+
+<h2 id="other-small-features">Other small features</h2>
+
+<p>This isn’t a comprehensive list, but some noteworthy or interesting
small features include:</p>
+
+<ul>
+<li>Support different timeout value for dag file parsing</li>
+<li><code>airflow dags reserialize</code> command to
reserialize dags</li>
+<li>Events Timetable</li>
+<li>SmoothOperator - Operator that does literally nothing except logging
a YouTube link to
+Sade&rsquo;s &ldquo;Smooth Operator&rdquo;. Enjoy!</li>
+</ul>
+
+<h2 id="contributors">Contributors</h2>
+
+<p>Thanks to everyone who contributed to this release: Ash
Berlin-Taylor, Brent Bovenzi, Daniel Standish, Elad, Ephraim Anierobi, Jarek
Potiuk, Jed Cunningham, Josh Fell, Kamil Breguła, Kanthi, Kaxil Naik, Khalid
Mammadov, Malthe Borch, Ping Zhang, Tzu-ping Chung and many others who keep
making Airflow better for everyone.</p>
+
+ </description>
+ </item>
+
<item>
<title>Blog: What's new in Apache Airflow 2.2.0</title>
<link>/blog/airflow-2.2.0/</link>
diff --git a/index.html b/index.html
index ffb77f75b..2e305ea88 100644
--- a/index.html
+++ b/index.html
@@ -1233,12 +1233,12 @@
<div id="integrations-container" class="list-items">
- <a class="list-item"
href="/docs/apache-airflow-providers-amazon/stable/index.html">
+ <a class="list-item"
href="/docs/apache-airflow-providers-google/stable/operators/cloud/bigquery.html">
<div class="card">
<div class="box-event box-event__integration">
- <span class="box-event__integration--name">AWS Step Functions</span>
+ <span class="box-event__integration--name">Google BigQuery</span>
</div>
</div>
@@ -1246,12 +1246,12 @@
- <a class="list-item"
href="/docs/apache-airflow-providers-amazon/stable/operators/glacier.html">
+ <a class="list-item"
href="/docs/apache-airflow-providers-microsoft-azure/stable/index.html">
<div class="card">
<div class="box-event box-event__integration">
- <span class="box-event__integration--name">Amazon Glacier</span>
+ <span class="box-event__integration--name">Microsoft Azure Files</span>
</div>
</div>
@@ -1259,12 +1259,12 @@
- <a class="list-item"
href="/docs/apache-airflow-providers-jdbc/stable/operators.html">
+ <a class="list-item"
href="/docs/apache-airflow-providers-oracle/stable/index.html">
<div class="card">
<div class="box-event box-event__integration">
- <span class="box-event__integration--name">Java Database Connectivity
(JDBC)</span>
+ <span class="box-event__integration--name">Oracle</span>
</div>
</div>
@@ -1272,12 +1272,12 @@
- <a class="list-item"
href="/docs/apache-airflow-providers-microsoft-azure/stable/index.html">
+ <a class="list-item"
href="/docs/apache-airflow-providers-google/stable/operators/cloud/compute.html">
<div class="card">
<div class="box-event box-event__integration">
- <span class="box-event__integration--name">Microsoft Azure Files</span>
+ <span class="box-event__integration--name">Google Compute Engine</span>
</div>
</div>
@@ -1285,12 +1285,12 @@
- <a class="list-item"
href="/docs/apache-airflow-providers-google/stable/operators/cloud/video_intelligence.html">
+ <a class="list-item"
href="/docs/apache-airflow-providers-cncf-kubernetes/stable/operators.html">
<div class="card">
<div class="box-event box-event__integration">
- <span class="box-event__integration--name">Google Cloud Video
Intelligence</span>
+ <span class="box-event__integration--name">Kubernetes</span>
</div>
</div>
@@ -1311,12 +1311,12 @@
- <a class="list-item"
href="/docs/apache-airflow-providers-google/stable/operators/firebase/firestore.html">
+ <a class="list-item"
href="/docs/apache-airflow-providers-microsoft-azure/stable/index.html">
<div class="card">
<div class="box-event box-event__integration">
- <span class="box-event__integration--name">Google Cloud
Firestore</span>
+ <span class="box-event__integration--name">Microsoft Azure</span>
</div>
</div>
@@ -1324,12 +1324,12 @@
- <a class="list-item"
href="/docs/apache-airflow-providers-microsoft-azure/stable/index.html">
+ <a class="list-item"
href="/docs/apache-airflow-providers-google/stable/operators/cloud/mlengine.html">
<div class="card">
<div class="box-event box-event__integration">
- <span class="box-event__integration--name">Microsoft Azure Container
Instances</span>
+ <span class="box-event__integration--name">Google Machine Learning
Engine</span>
</div>
</div>
diff --git a/index.xml b/index.xml
index fb17a146a..70b93dc30 100644
--- a/index.xml
+++ b/index.xml
@@ -12,6 +12,156 @@
+ <item>
+ <title>Blog: Apache Airflow 2.3.0 is here</title>
+ <link>/blog/airflow-2.3.0/</link>
+ <pubDate>Sat, 30 Apr 2022 00:00:00 +0000</pubDate>
+
+ <guid>/blog/airflow-2.3.0/</guid>
+ <description>
+
+
+
+
+<p>Apache Airflow 2.3.0 contains over 700 commits since 2.2.0 and
includes 50 new features, 99 improvements, 85 bug fixes, and several doc
changes.</p>
+
+<p><strong>Details</strong>:</p>
+
+<p>📦 PyPI: <a
href="https://pypi.org/project/apache-airflow/2.3.0/"
target="_blank">https://pypi.org/project/apache-airflow/2.3.0/</a>
<br />
+📚 Docs: <a
href="https://airflow.apache.org/docs/apache-airflow/2.3.0/"
target="_blank">https://airflow.apache.org/docs/apache-airflow/2.3.0/</a>
<br />
+🛠️ Release Notes: <a
href="https://airflow.apache.org/docs/apache-airflow/2.3.0/release_notes.html"
target="_blank">https://airflow.apache.org/docs/apache-airflow/2.3.0/release_notes.html</a>
<br />
+🐳 Docker Image: docker pull apache/airflow:2.3.0 <br />
+🚏 Constraints: <a
href="https://github.com/apache/airflow/tree/constraints-2.3.0"
target="_blank">https://github.com/apache/airflow/tree/constraints-2.3.0</a></p>
+
+<p>As the changelog is quite large, the following are some notable new
features that shipped in this release.</p>
+
+<h2 id="dynamic-task-mapping-aip-42">Dynamic Task
Mapping(AIP-42)</h2>
+
+<p>There&rsquo;s now first-class support for dynamic tasks in
Airflow. What this means is that you can generate tasks dynamically at runtime.
Much like using a <code>for</code> loop
+to create a list of tasks, here you can create the same tasks without having
to know the exact number of tasks ahead of time.</p>
+
+<p>You can have a <code>task</code> generate the list to
iterate over, which is not possible with a <code>for</code>
loop.</p>
+
+<p>Here is an example:</p>
+
+<pre><code class="language-python">@task
+def make_list():
+ # This can also be from an API call, checking a database, -- almost
anything you like, as long as the
+ # resulting list/dictionary can be stored in the current XCom backend.
+ return [1, 2, {&quot;a&quot;: &quot;b&quot;},
&quot;str&quot;]
+
+
+@task
+def consumer(arg):
+ print(list(arg))
+
+
+with DAG(dag_id=&quot;dynamic-map&quot;, start_date=datetime(2022, 4,
2)) as dag:
+ consumer.expand(arg=make_list())
+</code></pre>
+
+<p>More information can be found here: <a
href="https://airflow.apache.org/docs/apache-airflow/2.3.0/concepts/dynamic-task-mapping.html"
target="_blank">Dynamic Task Mapping</a></p>
+
+<h2 id="grid-view-replaces-tree-view">Grid View replaces Tree
View</h2>
+
+<p>Grid view replaces tree view in Airflow 2.3.0.</p>
+
+<p><strong>Screenshots</strong>:
+<img src="grid-view.png" alt="The new grid view"
/></p>
+
+<h2 id="purge-history-from-metadata-database">Purge history from
metadata database</h2>
+
+<p>Airflow 2.3.0 introduces a new <code>airflow db
clean</code> command that can be used to purge old data from the metadata
database.</p>
+
+<p>You would want to use this command if you want to reduce the size of
the metadata database.</p>
+
+<p>More information can be found here: <a
href="https://airflow.apache.org/docs/apache-airflow/2.3.0/usage-cli.html#purge-history-from-metadata-database"
target="_blank">Purge history from metadata
database</a></p>
+
+<h2
id="localkubernetesexecutor">LocalKubernetesExecutor</h2>
+
+<p>There is a new executor named LocalKubernetesExecutor. This executor
helps you run some tasks using LocalExecutor and run another set of tasks using
the KubernetesExecutor in the same deployment based on the task&rsquo;s
queue.</p>
+
+<p>More information can be found here: <a
href="https://airflow.apache.org/docs/apache-airflow/2.3.0/executor/local_kubernetes.html"
target="_blank">LocalKubernetesExecutor</a></p>
+
+<h2
id="dagprocessormanager-as-standalone-process-aip-43">DagProcessorManager
as standalone process (AIP-43)</h2>
+
+<p>As of 2.3.0, you can run the DagProcessorManager as a standalone
process. Because DagProcessorManager runs user code, separating it from the
scheduler process and running it as an independent process in a different host
is a good idea.</p>
+
+<p>The <code>airflow dag-processor</code> cli command will
start a new process that will run the DagProcessorManager in a separate
process. Before you can run the DagProcessorManager as a standalone process,
you need to set the <a
href="https://airflow.apache.org/docs/apache-airflow/stable/configurations-ref.html#standalone_dag_processor"
target="_blank">[scheduler] standalone_dag_processor</a> to
<code>True</code>.</p>
+
+<p>More information can be found here: <a
href="https://airflow.apache.org/docs/apache-airflow/2.3.0/cli-and-env-variables-ref.html#dag-processor"
target="_blank">dag-processor CLI command</a></p>
+
+<h2 id="json-serialization-for-connections">JSON serialization
for connections</h2>
+
+<p>You can now create connections using the
<code>json</code> serialization format.</p>
+
+<pre><code class="language-bash">airflow connections add
'my_prod_db' \
+ --conn-json '{
+ &quot;conn_type&quot;: &quot;my-conn-type&quot;,
+ &quot;login&quot;: &quot;my-login&quot;,
+ &quot;password&quot;: &quot;my-password&quot;,
+ &quot;host&quot;: &quot;my-host&quot;,
+ &quot;port&quot;: 1234,
+ &quot;schema&quot;: &quot;my-schema&quot;,
+ &quot;extra&quot;: {
+ &quot;param1&quot;: &quot;val1&quot;,
+ &quot;param2&quot;: &quot;val2&quot;
+ }
+ }'
+</code></pre>
+
+<p>You can also use <code>json</code> serialization format
when setting the connection in environment variables.</p>
+
+<p>More information can be found here: <a
href="https://airflow.apache.org/docs/apache-airflow/2.3.0/howto/connection.html"
target="_blank">JSON serialization for
connections</a></p>
+
+<h2
id="airflow-db-downgrade-and-offline-generation-of-sql-scripts">Airflow
<code>db downgrade</code> and Offline generation of SQL
scripts</h2>
+
+<p>Airflow 2.3.0 introduced a new command <code>airflow db
downgrade</code> that will downgrade the database to your chosen
version.</p>
+
+<p>You can also generate the downgrade/upgrade SQL scripts for your
database and manually run it against your database or just view the SQL queries
that would be run by the downgrade/upgrade command.</p>
+
+<p>More information can be found here: <a
href="https://airflow.apache.org/docs/apache-airflow/2.3.0/usage-cli.html#downgrading-airflow"
target="_blank">Airflow <code>db downgrade</code> and
Offline generation of SQL scripts</a></p>
+
+<h2 id="reuse-of-decorated-tasks">Reuse of decorated
tasks</h2>
+
+<p>You can now reuse decorated tasks across your dag files. A decorated
task has an <code>override</code> method that allows you to
override it&rsquo;s arguments.</p>
+
+<p>Here&rsquo;s an example:</p>
+
+<pre><code class="language-python">@task
+def add_task(x, y):
+ print(f&quot;Task args: x={x}, y={y}&quot;)
+ return x + y
+
+
+@dag(start_date=datetime(2022, 1, 1))
+def mydag():
+ start = add_task.override(task_id=&quot;start&quot;)(1, 2)
+ for i in range(3):
+ start &gt;&gt;
add_task.override(task_id=f&quot;add_start_{i}&quot;)(start, i)
+</code></pre>
+
+<p>More information can be found here: <a
href="https://airflow.apache.org/docs/apache-airflow/2.3.0/tutorial_taskflow_api.html#reusing-a-decorated-task"
target="_blank">Reuse of decorated DAGs</a></p>
+
+<h2 id="other-small-features">Other small features</h2>
+
+<p>This isn’t a comprehensive list, but some noteworthy or interesting
small features include:</p>
+
+<ul>
+<li>Support different timeout value for dag file parsing</li>
+<li><code>airflow dags reserialize</code> command to
reserialize dags</li>
+<li>Events Timetable</li>
+<li>SmoothOperator - Operator that does literally nothing except logging
a YouTube link to
+Sade&rsquo;s &ldquo;Smooth Operator&rdquo;. Enjoy!</li>
+</ul>
+
+<h2 id="contributors">Contributors</h2>
+
+<p>Thanks to everyone who contributed to this release: Ash
Berlin-Taylor, Brent Bovenzi, Daniel Standish, Elad, Ephraim Anierobi, Jarek
Potiuk, Jed Cunningham, Josh Fell, Kamil Breguła, Kanthi, Kaxil Naik, Khalid
Mammadov, Malthe Borch, Ping Zhang, Tzu-ping Chung and many others who keep
making Airflow better for everyone.</p>
+
+ </description>
+ </item>
+
<item>
<title>Blog: What's new in Apache Airflow 2.2.0</title>
<link>/blog/airflow-2.2.0/</link>
diff --git a/search/index.html b/search/index.html
index c00bfdfbe..e3140667f 100644
--- a/search/index.html
+++ b/search/index.html
@@ -35,12 +35,12 @@
<meta property="og:url" content="/search/" />
<meta property="og:image" content="/images/feature-image.png" />
-<meta property="article:modified_time" content="2022-04-30T23:44:13+01:00"
/><meta property="og:site_name" content="Apache Airflow" />
+<meta property="article:modified_time" content="2022-05-01T01:14:31+01:00"
/><meta property="og:site_name" content="Apache Airflow" />
<meta itemprop="name" content="Search Results">
<meta itemprop="description" content="">
-<meta itemprop="dateModified" content="2022-04-30T23:44:13+01:00" />
+<meta itemprop="dateModified" content="2022-05-01T01:14:31+01:00" />
<meta itemprop="wordCount" content="0">
diff --git a/sitemap.xml b/sitemap.xml
index e8441858d..2c0ffbf26 100644
--- a/sitemap.xml
+++ b/sitemap.xml
@@ -4,157 +4,162 @@
<url>
<loc>/docs/</loc>
- <lastmod>2022-04-30T23:44:13+01:00</lastmod>
+ <lastmod>2022-05-01T01:14:31+01:00</lastmod>
+ </url>
+
+ <url>
+ <loc>/blog/airflow-2.3.0/</loc>
+ <lastmod>2022-05-01T01:14:31+01:00</lastmod>
</url>
<url>
<loc>/blog/tags/release/</loc>
- <lastmod>2022-04-30T23:44:13+01:00</lastmod>
+ <lastmod>2022-05-01T01:14:31+01:00</lastmod>
</url>
<url>
<loc>/tags/</loc>
- <lastmod>2022-04-30T23:44:13+01:00</lastmod>
+ <lastmod>2022-05-01T01:14:31+01:00</lastmod>
</url>
<url>
<loc>/blog/airflow-2.2.0/</loc>
- <lastmod>2022-04-30T23:44:13+01:00</lastmod>
+ <lastmod>2022-05-01T01:14:31+01:00</lastmod>
</url>
<url>
<loc>/blog/tags/airflow-summit/</loc>
- <lastmod>2022-04-30T23:44:13+01:00</lastmod>
+ <lastmod>2022-05-01T01:14:31+01:00</lastmod>
</url>
<url>
<loc>/blog/airflow_summit_2021/</loc>
- <lastmod>2022-04-30T23:44:13+01:00</lastmod>
+ <lastmod>2022-05-01T01:14:31+01:00</lastmod>
</url>
<url>
<loc>/blog/tags/community/</loc>
- <lastmod>2022-04-30T23:44:13+01:00</lastmod>
+ <lastmod>2022-05-01T01:14:31+01:00</lastmod>
</url>
<url>
<loc>/blog/airflow-survey-2020/</loc>
- <lastmod>2022-04-30T23:44:13+01:00</lastmod>
+ <lastmod>2022-05-01T01:14:31+01:00</lastmod>
</url>
<url>
<loc>/blog/tags/survey/</loc>
- <lastmod>2022-04-30T23:44:13+01:00</lastmod>
+ <lastmod>2022-05-01T01:14:31+01:00</lastmod>
</url>
<url>
<loc>/blog/tags/users/</loc>
- <lastmod>2022-04-30T23:44:13+01:00</lastmod>
+ <lastmod>2022-05-01T01:14:31+01:00</lastmod>
</url>
<url>
<loc>/blog/airflow-two-point-oh-is-here/</loc>
- <lastmod>2022-04-30T23:44:13+01:00</lastmod>
+ <lastmod>2022-05-01T01:14:31+01:00</lastmod>
</url>
<url>
<loc>/blog/experience-with-airflow-as-an-outreachy-intern/</loc>
- <lastmod>2022-04-30T23:44:13+01:00</lastmod>
+ <lastmod>2022-05-01T01:14:31+01:00</lastmod>
</url>
<url>
<loc>/blog/airflow-1.10.12/</loc>
- <lastmod>2022-04-30T23:44:13+01:00</lastmod>
+ <lastmod>2022-05-01T01:14:31+01:00</lastmod>
</url>
<url>
<loc>/blog/apache-airflow-for-newcomers/</loc>
- <lastmod>2022-04-30T23:44:13+01:00</lastmod>
+ <lastmod>2022-05-01T01:14:31+01:00</lastmod>
</url>
<url>
<loc>/announcements/</loc>
- <lastmod>2022-04-30T23:44:13+01:00</lastmod>
+ <lastmod>2022-05-01T01:14:31+01:00</lastmod>
</url>
<url>
<loc>/blog/implementing-stable-api-for-apache-airflow/</loc>
- <lastmod>2022-04-30T23:44:13+01:00</lastmod>
+ <lastmod>2022-05-01T01:14:31+01:00</lastmod>
</url>
<url>
<loc>/blog/tags/rest-api/</loc>
- <lastmod>2022-04-30T23:44:13+01:00</lastmod>
+ <lastmod>2022-05-01T01:14:31+01:00</lastmod>
</url>
<url>
<loc>/blog/airflow-1.10.10/</loc>
- <lastmod>2022-04-30T23:44:13+01:00</lastmod>
+ <lastmod>2022-05-01T01:14:31+01:00</lastmod>
</url>
<url>
<loc>/blog/airflow-1.10.8-1.10.9/</loc>
- <lastmod>2022-04-30T23:44:13+01:00</lastmod>
+ <lastmod>2022-05-01T01:14:31+01:00</lastmod>
</url>
<url>
<loc>/blog/tags/documentation/</loc>
- <lastmod>2022-04-30T23:44:13+01:00</lastmod>
+ <lastmod>2022-05-01T01:14:31+01:00</lastmod>
</url>
<url>
<loc>/blog/experience-in-google-season-of-docs-2019-with-apache-airflow/</loc>
- <lastmod>2022-04-30T23:44:13+01:00</lastmod>
+ <lastmod>2022-05-01T01:14:31+01:00</lastmod>
</url>
<url>
<loc>/blog/airflow-survey/</loc>
- <lastmod>2022-04-30T23:44:13+01:00</lastmod>
+ <lastmod>2022-05-01T01:14:31+01:00</lastmod>
</url>
<url>
<loc>/blog/announcing-new-website/</loc>
- <lastmod>2022-04-30T23:44:13+01:00</lastmod>
+ <lastmod>2022-05-01T01:14:31+01:00</lastmod>
</url>
<url>
<loc>/blog/apache-con-europe-2019-thoughts-and-insights-by-airflow-committers/</loc>
- <lastmod>2022-04-30T23:44:13+01:00</lastmod>
+ <lastmod>2022-05-01T01:14:31+01:00</lastmod>
</url>
<url>
<loc>/blog/tags/development/</loc>
- <lastmod>2022-04-30T23:44:13+01:00</lastmod>
+ <lastmod>2022-05-01T01:14:31+01:00</lastmod>
</url>
<url>
<loc>/blog/documenting-using-local-development-environments/</loc>
- <lastmod>2022-04-30T23:44:13+01:00</lastmod>
+ <lastmod>2022-05-01T01:14:31+01:00</lastmod>
</url>
<url>
<loc>/blog/its-a-breeze-to-develop-apache-airflow/</loc>
- <lastmod>2022-04-30T23:44:13+01:00</lastmod>
+ <lastmod>2022-05-01T01:14:31+01:00</lastmod>
</url>
<url>
<loc>/use-cases/adobe/</loc>
- <lastmod>2022-04-30T23:44:13+01:00</lastmod>
+ <lastmod>2022-05-01T01:14:31+01:00</lastmod>
</url>
<url>
<loc>/use-cases/adyen/</loc>
- <lastmod>2022-04-30T23:44:13+01:00</lastmod>
+ <lastmod>2022-05-01T01:14:31+01:00</lastmod>
</url>
<url>
<loc>/use-cases/big-fish-games/</loc>
- <lastmod>2022-04-30T23:44:13+01:00</lastmod>
+ <lastmod>2022-05-01T01:14:31+01:00</lastmod>
</url>
<url>
<loc>/blog/</loc>
- <lastmod>2022-04-30T23:44:13+01:00</lastmod>
+ <lastmod>2022-05-01T01:14:31+01:00</lastmod>
</url>
<url>
@@ -163,77 +168,77 @@
<url>
<loc>/community/</loc>
- <lastmod>2022-04-30T23:44:13+01:00</lastmod>
+ <lastmod>2022-05-01T01:14:31+01:00</lastmod>
</url>
<url>
<loc>/use-cases/dish/</loc>
- <lastmod>2022-04-30T23:44:13+01:00</lastmod>
+ <lastmod>2022-05-01T01:14:31+01:00</lastmod>
</url>
<url>
<loc>/ecosystem/</loc>
- <lastmod>2022-04-30T23:44:13+01:00</lastmod>
+ <lastmod>2022-05-01T01:14:31+01:00</lastmod>
</url>
<url>
<loc>/use-cases/experity/</loc>
- <lastmod>2022-04-30T23:44:13+01:00</lastmod>
+ <lastmod>2022-05-01T01:14:31+01:00</lastmod>
</url>
<url>
<loc>/</loc>
- <lastmod>2022-04-30T23:44:13+01:00</lastmod>
+ <lastmod>2022-05-01T01:14:31+01:00</lastmod>
</url>
<url>
<loc>/install/</loc>
- <lastmod>2022-04-30T23:44:13+01:00</lastmod>
+ <lastmod>2022-05-01T01:14:31+01:00</lastmod>
</url>
<url>
<loc>/meetups/</loc>
- <lastmod>2022-04-30T23:44:13+01:00</lastmod>
+ <lastmod>2022-05-01T01:14:31+01:00</lastmod>
</url>
<url>
<loc>/use-cases/onefootball/</loc>
- <lastmod>2022-04-30T23:44:13+01:00</lastmod>
+ <lastmod>2022-05-01T01:14:31+01:00</lastmod>
</url>
<url>
<loc>/use-cases/plarium-krasnodar/</loc>
- <lastmod>2022-04-30T23:44:13+01:00</lastmod>
+ <lastmod>2022-05-01T01:14:31+01:00</lastmod>
</url>
<url>
<loc>/privacy-notice/</loc>
- <lastmod>2022-04-30T23:44:13+01:00</lastmod>
+ <lastmod>2022-05-01T01:14:31+01:00</lastmod>
</url>
<url>
<loc>/roadmap/</loc>
- <lastmod>2022-04-30T23:44:13+01:00</lastmod>
+ <lastmod>2022-05-01T01:14:31+01:00</lastmod>
</url>
<url>
<loc>/search/</loc>
- <lastmod>2022-04-30T23:44:13+01:00</lastmod>
+ <lastmod>2022-05-01T01:14:31+01:00</lastmod>
</url>
<url>
<loc>/use-cases/seniorlink/</loc>
- <lastmod>2022-04-30T23:44:13+01:00</lastmod>
+ <lastmod>2022-05-01T01:14:31+01:00</lastmod>
</url>
<url>
<loc>/use-cases/sift/</loc>
- <lastmod>2022-04-30T23:44:13+01:00</lastmod>
+ <lastmod>2022-05-01T01:14:31+01:00</lastmod>
</url>
<url>
<loc>/use-cases/</loc>
- <lastmod>2022-04-30T23:44:13+01:00</lastmod>
+ <lastmod>2022-05-01T01:14:31+01:00</lastmod>
</url>
</urlset>
\ No newline at end of file
diff --git a/tags/index.html b/tags/index.html
index 6cceaad3c..7e0f70b90 100644
--- a/tags/index.html
+++ b/tags/index.html
@@ -37,7 +37,7 @@
<meta property="og:image" content="/images/feature-image.png" />
-<meta property="og:updated_time" content="2021-10-11T00:00:00+00:00" /><meta
property="og:site_name" content="Apache Airflow" />
+<meta property="og:updated_time" content="2022-04-30T00:00:00+00:00" /><meta
property="og:site_name" content="Apache Airflow" />
<meta itemprop="name" content="Tags">
<meta itemprop="description" content="Platform created by the community to
programmatically author, schedule and monitor workflows.">
diff --git a/tags/index.xml b/tags/index.xml
index 779b1e12e..959674e76 100644
--- a/tags/index.xml
+++ b/tags/index.xml
@@ -4,7 +4,7 @@
<link>/tags/</link>
<description>Recent content in Tags on Apache Airflow</description>
<generator>Hugo -- gohugo.io</generator>
- <lastBuildDate>Mon, 11 Oct 2021 00:00:00 +0000</lastBuildDate>
+ <lastBuildDate>Sat, 30 Apr 2022 00:00:00 +0000</lastBuildDate>
<atom:link href="/tags/index.xml" rel="self"
type="application/rss+xml" />
diff --git a/use-cases/adobe/index.html b/use-cases/adobe/index.html
index b7b944801..7936359b2 100644
--- a/use-cases/adobe/index.html
+++ b/use-cases/adobe/index.html
@@ -35,12 +35,12 @@
<meta property="og:url" content="/use-cases/adobe/" />
<meta property="og:image" content="/images/feature-image.png" />
-<meta property="article:modified_time" content="2022-04-30T23:44:13+01:00"
/><meta property="og:site_name" content="Apache Airflow" />
+<meta property="article:modified_time" content="2022-05-01T01:14:31+01:00"
/><meta property="og:site_name" content="Apache Airflow" />
<meta itemprop="name" content="Adobe">
<meta itemprop="description" content="What was the problem? Modern big data
platforms need sophisticated data pipelines connecting to many backend services
enabling complex workflows. These workflows need to be deployed, monitored, and
run either on regular schedules or triggered by external events. Adobe
Experience Platform component services architected and built an orchestration
service to enable their users to author, schedule, and monitor complex
hierarchical (including sequential a [...]
-<meta itemprop="dateModified" content="2022-04-30T23:44:13+01:00" />
+<meta itemprop="dateModified" content="2022-05-01T01:14:31+01:00" />
<meta itemprop="wordCount" content="251">
diff --git a/use-cases/adyen/index.html b/use-cases/adyen/index.html
index e6ade97c6..5ef7c5dc0 100644
--- a/use-cases/adyen/index.html
+++ b/use-cases/adyen/index.html
@@ -35,12 +35,12 @@
<meta property="og:url" content="/use-cases/adyen/" />
<meta property="og:image" content="/images/feature-image.png" />
-<meta property="article:modified_time" content="2022-04-30T23:44:13+01:00"
/><meta property="og:site_name" content="Apache Airflow" />
+<meta property="article:modified_time" content="2022-05-01T01:14:31+01:00"
/><meta property="og:site_name" content="Apache Airflow" />
<meta itemprop="name" content="Adyen">
<meta itemprop="description" content="What was the problem? Many years ago we
started out with our own orchestration framework. Due to all the required
custom functionality it made sense at the time. However, quickly we realized
creating an orchestration tool is not to be underestimated. With the quickly
increasing number of users and teams, time spent on fixing issues increased,
severely limiting development speed. Furthermore, due to it not being open
source, we constantly had to make [...]
-<meta itemprop="dateModified" content="2022-04-30T23:44:13+01:00" />
+<meta itemprop="dateModified" content="2022-05-01T01:14:31+01:00" />
<meta itemprop="wordCount" content="405">
diff --git a/use-cases/big-fish-games/index.html
b/use-cases/big-fish-games/index.html
index 376bc1b3e..3a5c869a3 100644
--- a/use-cases/big-fish-games/index.html
+++ b/use-cases/big-fish-games/index.html
@@ -35,12 +35,12 @@
<meta property="og:url" content="/use-cases/big-fish-games/" />
<meta property="og:image" content="/images/feature-image.png" />
-<meta property="article:modified_time" content="2022-04-30T23:44:13+01:00"
/><meta property="og:site_name" content="Apache Airflow" />
+<meta property="article:modified_time" content="2022-05-01T01:14:31+01:00"
/><meta property="og:site_name" content="Apache Airflow" />
<meta itemprop="name" content="Big Fish Games">
<meta itemprop="description" content="What was the problem? The main challenge
is the lack of standardized ETL workflow orchestration tools. PowerShell and
Python-based ETL frameworks built in-house are currently used for scheduling
and running analytical workloads. However, there is no web UI through which we
can monitor these workflows and it requires additional effort to maintain this
framework. These scheduled jobs based on external dependencies are not well
suited to modern Big Data [...]
-<meta itemprop="dateModified" content="2022-04-30T23:44:13+01:00" />
+<meta itemprop="dateModified" content="2022-05-01T01:14:31+01:00" />
<meta itemprop="wordCount" content="336">
diff --git a/use-cases/dish/index.html b/use-cases/dish/index.html
index 200e7f7de..03241a5b7 100644
--- a/use-cases/dish/index.html
+++ b/use-cases/dish/index.html
@@ -35,12 +35,12 @@
<meta property="og:url" content="/use-cases/dish/" />
<meta property="og:image" content="/images/feature-image.png" />
-<meta property="article:modified_time" content="2022-04-30T23:44:13+01:00"
/><meta property="og:site_name" content="Apache Airflow" />
+<meta property="article:modified_time" content="2022-05-01T01:14:31+01:00"
/><meta property="og:site_name" content="Apache Airflow" />
<meta itemprop="name" content="Dish">
<meta itemprop="description" content="What was the problem? We faced
increasing complexity managing lengthy crontabs with scheduling being an issue,
this required carefully planning timing due to resource constraints, usage
patterns, and especially custom code needed for retry logic. In the last case,
having to verify success of previous jobs and/or steps prior to running the
next. Furthermore, time to results is important, but we were increasingly
relying on buffers for processing, wher [...]
-<meta itemprop="dateModified" content="2022-04-30T23:44:13+01:00" />
+<meta itemprop="dateModified" content="2022-05-01T01:14:31+01:00" />
<meta itemprop="wordCount" content="248">
diff --git a/use-cases/experity/index.html b/use-cases/experity/index.html
index 927eeab95..b76b2db64 100644
--- a/use-cases/experity/index.html
+++ b/use-cases/experity/index.html
@@ -36,13 +36,13 @@ How did Apache Airflow help to solve this problem?
Ultimately we decided flexibl
<meta property="og:url" content="/use-cases/experity/" />
<meta property="og:image" content="/images/feature-image.png" />
-<meta property="article:modified_time" content="2022-04-30T23:44:13+01:00"
/><meta property="og:site_name" content="Apache Airflow" />
+<meta property="article:modified_time" content="2022-05-01T01:14:31+01:00"
/><meta property="og:site_name" content="Apache Airflow" />
<meta itemprop="name" content="Experity">
<meta itemprop="description" content="What was the problem? We had to deploy
our complex, flagship app to multiple nodes in multiple ways. This required
tasks to communicate across Windows nodes and coordinate timing perfectly. We
did not want to buy an expensive enterprise scheduling tool and needed ultimate
flexibility.
How did Apache Airflow help to solve this problem? Ultimately we decided
flexible, multi-node, DAG capable tooling was key and airflow was one of the
few tools that fit that bill.">
-<meta itemprop="dateModified" content="2022-04-30T23:44:13+01:00" />
+<meta itemprop="dateModified" content="2022-05-01T01:14:31+01:00" />
<meta itemprop="wordCount" content="191">
diff --git a/use-cases/onefootball/index.html b/use-cases/onefootball/index.html
index 5b80c064f..e1537b5fc 100644
--- a/use-cases/onefootball/index.html
+++ b/use-cases/onefootball/index.html
@@ -36,13 +36,13 @@ On top of that, new data tools appear each month: third
party data sources, clou
<meta property="og:url" content="/use-cases/onefootball/" />
<meta property="og:image" content="/images/feature-image.png" />
-<meta property="article:modified_time" content="2022-04-30T23:44:13+01:00"
/><meta property="og:site_name" content="Apache Airflow" />
+<meta property="article:modified_time" content="2022-05-01T01:14:31+01:00"
/><meta property="og:site_name" content="Apache Airflow" />
<meta itemprop="name" content="Onefootball">
<meta itemprop="description" content="What was the problem? With millions of
daily active users, managing the complexity of data engineering at Onefootball
is a constant challenge. Lengthy crontabs, multiplication of custom API
clients, erosion of confidence in the analytics served, increasing heroism
(“only one person can solve this issue”). Those are the challenges
that most teams face unless they consciously invest in their tools and
processes.
On top of that, new data tools appear each month: third party data sources,
cloud providers solutions, different storage technologies… Managing all
those integrations is costly and brittle, especially for small data engineering
teams that are trying to do more with less.">
-<meta itemprop="dateModified" content="2022-04-30T23:44:13+01:00" />
+<meta itemprop="dateModified" content="2022-05-01T01:14:31+01:00" />
<meta itemprop="wordCount" content="294">
diff --git a/use-cases/plarium-krasnodar/index.html
b/use-cases/plarium-krasnodar/index.html
index 60ca6e577..477dfc612 100644
--- a/use-cases/plarium-krasnodar/index.html
+++ b/use-cases/plarium-krasnodar/index.html
@@ -35,12 +35,12 @@
<meta property="og:url" content="/use-cases/plarium-krasnodar/" />
<meta property="og:image" content="/images/feature-image.png" />
-<meta property="article:modified_time" content="2022-04-30T23:44:13+01:00"
/><meta property="og:site_name" content="Apache Airflow" />
+<meta property="article:modified_time" content="2022-05-01T01:14:31+01:00"
/><meta property="og:site_name" content="Apache Airflow" />
<meta itemprop="name" content="Plarium Krasnodar">
<meta itemprop="description" content="What was the problem? Our Research &
Development department carries out various experiments, and in all of them, we
need to create workflow orchestrations for solving tasks in game dev.
Previously, we didn’t have any suitable tools with a sufficient number of
built-in functions, and we had to orchestrate processes manually and entirely
from scratch every time. This led to difficulties with dependencies and
monitoring when building complex w [...]
-<meta itemprop="dateModified" content="2022-04-30T23:44:13+01:00" />
+<meta itemprop="dateModified" content="2022-05-01T01:14:31+01:00" />
<meta itemprop="wordCount" content="257">
diff --git a/use-cases/seniorlink/index.html b/use-cases/seniorlink/index.html
index 89d8b925e..c5a34fa48 100644
--- a/use-cases/seniorlink/index.html
+++ b/use-cases/seniorlink/index.html
@@ -36,13 +36,13 @@ We had built a robust stack of batch processes to deliver
value to the business,
<meta property="og:url" content="/use-cases/seniorlink/" />
<meta property="og:image" content="/images/feature-image.png" />
-<meta property="article:modified_time" content="2022-04-30T23:44:13+01:00"
/><meta property="og:site_name" content="Apache Airflow" />
+<meta property="article:modified_time" content="2022-05-01T01:14:31+01:00"
/><meta property="og:site_name" content="Apache Airflow" />
<meta itemprop="name" content="Seniorlink">
<meta itemprop="description" content="What was the problem? Here at
Seniorlink, we provide services, support, and technology that engages family
caregivers. One of our focuses is using data to bolster our knowledge and
improve the experience of our users. Like many looking to build an effective
data stack, we adopted a Python, Spark, Redshift, and Tableau core toolset.
We had built a robust stack of batch processes to deliver value to the
business, deploying these data services in AWS using a mixture of EMR, ECS,
Lambda, and EC2.">
-<meta itemprop="dateModified" content="2022-04-30T23:44:13+01:00" />
+<meta itemprop="dateModified" content="2022-05-01T01:14:31+01:00" />
<meta itemprop="wordCount" content="400">
diff --git a/use-cases/sift/index.html b/use-cases/sift/index.html
index 766c9a5c1..78740c1b4 100644
--- a/use-cases/sift/index.html
+++ b/use-cases/sift/index.html
@@ -35,12 +35,12 @@
<meta property="og:url" content="/use-cases/sift/" />
<meta property="og:image" content="/images/feature-image.png" />
-<meta property="article:modified_time" content="2022-04-30T23:44:13+01:00"
/><meta property="og:site_name" content="Apache Airflow" />
+<meta property="article:modified_time" content="2022-05-01T01:14:31+01:00"
/><meta property="og:site_name" content="Apache Airflow" />
<meta itemprop="name" content="Sift">
<meta itemprop="description" content="What was the problem? At Sift, we’re
constantly training machine learning models that feed into the core of Sift’s
Digital Trust & Safety platform. The platform gives our customers a way to
discern suspicious online behavior from trustworthy behavior, allowing our
customers to protect their online transactions, maintain the integrity of their
content platforms, and keep their users’ accounts secure. To make this
possible, we’ve built model traini [...]
-<meta itemprop="dateModified" content="2022-04-30T23:44:13+01:00" />
+<meta itemprop="dateModified" content="2022-05-01T01:14:31+01:00" />
<meta itemprop="wordCount" content="641">
