[VOTE] Release Apache Airflow Helm Chart 1.14.0 based on 1.14.0rc1

[Jed Cunningham]

Hello Apache Airflow Community,

This is a call for the vote to release Helm Chart version 1.14.0.

The release candidate is available at:
https://dist.apache.org/repos/dist/dev/airflow/helm-chart/1.14.0rc1/

airflow-chart-1.14.0-source.tar.gz - is the "main source release" that
comes with INSTALL instructions.
airflow-1.14.0.tgz - is the binary Helm Chart release.

Public keys are available at: https://www.apache.org/dist/airflow/KEYS

For convenience "index.yaml" has been uploaded (though excluded from
voting), so you can also run the below commands.

helm repo add apache-airflow-dev
https://dist.apache.org/repos/dist/dev/airflow/helm-chart/1.14.0rc1/
helm repo update
helm install airflow apache-airflow-dev/airflow

airflow-1.14.0.tgz.prov - is also uploaded for verifying Chart Integrity,
though not strictly required for releasing the artifact based on ASF
Guidelines.

$ helm gpg verify airflow-1.14.0.tgz
gpg: Signature made Fri Jun 14 14:41:31 2024 MDT
gpg:using RSA key E1A1E984F55B8F280BD9CBA20BB7163892A2E48E
gpg:issuer "jedcunning...@apache.org"
gpg: Good signature from "Jed Cunningham "
[ultimate]
plugin: Chart SHA verified.
sha256:206d7eae00697bfd2fbe896b58adf9d66f928f70ee75a07e5772be86c9ed6185

The vote will be open for at least 72 hours (2024-06-17 20:59 UTC) or until
the necessary number of votes is reached.

https://www.timeanddate.com/countdown/to?iso=20240617T2059=136=cursive

Please vote accordingly:

[ ] +1 approve
[ ] +0 no opinion
[ ] -1 disapprove with the reason

Only votes from PMC members are binding, but members of the community are
encouraged to test the release and vote with "(non-binding)".

Consider this my (binding) +1.

For license checks, the .rat-excludes files is included, so you can run the
following to verify licenses (just update your path to rat):

tar -xvf airflow-chart-1.14.0-source.tar.gz
cd airflow-chart-1.14.0
java -jar apache-rat-0.13.jar chart -E .rat-excludes

Please note that the version number excludes the `rcX` string, so it's now
simply 1.14.0. This will allow us to rename the artifact without modifying
the artifact checksums when we actually release it.

The status of testing the Helm Chart by the community is kept here:
https://github.com/apache/airflow/issues/40248

Thanks,
Jed

Re: Call with Nielsen team demoing their DAG debugging feature

[Jed Cunningham]

Interesting. Can you forward to me as well Jarek? Thanks!

Re: [DISCUSS] AIP-71 Generalizing DAG Loader and Processor for Ephemeral Storage

[Jed Cunningham]

Sorry for the delayed reply here. I've been chewing on this one a bit
though.

One concern I have is that I highly value having a provider agnostic remote
git integration. fsspec, however, has local git or github - no arbitrary
remote git support. That means Airflow, in my view, can't just rely on
fsspec alone, but will have to also deal with cloning arbitrary repos. Or
we expect a "ready to go" cloned repo on local disk, which might be a good
tradeoff. This was something we'd have to tackle for AIP-63 anyways, though.

Another concern, as I've been considering Airflow 3, is that I see wanting
to version "more" than just the DAG folder - possibly a whole venv (or
similar). Doing that versioning file-by-file doesn't feel particularly
practical to me. Which means that, I think, we could end up using fsspec to
get zips/tarballs/"assets", which is still very helpful imo. But it does
limit the "slickness" of being sorta behind the scenes like a custom module
loader and resource loader would be.

I actually see this as also being coupled to Ash's upcoming “Task Execution
interface”.

I admittedly didn't quite follow the VersionedFS idea you all were
discussing. I don't see how it could easily allow Airflows goal of "this
version of the dag folder and maybe more" to a backend that is versioned by
file. Maybe you can eli5 and connect the dots for me?

All that said, my goal this next week is to spend some quality time in this
area so we can all start nailing down options for Airflow 3.

Re: [VOTE] Airflow Providers prepared on June 07, 2024

[Jed Cunningham]

+1 (binding)

Checked reproducibility, signatures, checksums, licences. Looks good!

Re: [DISCUSS] Restore the SQL server backend

[Jed Cunningham]

Just for context, here were (roughly) the results from the 2023 Airflow
survey:

PostgreSQL: 75%
MySQL: 15%
MSSQL: 5%

Also, there are already discussions about potentially dropping MySQL
support in Airflow 3. Given all that and the points from the past vote, I
don't think it makes much sense to bring MSSQL back.

Re: [VOTE] May 2024 PR of the Month

[Jed Cunningham]

Some good contenders this month, but my vote also goes to #39336.

Re: Need information regarding the Apache airflow server down

[Jed Cunningham]

Hi Nikhil,

What "Airflow server" are you referring to? I'm not aware of anything that
the community provides being down currently.

Did you mean to send this to $work, not the community, perhaps?

Jed

Re: [VOTE] Proposal for adding Telemetry via Scarf

[Jed Cunningham]

+1 binding

Re: [VOTE] Release Airflow 2.9.1 from 2.9.1rc2

[Jed Cunningham]

+1 (binding)

Checked reproducibility, signatures, checksums, licences. Used it with the
helm chart with a few different configs.

Re: [HUGE DISCUSSION] Airflow3 and tactical (Airflow 2) vs strategic (Airflow 3) approach

[Jed Cunningham]

Very exciting! Looks like we will have a busy period of time ahead of us.
Overall I like the plan so far, especially using this year's Airflow Summit
as an opportunity to announce and gather feedback, and the 2025 version to
pitch upgrading.

I left a comment in the doc, but we might want to iterate on the
terminology we use for high priority or "synchronous" DAG runs to serve LLM
responses - I find "interactive DAG runs" a bit confusing.

[ANNOUNCE] Apache Airflow Python Client 2.9.0 Released

[Jed Cunningham]

Dear Airflow community,

I'm happy to announce that Apache Airflow Python Client 2.9.0 was just
released.

We made this version available on PyPI for convenience:
`pip install apache-airflow-client`
https://pypi.org/project/apache-airflow-client/2.9.0/

The documentation is available at:
https://github.com/apache/airflow-client-python/

Find the changelog here for more details:
https://github.com/apache/airflow-client-python/blob/main/CHANGELOG.md

Thanks,
Jed

[ANNOUNCE] Apache Airflow Python Client 2.9.0 Released

[Jed Cunningham]

Dear Airflow community,

I'm happy to announce that Apache Airflow Python Client 2.9.0 was just
released.

We made this version available on PyPI for convenience:
`pip install apache-airflow-client`
https://pypi.org/project/apache-airflow-client/2.9.0/

The documentation is available at:
https://github.com/apache/airflow-client-python/

Find the changelog here for more details:
https://github.com/apache/airflow-client-python/blob/main/CHANGELOG.md

Thanks,
Jed

[RESULT][VOTE] Release Apache Airflow Python Client 2.9.0 from 2.9.0rc1

[Jed Cunningham]

Hello,

Apache Airflow Python Client 2.9.0 (based on RC1) has been accepted.

4 "+1" binding votes received:
- Jed Cunningham
- Jarek Potiuk
- Kaxil Naik
- Hussein Awala

2 "+1" non-binding votes received:

- Pankaj Koti
- Amogh Desai

Vote thread:
https://lists.apache.org/thread/okybvk22mzzskflz8vbkvrd07cgj38gw

I'll continue with the release process, and the release announcement will
follow shortly.

Thanks,
Jed

Re: [VOTE] April 2024 PR of the Month

[Jed Cunningham]

I'll also vote for 38674 - cool stuff!

[VOTE] Release Apache Airflow Python Client 2.9.0 from 2.9.0rc1

[Jed Cunningham]

Hey fellow Airflowers,

I have cut the first release candidate for the Apache Airflow Python Client
2.9.0.
This email is calling for a vote on the release,
which will last for 72 hours. Consider this my (binding) +1.

Airflow Client 2.9.0rc1 is available at:
https://dist.apache.org/repos/dist/dev/airflow/clients/python/2.9.0rc1/

The apache_airflow_client-2.9.0.tar.gz is an sdist release that contains
INSTALL instructions, and also is the official source release.

The apache_airflow_client-2.9.0-py3-none-any.whl is a binary wheel release
that pip can install.

Those packages do not contain .rc* version as, when approved, they will be
released as the final version.

The rc packages are also available at PyPI (with rc suffix) and you can
install it with pip as usual:
https://pypi.org/project/apache-airflow-client/2.9.0rc1/

Public keys are available at:
https://dist.apache.org/repos/dist/release/airflow/KEYS

Only votes from PMC members are binding, but all members of the community
are encouraged to test the release and vote with "(non-binding)".

The test procedure for PMC members is described in:
https://github.com/apache/airflow/blob/main/dev/README_RELEASE_PYTHON_CLIENT.md#verify-the-release-candidate-by-pmc-members

The test procedure for contributors and members of the community who would
like to test this RC is described in:
https://github.com/apache/airflow/blob/main/dev/README_RELEASE_PYTHON_CLIENT.md#verify-the-release-candidate-by-contributors

*Changelog:*

*Major changes:*

- Allow users to write dag_id and task_id in their national characters,
added display name for dag / task (v2) (#38446)
- Add dataset_expression to grid dag details (#38121)
- Adding run_id column to log table (#37731)
- Show custom instance names for a mapped task in UI (#36797)
- Add excluded/included events to get_event_logs api (#37641)
- Filter Datasets by associated dag_ids (GET /datasets) (#37512)
- Add data_interval_start and data_interval_end in dagrun create API
endpoint (#36630)
- Return the specified field when get dag/dagRun (#36641)

*New API supported:*

- Add post endpoint for dataset events (#37570)
- Add "queuedEvent" endpoint to get/delete DatasetDagRunQueue (#37176)

Thanks,
Jed

Re: [DISCUSS] Proposal for adding Telemetry via Scarf

[Jed Cunningham]

+1, looking forward to having better data!

Re: [VOTE] Release Airflow 2.9.0 from 2.9.0rc3

[Jed Cunningham]

+1 (binding)

Checked reproducibility, signatures, checksums, licences. Used it with the
helm chart with a few different configs.

Re: [VOTE] Release Airflow 2.9.0 from 2.9.0rc2

[Jed Cunningham]

+1 (binding)

Checked reproducibility, signatures, checksums, licences. Used it with the
helm chart with a few different configs. All looks good!

[RESULT][VOTE] AIP-64: Keep TaskInstance try history

[Jed Cunningham]

Hello Airflow Community,

The vote for AIP-64 has completed and it has been accepted.

14 "+1" binding votes received:
- Jed Cunningham
- Tzu-ping Chung
- Pierra Jeambrun
- Kaxil Naik
- Jens Scheffler
- Jarek Potiuk
- Vikram Koka
- Amogh Desai
- Phani Kumar
- Niko Oliveira
- Brent Bovenzi
- Utkarsh Sharma
- Josh Fell
- Hussein Awala

6 "+1" non-binding received:
- Arita Basu
- Igor Kholopov
- Wei Lee
- Shubham Mehta
- Rahul Vats
- Ankit Chaurasia

Vote thread:
https://lists.apache.org/thread/cdrxd4tsq982gjmbbl32vp2ygt9dxgpk

Thanks everyone that took the time to vote!

Thanks,
Jed

[ANNOUNCE] Apache Airflow 2.8.4 Released

[Jed Cunningham]

Dear Airflow community,

I'm happy to announce that Airflow 2.8.4 was just released.

The released sources and packages can be downloaded via
https://airflow.apache.org/docs/apache-airflow/stable/installation/installing-from-sources.html

Other installation methods are described in
https://airflow.apache.org/docs/apache-airflow/stable/installation/

We also made this version available on PyPI for convenience:
`pip install apache-airflow`
https://pypi.org/project/apache-airflow/2.8.4/

The documentation is available at:
https://airflow.apache.org/docs/apache-airflow/2.8.4/

Find the release notes here for more details:
https://airflow.apache.org/docs/apache-airflow/2.8.4/release_notes.html

Container images are published at:
https://hub.docker.com/r/apache/airflow/tags/?page=1=2.8.4

Thanks,
Jed

[ANNOUNCE] Apache Airflow 2.8.4 Released

[Jed Cunningham]

Dear Airflow community,

I'm happy to announce that Airflow 2.8.4 was just released.

The released sources and packages can be downloaded via
https://airflow.apache.org/docs/apache-airflow/stable/installation/installing-from-sources.html

Other installation methods are described in
https://airflow.apache.org/docs/apache-airflow/stable/installation/

We also made this version available on PyPI for convenience:
`pip install apache-airflow`
https://pypi.org/project/apache-airflow/2.8.4/

The documentation is available at:
https://airflow.apache.org/docs/apache-airflow/2.8.4/

Find the release notes here for more details:
https://airflow.apache.org/docs/apache-airflow/2.8.4/release_notes.html

Container images are published at:
https://hub.docker.com/r/apache/airflow/tags/?page=1=2.8.4

Thanks,
Jed

[ANNOUNCE] Apache Airflow Helm Chart version 1.13.1 Released

[Jed Cunningham]

Dear Airflow community,

I am pleased to announce that we have released Apache Airflow Helm chart
1.13.1  

The source release, as well as the "binary" Helm Chart release, are
available:

   Official Sources:
https://airflow.apache.org/docs/helm-chart/1.13.1/installing-helm-chart-from-sources.html
   ArtifactHub:
https://artifacthub.io/packages/helm/apache-airflow/airflow
   Docs: https://airflow.apache.org/docs/helm-chart/1.13.1/
   Quick Start Installation Guide:
https://airflow.apache.org/docs/helm-chart/1.13.1/quick-start.html
️Release Notes:
https://airflow.apache.org/docs/helm-chart/1.13.1/release_notes.html

Thanks to all the contributors who made this possible.

Thanks,
Jed

[ANNOUNCE] Apache Airflow Helm Chart version 1.13.1 Released

[Jed Cunningham]

Dear Airflow community,

I am pleased to announce that we have released Apache Airflow Helm chart
1.13.1  

The source release, as well as the "binary" Helm Chart release, are
available:

   Official Sources:
https://airflow.apache.org/docs/helm-chart/1.13.1/installing-helm-chart-from-sources.html
   ArtifactHub:
https://artifacthub.io/packages/helm/apache-airflow/airflow
   Docs: https://airflow.apache.org/docs/helm-chart/1.13.1/
   Quick Start Installation Guide:
https://airflow.apache.org/docs/helm-chart/1.13.1/quick-start.html
️Release Notes:
https://airflow.apache.org/docs/helm-chart/1.13.1/release_notes.html

Thanks to all the contributors who made this possible.

Thanks,
Jed

[RESULT][VOTE] Release Airflow 2.8.4 from 2.8.4rc1

[Jed Cunningham]

Hello,

Apache Airflow 2.8.4 (based on RC1) has been accepted.

4 "+1" binding votes received:
- Jed Cunningham
- Jarek Potiuk
- Hussein Awala
- Andrey Anshin

3 "+1" non-binding votes received:

- Rahul Vats
- Amogh Desai
- Wei Lee

Vote thread:
https://lists.apache.org/thread/o7d198j2q5nxjm3osxz6mgglqgcof1jz

I'll continue with the release process, and the release announcement will
follow shortly.

Thanks,
Jed

Re: [VOTE] Release Airflow 2.8.4 from 2.8.4rc1

[Jed Cunningham]

Hey Andrey,

If you reimport from KEYS, I've already updated the expiration date a while
back :). But yes, that would be problematic otherwise.

Jed

On Sat, Mar 23, 2024 at 10:41 AM Andrey Anshin 
wrote:

> +1 binding
>
> Checked files, licences, signatures and also my changes.
>
> One small nit that I've found is that the key which is used for sign
> releases is expired, I'm not sure if it should be considered as a
> showstopper.
>
> ❯ gpg --verify apache_airflow-2.8.4-py3-none-any.whl.asc
> apache_airflow-2.8.4-py3-none-any.whl
> gpg: Signature made Wed Mar 20 19:17:26 2024 +04
> gpg:using RSA key E1A1E984F55B8F280BD9CBA20BB7163892A2E48E
> gpg:issuer "jedcunning...@apache.org"
> gpg: Good signature from "Jed Cunningham "
> [ultimate]
> gpg: Note: This key has expired!
> Primary key fingerprint: A020 DD36 34F1 A4D1 26A1  C554 7774 A4E5 90CB 0351
>  Subkey fingerprint: E1A1 E984 F55B 8F28 0BD9  CBA2 0BB7 1638 92A2 E48E
>
> ❯ gpg --list-options show-unusable-subkeys --list-keys
> A020DD3634F1A4D126A1C5547774A4E590CB0351
> pub   rsa4096 2022-01-05 [C]
>   A020DD3634F1A4D126A1C5547774A4E590CB0351
> uid   [ultimate] Jed Cunningham 
> sub   rsa4096 2022-01-05 [A] [expired: 2024-01-05]
> sub   rsa4096 2022-01-05 [E] [expired: 2024-01-05]
> sub   rsa4096 2022-01-05 [S] [expired: 2024-01-05]
>
>
>
>
> On Fri, 22 Mar 2024 at 15:24, Rahul Vats  wrote:
>
> > +1 (non-binding)
> >
> > Verified running our regression DAGS and API tests. LGTM
> >
> > Regards,
> > Rahul Vats
> > 9953794332
> >
> >
> > On Fri, 22 Mar 2024 at 15:24, Hussein Awala  wrote:
> >
> > > +1 (binding) checked licences, checksums, signatures, and source codes,
> > and
> > > ran some testing dags. All looks good.
> > >
> > > On Wed, Mar 20, 2024 at 7:54 PM Jarek Potiuk  wrote:
> > >
> > > > +1 (binding) - tested / verified all changes I was involved (either
> as
> > > > fixer, bug introducer or both, particularly when both), verified
> > > > reproducibility, licences, checksums, signatures, run a few DAGs -
> all
> > > > looks good.
> > > >
> > > > On Wed, Mar 20, 2024 at 4:56 PM Jed Cunningham <
> > jedcunning...@apache.org
> > > >
> > > > wrote:
> > > >
> > > > > Hey fellow Airflowers,
> > > > >
> > > > > I have cut Airflow 2.8.4rc1. This email is calling a vote on the
> > > release,
> > > > > which will last at least 72 hours, from Wednesday, March 20, 2024
> at
> > > 4:00
> > > > > pm UTC
> > > > > until Saturday, March 23, 2024 at 4:00 pm UTC, and until 3 binding
> +1
> > > > votes
> > > > > have been received.
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
> https://www.timeanddate.com/worldclock/fixedtime.html/?msg=8=20240323T1600=1440
> > > > >
> > > > > Status of testing of the release is kept in
> > > > > https://github.com/apache/airflow/issues/38334
> > > > >
> > > > > Consider this my (binding) +1.
> > > > >
> > > > > Airflow 2.8.4rc1 is available at:
> > > > > https://dist.apache.org/repos/dist/dev/airflow/2.8.4rc1/
> > > > >
> > > > > *apache-airflow-2.8.4-source.tar.gz* is a source release that comes
> > > with
> > > > > INSTALL instructions.
> > > > > *apache-airflow-2.8.4.tar.gz* is the binary Python "sdist" release.
> > > > > *apache_airflow-2.8.4-py3-none-any.whl* is the binary Python wheel
> > > > "binary"
> > > > > release.
> > > > >
> > > > > Public keys are available at:
> > > > > https://dist.apache.org/repos/dist/release/airflow/KEYS
> > > > >
> > > > > Please vote accordingly:
> > > > >
> > > > > [ ] +1 approve
> > > > > [ ] +0 no opinion
> > > > > [ ] -1 disapprove with the reason
> > > > >
> > > > > Only votes from PMC members are binding, but all members of the
> > > community
> > > > > are encouraged to test the release and vote with "(non-binding)".
> > > > >
> > > > > The test procedure for PMC members is described in:
> > > > >
> > > > >
> > > >
> > >
> >
> https://gi

[RESULT][VOTE] Release Apache Airflow Helm Chart 1.13.1 based on 1.13.1rc1

[Jed Cunningham]

Hello all,

The vote to release Apache Airflow Helm Chart version 1.13.1 based on
1.13.1rc1 is now closed.

The vote PASSED with 3 binding "+1", 1 non-binding "+1" and 0 "-1" votes:

"+1" Binding votes:

  - Jed Cunningham
  - Jarek Potiuk
  - Hussein Awala

"+1" Non-Binding votes:

  - Amogh Desai

Vote thread:
https://lists.apache.org/thread/zsx1bv24c3mx362ctny4tvlrzjo25j7p

I'll continue with the release process and the release announcement will
follow shortly.

Thanks,
Jed

[VOTE] AIP-64: Keep TaskInstance try history

[Jed Cunningham]

Hello Airflow Community,

I would like to start a vote on AIP-64: Keep TaskInstance try history.

You can find the AIP here:
https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-64%3A+Keep+TaskInstance+try+history

Discussion Thread:
https://lists.apache.org/thread/vvm43tfchyo92hmf40fqvmq0f5845bjr

This is the first step in the AIP-63 DAG Versioning journey, though this
provides value in isolation as well:
https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-63%3A+DAG+Versioning

The vote will last until 2024-03-28 17:30 UTC and until at least 3 binding
votes have been cast.

Consider this my binding +1.

Please vote accordingly:

[ ] + 1 approve
[ ] + 0 no opinion
[ ] - 1 disapprove with the reason

Only votes from PMC members and committers are binding, but other members
of the community are encouraged to check the AIP and vote with
"(non-binding)".

Thanks,
Jed

[VOTE] Release Apache Airflow Helm Chart 1.13.1 based on 1.13.1rc1

[Jed Cunningham]

Hello Apache Airflow Community,

This is a call for the vote to release Helm Chart version 1.13.1.

The release candidate is available at:
https://dist.apache.org/repos/dist/dev/airflow/helm-chart/1.13.1rc1/

airflow-chart-1.13.1-source.tar.gz - is the "main source release" that
comes with INSTALL instructions.
airflow-1.13.1.tgz - is the binary Helm Chart release.

Public keys are available at: https://www.apache.org/dist/airflow/KEYS

For convenience "index.yaml" has been uploaded (though excluded from
voting), so you can also run the below commands.

helm repo add apache-airflow-dev
https://dist.apache.org/repos/dist/dev/airflow/helm-chart/1.13.1rc1/
helm repo update
helm install airflow apache-airflow-dev/airflow

airflow-1.13.1.tgz.prov - is also uploaded for verifying Chart Integrity,
though not strictly required for releasing the artifact based on ASF
Guidelines.

$ helm gpg verify airflow-1.13.1.tgz
gpg: Signature made Thu Mar 21 12:13:02 2024 MDT
gpg:using RSA key E1A1E984F55B8F280BD9CBA20BB7163892A2E48E
gpg:issuer "jedcunning...@apache.org"
gpg: Good signature from "Jed Cunningham "
[ultimate]
plugin: Chart SHA verified.
sha256:ed6b2dea0d8f99eb9bd9cd6bc418db95f88a7bb3b8d1afb7fdc266b1ea411a15

The vote will be open for at least 72 hours (2024-03-24 18:29 UTC) or until
the necessary number of votes is reached.

https://www.timeanddate.com/countdown/to?iso=20240324T1829=136=cursive

Please vote accordingly:

[ ] +1 approve
[ ] +0 no opinion
[ ] -1 disapprove with the reason

Only votes from PMC members are binding, but members of the community are
encouraged to test the release and vote with "(non-binding)".

Consider this my (binding) +1.

For license checks, the .rat-excludes files is included, so you can run the
following to verify licenses (just update your path to rat):

tar -xvf airflow-chart-1.13.1-source.tar.gz
cd airflow-chart-1.13.1
java -jar apache-rat-0.13.jar chart -E .rat-excludes

Please note that the version number excludes the `rcX` string, so it's now
simply 1.13.1. This will allow us to rename the artifact without modifying
the artifact checksums when we actually release it.

The status of testing the Helm Chart by the community is kept here:
https://github.com/apache/airflow/issues/38382

Thanks,
Jed

[VOTE] Release Airflow 2.8.4 from 2.8.4rc1

[Jed Cunningham]

Hey fellow Airflowers,

I have cut Airflow 2.8.4rc1. This email is calling a vote on the release,
which will last at least 72 hours, from Wednesday, March 20, 2024 at 4:00
pm UTC
until Saturday, March 23, 2024 at 4:00 pm UTC, and until 3 binding +1 votes
have been received.

https://www.timeanddate.com/worldclock/fixedtime.html/?msg=8=20240323T1600=1440

Status of testing of the release is kept in
https://github.com/apache/airflow/issues/38334

Consider this my (binding) +1.

Airflow 2.8.4rc1 is available at:
https://dist.apache.org/repos/dist/dev/airflow/2.8.4rc1/

*apache-airflow-2.8.4-source.tar.gz* is a source release that comes with
INSTALL instructions.
*apache-airflow-2.8.4.tar.gz* is the binary Python "sdist" release.
*apache_airflow-2.8.4-py3-none-any.whl* is the binary Python wheel "binary"
release.

Public keys are available at:
https://dist.apache.org/repos/dist/release/airflow/KEYS

Please vote accordingly:

[ ] +1 approve
[ ] +0 no opinion
[ ] -1 disapprove with the reason

Only votes from PMC members are binding, but all members of the community
are encouraged to test the release and vote with "(non-binding)".

The test procedure for PMC members is described in:
https://github.com/apache/airflow/blob/main/dev/README_RELEASE_AIRFLOW.md#verify-the-release-candidate-by-pmc-members

The test procedure for and Contributors who would like to test this RC is
described in:
https://github.com/apache/airflow/blob/main/dev/README_RELEASE_AIRFLOW.md#verify-the-release-candidate-by-contributors


Please note that the version number excludes the `rcX` string, so it's now
simply 2.8.4. This will allow us to rename the artifact without modifying
the artifact checksums when we actually release.

Release Notes:
https://github.com/apache/airflow/blob/2.8.4rc1/RELEASE_NOTES.rst

For information on what goes into a release please see:
https://github.com/apache/airflow/blob/main/dev/WHAT_GOES_INTO_THE_NEXT_RELEASE.md

Changes since 2.8.3:
*Bugs*:
- Fix incorrect serialization of ``FixedTimezone`` (#38139)
- Fix excessive permission changing for log task handler (#38164)
- Fix task instances list link (#38096)
- Fix a bug where scheduler heartrate parameter was not used (#37992)
- Add padding to prevent grid horizontal scroll overlapping tasks (#37942)
- Fix hash caching in ``ObjectStoragePath`` (#37769)

*Miscellaneous*:
- Limit importlib_resources as it breaks ``pytest_rewrites`` (#38095,
#38139)
- Limit ``pandas`` to ``<2.2`` (#37748)
- Bump ``croniter`` to fix an issue with 29 Feb cron expressions (#38198)

*Doc-only Change*:
- Tell users what to do if their scanners find issues in the image (#37652)
- Add a section about debugging in Docker Compose with PyCharm (#37940)
- Update deferrable docs to clarify kwargs when trigger resumes operator
(#38122)

Thanks,
Jed

Re: [VOTE] Remove experimental API

[Jed Cunningham]

I forgot to add the "why" - I view this as a breaking change still.

Re: [VOTE] Remove experimental API

[Jed Cunningham]

-1. Even though it's been deprecated for a really long time now, I don't
think we should remove it in a minor 2 release. I think we should wait
until the next major.

[DISCUSS] AIP-63, AIP-64, and AIP-65: DAG Versioning

[Jed Cunningham]

Hello everyone!

I'm excited to start a discussion around DAG Versioning in Airflow. It's
been the most requested feature in the last 3 community surveys!

AIP-63: DAG Versioning


As this topic quickly becomes rather large, I've made AIP-63 an umbrella
AIP and split the specifics into separate AIPs:

AIP-64: Keep TaskInstance try history

AIP-65: Improve DAG history in UI

[WIP] AIP-66: Execution of specific DAG code versions


AIP-64 and AIP-65 are ready to be discussed in depth, while AIP-66 is there
to provide an intentionally high level vision of what we may want to tackle
before Airflow's "DAG versioning" story is complete.

Thanks,
Jed

[ANNOUNCE] Apache Airflow Helm Chart version 1.13.0 Released

[Jed Cunningham]

Dear Airflow community,

I am pleased to announce that we have released Apache Airflow Helm chart
1.13.0  

The source release, as well as the "binary" Helm Chart release, are
available:

   Official Sources:
https://airflow.apache.org/docs/helm-chart/1.13.0/installing-helm-chart-from-sources.html
   ArtifactHub:
https://artifacthub.io/packages/helm/apache-airflow/airflow
   Docs: https://airflow.apache.org/docs/helm-chart/1.13.0/
   Quick Start Installation Guide:
https://airflow.apache.org/docs/helm-chart/1.13.0/quick-start.html
️   Release Notes:
https://airflow.apache.org/docs/helm-chart1.13.0/release_notes.html

Thanks to all the contributors who made this possible.

Thanks,
Jed

[ANNOUNCE] Apache Airflow Helm Chart version 1.13.0 Released

[Jed Cunningham]

Dear Airflow community,

I am pleased to announce that we have released Apache Airflow Helm chart
1.13.0  

The source release, as well as the "binary" Helm Chart release, are
available:

   Official Sources:
https://airflow.apache.org/docs/helm-chart/1.13.0/installing-helm-chart-from-sources.html
   ArtifactHub:
https://artifacthub.io/packages/helm/apache-airflow/airflow
   Docs: https://airflow.apache.org/docs/helm-chart/1.13.0/
   Quick Start Installation Guide:
https://airflow.apache.org/docs/helm-chart/1.13.0/quick-start.html
️   Release Notes:
https://airflow.apache.org/docs/helm-chart1.13.0/release_notes.html

Thanks to all the contributors who made this possible.

Thanks,
Jed

[RESULT][VOTE] Release Apache Airflow Helm Chart 1.13.0 based on 1.13.0rc1

[Jed Cunningham]

Hello all,

The vote to release Apache Airflow Helm Chart version 1.13.0 based on
1.13.0rc1 is now closed.

The vote PASSED with 3 binding "+1", 2 non-binding "+1" and 0 "-1" votes:

"+1" Binding votes:

  - Jed Cunningham
  - Jarek Potiuk
  - Ephraim Anierobi

"+1" Non-Binding votes:

  - Amogh Desai
  - Utkarsh Sharma

Vote thread:https://lists.apache.org/thread/0xgxsjwsbd8qk3h1tn4gj01hoplz1p5k

I'll continue with the release process and the release announcement
will follow shortly.

Thanks,
Jed

[VOTE] Release Apache Airflow Helm Chart 1.13.0 based on 1.13.0rc1

[Jed Cunningham]

Hello Apache Airflow Community,

This is a call for the vote to release Helm Chart version 1.13.0.

The release candidate is available at:
https://dist.apache.org/repos/dist/dev/airflow/helm-chart/1.13.0rc1/

airflow-chart-1.13.0-source.tar.gz - is the "main source release" that
comes with INSTALL instructions.
airflow-1.13.0.tgz - is the binary Helm Chart release.

Public keys are available at: https://www.apache.org/dist/airflow/KEYS

For convenience "index.yaml" has been uploaded (though excluded from
voting), so you can also run the below commands.

helm repo add apache-airflow-dev
https://dist.apache.org/repos/dist/dev/airflow/helm-chart/1.13.0rc1/
helm repo update
helm install airflow apache-airflow-dev/airflow

airflow-1.13.0.tgz.prov - is also uploaded for verifying Chart Integrity,
though not strictly required for releasing the artifact based on ASF
Guidelines.

$ helm gpg verify airflow-1.13.0.tgz
gpg: Signature made Fri Mar  1 21:16:51 2024 MST
gpg:using RSA key E1A1E984F55B8F280BD9CBA20BB7163892A2E48E
gpg:issuer "jedcunning...@apache.org"
gpg: Good signature from "Jed Cunningham "
[ultimate]
plugin: Chart SHA verified.
sha256:23155cf90b66c8ec6d49d2060686f90d23329eecf71c5368b1f0b06681b816cc

The vote will be open for at least 72 hours (2024-03-05 04:35 UTC) or until
the necessary number of votes is reached.

https://www.timeanddate.com/countdown/to?iso=20240305T0435=136=cursive

Please vote accordingly:

[ ] +1 approve
[ ] +0 no opinion
[ ] -1 disapprove with the reason

Only votes from PMC members are binding, but members of the community are
encouraged to test the release and vote with "(non-binding)".

Consider this my (binding) +1.

For license checks, the .rat-excludes files is included, so you can run the
following to verify licenses (just update your path to rat):

tar -xvf airflow-chart-1.13.0-source.tar.gz
cd airflow-chart-1.13.0
java -jar apache-rat-0.13.jar chart -E .rat-excludes

Please note that the version number excludes the `rcX` string, so it's now
simply 1.13.0. This will allow us to rename the artifact without modifying
the artifact checksums when we actually release it.

The status of testing the Helm Chart by the community is kept here:
https://github.com/apache/airflow/issues/37844

Thanks,
Jed

Re: [VOTE] January 2024 PR of the Month

[Jed Cunningham]

I'm down for #37101 too, but we might consider including #37016
 as well. The former is just
a simpler interface for the latter.

[ANNOUNCE] Apache Airflow Helm Chart version 1.12.0 Released

[Jed Cunningham]

Dear Airflow community,

I am pleased to announce that we have released Apache Airflow Helm chart
1.12.0  

The source release, as well as the "binary" Helm Chart release, are
available:

   Official Sources:
https://airflow.apache.org/docs/helm-chart/1.12.0/installing-helm-chart-from-sources.html
   ArtifactHub:
https://artifacthub.io/packages/helm/apache-airflow/airflow
   Docs: https://airflow.apache.org/docs/helm-chart/1.12.0/
   Quick Start Installation Guide:
https://airflow.apache.org/docs/helm-chart/1.12.0/quick-start.html
️   Release Notes:
https://airflow.apache.org/docs/helm-chart/1.12.0/release_notes.html

Thanks to all the contributors who made this possible.

Thanks,
Jed

[ANNOUNCE] Apache Airflow Helm Chart version 1.12.0 Released

[Jed Cunningham]

Dear Airflow community,

I am pleased to announce that we have released Apache Airflow Helm chart
1.12.0  

The source release, as well as the "binary" Helm Chart release, are
available:

   Official Sources:
https://airflow.apache.org/docs/helm-chart/1.12.0/installing-helm-chart-from-sources.html
   ArtifactHub:
https://artifacthub.io/packages/helm/apache-airflow/airflow
   Docs: https://airflow.apache.org/docs/helm-chart/1.12.0/
   Quick Start Installation Guide:
https://airflow.apache.org/docs/helm-chart/1.12.0/quick-start.html
️   Release Notes:
https://airflow.apache.org/docs/helm-chart/1.12.0/release_notes.html

Thanks to all the contributors who made this possible.

Thanks,
Jed

[RESULT][VOTE] Release Apache Airflow Helm Chart 1.12.0 based on 1.12.0rc1

[Jed Cunningham]

Hello all,

The vote to release Apache Airflow Helm Chart version 1.12.0 based on
1.12.0rc1 is now closed.

The vote PASSED with 3 binding "+1" votes:

"+1" Binding votes:

  - Jed Cunningham
  - Jarek Potiuk
  - Hussein Awala

Vote thread:
https://lists.apache.org/thread/pqz480pfy5klbvxbf46s4thgjyooz1h9

I'll continue with the release process and the release announcement will
follow shortly.

Thanks,
Jed

[VOTE] Release Apache Airflow Helm Chart 1.12.0 based on 1.12.0rc1

[Jed Cunningham]

Hello Apache Airflow Community,

This is a call for the vote to release Helm Chart version 1.12.0.

The release candidate is available at:
https://dist.apache.org/repos/dist/dev/airflow/helm-chart/1.12.0rc1/

airflow-chart-1.12.0-source.tar.gz - is the "main source release" that
comes with INSTALL instructions.
airflow-1.12.0.tgz - is the binary Helm Chart release.

Public keys are available at: https://www.apache.org/dist/airflow/KEYS

For convenience "index.yaml" has been uploaded (though excluded from
voting), so you can also run the below commands.

helm repo add apache-airflow-dev
https://dist.apache.org/repos/dist/dev/airflow/helm-chart/1.12.0rc1/
helm repo update
helm install airflow apache-airflow-dev/airflow

airflow-1.12.0.tgz.prov - is also uploaded for verifying Chart Integrity,
though not strictly required for releasing the artifact based on ASF
Guidelines.

$ helm gpg verify airflow-1.12.0.tgz
gpg: Signature made Thu Feb  8 12:21:08 2024 MST
gpg:using RSA key E1A1E984F55B8F280BD9CBA20BB7163892A2E48E
gpg:issuer "jedcunning...@apache.org"
gpg: Good signature from "Jed Cunningham "
[ultimate]
plugin: Chart SHA verified.
sha256:a261081a40cb88c9052ad9c90968ddb701a2205d0eae816e7945a5b15466f054

The vote will be open for at least 72 hours (2024-02-11 19:45 UTC) or until
the necessary number of votes is reached.

https://www.timeanddate.com/countdown/to?iso=20240211T1945=136=cursive

Please vote accordingly:

[ ] +1 approve
[ ] +0 no opinion
[ ] -1 disapprove with the reason

Only votes from PMC members are binding, but members of the community are
encouraged to test the release and vote with "(non-binding)".

Consider this my (binding) +1.

For license checks, the .rat-excludes files is included, so you can run the
following to verify licenses (just update your path to rat):

tar -xvf airflow-chart-1.12.0-source.tar.gz
cd airflow-chart-1.12.0
java -jar apache-rat-0.13.jar chart -E .rat-excludes

Please note that the version number excludes the `rcX` string, so it's now
simply 1.12.0. This will allow us to rename the artifact without modifying
the artifact checksums when we actually release it.

The status of testing the Helm Chart by the community is kept here:
https://github.com/apache/airflow/issues/37259

Thanks,
Jed

Re: [DISCUSS] Rename channels on slack

[Jed Cunningham]

That's not what I'm proposing. I should have been more verbose before,
sorry. Let me rephrase.

Don't rename #development. Leave it where it is, restrict posting to admins
only. Create a new, fresh #contributing where everyone is still welcome.
One of us admins will post in #development with links to #troubleshooting,
#random, #contributing, etc. But #development is "dead" at that point for
everyone, just sticking around for a while so folks have time to
navigate/join the channels they care about going forward.

Basically, don't just rename #development -> #contributing, and carry 14k
folks along with us. But leave a map in #development for folks.

Re: [DISCUSS] Rename channels on slack

[Jed Cunningham]

Sounds good to me. We already have much more niche channels than best
practices would be. Worst case no one uses it and we can purge it down the
road, no harm no foul.

One thing we should consider is not renaming development, but starting with
a fresh channel for contributing. There are nearly 14k people in
development today. Maybe we restrict who can post in development for a
period of time with a message directing folks to the right places?

Re: [VOTE] "Require conversation resolution" setting in PRs as permanent solution

[Jed Cunningham]

-0.5

Re: [VOTE] Airflow Providers prepared on January 22, 2024

[Jed Cunningham]

+1 (binding) checked binary reproduction, licences, signatures, and
checksums.

On my system the binary reproduction check for the source tarball did fail,
but I spot verified the tarball contents are correct. I'm still
investigating, but no reason to hold the release for this.

Re: [VOTE] Release Airflow 2.8.1 from 2.8.1rc1

[Jed Cunningham]

+1 (binding)

Checked signatures, checksums, licences. Used it with the helm chart with a
few different configs

Re: [VOTE] October 2023 PR of the Month

[Jed Cunningham]

I'm slightly embarrassed I forgot about 34729, but yeah changing my vote to
that as well. Now (at least so far) it's unanimous :)

Re: [VOTE] October 2023 PR of the Month

[Jed Cunningham]

The new OpenSearch provider gets my vote - 34705.

Re: [VOTE] Add Multiple PR's of the Month to the Airflow Newsletter

[Jed Cunningham]

Just to clarify, I didn't mean we should highlight multiple PRs every
month. In that month none of @eumiro's PRs were individually enough to be
highlighted but in bulk they were. That was an unusual situation though.

I think we should be flexible with it. I'm not opposed to having multiple
per month, but if we do I think they should be equally notable. If we have
a clear winner, we should just highlight that one PR. My 2c.

I'll also call out that we (committers, community members, anyone really!)
should toss protm on good candidates as we are reviewing stuff day-to-day.
That bumps up the score in the script and will help ensure the good stuff
bubbles to the top of the scripts output! A quick search shows we've done
this less than a dozen times so far.

Re: [VOTE] Add providers for Pinecone, OpenAI & Cohere to enable first-class LLMOps

[Jed Cunningham]

+1 (binding)

Re: [VOTE] Airflow Providers prepared on October 13, 2023

[Jed Cunningham]

1 (binding)

Checked signatures, checksums, licences

Re: [VOTE] Release Airflow 2.7.2 from 2.7.2rc1

[Jed Cunningham]

+1 (binding)

Checked signatures, checksums, licences, compared sources. Used it with the
helm chart with a few different configs

[ANNOUNCE] Apache Airflow Helm Chart version 1.11.0 Released

[Jed Cunningham]

Dear Airflow community,

I am pleased to announce that we have released Apache Airflow Helm chart
1.11.0  

The source release, as well as the "binary" Helm Chart release, are
available:

   Official Sources:
https://airflow.apache.org/docs/helm-chart/1.11.0/installing-helm-chart-from-sources.html
   ArtifactHub:
https://artifacthub.io/packages/helm/apache-airflow/airflow
   Docs: https://airflow.apache.org/docs/helm-chart/1.11.0/
   Quick Start Installation Guide:
https://airflow.apache.org/docs/helm-chart/1.11.0/quick-start.html
️   Release Notes:
https://airflow.apache.org/docs/helm-chart/1.11.0/release_notes.html

Thanks to all the contributors who made this possible.

Cheers,
Jed

[ANNOUNCE] Apache Airflow Helm Chart version 1.11.0 Released

[Jed Cunningham]

Dear Airflow community,

I am pleased to announce that we have released Apache Airflow Helm chart
1.11.0  

The source release, as well as the "binary" Helm Chart release, are
available:

   Official Sources:
https://airflow.apache.org/docs/helm-chart/1.11.0/installing-helm-chart-from-sources.html
   ArtifactHub:
https://artifacthub.io/packages/helm/apache-airflow/airflow
   Docs: https://airflow.apache.org/docs/helm-chart/1.11.0/
   Quick Start Installation Guide:
https://airflow.apache.org/docs/helm-chart/1.11.0/quick-start.html
️   Release Notes:
https://airflow.apache.org/docs/helm-chart/1.11.0/release_notes.html

Thanks to all the contributors who made this possible.

Cheers,
Jed

[RESULT][VOTE] Release Apache Airflow Helm Chart 1.11.0 based on 1.11.0rc2

[Jed Cunningham]

Hello all,

The vote to release Apache Airflow Helm Chart version 1.11.0 based on
1.11.0rc2 is now closed.

The vote PASSED with 3 binding "+1", 2 non-binding "+1" and 0 "-1" votes:

"+1" Binding votes:

  - Jed Cunningham
  - Hussein Awala
  - Daniel Standish

"+1" Non-Binding votes:

  - Amogh Desai
  - Utkarsh Sharma

Vote thread:https://lists.apache.org/thread/wyoz9g7cldym3f6x4cnhvzsy76hgq0h5

I'll continue with the release process and the release announcement
will follow shortly.

Thanks,
Jed

Re: [VOTE] Release Apache Airflow Helm Chart 1.11.0 based on 1.11.0rc2

[Jed Cunningham]

> the `Chart.yaml` file which does not have a license

Quirk of helm ultimately. The source tarball does have the license, just
the convenience package tarball is missing it - helm processes the file and
removes the comment when it packages the chart. It's been this way forever
unfortunately.

However, I just had an idea which we can try out for future releases 

[VOTE] Release Apache Airflow Helm Chart 1.11.0 based on 1.11.0rc2

[Jed Cunningham]

Hello Apache Airflow Community,

This is a call for the vote to release Helm Chart version 1.11.0.

The release candidate is available at:
https://dist.apache.org/repos/dist/dev/airflow/helm-chart/1.11.0rc2/

airflow-chart-1.11.0-source.tar.gz - is the "main source release" that
comes with INSTALL instructions.
airflow-1.11.0.tgz - is the binary Helm Chart release.

Public keys are available at: https://www.apache.org/dist/airflow/KEYS

For convenience "index.yaml" has been uploaded (though excluded from
voting), so you can also run the below commands.

helm repo add apache-airflow-dev
https://dist.apache.org/repos/dist/dev/airflow/helm-chart/1.11.0rc2/
helm repo update
helm install airflow apache-airflow-dev/airflow

airflow-1.11.0.tgz.prov - is also uploaded for verifying Chart Integrity,
though not strictly required for releasing the artifact based on ASF
Guidelines.

$ helm gpg verify airflow-1.11.0.tgz
gpg: Signature made Sat Sep 30 11:45:11 2023 MDT
gpg:using RSA key E1A1E984F55B8F280BD9CBA20BB7163892A2E48E
gpg:issuer "jedcunning...@apache.org"
gpg: Good signature from "Jed Cunningham "
[ultimate]
plugin: Chart SHA verified.
sha256:4b3b696c323cd5b001fc03c2749ed096fc2967e7ad2c0d2ae42e68e2b6eeffa2

The vote will be open for at least 24 hours (2023-10-01 18:04 UTC) or until
the necessary number of votes is reached.

https://www.timeanddate.com/countdown/to?iso=20231001T1804=136=cursive

Please vote accordingly:

[ ] +1 approve
[ ] +0 no opinion
[ ] -1 disapprove with the reason

Only votes from PMC members are binding, but members of the community are
encouraged to test the release and vote with "(non-binding)".

Consider this my (binding) +1.

For license checks, the .rat-excludes files is included, so you can run the
following to verify licenses (just update your path to rat):

tar -xvf airflow-chart-1.11.0-source.tar.gz
cd airflow-chart-1.11.0
java -jar apache-rat-0.13.jar chart -E .rat-excludes

Please note that the version number excludes the `rcX` string, so it's now
simply 1.11.0. This will allow us to rename the artifact without modifying
the artifact checksums when we actually release it.

The status of testing the Helm Chart by the community is kept here:
https://github.com/apache/airflow/issues/34660

Changes since 1.11.0rc1:
remove lifecycle from init containers (#34677

Thanks,
Jed

Re: [VOTE] Release Apache Airflow Helm Chart 1.11.0 based on 1.11.0rc1

[Jed Cunningham]

A bug was found in rc1 that'll necessitate an rc2. I'm canceling this vote
and will create 1.11.0rc2 soon.

[VOTE] Release Apache Airflow Helm Chart 1.11.0 based on 1.11.0rc1

[Jed Cunningham]

Hello Apache Airflow Community,

This is a call for the vote to release Helm Chart version 1.11.0.

The release candidate is available at:
https://dist.apache.org/repos/dist/dev/airflow/helm-chart/1.11.0rc1/

airflow-chart-1.11.0-source.tar.gz - is the "main source release" that
comes with INSTALL instructions.
airflow-1.11.0.tgz - is the binary Helm Chart release.

Public keys are available at: https://www.apache.org/dist/airflow/KEYS

For convenience "index.yaml" has been uploaded (though excluded from
voting), so you can also run the below commands.

helm repo add apache-airflow-dev
https://dist.apache.org/repos/dist/dev/airflow/helm-chart/1.11.0rc1/
helm repo update
helm install airflow apache-airflow-dev/airflow

airflow-1.11.0.tgz.prov - is also uploaded for verifying Chart Integrity,
though not strictly required for releasing the artifact based on ASF
Guidelines.

gpg: Signature made Wed Sep 27 17:24:14 2023 MDT


gpg:using RSA key E1A1E984F55B8F280BD9CBA20BB7163892A2E48E


gpg:issuer "jedcunning...@apache.org"


gpg: Good signature from "Jed Cunningham "
[ultimate]

plugin: Chart SHA verified.
sha256:624ed6fa0de7bea9b1b56a65088d62d9786342b831f51ae3eee39542ddd30f91

The vote will be open for at least 72 hours (2023-09-30 23:44 UTC) or until
the necessary number of votes is reached.

https://www.timeanddate.com/countdown/to?iso=20230930T2344=136=cursive

Please vote accordingly:

[ ] +1 approve
[ ] +0 no opinion
[ ] -1 disapprove with the reason

Only votes from PMC members are binding, but members of the community are
encouraged to test the release and vote with "(non-binding)".

Consider this my (binding) +1.

For license checks, the .rat-excludes files is included, so you can run the
following to verify licenses (just update your path to rat):

tar -xvf airflow-chart-1.11.0-source.tar.gz
cd airflow-chart-1.11.0
java -jar apache-rat-0.13.jar chart -E .rat-excludes

Please note that the version number excludes the `rcX` string, so it's now
simply 1.11.0. This will allow us to rename the artifact without modifying
the artifact checksums when we actually release it.

The status of testing the Helm Chart by the community is kept here:
https://github.com/apache/airflow/issues/34660

Thanks,
Jed

Re: [DISCUSS] move from semver to a more "rolling" release cycle for core

[Jed Cunningham]

Shubham touched on it a bit, but I want to bring providers into the fold a
bit as well. I don't think there is enough focus on provider versions. Us
maintainers of Airflow have greatly benefited from being able to have
breaking provider changes, but I always get the impression the average user
just looks at the core version and ignores the provider versions almost
completely.

I'm not saying our position today is "wrong", but I think we (knowingly)
overlook how semver compliant "Airflow" is overall. Sure, core is backwards
compatible, but Airflow overall isn't, and the core version is the focal
point... Sucking more things out of core (like executors) just makes this
problem worse.

I don't have a fix here though, just want to express my thoughts that we
aren't as strict overall as the one version string implies. I'd be in favor
of allowing major version releases with breaking changes in core as well.

Now, one elephant in the room - the 5 year security patches thing Jarek
brought up. I freely admit I haven't tracked this at all, so please correct
me if I'm wrong. If that ends up panning out though, I think we will have
to reassess our strategy with providers too. As of today, we have 294 major
versions of providers, all released within the last few years. Being on the
hook for maintaining all of those for a 5 year period is a pretty good
reason to be much more picky on when we have major provider releases I'd
say.

Re: [DISCUSS] Preventing users from misusing _PIP_ADDITIONAL_REQUIREMENTS ?

[Jed Cunningham]

I also don't like the 10 minute thing. I'd rather we remove it, or display
a message like we do sequential executor (we can only do so much, this is
as visible as we can make it really), I think in that order?

Re: [VOTE] August 2023 PR of the Month

[Jed Cunningham]

My vote is for a batch of PRs, not just a single one (which would have been
impossible to review if they were all combined) - the refactoring work
@eumiro has been doing recently. He's done 69 this month already!

https://github.com/apache/airflow/pulls?q=is%3Apr+author%3Aeumiro+merged%3A2023-08-01..2023-08-31

Re: [VOTE] Drop MsSQL as supported backend

[Jed Cunningham]

+1 binding

Re: [VOTE] Release Airflow 2.7.0 from 2.7.0rc2

[Jed Cunningham]

+1 (binding)

Checked signatures, checksums, licences, compared sources. Used it with the
helm chart with a few different configs

Re: [VOTE] The daskexecutor, cncf.provider, celery providers in 2.7.0. as regular not-preinstalled providers

[Jed Cunningham]

I'm also in favor of option a.

Re: Allowing providers to load the respective default connections

[Jed Cunningham]

I like migrate as well.

Re: Allowing providers to load the respective default connections

[Jed Cunningham]

I'm not sure I like sync over upgrade, personally. Everything else
discussed sounds good though.

[DISCUSS] Should we pre-install celery/k8s providers?

[Jed Cunningham]

Hello everyone,

After thinking through whether the dask provider should be pre-installed or
not, it got me thinking about whether we should even be pre-installing the
celery and k8s providers. Core Airflow so far hasn't had the dependencies
required to make those executors functional either - users either had to
use the extra or install the provider directly. So that doesn't really
change if we choose not to preinstall the providers.

Why would it be beneficial to not pre-install them? The most obvious
benefit is you don't get a handful of new required deps you aren't using
(like celery/flower/billiard/etc if you don't use CeleryExecutor). However
another benefit is we wouldn't have to set a minimum version for these 2
providers to ensure we have the version with the executors, allowing an
"old" k8s provider (KPO itself is pretty common after all) to be used if
you aren't using KubernetesExecutor.

The one downside here is how do we make sure users have a new enough
provider, one that has their executor of choice in it. After chatting with
Jarek, he brought up that we could also make the error messaging more
helpful in this situation. This messaging would basically solve the problem
without needing minimum version pinning in core.

All that said, I think we should strongly consider not pre-installing the
celery and k8s providers as well.

Thoughts?

Jed

Re: [VOTE] Make (soon coming) dask provider preinstalled

[Jed Cunningham]

-1 binding

(btw Vincent, your vote is binding in this vote )

On Fri, Jul 21, 2023 at 7:52 AM Beck, Vincent 
wrote:

> -1 (non-binding)
>
> On 2023-07-21, 5:46 AM, "Michał Modras"  LID> wrote:
>
>
> CAUTION: This email originated from outside of the organization. Do not
> click links or open attachments unless you can confirm the sender and know
> the content is safe.
>
>
>
>
>
>
> -1 (non-binding)
>
>
> Dask is more niche than Celery or K8s - let's err on the side of making
> Airflow core dependencies more light-weight and not preinstall it.
>
>
> On Fri, Jul 21, 2023 at 10:57 AM Pankaj Koti
> mailto:pankaj.k...@astronomer.io.inva>lid>
> wrote:
>
>
> > 0 (non-binding)
> >
> > I agree with most of what has been discussed in the thread
> > https://lists.apache.org/thread/0d9x4kl7hc2qzvho2mbdf35ohn5w12l6 <
> https://lists.apache.org/thread/0d9x4kl7hc2qzvho2mbdf35ohn5w12l6>
> > and this email thread.
> >
> > I did a check on Airflow discussions (
> > https://github.com/apache/airflow/discussions?discussions_q=dask+ <
> https://github.com/apache/airflow/discussions?discussions_q=dask+>)
> > and Airflow issues(
> > https://github.com/apache/airflow/issues?q=is%3Aissue+dask+ <
> https://github.com/apache/airflow/issues?q=is%3Aissue+dask+>)
> > and upon reading their descriptions, see that there are some users using
> > the Dask executor although I personally have never tried it.
> >
> > If we decide to not pre-install it, in my opinion, a detailed and
> top-level
> > news fragment / Changelog would help us here which mentions it to install
> > the provider additionally in case they would like to continue using it.
> >
> > Regards,
> >
> >
> >
> > Pankaj Koti
> >
> > *Senior Software Engineer, *OSS Engineering Team.
> > Location: Pune, India
> >
> > Timezone: Indian Standard Time (IST)
> >
> > Email: pankaj.k...@astronomer.io 
> >
> > Mobile: +91 9730079985 <+91%2097300%2079985>
> >
> >
> > On Fri, Jul 21, 2023 at 2:20 PM Jarek Potiuk  ja...@potiuk.com>> wrote:
> >
> > > > One possible solution could be allowing users to instruct Airflow not
> > to
> > > pre-install executors providers by passing an argument via
> > > `--install-option` during pip install, and add install them explicitly
> by
> > > providing the extras.
> > >
> > > Yes it would be nice. Unfortunately the dependency mechanisms of
> > > Python (and `pip`) are not as flexible. You cannot individually remove
> > > requirements, and having requirements is the only way you can have
> > > "install also that other package" feature (extras is the mechanism to
> > > install optional dependencies, but it's not something we should ask
> > > regular users to do
> > >
> > >
> > >
> > > On Fri, Jul 21, 2023 at 10:46 AM Jarek Potiuk  > wrote:
> > > >
> > > > And just to clarify: The voting will last till Tuesday 25 July 20023
> > > 11am CST.
> > > >
> > > > On Fri, Jul 21, 2023 at 10:43 AM Jarek Potiuk  >
> > wrote:
> > > > >
> > > > > Clarification: Ash is right (I totally forgot about it).
> > > > >
> > > > > We do have dask in
> > > > >
> > >
> >
> https://airflow.apache.org/docs/apache-airflow/stable/extra-packages-ref.html#core-airflow-extras
> <
> https://airflow.apache.org/docs/apache-airflow/stable/extra-packages-ref.html#core-airflow-extras
> >
> > > > > and it even explicitly mentions it:
> > > > >
> > > > > * "dask pip install 'apache-airflow[dask]' DaskExecutor"
> > > > >
> > > > > So technically speaking you **should** use the dask extra to
> install
> > > > > distributed, dask and cloudpickle package - all needed to run the
> > dask
> > > > > executor. And we will keep it this way - it will just move from
> "core
> > > > > extras" to "providers". Actually it will move to "deprecated" and
> we
> > > > > will alias "dask" it with the new "daskexecutor", because the name
> > > > > "apache-airflow-providers-dask" is not allowed by PyPI (I guess the
> > > > > -dask suffix is considered potentially harmful for typosquatting)
> so
> > I
> > > > > reserved "apache-airflow-providers-daskexecutor".
> > > > >
> > > > > J.
> > > > >
> > > > >
> > > > >
> > > > > On Fri, Jul 21, 2023 at 10:34 AM Hussein Awala  >
> > > wrote:
> > > > > >
> > > > > > > based on the premise that your had to install the `dask` extra
> in
> > > the
> > > > > > first place to get dask module of the right version
> > > > > >
> > > > > > After reviewing the Dask Executor
> > > > > > <
> > >
> >
> https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/executor/dask.html
> <
> https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/executor/dask.html
> >
> > > >
> > > > > > section in Airflow documentation, I didn't find any requirement
> or
> > > > > > recommendation to add the `dask` extra when using it. So,
> including
> > > the
> > > > > > dependencies explicitly in the requirements file should suffice
> to
> > 

Re: [DISCUSS] Contributing "core" options by providers configuration ?

[Jed Cunningham]

I don't necessarily mind providers adding options to existing sections, but
I can see the appeal and simplicity of enforcing it at the section level.

 Either way, I think in the case of multiple, it should fail loudly, not
just warn.

Re: [VOTE] Airflow Providers prepared on July 09, 2023

[Jed Cunningham]

+1 (binding)

Checked signatures, checksums, licences.

Re: [VOTE] Release Airflow 2.6.3 from 2.6.3rc1

[Jed Cunningham]

+1 (binding)

Checked signatures, checksums, licences, and compared code to the tag. Used
it with the helm chart with a few different config.

[ANNOUNCE] Apache Airflow Helm Chart version 1.10.0 Released

[Jed Cunningham]

Dear Airflow community,

I am pleased to announce that we have released Apache Airflow Helm chart
1.10.0  

The source release, as well as the "binary" Helm Chart release, are
available:

   Official Sources:
https://airflow.apache.org/docs/helm-chart/1.10.0/installing-helm-chart-from-sources.html
   ArtifactHub:
https://artifacthub.io/packages/helm/apache-airflow/airflow
   Docs: https://airflow.apache.org/docs/helm-chart/1.10.0/
   Quick Start Installation Guide:
https://airflow.apache.org/docs/helm-chart/1.10.0/quick-start.html
️   Release Notes:
https://airflow.apache.org/docs/helm-chart/1.10.0/release_notes.html

Thanks to all the contributors who made this possible.

Thanks,
Jed

[ANNOUNCE] Apache Airflow Helm Chart version 1.10.0 Released

[Jed Cunningham]

Dear Airflow community,

I am pleased to announce that we have released Apache Airflow Helm chart
1.10.0  

The source release, as well as the "binary" Helm Chart release, are
available:

   Official Sources:
https://airflow.apache.org/docs/helm-chart/1.10.0/installing-helm-chart-from-sources.html
   ArtifactHub:
https://artifacthub.io/packages/helm/apache-airflow/airflow
   Docs: https://airflow.apache.org/docs/helm-chart/1.10.0/
   Quick Start Installation Guide:
https://airflow.apache.org/docs/helm-chart/1.10.0/quick-start.html
️   Release Notes:
https://airflow.apache.org/docs/helm-chart/1.10.0/release_notes.html

Thanks to all the contributors who made this possible.

Thanks,
Jed

[RESULT][VOTE] Release Apache Airflow Helm Chart 1.10.0 based on 1.10.0rc1

[Jed Cunningham]

Hello all,

The vote to release Apache Airflow Helm Chart version 1.10.0 based on
1.10.0rc1 is now closed.

The vote PASSED with 3 binding "+1", 4 non-binding "+1" and 0 "-1" votes:

"+1" Binding votes:

  - Jed Cunningham
  - Jarek Potiuk
  - Elad Kalif

"+1" Non-Binding votes:

  - Pankaj Koti
  - Hussein Awala

Vote thread:
https://lists.apache.org/thread/4mk3t0krfj8546zrpmxhxxo0rssfw4ql

I'll continue with the release process and the release announcement will
follow shortly.

Thanks,
Jed

[VOTE] Release Apache Airflow Helm Chart 1.10.0 based on 1.10.0rc1

[Jed Cunningham]

Hello Apache Airflow Community,

This is a call for the vote to release Helm Chart version 1.10.0.

The release candidate is available at:
https://dist.apache.org/repos/dist/dev/airflow/helm-chart/1.10.0rc1/

airflow-chart-1.10.0-source.tar.gz - is the "main source release" that
comes with INSTALL instructions.
airflow-1.10.0.tgz - is the binary Helm Chart release.

Public keys are available at: https://www.apache.org/dist/airflow/KEYS

For convenience "index.yaml" has been uploaded (though excluded from
voting), so you can also run the below commands.

helm repo add apache-airflow-dev
https://dist.apache.org/repos/dist/dev/airflow/helm-chart/1.10.0rc1/
helm repo update
helm install airflow apache-airflow-dev/airflow

airflow-1.10.0.tgz.prov - is also uploaded for verifying Chart Integrity,
though not strictly required for releasing the artifact based on ASF
Guidelines.

$ helm gpg verify airflow-1.10.0.tgz
gpg: Signature made Fri Jun 23 09:26:49 2023 EDT
gpg:using RSA key E1A1E984F55B8F280BD9CBA20BB7163892A2E48E
gpg:issuer "jedcunning...@apache.org"
gpg: Good signature from "Jed Cunningham "
[ultimate]
plugin: Chart SHA verified.
sha256:6ed4828a04b7948a7cf861999ee9f6f94dcac0e63f0fcb9816da21be0ff60f57

The vote will be open for at least 72 hours (2023-06-26 13:51 UTC) or until
the necessary number of votes is reached.

https://www.timeanddate.com/countdown/to?iso=20230626T1351=136=cursive

Please vote accordingly:

[ ] +1 approve
[ ] +0 no opinion
[ ] -1 disapprove with the reason

Only votes from PMC members are binding, but members of the community are
encouraged to test the release and vote with "(non-binding)".

Consider this my (binding) +1.

For license checks, the .rat-excludes files is included, so you can run the
following to verify licenses (just update your path to rat):

tar -xvf airflow-chart-1.10.0-source.tar.gz
cd airflow-chart-1.10.0
java -jar apache-rat-0.13.jar chart -E .rat-excludes

Please note that the version number excludes the `rcX` string, so it's now
simply 1.10.0. This will allow us to rename the artifact without modifying
the artifact checksums when we actually release it.

The status of testing the Helm Chart by the community is kept here:
https://github.com/apache/airflow/issues/32096

Thanks,
Jed

Re: AIP-56 Extensible user management

[Jed Cunningham]

Sounds good to me. Hopefully we can make it happen, but we give ourselves
an escape hatch :)

On Fri, Jun 16, 2023 at 9:45 AM Beck, Vincent 
wrote:

> Thanks for the feedback Jed, that's something I confess I had not thought
> about. It is a valid concern. What I can propose as a compromise is to
> leave the "move FAB auth manager to a separate provider" task to the end of
> the project and mark it as optional. In other words, we can build
> everything as described in the AIP but keep it in core Airflow as opposed
> to a separate provider. All the user management logic would still be
> "packaged" in the auth manager, and then when everything is implemented and
> done we can decide whether moving this FAB auth manager to a separate
> provider is feasible/a good idea.
>
> What do you think?
>
> On 2023-06-12, 11:18 AM, "Jed Cunningham"  <mailto:j...@astronomer.io.inva>LID> wrote:
>
>
>
> Overall I'm happy with the proposal. One thing that concerns me though is
> moving the FAB auth manager into a separate provider. That auth manager
> will need to be able to hook into the db migration tooling, and we don't
> expose that to providers or plugins today. So if we do want to move it, we
> have to account for that as well.
>
>
> I feel the vast majority of the benefit here comes from being able to sub
> in another auth manager, but having the FAB one "not in core" isn't all
> that consequential at the end of the day. It would keep the strict version
> requirements of FAB in core though - pick your poison I guess :)
>
>
>
>

Re: [VOTE] Release Airflow 2.6.2 from 2.6.2rc2

[Jed Cunningham]

+1 (binding)

Checked signatures, checksums, licences. Used it with the helm chart with a
few different configs

Re: AIP-56 Extensible user management

[Jed Cunningham]

Overall I'm happy with the proposal. One thing that concerns me though is
moving the FAB auth manager into a separate provider. That auth manager
will need to be able to hook into the db migration tooling, and we don't
expose that to providers or plugins today. So if we do want to move it, we
have to account for that as well.

I feel the vast majority of the benefit here comes from being able to sub
in another auth manager, but having the FAB one "not in core" isn't all
that consequential at the end of the day. It would keep the strict version
requirements of FAB in core though - pick your poison I guess :)

Re: [VOTE] Airflow Providers prepared on May 19, 2023

[Jed Cunningham]

+1 (binding)

Checked signatures, checksums, licences.

Re: [VOTE] Release Airflow 2.6.0 from 2.6.0rc5

[Jed Cunningham]

+1 (binding)

Checked signatures, checksums, licences. Used it with the helm chart with a
few different configs

Re: [VOTE] Release Airflow 2.6.0 from 2.6.0rc4

[Jed Cunningham]

+1 (binding)

Checked signatures, checksums, licences. Used it with the helm chart with a
few different configs

Re: User Actions Logging?

[Jed Cunningham]

While it certainly isn't exhaustive, what we do have ends up in the audit
logs. They can be found under Browse -> Audit logs, or a filtered view is
available for a given DAG in later versions in the normal DAG menu.

Re: [VOTE] Airflow Providers prepared on April 25, 2023

[Jed Cunningham]

+1 (binding)

Checked signatures, checksums, licences.

Re: [VOTE] Airflow Providers prepared on April 21, 2023

[Jed Cunningham]

+1 (binding)

Checked signatures, checksums, licences.

[ANNOUNCE] Apache Airflow Helm Chart version 1.9.0 Released

[Jed Cunningham]

Dear Airflow community,

I am pleased to announce that we have released Apache Airflow Helm chart
1.9.0  

The source release, as well as the "binary" Helm Chart release, are
available:

   Official Sources:
https://airflow.apache.org/docs/helm-chart/1.9.0/installing-helm-chart-from-sources.html
   ArtifactHub:
https://artifacthub.io/packages/helm/apache-airflow/airflow
   Docs: https://airflow.apache.org/docs/helm-chart/1.9.0/
   Quick Start Installation Guide:
https://airflow.apache.org/docs/helm-chart/1.9.0/quick-start.html
️   Release Notes:
https://airflow.apache.org/docs/helm-chart/1.9.0/release_notes.html

Thanks to all the contributors who made this possible.

Thanks,
Jed

[ANNOUNCE] Apache Airflow Helm Chart version 1.9.0 Released

[Jed Cunningham]

Dear Airflow community,

I am pleased to announce that we have released Apache Airflow Helm chart
1.9.0  

The source release, as well as the "binary" Helm Chart release, are
available:

   Official Sources:
https://airflow.apache.org/docs/helm-chart/1.9.0/installing-helm-chart-from-sources.html
   ArtifactHub:
https://artifacthub.io/packages/helm/apache-airflow/airflow
   Docs: https://airflow.apache.org/docs/helm-chart/1.9.0/
   Quick Start Installation Guide:
https://airflow.apache.org/docs/helm-chart/1.9.0/quick-start.html
️   Release Notes:
https://airflow.apache.org/docs/helm-chart/1.9.0/release_notes.html

Thanks to all the contributors who made this possible.

Thanks,
Jed

[RESULT][VOTE] Release Apache Airflow Helm Chart 1.9.0 based on 1.9.0rc1

[Jed Cunningham]

Hello all,

The vote to release Apache Airflow Helm Chart version 1.9.0 based on
1.9.0rc1 is now closed.

The vote PASSED with 3 binding "+1", 2 non-binding "+1" and 0 "-1" votes:

"+1" Binding votes:

  - Jed Cunningham
  - Ephraim Anierobi
  - Jarek Potiuk

"+1" Non-Binding votes:

  - Jules Triomphe
  - Hussein Awala

Vote thread:https://lists.apache.org/thread/jypl0nk94zf1cs5r7tc93qlwyjw497zy

I'll continue with the release process and the release announcement
will follow shortly.

Thanks,
Jed

Re: [VOTE] Airflow Providers prepared on April 09, 2023

[Jed Cunningham]

+1 (binding)

Checked signatures, checksums, licences.

[VOTE] Release Apache Airflow Helm Chart 1.9.0 based on 1.9.0rc1

[Jed Cunningham]

Hello Apache Airflow Community,

This is a call for the vote to release Helm Chart version 1.9.0.

The release candidate is available at:
https://dist.apache.org/repos/dist/dev/airflow/helm-chart/1.9.0rc1/

airflow-chart-1.9.0-source.tar.gz - is the "main source release" that comes
with INSTALL instructions.
airflow-1.9.0.tgz - is the binary Helm Chart release.

Public keys are available at: https://www.apache.org/dist/airflow/KEYS

For convenience "index.yaml" has been uploaded (though excluded from
voting), so you can also run the below commands.

helm repo add apache-airflow-dev
https://dist.apache.org/repos/dist/dev/airflow/helm-chart/1.9.0rc1/
helm repo update
helm install airflow apache-airflow-dev/airflow

airflow-1.9.0.tgz.prov - is also uploaded for verifying Chart Integrity,
though not strictly required for releasing the artifact based on ASF
Guidelines.

$ helm gpg verify airflow-1.9.0.tgz
gpg: Signature made Tue Apr 11 09:41:29 2023 MDT
gpg:using RSA key E1A1E984F55B8F280BD9CBA20BB7163892A2E48E
gpg:issuer "jedcunning...@apache.org"
gpg: Good signature from "Jed Cunningham "
[ultimate]
plugin: Chart SHA verified.
sha256:750ff03fa93049bb049c1dd91722530ba39c8e2af951883bf3fb8b2630fd0684

The vote will be open for at least 72 hours (2023-04-14 16:12 UTC) or until
the necessary number of votes is reached.

https://www.timeanddate.com/countdown/to?iso=20230414T1612=136=cursive

Please vote accordingly:

[ ] +1 approve
[ ] +0 no opinion
[ ] -1 disapprove with the reason

Only votes from PMC members are binding, but members of the community are
encouraged to test the release and vote with "(non-binding)".

Consider this my (binding) +1.

For license checks, the .rat-excludes files is included, so you can run the
following to verify licenses (just update your path to rat):

tar -xvf airflow-chart-1.9.0-source.tar.gz
cd airflow-chart-1.9.0
java -jar apache-rat-0.13.jar chart -E .rat-excludes

Please note that the version number excludes the `rcX` string, so it's now
simply 1.9.0. This will allow us to rename the artifact without modifying
the artifact checksums when we actually release it.

The status of testing the Helm Chart by the community is kept here:
https://github.com/apache/airflow/issues/30585

Thanks,
Jed

Re: [VOTE] March PR of the Month

[Jed Cunningham]

Another for 29413.

Re: [DISCUSS] AIP-52 updates - setup / teardown tasks

[Jed Cunningham]

I've been working closely with Daniel on AIP-52 for a while now, but I'll
still share my thoughts here.

I'm also in favor of only supporting explicit relationships.

In my opinion, even if multiple setup/teardown per scope never materialize,
explicit relationships are still a better choice due to the clarity it
brings today.

Re: [VOTE] Release Airflow 2.5.2 from 2.5.2rc2

[Jed Cunningham]

+1 (binding)

Checked signatures, checksums, licences.

Re: [VOTE] Release Airflow 2.5.2 from 2.5.2rc1

[Jed Cunningham]

+1 (binding)

Checked signatures, checksums, licences. Used it with the helm chart with a
few different configs.

Re: [VOTE] Airflow Providers prepared on March 03, 2023

[Jed Cunningham]

+1 (binding)

Checked signatures, checksums, licences.

Re: Apache Airflow Newsletter | February 2023

[Jed Cunningham]

Thanks for putting this together John.

I'd just like to mention to readers here that the old Airflow docs were
only taken down for about 9 hours. They are back!

Re: [VOTE] February PR of the Month

[Jed Cunningham]

+1 to #27758

Re: oauth configuration for airflow in k8s

[Jed Cunningham]

Daniel means adding the webserver_config via your Dockerfile, so it is
always in any container from the start, e.g.

```
FROM apache/airflow:2.5.1

COPY webserver_config.py /opt/airflow
```

My "don't worry about reload, delete" was more of a general comment.
However you get your webserver_config into the container, don't spend time
trying to reload it in a running webserver, just delete the pod and let a
fresh one come up. You still need to get the webserver_config in the
container somehow (and not via `kubectl cp` or `kubectl exec`, but some way
that'll happen in new pods automatically). If you use `webserverConfig` in
the chart, or bake it into an image and you tag changes, the helm chart
will do the restart for you automatically.

I take it you are new to k8s, so let me say that `kubectl cp` and `kubectl
exec` should only be debugging aids. You really need things to be
durable/resilient to the unexpected happening. Your pods could be killed
because the node it's on comes under resource pressure, or the node itself
may die. Fresh pods should be ready to go by their config alone, with no
manual intervention needed. In this case, this means getting your
webserver_config into the image itself, or mounted into the container (via
a configmap or secret).

Hope that helps!

(p.s. I'm happy to chat on slack in #helm-chart-official about this stuff
as well)

Re: oauth configuration for airflow in k8s

[Jed Cunningham]

Don't worry about "reload" in k8s, just delete the pod and let another come
up. Or if you bake it into the image like Daniel suggested, it'll be
handled for you!

Otherwise, the helm chart also supports a string in your values file, or
using a ConfigMap:
https://github.com/apache/airflow/blob/2cd12fc86950646d10dfb5fb7f9c76c529715b46/chart/values.yaml#L997-L1006

It sounds like you found the right spot in the Airflow core docs, but for
completeness/others benefit:
https://airflow.apache.org/docs/apache-airflow/stable/administration-and-deployment/security/webserver.html#other-methods

Re: [URGENT] Remove old versions of Airflow docs (<1.10.14) as stop-gap measure for doc builds

[Jed Cunningham]

Here it is: https://github.com/apache/airflow-site/pull/742