Can I get added to the invitation as well? (mailbow...@gmail.com)
Thanks
On Fri, Mar 24, 2023 at 2:37 AM Jarek Potiuk wrote:
> did
>
> On Thu, Mar 23, 2023 at 9:22 PM c c wrote:
> >
> > Can I be added to the invitation as well(changcheng12...@gmail.com)?
> > thanks!
> >
> > On Thu, Mar 23,
It would not help with kubernetes executor. Did you try with local
executor?
Another option to consider is to implement this specifically on the AWS
secrets manager secrets backend itself.
Thank you so much for the valuable context Damian. I’m glad that you were still
able to identify some positives in being able to introduce more custom SLA
monitoring requirements to your DAG with our proposed workaround.
I feel like task-level SLAs sit in a grey area between what seems to be a
Considering the current framing of the problem/solution - as a means to address
poorly written DAGs - I agree with Elad's perspective. Nevertheless, I believe
that caching for the secrets backend has the potential to enhance Airflow
performance even for DAGs adhering to best practices,
re
> 2. `task1 >> task2 >> teardown_task` to me falsely implies that teardown
> depends on task2, But it doesn't. It only depends on the "scope being
> exited".
So that's not quite the case. With the proposed implementation, there's no
such scope concept. They're just normal tasks, with
I second Elad's view here.
I would also propose an alternative fix: let's come up with a way to tell
Airflow to not continuously reparse a file!
A strawman example:
```
from airflow import ReparseMode, DAG
AIRFLOW_REPARSE = ReparseMode.ON_FILE_CHANGED
with DAG(...):
```
We could
I'm obviously in favour of the way the AIP was written, and that's for two
primary reasons.
1. It's analogous to setup and teardown in testing frameworks where you don't
ever explicitly call them - the framework handles it for you.
2. `task1 >> task2 >> teardown_task` to me falsely implies that
To me Airflow is not "open and play". It is not a closed system that guides
you how to develop. Workflow as code requires more skills.
There are stuff to learn before authoring dags.
This discussion is about Variables but I might ask similar question about
users who query API as part of top level
Hello,
I’d like to submit to discussion the idea of having a cache on Airflow
Variables.
The aim is to reduce DAG parsing time and Secret Manager API bill for users who
make a liberal use of Variable.get in their DAG files.
The drawback, of course, is that caches introduce a delay in how fast
I am also in favor of explicit relationships only.
>From a person who didn't work on AIP-52, it seems easier to understand what
is going on without having to dive into the setup/teardown documentation.
Le jeu. 23 mars 2023 à 22:53, Jed Cunningham a
écrit :
> I've been working closely with
I've been working closely with Daniel on AIP-52 for a while now, but I'll
still share my thoughts here.
I'm also in favor of only supporting explicit relationships.
In my opinion, even if multiple setup/teardown per scope never materialize,
explicit relationships are still a better choice due to
did
On Thu, Mar 23, 2023 at 9:22 PM c c wrote:
>
> Can I be added to the invitation as well(changcheng12...@gmail.com)?
> thanks!
>
> On Thu, Mar 23, 2023 at 12:59 PM Jarek Potiuk wrote:
>
> > I added all those who asked. It's really cool we have so much interest :).
> >
> > Julien, Maciej: NO
Disclaimer - I've spent some time with Daniel discussing the options
and brainstorming some consequences of the change over the last few
days (er... evenings) and that was quite a brain-teaser. So I
perfectly understand if it takes time and effort to digest.
But here is the digest of my thoughts
Can I be added to the invitation as well(changcheng12...@gmail.com)?
thanks!
On Thu, Mar 23, 2023 at 12:59 PM Jarek Potiuk wrote:
> I added all those who asked. It's really cool we have so much interest :).
>
> Julien, Maciej: NO PRESSURE
>
>
>
I added all those who asked. It's really cool we have so much interest :).
Julien, Maciej: NO PRESSURE
-
To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org
For additional commands, e-mail: dev-h...@airflow.apache.org
I'd like to join as well! (mesmaco...@gmail.com)
On Thu, 23 Mar 2023 at 19:23 Oliveira, Niko
wrote:
> I'd like to join as well! (oliveira...@gmail.com)
>
>
> From: Igor Kholopov
> Sent: Wednesday, March 22, 2023 4:01:40 PM
> To: dev@airflow.apache.org
>
I'd like to join as well! (oliveira...@gmail.com)
From: Igor Kholopov
Sent: Wednesday, March 22, 2023 4:01:40 PM
To: dev@airflow.apache.org
Subject: RE: [EXTERNAL]Request for feedback on proposal for new OpenLineage
provider in Airflow
CAUTION: This email
Hi, would like to clarify, in this thread we're specifically hoping to get
community feedback on the proposal to drop the "implicit" logic.
In the original AIP, if you instantiate a setup task in a group, in effect
it's made the setup task for all tasks in the group. And the proposal up
for
This AIP comes out of a previous discussion on skipping tasks based on
a rule-based schedule, e.g. excluding holidays except if it's Monday.
The central idea is to define a schedule based on logical composition
(and, or, not) – using a small number of primitives.
I’m part of a group working on the implementation of AIP-52. We would like
to update the community on some changes to the implementation approach, the
planned roadmap, and give an opportunity to provide feedback.
First though, let’s recap briefly what are the main benefits of adding
setup and
20 matches
Mail list logo
