; of the AIP and we can easily turn it into
implementation detail after extended POC is complete.
On Thu, Aug 11, 2022 at 9:08 AM Jarek Potiuk wrote:
> Hey Ash, Andrew,
>
> TL;DR; I slept over it to try to understand what just happened, and I have
> a proposal.
>
> * I am ha
nfusing. I just mean
>> "ship JSON in a POST and receive JSON in the response".
>>
>> As you said before though, the line where you draw the abstraction is
>> what matters more than the transport layer, and "fat endpoints" (doing
>> transactions a
t; schemas need to change to add namespacing!
>
> Andrew
>
> On Wed, Aug 10, 2022 at 12:52 PM Jarek Potiuk wrote:
>
>> Sure I can explain - the main reasons are:
>>
>> - 1) binary representation performance - impact of this is rather limited
>> because our AP
would win over this option. Apologies again for the late
> realisation that gRPC got chosen and was being voted on - it's been a very
> busy summer.
>
> Thanks,
> Andrew
>
> On Wed, Aug 10, 2022 at 12:12 PM Jarek Potiuk wrote:
>
>> Just let me express my rather strong
:
> Sorry to weigh in at the last minute, but I'm wary of gRPC over just JSON,
> so -1 to that specific choice. Everything else I'm happy with.
>
> I (or Andrew G) will follow up with more details shortly.
>
> -ash
>
> On Wed, Aug 10 2022 at 19:38:59 +02:00:00, Jarek Potiuk
ing)".
>
>
>
> +1 (binding)
>
> Thanks,
>
> Ping
>
>
> On Wed, Aug 10, 2022 at 10:20 AM Jarek Potiuk wrote:
>
>> Thank you . And BTW. It's binding Ping :). For AIP's commiter's votes are
>> binding. See
>> https://github.com/apache/airflow/blob/main/
, Aug 4, 2022 at 1:42 AM Jarek Potiuk wrote:
>
>> Hey everyone,
>>
>> I would like to cast a vote for "AIP-44 - Airflow Internal API".
>>
>> The AIP-44 is here:
>> https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-44+Airflow
Hey all,
> Yes, this is an RC3 (!) - don't ask :). Forgot to run `git pull` when
preparing rc2 :facepalm: :)..
I have just cut the new wave Airflow Providers packages. This email is
calling a vote on the release,
which will last for 72 hours - which means that it will end on Sat 13 Aug
4pm
This has been discussed several times, and I think you should rather take a
look and focus on those proposals already there:
* https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-20+DAG+manifest
*
https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-5+Remote+DAG+Fetcher
Both proposals are
, Aug 10, 2022 at 12:48 PM Jarek Potiuk wrote:
> Hey all,
>
> I have just cut the new wave Airflow Providers packages. This email is
> calling a vote on the release,
> which will last for 72 hours - which means that it will end on Sat 13 Aug
> 1pm CEST 2022.
>
> Consi
Hey all,
I have just cut the new wave Airflow Providers packages. This email is
calling a vote on the release,
which will last for 72 hours - which means that it will end on Sat 13 Aug
1pm CEST 2022.
Consider this my (binding) +1.
*Summary of this wave*
Apart of "regular" bugfix/features
ks for forcing us to make the case :)
>>>
>>> I'll initiate a lazy consensus "vote"
>>>
>>>
>>> On Fri, Aug 5, 2022 at 1:01 PM Vikram Koka
>>> wrote:
>>>
>>>> Fair points all.
>>>> I withdraw my objectio
What do you think, Pablo about the "being out" vs. "being in" the
official repo?
On Thu, Jul 28, 2022 at 3:51 PM Jarek Potiuk wrote:
> Anyone :) ?
>
> On Mon, Jul 18, 2022 at 10:38 AM Jarek Potiuk wrote:
>
>> I would love to hear what others think about t
Crearted PR here: https://github.com/apache/airflow/pull/25564
On Sat, Aug 6, 2022 at 9:26 AM Jarek Potiuk wrote:
> Free text field seems better for those. But we should make it required :)
>
> On Sat, Aug 6, 2022 at 1:40 AM Jed Cunningham
> wrote:
>
>> So far we've been
Yeah. I feel like having one place where you can add custom code is better,
especially that there are a number of conf settings that you can also add
your custom code (think hostname_callable) so having one "dedicated" module
at least as an entrypoint sounds reasonable and yeah - you can import
5:44 PM Jeambrun Pierre
> wrote:
>
>> Same, great idea :)
>>
>> On Fri 5 Aug 2022 at 01:43, Vikram Koka
>> wrote:
>>
>>> +1 on this approach with this approach in the PR
>>> https://github.com/apache/airflow/pull/25542
>>>
>>>
>
may
>>>> have to make to deal with it.
>>>>
>>>> Thanks for raising this topic and the discussion,
>>>>
>>>> Vikram
>>>>
>>>>
>>>> On Thu, Aug 4, 2022 at 10:24 AM Daniel Standish
>>>> w
https://lists.apache.org/thread/cp9n8r9x75xzzsdjgdqd82p8nmyn1nd5 ->
non-broken link here.
On Fri, Aug 5, 2022 at 1:15 PM Jarek Potiuk wrote:
> And I agree with Ash - two years ago it would be a bad choice but we are
> past the time when we should be "gentle" with it :).
looks like in my IDE now: https://imgur.com/a/OnRjiKs
>>>
>>> pt., 5 sie 2022 o 00:28 Ferruzzi, Dennis
>>> napisał(a):
>>>
>>>> > add dynamic attributes to __init__ of airflow.contrib,
>>>> airflow.operators, airflow.hooks. (to resolve to provide
>> Hi Jarek,
>>
>> Thanks for the thorough explanation. They all make sense, especially the
>> performance impact and scalability part. I believe with the internal API,
>> it should reduce lots of db connections.
>>
>> Looking forward to it!
>>
>>
Love it.
On Thu, Aug 4, 2022 at 9:28 PM Jed Cunningham
wrote:
> Sorry, coming to this a little late. I tend to agree with Elad that we
> might be better off not even having 2.0/2.1, and I'll go further that we
> should consider only listing main/latest/possibly n-1(e.g. 2.3.2).
>
> I'm also
No problem - we can rename it - whoever gets to the old name will get
information that the page is gone and they will see the link to the new
page.
On Thu, Aug 4, 2022 at 7:28 PM Daniel Standish
wrote:
> I noticed that our AIP page says "Airflow *Improvements* Proposals" at
> the header at the
in helping. Let me know if I can help in anyway.
>
> Regards,
> *Ankit Chaurasia*
> HomePage <https://ankitchaurasia.info/> | LinkedIn
> <https://www.linkedin.com/in/sunank200/> | +91-9987351649
>
>
>
>
>
>
> On Thu, Aug 4, 2022 at 9:47 PM Jarek Po
Cool :)
On Thu, Aug 4, 2022 at 5:50 PM Beck, Vincent
wrote:
> Hey Jarek,
>
>
>
> I’ll be working today on cleaning up deprecated modules from Amazon
> package. I’ll ping you when I have a PR ready (should be today).
>
>
>
> Thanks,
>
> Vincent
>
>
&g
Hello Everyone,
I am just finishing reviewing and merging some last remaining PRs for
providers and I am starting to prepare the next wave of providers, so if
you have something that you think would be good to be included - ping me
today/tomorrow.
I think we might have at least one (Amazon) -
And there is a possibility of having a switch to turn those deprecation
warnings into errors in this case - still giving the admin of
Airflow installation (even selectively) to migrate out of contrib for their
users.
On Thu, Aug 4, 2022 at 3:52 PM Jarek Potiuk wrote:
> > This is the
2 at 3:47 PM Ash Berlin-Taylor wrote:
> On Thu, Aug 4 2022 at 13:59:23 +02:00:00, Jarek Potiuk
> wrote:
>
> When you migrate to Airflow 2.4 and it links to the 8.0 version of Google
> provider (assume 2.3 linked to 7) you have to make changes to make it work
> in bac
And BTW. airflow/contrib/operators/ecs_operator.py is potentially
already broken multiple times - Every time we release Amazon major
version of provider, it might break if you relied on it being backwards
compatible.
On Thu, Aug 4, 2022 at 12:22 PM Jarek Potiuk wrote:
> Not if you add inst
> wrote:
>
> One question: Are you talking about removing things from airflow.contrib,
> or things already with in airflow.providers.*?
>
> -a
>
> On Thu, Aug 4 2022 at 11:55:52 +02:00:00, Jarek Potiuk
> wrote:
>
> Hello everyone,
>
> Following the discussio
out removing things from airflow.contrib,
> or things already with in airflow.providers.*?
>
> -a
>
> On Thu, Aug 4 2022 at 11:55:52 +02:00:00, Jarek Potiuk
> wrote:
>
> Hello everyone,
>
> Following the discussion in https://github.com/apache/airflow/pull/25413
> I have
Hello everyone,
Following the discussion in https://github.com/apache/airflow/pull/25413 I
have a proposal.
Why don't we remove all "contrib" and other Airflow 1.10 deprecated classes
to a separate package and add dependency to that package as [contrib] or
[deprecated] extra in Airflow - and
Hey everyone,
I would like to cast a vote for "AIP-44 - Airflow Internal API".
The AIP-44 is here:
https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-44+Airflow+Internal+API
Discussion thread:
https://lists.apache.org/thread/nsmo339m618kjzsdkwq83z8omrt08zh3
The voting will last for 5 days
access in those untrusted components, we need to do it for both
> db session and internal API. What's more, with the Internal API, we can
> even remove the db-proxy :)
>
> Thanks,
>
> Ping
>
>
> On Tue, Aug 2, 2022 at 6:56 AM Jarek Potiuk wrote:
>
>> Hello,
&
To focus on a possible solution - I created a PR with the removal of all
the versions but the latest from a given branch and updated the description
- https://github.com/apache/airflow/pull/25480
Let me know what you think.
J.
On Wed, Jul 27, 2022 at 4:52 PM Jarek Potiuk wrote:
> Other id
Hello,
I have updated AIP-44 with the findings from the POC. If there are any more
questions/comments, please let me know, otherwise I plan to start voting
tomorrow.
https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-44+Airflow+Internal+API
On Mon, Jul 25, 2022 at 10:18 PM Jarek Potiuk
Less is more. I like the lack of "as dag".
I think it's not really a breaking change.
We can easily argue we are adding functionality rather than introducing a
breaking change. I think the only reason why someone would create a DAG in
a function is to return it and eventually add it to
Also check out https://github.com/jghoman/awesome-apache-airflow
On Mon, Aug 1, 2022 at 6:25 PM Dennis Akpenyi
wrote:
> Hello Mustapha,
>
> Apache Airflow docs (
> https://airflow.apache.org/docs/apache-airflow/stable/index.html) is a
> great place to start. And Marc Lamberti (Astronomer
t-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466=6213fd2ecd=ce16eef4ef>
> by Samhita Alla, BetterProgramming/Medium
> -
>
>“Deutsche Bank uses Cloud Composer Workload Automation,”
>
> <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a1046
(mostly) not
> implemented at the moment.
>
> On Fri, 29 Jul 2022 at 08:34, Sumit Maheshwari
> wrote:
>
>> Yeah, make sense about updating the links. Once I move tests and docs one
>> level up (prolly in the next client release) we will update the links
>> again.
>&
+1 for new "schedule" param to rule them all and deprecate the rest.
On Wed, Jul 27, 2022 at 10:36 PM Daniel Standish
wrote:
> As of airflow 2.3, we have two mutually-exclusive scheduling params,
> `schedule_interval` and `timetable`. As it stands now, AIP-48 will be
> adding a *third* such
> follow the same, i.e. Github-based API documentation.
>
>
>
> On Thu, Jul 28, 2022 at 6:31 PM Jarek Potiuk wrote:
>
>> +1 (binding) I checked sources/signatures/checksums and runs the API
>> calls using the
>> https://github.com/apache/airflow-client-python/blo
a PR just one month out of
> twelve, it’s worth it.
>
> We’ll give it a try this next month. Why not? :)
>
> Ross
>
>
> On Jul 26, 2022, at 5:36 PM, Jarek Potiuk wrote:
>
> Might be difficult. Adding new behaviours (especially for people who have
> plenty to do) i
Anyone :) ?
On Mon, Jul 18, 2022 at 10:38 AM Jarek Potiuk wrote:
> I would love to hear what others think about the "in/out" approach - mine
> is just the line of thoughts I've been exploring during the last few months
> where I prepared my own line of thought about pro
+1 (binding) I checked sources/signatures/checksums and runs the API calls
using the
https://github.com/apache/airflow-client-python/blob/master/dev/test_python_client.py
against running Airflow 2.3.3 installation.
However, when using the test client script, I have found that we still have
the
Other ideas/opinions here?
On Thu, Jul 21, 2022 at 2:05 PM Jarek Potiuk wrote:
> Just to comment. I do not think everyone will be able to upgrade (and
> if they don't the "discussion" route is still possible).
>
> This is more of a "psychological barrier" that
Might be difficult. Adding new behaviours (especially for people who have
plenty to do) is difficult. I think it's better to make smart use of
existing behaviours that are already happening regardless.
On Tue, Jul 26, 2022 at 11:28 PM Ross Turk
wrote:
>
> > On Jul 22, 2022, at 3:08
>
> Thanks,
>
> Ping
>
>
>
> Thanks,
>
> Ping
>
>
> On Tue, Jul 26, 2022 at 6:53 AM Jarek Potiuk wrote:
>
>> If I understand correctly, the Idea is to run an additional set of stress
>> tests before releasing a version - without impacting the p
act like a "Newcomer Spotlight" or something like that.
>>
>>
>> --
>> *From:* Jarek Potiuk
>> *Sent:* Friday, July 22, 2022 12:08 PM
>> *To:* dev@airflow.apache.org
>> *Subject:* RE: [EXTERNAL][VOTE] July 2022 PR of
heavily used by DE/DS, but not the
> admin of the clusters, or it can be in the Admin tab?
>
> Also, what's our longer term plan for MySQL 5.7 once it reaches EOL?
>
> Thanks,
>
> Ping
>
>
> On Sun, Jul 24, 2022 at 8:44 AM Jarek Potiuk wrote:
>
>> Hello everyo
image
(with the goal that we will be able to fully automate it). Until then
releasing of the images was not "community" duty, but "Jarek Potiuk"'s duty
(I did automate it from the very beginning, but it took some time and
effort to implement - but we finally got this nice and simp
hich
> DB- access is used most (and what object types are required there -
> small/big or maybe just primitives?).
>
> Nonetheless, I am looking forward to starting the voting process!
>
> Best regards,
> Mateusz Henc
>
>
> On Fri, Jul 15, 2022 at 4:48 PM Jarek Potiuk wrote:
>
Hello everyone,
TL;DR; I wanted to propose to start treating MySQL 5.7 in a special way
(and add warning about upcoming disabling of support for Older DBs users
use). We have still more than year of support for MySQL 5.7 (October 2023)
by Oracle, but mySQL 5.7 has some behaviours that make it
Side comment: I wonder when we will start employing AI to select "best pr"...
On Fri, Jul 22, 2022 at 8:39 PM Daniel Standish
wrote:
>
> my write-in vote:
>
> Implement expand_kwargs() (#24989) (uranusjr)
>
> honorable mention:
> Patch getfqdn with more resilient version (#24981) (potiuk)
> Fix
Was it really *puggable* ? I would have never guessed. Not so lazy
consensus here
On Thu, Jul 21, 2022 at 11:44 PM Daniel Standish
wrote:
>
> Sorry --- sunday july 24
Michael, others.
Even if I am the author of the simple heuristics I found those a bit
of a weird choice :). None of those PR really stands out to me as
something "cool" (well they are all cool, but not "REALLY COOL"). And
I am pretty sure there were some REALLY cool ones.
Ross - you mentioned
the issue on the latest version and submit the bug report on
> that version.
>
>
> On Thu, Jul 21, 2022 at 12:35 PM Jarek Potiuk wrote:
>>
>> (second line should be 2.3.2 -> for a few days after 2.3.3 is released
>>
>> On Thu, Jul 21, 2022 at 11:34 AM Jare
(second line should be 2.3.2 -> for a few days after 2.3.3 is released
On Thu, Jul 21, 2022 at 11:34 AM Jarek Potiuk wrote:
>
> Hello everyone,
>
> I've looked at some issues raised recently and I have an idea on how
> to make diagnosis of problems raised by our users a bit
Hello everyone,
I've looked at some issues raised recently and I have an idea on how
to make diagnosis of problems raised by our users a bit more
"efficient". It should also help to address "version" proliferation ve
have in the issue template.
The list grows longer and longer the more releases
~35
> lines, this is a nice quality of life improvement :)
>
> Le mer. 20 juil. 2022 à 21:41, Jarek Potiuk a écrit :
>
>> Glad you liked it. Happy to help in adding more of those if we have an
>> idea on how to improve the experience of webserver devs :).
>>
>> J.
&
he PR before it was merged, sorry for that.
>
> I've run it locally and everything is working fine. I'm glad to see that
> it simplifies the Dockerfiles a lot.
>
> Best,
>
> Le mer. 20 juil. 2022 à 20:54, Jarek Potiuk a écrit :
>
>> The change is merged.
>>
>
- first time you run Breeze, but you can also force it
with `breeze fix-ownership` command.
J.
On Wed, Jul 20, 2022 at 8:19 PM Ferruzzi, Dennis
wrote:
>
> I do like the sound of this. :thumbs-up:
>
>
>
> ________
> From: Jarek Potiuk
> Sent: Tue
Dear Airflow community,
I'm happy to announce that new versions of Airflow Providers packages
were just released.
This is an ad-hoc release of providers that were removed from previous
release due to bugs found:
https://pypi.org/project/apache-airflow-providers-tabular/1.0.1/
Hello,
Apache Airflow Providers (based on RC1) have been accepted.
3 "+1" binding votes received:
- Jarek Potiuk (binding)
- Jed Cunningham (binding)
- Elad Kalif (binding)
Vote thread:
https://lists.apache.org/thread/1tjbfrsf6vf6m9xr9znghtthz7x6khss
I'll continue with the relea
Hello everyone.
I wanted to propose a slight change (but also simplification and speedup)
of our dev env for the www asset compilation.
I am on a spree of optimizing our CI/Dev environment (with quite a success
so far - the new Python-based breeze is a wonderful tool that allows all
kinds of
Just two providers looking for a bit of warm feelings from PMC members :)
On Sun, Jul 17, 2022 at 11:25 AM Jarek Potiuk wrote:
> Wed 20th of July 2022 of course :)
>
> On Sun, Jul 17, 2022 at 11:24 AM Jarek Potiuk wrote:
> >
> > Hey all,
> >
> > I have just c
FYI. I've just updated my "robust" implementation
https://github.com/apache/airflow/pull/25161 to be such "future-possible".
J.
On Tue, Jul 19, 2022 at 5:32 PM Jarek Potiuk wrote:
> Actually this is even more complex as I found another case:
>
&g
Actually this is even more complex as I found another case:
I thought about it too, but this will be only for 2.4+ and a ton of people
could benefit from that even now.
However I have a thought. I was waiting for something like that to appear
to have a very good reason to implement the
y.
>
> Please let me know your thoughts on this.
>
> Thanks,
>
> Ping
>
>
> On Tue, Jul 12, 2022 at 12:45 AM Jarek Potiuk wrote:
>
>> Not interesting :) ?
>>
>> On Thu, Jul 7, 2022 at 10:41 AM Jarek Potiuk wrote:
>> >
>> > Hello e
;
> On 12-Jul-2022 at 7:44:35 AM, Jarek Potiuk wrote:
>
>> Not interesting :) ?
>>
>> On Thu, Jul 7, 2022 at 10:41 AM Jarek Potiuk wrote:
>>
>>
>> Hello everyone,
>>
>>
>> We have just published a blog on our medium -
>> https://med
especially -
expectations of the users that it creates. But those are just my thoughts
and I'd love to hear what others think about it.
On Mon, Jul 18, 2022 at 10:33 AM Jarek Potiuk wrote:
> I had some thoughts about it - this also connected with recent discussions
> about mixed governance for
It means you can release and iterate quickly without being beholden to
> the Airflow release process (voting, timelines etc)
> 2. It means we can see how popular it is before we (Airflow maintainers)
> have to commit to supporting it long term.
>
> -a
>
> On 17 July 2022
itten here : )
>
> LMK what y'all think. I am also happy to publish this as a separate
> library if y'all wanna be cautious about adding it directly to Airflow.
> -P.
>
> [1]
> https://github.com/apache/airflow/pull/25112/files#diff-b1f30afa38d247f9204790392ab6888b04288603ac4d38154d05e6c5b998cf85R2
Wed 20th of July 2022 of course :)
On Sun, Jul 17, 2022 at 11:24 AM Jarek Potiuk wrote:
>
> Hey all,
>
> I have just cut the new wave Airflow Providers packages. This email is
> calling a vote on the release,
> which will last for 72 hours - which means that it will end on We
Hey all,
I have just cut the new wave Airflow Providers packages. This email is
calling a vote on the release,
which will last for 72 hours - which means that it will end on Wed
11th July 2022, 12:30 CEST.
Consider this my (binding) +1.
This is a follow-up release of sftp and tabular providers:
Yep. Just outline your proposal on devlist, Pablo :).
On Sun, Jul 17, 2022 at 10:35 AM Ash Berlin-Taylor wrote:
>
> Hi Pablo,
>
> Could you describe at a high level what you are thinking of? It's entirely
> possible it doesn't need any changes to core Airflow, or isn't significant
> enough to
Dear Airflow community,
I 'm happy to announce that new versions of Airflow Providers packages
were just released.
This is more than a regular set of providers. We released 41 providers
in total this month. Additionally to regular providers we released
first version of Tabular provider and
Pankaj Singh Hello,
Apache Airflow Providers (based on RC1) have been accepted.
Note that SFTP 3.1.0 is removed from this wave as it contains breaking
changes and 4.0.0rc1 will be released shortly instead.
4 "+1" binding votes received:
- Jarek Potiuk (binding)
- Jed Cunningha
Need one more binding vote :)
On Fri, Jul 15, 2022 at 11:02 AM Pankaj Singh
wrote:
> Tested my changes working fine.
>
> +1 (non-binding)
>
> On Fri, Jul 15, 2022 at 4:57 AM Jed Cunningham
> wrote:
>
>> +1 (binding)
>>
>> Checked signatures, checksums, and licences.
>>
>
ple people and make it a true community
effort to complete - no need for major refactorings or changes across the
whole airflow code.
J.
On Fri, Jul 15, 2022 at 4:32 PM Jarek Potiuk wrote:
> Hello Everyone,
>
> First of all - apologies for those who waited for it, I've been dragged in
&g
would like
to focus on answering the two questions:
* Does it look plausible?
* Does it look like it's almost ready to vote on it? (I will update the AIP
before starting voting, of course).
Let me know what you think.
J.
On Tue, Feb 1, 2022 at 3:11 PM Jarek Potiuk wrote:
>
ul 2, 2022 at 3:33 AM Jarek Potiuk wrote:
>>
>> Hey Everyone,
>>
>> The registration for ONSITE ApacheCon in New Orleans, Louisiana,
>> October 3-6, is now open.
>>
>> This is the first time we have a 2-full-day "DataEngineering" track
Hey all,
I have just cut the new wave Airflow Providers packages. This email is
calling a vote on the release,
which will last for 72 hours - which means that it will end on
Saturday 16th of July 2022, 23:45 CEST.
Consider this my (binding) +1.
This is a bit more than regular release of
>
>> Thanks for bringing this up.
>>
>> I agree the SLA feature needs some work. However, I think we want an
>> equivalent SLA feature as it is still very useful.
>>
>> Thanks,
>>
>> Ping
>>
>>
>> On Tue, Jul 12, 2022 at 12:42 AM Ja
Correction of course.
On Tue, Jul 12, 2022 at 8:01 PM Jarek Potiuk wrote:
>
> I think if we limit it to stress tests, this could be an "extra"
> addition - not even necessarily part of Airflow codebase and adding
> triggers with a script, on a single database, some
erent airflow versions.
>
> Thanks for pointing out the OpenTelemetry, let me check it out.
>
> Thanks,
>
> Ping
>
>
> On Mon, Jul 11, 2022 at 9:44 AM Jarek Potiuk wrote:
>>
>> Sorry for the late reply - Ping.
>>
>> TL;DR; I think the metrics m
Not interesting :) ?
On Thu, Jul 7, 2022 at 10:41 AM Jarek Potiuk wrote:
>
> Hello everyone,
>
> We have just published a blog on our medium -
> https://medium.com/apache-airflow/airflows-magic-loop-ec424b05b629 - that is
> a blog of one of our users Itay Bittan (thanks!) who
Hey everyone,
I keep on being involved in discussions where people are complaining
about how bad and useless the SLA feature of Airflow is. And yeah, I
pretty much agree with it.
Without getting into details of why it is bad - should we possibly
just, well, deprecate it? I think that would give
Sorry for the late reply - Ping.
TL;DR; I think the metrics might be useful but I think using triggers
is asking for troubles.
While using triggers sounds like a common approach in a number of
installations, we do not use triggers so far.
Using Triggers moves some logic to the database, and in
Yes. Using ProvidersManager is the best option. And the easiest way is
to do (example for databricks):
```
info = ProvidersManager().providers['apache-airflow-providers-databricks']
```
this will return ProviderInfo class (providers is a str ->
ProviderInfo dict) which keeps information about
Hey everyone.
I am about to start preparing a new wave of providers.
Following the recent discussion and agreed "Release Process for
Providers"
https://github.com/apache/airflow/blob/main/README.md#release-process-for-providers.
If there is someone interested in doing cherry-picks for some
+1 (binding).
I double checked most of the changes I was involved with (updated
https://github.com/apache/airflow/issues/24863), and reached out to users
who could help with verifying a rare deadlock fix. I verified that
dependencies look good and are nicely updated (this is a nice one with FAB
4
Hello everyone,
We have just published a blog on our medium -
https://medium.com/apache-airflow/airflows-magic-loop-ec424b05b629 - that
is a blog of one of our users Itay Bittan (thanks!) who had been inspired
by our discussion on Slack on how they struggle with delays of loading
dynamic dags in
;
> *isort* can enforce automatic imports addition or deletion via the
> *add_imports* and *remove_imports* options. (might be good to check if
> this can help achieve what we want):
> https://pycqa.github.io/isort/docs/configuration/options.html
>
> Best,
> Pierre
>
> Le
of right now?
>
> -ash
>
> On Mon, Jul 4 2022 at 00:12:41 +0200, Jarek Potiuk
> wrote:
>
> Hey everyone,
>
> *TL;DR;* I wanted to discuss that maybe (not really sure- but I wanted to
> check with others) we could migrate the whole codebase of airflow to
> suppo
Too late :D. But it seems we could have two winners :)
On Mon, Jul 4, 2022 at 8:32 AM Rajath Srinivasaiah
wrote:
> +1 for #24284
>
>
> Thanks and Regards,
> Rajath S
> rajath.srinivasa...@astronomer.io
>
> [image: image.png]
>
>
> On Sat, Jul 2, 2022 at 3:51 PM Pankaj Singh
> wrote:
>
>> +1
Hey everyone,
*TL;DR;* I wanted to discuss that maybe (not really sure- but I wanted to
check with others) we could migrate the whole codebase of airflow to
support PEP 563 - postponed evaluation of annotations. PEP 563 is nicely
supported in our case (Python 3.7+) by `from __future__ import
Thanks to everyone who helped and contributed to the discussion!
On Sun, Jul 3, 2022 at 6:55 PM Jarek Potiuk wrote:
>
> The Lazy consensus have been reached.
>
> The https://github.com/apache/airflow/pull/24680 is merged now and the
> process we are going to follow regarding rel
-for-providers
I will follow up with those who wish to contribute their cherry-picks
and work out more detailed instructions based on our experiences with
cherry-picking bugfixes to Airflow releases. Watch out for the next
provider's release.
J.
On Thu, Jun 30, 2022 at 8:23 AM Jarek Potiuk wrote
And there is a discount for committers for a bargain price :)
-- Forwarded message -
From: Brian Proffitt
Date: Fri, Jul 1, 2022 at 7:48 PM
Subject: ApacheCon Registration Available, Committers Discount
To:
The ApacheCon planners and the Apache Software Foundation (ASF) are
Hey Everyone,
The registration for ONSITE ApacheCon in New Orleans, Louisiana,
October 3-6, is now open.
This is the first time we have a 2-full-day "DataEngineering" track -
which I am co-chairing with Ismael (PMC member of Apache Beam and
Apache Avro). We have some great talks - including
901 - 1000 of 2965 matches
Mail list logo
