Re: [DISCUSS] consolidate dag scheduling params

2022-07-29 Thread Vikram Koka
-1 from me. Though I agree in principle with the idea of consolidation, I don't think we should be doing this yet until we understand the implications completely. I am really not in favor of deprecation of the existing params, unless there is really no alternative. On Fri, Jul 29, 2022 at 2:37

Re: [DISCUSS] consolidate dag scheduling params

2022-07-29 Thread Daniel Standish
So far, seems all in favor. I will just highlight, in case it's not clear when we release this (presumably 2.4), basically every single dag in existence will start emitting deprecation warnings, and prior to 3.0, basically every dag in existence will need to be updated. Thankfully, for most

Re: [DISCUSS] consolidate dag scheduling params

2022-07-29 Thread Brent Bovenzi
+1 to consolidating and "schedule". On Fri, Jul 29, 2022, 10:12 PM Drew Hubl wrote: > +1 for ‘schedule', and another +1 to the importance of consolidating to > one being more important than the name of that one > > On Jul 29, 2022, at 1:58 PM, Josh Fell > wrote: > > Consolidating to a single

Re: [DISCUSS] consolidate dag scheduling params

2022-07-29 Thread Josh Fell
Consolidating to a single scheduling parameter is a big +1 from me. `schedule` seems like a nice catch-all. On Fri, Jul 29, 2022 at 9:10 AM Jed Cunningham wrote: > +1 on moving to a single `schedule` param. >

Re: Apache Airflow Newsletter | July 2022

2022-07-29 Thread Jarek Potiuk
Nice! On Fri, Jul 29, 2022 at 9:32 PM Michael Robinson via users < us...@airflow.apache.org> wrote: > View this email in your browser > > > This month in the Airflow community, we released Airflow 2.3.3 >

Fwd: Apache Airflow Newsletter | July 2022

2022-07-29 Thread Michael Robinson
> View this email in your browser > > > > This month in the Airflow community, we released Airflow 2.3.3 > >

Re: [DISCUSS] consolidate dag scheduling params

2022-07-29 Thread Jed Cunningham
+1 on moving to a single `schedule` param.

Re: [VOTE] Release Airflow Python Client 2.3.0 based on 2.3.0rc1

2022-07-29 Thread Phani Kumar
+1 non-binding. I am able to install the RC package , made changes to the local airflow config to use basic authentication to run some basic client calls . Observed the same issue as Jarek while executing the test_python_client.py. On Fri, Jul 29, 2022 at 4:47 PM Jarek Potiuk wrote: > Thanks

Re: [VOTE] Release Airflow Python Client 2.3.0 based on 2.3.0rc1

2022-07-29 Thread Jarek Potiuk
Thanks Pankaj for bringing it up. I think I will take a bit closer look at the tests/integration as part of implementing the AIP-44 - Internal API (hopefully I will get some comments and once we get it implemented - probably on 2.5). This will cover better integration and a bit more (smoke)

Re: [VOTE] Release Airflow Python Client 2.3.0 based on 2.3.0rc1

2022-07-29 Thread Pankaj Koti
+1 non-binding. The RC package installs fine, am able to run some basic client calls (using basic auth). Facing the same issue as Jarek while running the test_python_client.py. Is it planned to have a document helping on how to make client calls corresponding to the REST APIs? The test suite

Re: SparkSubmitHook - high memory consumption in YARN cluster mode

2022-07-29 Thread Jeff Zhang
You don't need to kill spark-submit process by yourself, just configure this spark conf spark.yarn.submit.waitAppCompletion to be false, then spark submit process will exit right after yarn accepts it. On Fri, Jul 29, 2022 at 5:23 AM Tornike Gurgenidze wrote: > Hi all, > > I opened a ticket