Re: [DISCUSSION] Assessing what is a breaking change for Airflow (SemVer context)

2022-11-22 Thread Darren Weber
Deprecate first to discover the possible footprint of any public API. With the deprecation, bump the minor version. Add documentation about the deprecation and appropriate deprecation code and warnings, with a notice for the EOL date or an EOL sem ver. — Darren On Tue, Nov 22, 2022 at 1:32 AM

Re: [VOTE] AIP-51 - Removing Executor Coupling from Core Airlfow

2022-11-16 Thread Darren Weber
+1 This turned out to be a sticking point while considering AIP-28 (replaced by AIP-40) On Wed, Nov 16, 2022 at 8:44 AM Xiaodong Deng wrote: > Just read through the AIP. It reads nice & solid  > > +1 binding. > > > XD > > On Wed, Nov 16, 2022 at 2:16 AM Tomasz Urbaszek > wrote: > >> +1

Re: [VOTE] Approach for supported python versions

2020-12-03 Thread Darren Weber
+1 On Thu, Dec 3, 2020 at 3:23 PM Kaxil Naik wrote: > +1 > > On Thu, Dec 3, 2020 at 10:14 PM Deng Xiaodong wrote: > >> Yep, it clarifies. I find it important because people may have different >> interpretations on "*support*" and may lead to confusion later. But this >> supplementary statement

Re: [VOTE] Switch from using Jira to Github Issues

2020-03-16 Thread Darren Weber
+1 What about AIPs - will they migrate also? On Mon, Mar 16, 2020 at 8:01 AM Maxime Beauchemin < maximebeauche...@gmail.com> wrote: > +1 binding > > On Mon, Mar 16, 2020, 7:45 AM Jarek Potiuk > wrote: > > > +1 binding. > > > > Re: contention: I think we can split reviewers by components. I

Re: Remove Dask Executor in Airflow 2.0 ?

2020-01-22 Thread Darren Weber
be added to CONTRIBUTING.md. > > > On Mon, Jan 20, 2020 at 10:18 PM Darren Weber > > wrote: > > > Via the GSOC thread, I found > > > > > https://cwiki.apache.org/confluence/display/AIRFLOW/First+time+contributor%27s+workshop > > - agree wi

Re: Remove Dask Executor in Airflow 2.0 ?

2020-01-20 Thread Darren Weber
> J. > > > On Mon, Jan 20, 2020 at 5:25 PM Darren Weber > wrote: > > > Thanks for the ping on https://github.com/dask/dask/issues/5803 > > > > I'm curious about how dask async features might be low-hanging fruit for > > Airflow scaling > > - https://

Re: Remove Dask Executor in Airflow 2.0 ?

2020-01-20 Thread Darren Weber
y list compared to things like autoscaling, DAG > >>> serialization, > >>> > etc. > >>> > > >>> > via Newton Mail [ > >>> > > >>> > https://cloudmagic.com/k/d/mailapp?ct=dx=10.0.32=10.14.6=email_footer_2 > >&g

Re: Remove Dask Executor in Airflow 2.0 ?

2020-01-13 Thread Darren Weber
+1 for keeping it and fixing tests PS, I also noticed the skipped tests while looking at an option to use the async client feature; if/when I get time to get back on that and figure out how the test setup needs to work, I might also discover how to enable tests for the non-async executor. No

Re: [Discussion] In Prep for AIP: Stateful XComs and Poke Rescheduling in Operators

2020-01-09 Thread Darren Weber
Not sure whether to add to this email thread or the google-doc (not sure if that google-doc is just a meeting-notes or if it should evolve into a spec :grin:). Maybe a stupid suggestion, but here it is anyway: XCom - communication between elements of a DAG XState - key/value store available for

Re: [DISCUSS] Revenge of the asserts

2020-01-02 Thread Darren Weber
Judicious use of: # pylint: disable=do-not-use-asserts ^^ helps with interim use of pytest `assert` On Wed, Jan 1, 2020 at 10:39 AM Kamil Breguła wrote: > Hello, > > As I mentioned earlier, I would prefer to maintain one convention for the > project. The migration of the unittest assertions

Re: Grouping tests using pytest markers

2019-12-29 Thread Darren Weber
The link to https://docs.pytest.org/en/latest/example/markers.html#custom-marker-and-command-line-option-to-control-test-runs helps to clarify some of the customization required to add CLI options that select test sets based on markers. +1 for a common default with *no marker*. (It's hard to

Re: Grouping tests using pytest markers

2019-12-28 Thread Darren Weber
60s : 0.0%, time together: 0.0m > > > Total time of the example build: 28m. I am not sure when a test is "slow". > Moreover, I think there could be > a difference in times between local environment (where developer will > decide to use such marker) and > the CI envir

Re: Grouping tests using pytest markers

2019-12-27 Thread Darren Weber
Consider all the options for filtering tests: - http://doc.pytest.org/en/latest/example/markers.html The `pytest -k` filters are very useful. Provide guidelines on how to name things so that `pytest -k` can be used to filter categories of tests. Use markers for tests that might be the

AIP write-permission on Confluence

2019-12-24 Thread Darren Weber
Hi, I'm writing to request permission to create a new AIP at - https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals?show-miniview My Wiki-ID is `dazza`. Thanks, Darren