Re: [ANNOUNCEMENT] Looking for potential new security team members

2024-01-08 Thread Jarek Potiuk
> > not so sure if this will be the right forum for "experimentation". > > > > So, not so sure if I cut the criteria here. > > > > Thanks & Regards, > > Amogh Desai > > > > On Sat, Jan 6, 2024 at 2:14 AM Jarek Potiuk wrote: >

Re: [PROPOSAL] Standardize Airflow Packaging and build process (modern appropriate PEP-compliant tooling)

2024-01-07 Thread Jarek Potiuk
/pep-0561/>`__ * `PEP-621 Storing project metadata in pyproject.toml <https://www.python.org/dev/peps/pep-0621/>`__ * `PEP-685 Comparison of extra names for optional distribution dependencies <https://www.python.org/dev/peps/pep-0622/>`__ J. On Sun, Jan 7, 2024 at 11:27 PM Jarek Potiu

Re: [PROPOSAL] Standardize Airflow Packaging and build process (modern appropriate PEP-compliant tooling)

2024-01-07 Thread Jarek Potiuk
ve. That's the in-progress experiment with conversations which I personally like a lot so far. It already saved me from merging a PR that still had things to resolve. J. On Thu, Jan 4, 2024 at 8:04 PM Jarek Potiuk wrote: > > I slept over it a few nights and got away of it and I have an idea to > simp

Re: [VOTE] Airflow Providers prepared on January 07, 2024

2024-01-07 Thread Jarek Potiuk
+1 (binding) - > checked reproducible builds, signatures, licences, checksums, all look good. On Sun, Jan 7, 2024 at 1:11 PM Elad Kalif wrote: > > Hey all, > > I have just cut the new wave Airflow Providers packages. This email is > calling a vote on the release, > which will last for 72 hours -

Re: [ANNOUNCEMENT] Looking for potential new security team members

2024-01-05 Thread Jarek Potiuk
as I'm quite confident I do not fit > > the criteria  > > > > On Fri, Jan 5, 2024 at 9:21 AM Jarek Potiuk wrote: > > > > > Hello everyone, > > > > > > TL;DR; In short - we are looking for candidates to join our security > > > team. Please sen

Re: [PROPOSAL] let's update the community content on the landing pages

2024-01-04 Thread Jarek Potiuk
I am in. And would even like to see this more "live" over time. But Maybe a proposal here. Looking at things like https://airflow.apache.org/survey/ I - for one - would be perfectly fine if someone - you and Briana - take more "stewardship' for those pages Maybe that's a good idea that we

Re: [DISCUSSION] Enabling `pre-commit.ci` application for Airflow

2024-01-04 Thread Jarek Potiuk
gt; Though I am not sure whether the plugin is able to use all the custom > >>>> stuff as we also depend during execution on the CI image and docker. > >>>> Besides security things this would be something that needs testing if it > >>>> works. &g

Re: [PROPOSAL] Standardize Airflow Packaging and build process (modern appropriate PEP-compliant tooling)

2024-01-04 Thread Jarek Potiuk
two big pros for me are its ease of use (backend and front > end) as well as the security covered aspects (reproducible builds to name > one). > > I will take a look at the PR later this week, but it definitely sounds > exciting. > > > > On Tue 2 Jan 2024 at 20:26, Ja

Re: [Discussion] AIP-60 Standard URI representation for Airflow Datasets

2024-01-03 Thread Jarek Potiuk
Happy 2024, Added some nits - mostly informational, generally LGTM On Tue, Jan 2, 2024 at 11:23 AM Tzu-ping Chung wrote: > > Happy 2024 everyone! > > I’m going to kick off the new year by formally proposing a new AIP. This > attempts to standardise the URI format used by Dataset events. This

Re: [DISCUSS] Interested in joining/leaving the triage team ?

2024-01-03 Thread Jarek Potiuk
And Aritra Basu as well :). On Wed, Jan 3, 2024 at 6:35 PM Jarek Potiuk wrote: > > Updating collaborators - removing Utkarsh and Adding Rahul Vats and > Gopal Dirisala who wanted to join. > > On Tue, Jan 2, 2024 at 4:09 PM Wei Lee wrote: > > > > Hi Jarek, > >

Re: [DISCUSS] Interested in joining/leaving the triage team ?

2024-01-03 Thread Jarek Potiuk
; I would want to continue being a "collaborators" team for the triage. > >>> > >>> *Ankit Chaurasia* > >>> HomePage <https://ankitchaurasia.info/> | LinkedIn > >>> <https://www.linkedin.com/in/sunank200/> > >>> > >>&

Re: [EXTERNAL] Re: [Discuss] New Airflow Community Provider: Teradata

2024-01-03 Thread Jarek Potiuk
You get to the voting stage (or lazy consensus) when you decide it's ready to vote. Up to you to see if you think it's going to pass the vote/consensus or not. I'd say - so far the response is positive, so I'd wait a few days and give a chance to people to react after they came back from holidays

Re: [VOTE] December PR of the Month

2024-01-02 Thread Jarek Potiuk
24 at 7:51 AM Wei Lee wrote: > > > I want to vote for https://github.com/apache/airflow/pull/35926, and > > thank Jarek for bringing it up. > > https://github.com/apache/airflow/pull/35719 is also great, so it's a > > difficult choice. > > > > Best

Re: [VOTE] December PR of the Month

2024-01-02 Thread Jarek Potiuk
I'd like to propose another one: https://github.com/apache/airflow/pull/35926 - Create FAB provider and move FAB auth manager in it I think that one is something some of us have been talking about getting rid of FAB for a long time. While FAB was really cool when Airflow started, it has not aged

Re: Looking for co-mentor in Major League Hacking Spring 2024 Fellows

2024-01-02 Thread Jarek Potiuk
ounder. > > There are a bunch of improvements we can make to the REST API that I'd love > to see. > > On Fri, Dec 29, 2023 at 3:11 AM Jarek Potiuk wrote: > > > Absolutely - devlist is the best place :) > > > > On Fri, Dec 29, 2023 at 7:22 AM Amogh Desai > >

[PROPOSAL] Standardize Airflow Packaging and build process (modern appropriate PEP-compliant tooling)

2024-01-02 Thread Jarek Potiuk
Hello everyone. Tl;DR; I have a proposal to adopt Hatchling as a build backend (and recommend, but not require Hatch as frontend) for Airflow as our way of switching to PEP-standard compliant pyproject.toml way of installing Airflow (including local venvs) and building the Airflow package. I

[ANNOUNCE] Apache Airflow Providers prepared on December 31, 2023 are released

2024-01-02 Thread Jarek Potiuk
Dear Airflow community, I'm happy to announce that new versions of Airflow Providers packages prepared on December 31, 2023 were just released. Full list of PyPI packages released is added at the end of the message. The source release, as well as the binary releases, are available here:

[RESULT][VOTE] Airflow Providers - release of December 31, 2023

2024-01-02 Thread Jarek Potiuk
Hello, Apache Airflow Providers prepared on DATE OF RELEASE have been accepted. 3 "+1" binding votes received: - Jarek Potiuk (binding) - Husein Awala (binding) - Elad Kalif (binding) 5 "+1" non-binding votes received: - Utkarsh Sharma - Phani Kumar - Amogh Desai - Wei Lee

Re: [VOTE] Airflow Providers prepared on December 31, 2023

2024-01-01 Thread Jarek Potiuk
> > > > >> +1 non binding. > > >> > > >> On Sun, Dec 31, 2023 at 9:48 PM Hussein Awala > wrote: > > >> > > >>> +1 (binding) > > >>> > > >>> On Sun, Dec 31, 2023 at 5:14 PM Jarek Potiuk > wrote

Re: [VOTE] Airflow Providers prepared on December 31, 2023

2023-12-31 Thread Jarek Potiuk
Cool. Now I just need 2 binding +1s :) On Sun, Dec 31, 2023 at 5:04 PM utkarsh sharma wrote: > +1(non-binding) tested my changes. > > On Sun, Dec 31, 2023 at 6:16 PM Jarek Potiuk wrote: > > > Hey all, > > > > I have just cut the new wave Airflow Providers packa

[VOTE] Airflow Providers prepared on December 31, 2023

2023-12-31 Thread Jarek Potiuk
Hey all, I have just cut the new wave Airflow Providers packages. This email is calling a vote on the release, which will last for 24 hours - accelerated vote which means that it will end on January 01, 2024 12:42 PM UTC and until 3 binding +1 votes have been received. Consider this my (binding)

[ANNOUNCE] Apache Airflow Providers prepared on 28th December 2023 are released

2023-12-31 Thread Jarek Potiuk
Dear Airflow community, I'm happy to announce that new versions of Airflow Providers packages prepared on 28th of December 2023 were just released. Full list of PyPI packages released is added at the end of the message. The source release, as well as the binary releases, are available here:

[RESULT][VOTE] Airflow Providers - release of 28th December 2023

2023-12-31 Thread Jarek Potiuk
Hello, Apache Airflow Providers prepared on 28th of December 2023 have been accepted (except Weaviate provider for which RC2 with accelerated voting will follow shortly. 3 "+1" binding votes received (for all providers except Waeviate): - Jarek Potiuk (binding) - Hussein Awala (bindi

Re: [ANNOUNCE] Starting experimenting with "Require conversation resolution" setting

2023-12-30 Thread Jarek Potiuk
After an initial indentation problem in .asf.yaml it's not working as expected. So let's see how resolving conversations will work for us. On Sat, Dec 30, 2023 at 12:17 PM Amogh Desai wrote: > Wooho! Looking to see how this turns out for airflow  > > On Sat, 30 Dec 2023 at 1:35 

Re: [VOTE] Airflow Providers prepared on December 28, 2023

2023-12-30 Thread Jarek Potiuk
Need 2 binding (+1) to proceed on that one (and get RC2 for Waeviate) On Thu, Dec 28, 2023 at 6:48 PM Jarek Potiuk wrote: > We will get an rc2 for weaviate. Depending on how many other changes will > be merged in the meantime I will either release ad-hoc for just weaviate > (and po

[ANNOUNCE] Starting experimenting with "Require conversation resolution" setting

2023-12-30 Thread Jarek Potiuk
Hello everyone, As discussed in https://lists.apache.org/thread/cs6mcvpn2lk9w2p4oz43t20z3fg5nl7l I just enabled "require conversation resolution" for our main/stable branches. We have not used it in the past so it might not work as we think or we might need to tweak something. Generally speaking

[DISCUSS] Interested in joining/leaving the triage team ?

2023-12-29 Thread Jarek Potiuk
A regular question if there are people who are interested in joining the "collaboators" team for our triage team (or would like to be removed) ? We regularly review/cleanup the "collaborators" team of ours https://github.com/apache/airflow/blob/main/.asf.yaml#L78 . Being collaborator gives a bit

Re: [DISCUSSION] Enabling `pre-commit.ci` application for Airflow

2023-12-29 Thread Jarek Potiuk
intainers of it to adapt it to our needs). 2) Also, there is a "Pre-commit Lite Github Action" that we **might** be able to use to achieve a similar effect (with some added complexity to our `Pull Request Target` workflow. So I would still love to hear from others :) J. On Fri, Dec 29, 2023 at

[DISCUSSION] Enabling `pre-commit.ci` application for Airflow

2023-12-29 Thread Jarek Potiuk
Hello everyone, TL;DRl; I'd like to propose that we enable the pre-commit-ci GitHub application for Airflow repo. According to how I understand it works, it should greatly reduce friction (especially for new contributors) for passing the quality gates for our pre-commits. That is - if we get the

Re: Looking for co-mentor in Major League Hacking Spring 2024 Fellows

2023-12-29 Thread Jarek Potiuk
t; > > > On Thu, 28 Dec 2023 at 11:29 PM, Jarek Potiuk wrote: > > > Oh. Wow. I did not know you followed it Amogh after I had to pass it on > :). > > That's cool. I think we should be more vocal about those kinds of > > mentorships / internships. This is reall

Re: Looking for co-mentor in Major League Hacking Spring 2024 Fellows

2023-12-28 Thread Jarek Potiuk
right now, I'd love to do it again in > the future. Meanwhile, I encourage anyone with moderate knowledge to > consider joining this fantastic program. It's a chance to make a > difference, learn, and be part of an awesome community. > > Feel free to reach out if you have any quest

Re: [VOTE] Airflow Providers prepared on December 28, 2023

2023-12-28 Thread Jarek Potiuk
ing) for Weavaite provider. I found an issue with the provider, > which I unfortunately missed in my initial testing. Raised a PR > <https://github.com/apache/airflow/pull/36475> for the same. > > Thanks, > Utkarsh Sharma > > On Thu, Dec 28, 2023 at 2:40 PM Jarek Potiuk wrote:

Re: [DISCUSS] "Require conversation resolution" in our PRs before merge?

2023-12-28 Thread Jarek Potiuk
to announce it on #development in slack and in devlist here - linking to that part of our CONTRIBUTING docs. J. On Fri, Dec 22, 2023 at 8:09 PM Jarek Potiuk wrote: > Hey Elad, Daniek, > > I understand your concerns, and yes I agree visibility of "resolved" > threads is lower

Re: Looking for co-mentor in Major League Hacking Spring 2024 Fellows

2023-12-28 Thread Jarek Potiuk
The fellowship starts 29th of Jan *2024 * of course :) On Thu, Dec 28, 2023 at 10:53 AM Jarek Potiuk wrote: > Hello Airflow Community, > > TL;DR; I am looking for a (volunteer) co-mentor to join me in > mentoring 2 Fellows from Major League Hacking Spring 2024. From the > past

Looking for co-mentor in Major League Hacking Spring 2024 Fellows

2023-12-28 Thread Jarek Potiuk
Hello Airflow Community, TL;DR; I am looking for a (volunteer) co-mentor to join me in mentoring 2 Fellows from Major League Hacking Spring 2024. From the past experiences, I know it's a very important thing to have two co-mentors, because that allows them to share the (small) extra load and be

[VOTE] Airflow Providers prepared on December 28, 2023

2023-12-28 Thread Jarek Potiuk
Hey all, I have just cut the new wave Airflow Providers packages. This email is calling a vote on the release, which will last for 72 hours - which means that it will end on December 31, 2023 09:08 AM UTC and until 3 binding +1 votes have been received. Consider this my (binding) +1. Airflow

[ANNOUNCE] Apache Airflow Providers prepared on 23rd December 2023 are released

2023-12-27 Thread Jarek Potiuk
Dear Airflow community, I'm happy to announce that new versions of Airflow Providers packages were just released. There are 49 provider packages released this time. The full list follows at the end. The source release, as well as the binary releases, are available here:

[RESULT][VOTE] Airflow Providers - release of 23rd of December 2023

2023-12-27 Thread Jarek Potiuk
Hello, Apache Airflow Providers prepared on 23rd of December 2023 have been accepted. 3 "+1" binding votes received: - Jarek Potiuk (binding) - Hussein Awala (binding) - Ephraim Anierobi (binding) 6 "+1" non-binding votes received: - Wel Lee - Rahul Vats - Utkarsh

Re: [VOTE] Airflow Providers prepared on December 23, 2023

2023-12-27 Thread Jarek Potiuk
One more binding vote needed :) On Wed, Dec 27, 2023 at 7:30 PM Hussein Awala wrote: > +1 (binding) I checked the checksums, the signatures, the licences, and the > sources and I tested all my changes, all look good. > > On Tue, Dec 26, 2023 at 11:56 PM Jarek Potiuk wrote: &

Re: [VOTE] Airflow Providers prepared on December 23, 2023

2023-12-26 Thread Jarek Potiuk
ve tested the following provider RCs with our example DAGs > > without > > > > > > encountering issues. > > > > > > > > > > > > apache-airflow-providers-microsoft-azure==8.5.0rc1 > > > > > > apache-airflow-providers-sf

[VOTE] Airflow Providers prepared on December 23, 2023

2023-12-22 Thread Jarek Potiuk
Hey all, I have just cut the new wave Airflow Providers packages. This email is calling a vote on the release, which will last for 72 hours - which means that it will end on December 27th, 2023 02:24 AM UTC and until 3 binding +1 votes have been received. This accounts for upcoming Holidays.

Re: [VOTE] Release Airflow Python Client 2.8.0 from 2.8.0rc1

2023-12-22 Thread Jarek Potiuk
+1 (binding). Tested, licences, sources, checksums. Verified source code by regenerating the client and comparing the output. Tested `test_python_client.py` with it. All looks fine. On Tue, Dec 19, 2023 at 3:36 PM Wei Lee wrote: > +1 (non-binding) > > Best, > Wei > > > On Dec 19, 2023, at 8:20 

Re: [Discuss] New Airflow Community Provider: Teradata

2023-12-22 Thread Jarek Potiuk
Thanks a lot Satish, Mishra, Adam for all the work put in this. I clearly see that you took seriously our expectations for a new provider we set. By a (quick) look at the arguments, code, dependencies, code quality, complexity, and system test dashboard (yay!) I personally think it's quite a good

Re: [DISCUSS] "Require conversation resolution" in our PRs before merge?

2023-12-22 Thread Jarek Potiuk
ek. > > > > > > We can try it out since year ends are low traffic periods, we get some > > > experimental phase with this. > > > > > > Thanks & Regards, > > > Amogh Desai > > > > > > On Wed, Dec 20, 2023 at 7:45 PM Aritra Basu

Re: [DISCUSSION] Do the resources on the Community landing page need reordering/reprioritizing?

2023-12-20 Thread Jarek Potiuk
Yep - separating into "for developers" and "for users" is I think a good idea. I Imagine right now someone who just comes with troubleshooting is - I am sure - a bit confused where to look for help. Not sure how to nicely design it but I think we should think about "UX experience" here and

Re: [DISCUSS] "Require conversation resolution" in our PRs before merge?

2023-12-20 Thread Jarek Potiuk
sume isn't restricting a commit author from resolving the > comments in cases where they either think it doesn't need fixing or if it's > perhaps something for a follow-up pr? What I mean is it isn't gonna enforce > who can resolve a comment thread? > > -- > Regards, > Aritra

Re: Issue: MappedArgument Object is not Subscriptable

2023-12-20 Thread Jarek Potiuk
Please kindly use other channels for troubleshooting. Devlist is used for development discussions, users@ is more for user's questions, Slack and Github Discussions are mostly used for troubleshooting and issues is where you have reproducible issue that you have quite a certainty it's an Airflow

Re: [DISCUSS] "Require conversation resolution" in our PRs before merge?

2023-12-20 Thread Jarek Potiuk
to communicate it in a few of our channels that we are experimenting with and be on the lookout to help people if they will have problems with it. J. On Wed, Dec 20, 2023 at 8:31 AM Jarek Potiuk wrote: > Amogh: > > > Should we enforce creating a follow up Github issue in our case fo

Re: [DISCUSS] "Require conversation resolution" in our PRs before merge?

2023-12-19 Thread Jarek Potiuk
and then follow up on > it later. > > Should we enforce creating a follow up Github issue in our case for the > same? This would solve the issue of unfinished PRs > while also giving confidence that the comments won't be ignored. > > > Thanks & Regards, > Amogh D

Re: [DISCUSS] "Require conversation resolution" in our PRs before merge?

2023-12-19 Thread Jarek Potiuk
o > get blocked because of a drive-by comment. > > Seems like this is easily reversible and we can give it a trial run and > decide later if we want to keep it. > > > - ferruzzi > > > > From: Jarek Potiuk > Sent: Tuesday

Re: [DISCUSS] "Require conversation resolution" in our PRs before merge?

2023-12-19 Thread Jarek Potiuk
23 20:33:23 GMT, Jarek Potiuk wrote: > >Ash: > > > >> I.e. Convention over enforcement and treating people as mature adults > not > >children who need guard rails. > > > >I think it's quite the opposite, Both 1) 2) and 3) reasoning is more of > the

Re: [DISCUSS] "Require conversation resolution" in our PRs before merge?

2023-12-19 Thread Jarek Potiuk
ugh). Why not everyone who actively participates in the PR review process brings their experience and explains if it caused them unnecessary burden for no gain (also the opposite - where it helped). J. On Tue, Dec 19, 2023 at 9:15 PM Jarek Potiuk wrote: > Answering some of the recent ques

Re: [DISCUSS] "Require conversation resolution" in our PRs before merge?

2023-12-19 Thread Jarek Potiuk
> > > > > > > > > > > Best, > > > > > > Wei > > > > > > > > > > > >> On Dec 19, 2023, at 9:28 PM, Andrey Anshin < > > > andrey.ans...@taragol.is> > > > > wrote: > > > > > >> >

Re: [DISCUSS] "Require conversation resolution" in our PRs before merge?

2023-12-19 Thread Jarek Potiuk
reason we always might > > >> revert it back. > > >> > > >> Just one suggestion, sometimes discussion contains some useful > > >> information, > > >> e.g. "What the reason of finally decision", "Useful information why it > &g

Re: [DISCUSS] Future of Pendulum in Airflow

2023-12-19 Thread Jarek Potiuk
gt; and the potential of us being in the same place again potentially in the > > > future. > > > > > > -- > > > Regards, > > > Aritra Basu > > > > > > On Fri, Dec 1, 2023, 10:27 PM Jarek Potiuk wrote: > > > > > > >

[DISCUSS] "Require conversation resolution" in our PRs before merge?

2023-12-19 Thread Jarek Potiuk
Hey everyone, TL;DR; I have a small proposal/discussion proposal to modify a bit the branch protection rules for Airflow. Why don't we add a protection rule in our PRs that requires all the comments in the PRs to be "marked as resolved" before merging the PR ? I have been following myself - for

Re: [ANNOUNCE] Apache Airflow 2.8.0 Released

2023-12-18 Thread Jarek Potiuk
Wooohooo! On Mon, Dec 18, 2023 at 8:05 PM Ephraim Anierobi wrote: > > Dear Airflow community, > > I'm happy to announce that Airflow 2.8.0 was just released. > > The released sources and packages can be downloaded via >

Re: [LAZY CONSENSUS] Switch 2.8.0 images to use MariaDB client

2023-12-18 Thread Jarek Potiuk
Yep. Reached :) On Mon, Dec 18, 2023 at 5:37 PM Ephraim Anierobi wrote: > > The lazy consensus has been reached > > On 2023/12/15 15:45:18 Jarek Potiuk wrote: > > Hello everyone, > > > > This is a bit hurried, but the circumstances are unusual. > > > > E

Re: [VOTE] Release Airflow 2.8.0 from 2.8.0rc4

2023-12-18 Thread Jarek Potiuk
+1 (binding). Checked checksums, signatures, sources, licences, run some tests with Breeze with a few python versions and databases and both local and celery executor and run a few dags in each - all looks good. On Mon, Dec 18, 2023 at 12:50 PM Utkarsh Sharma wrote: > > +1 Non-binding > > On

[ANNOUNCE] Apache Airflow Providers prepared on 12 December 2023 are released

2023-12-17 Thread Jarek Potiuk
Dear Airflow community, I'm happy to announce that new versions of Airflow Providers packages were just released. https://pypi.org/project/apache-airflow-providers-odbc/4.3.0/ https://pypi.org/project/apache-airflow-providers-docker/3.9.0/

Re: Making Listeners API non-experimental

2023-12-17 Thread Jarek Potiuk
; > > > > wrote: > > > > > > > > > > > +1 > > > > > > > > > > > > On Sat, 16 Dec 2023 at 10:56, Amogh Desai < > > amoghdesai@gmail.com> > > > > > > wrote: > > > > > > > &g

[RESULT][VOTE] Airflow Providers - release of 12 December 2023

2023-12-17 Thread Jarek Potiuk
Hello, Apache Airflow Providers (based on RC2) have been accepted (while excluding databricks and google providers which got 5 non-binding -1 and will get RC3 soon). The remaining providers received those votes: 3 "+1" binding votes received: - Jarek Potiuk (binding) - Hussein Awal

Re: [VOTE] Airflow Providers prepared on December 12, 2023

2023-12-17 Thread Jarek Potiuk
harma < > utkarshar...@gmail.com> > > > wrote: > > > >> > > > >> +1 non-binding > > > >> > > > >> On Sun, 17 Dec 2023 at 1:15 PM, Pankaj Singh < > > ags.pankaj1...@gmail.com> > > > >> wrote: &g

Re: [VOTE] Airflow Providers prepared on December 12, 2023

2023-12-16 Thread Jarek Potiuk
Bump .. On Fri, Dec 15, 2023 at 6:26 PM Jarek Potiuk wrote: > One more +1 needed, please :) > > On Thu, Dec 14, 2023 at 1:39 PM Hussein Awala wrote: > >> +1 (binding) for daskexecutor, docker and odbc providers >> >> On Thu 14 Dec 2023 at 13:37, Jarek Potiuk w

Re: [LAZY CONSENSUS] Switch 2.8.0 images to use MariaDB client

2023-12-15 Thread Jarek Potiuk
ast 2 days because of this bug). Lazy consensus thread here: https://lists.apache.org/list.html?dev@airflow.apache.org I hope - for the sake of your users losing days due to such issues, you will reconsider your policies around signing your APT repos. On Fri, Dec 15, 2023 at 4:47 PM Jarek Pot

Re: [VOTE] Airflow Providers prepared on December 12, 2023

2023-12-15 Thread Jarek Potiuk
One more +1 needed, please :) On Thu, Dec 14, 2023 at 1:39 PM Hussein Awala wrote: > +1 (binding) for daskexecutor, docker and odbc providers > > On Thu 14 Dec 2023 at 13:37, Jarek Potiuk wrote: > > > A small reminder. Despite -1s on Google, databricks, i would love t

Re: [LAZY CONSENSUS] Switch 2.8.0 images to use MariaDB client

2023-12-15 Thread Jarek Potiuk
BTW. Big shoutout to Andrey who implemented the easy switch between the clients, a few months ago (for ARM case) in the way that we could implement the switch **really** quickly. Kudos for anticipating the need :D On Fri, Dec 15, 2023 at 4:45 PM Jarek Potiuk wrote: > Hello every

[LAZY CONSENSUS] Switch 2.8.0 images to use MariaDB client

2023-12-15 Thread Jarek Potiuk
Hello everyone, This is a bit hurried, but the circumstances are unusual. Even if we use MySQL and not MariaDB as our "production ready" database we've lost a bit of faith in Oracle being a good steward of MySQL apt repos. More details can be found in:

Re: [PROPOSAL] NEED URGENT FEEDBACK! MySQL Badly signed repo breaks Airflow images

2023-12-15 Thread Jarek Potiuk
PR with complete fix for "main" and "2.8.0": https://github.com/apache/airflow/pull/36243 Once it passes all the tests and we will know the solution there works, I will also start removing the repos for old released images. J On Fri, Dec 15, 2023 at 3:09 PM Jarek Pot

Re: [PROPOSAL] NEED URGENT FEEDBACK! MySQL Badly signed repo breaks Airflow images

2023-12-15 Thread Jarek Potiuk
lient. The Client is generally very thin and it follows well-defined and rather simple protocol, so I do not expect some "real" issues with it. I will send a LAZY CONSENSUS soon and PR with proposal of permanent fix J. On Fri, Dec 15, 2023 at 1:46 PM Jarek Potiuk wrote: > > Isn

Re: Making Listeners API non-experimental

2023-12-15 Thread Jarek Potiuk
+1. ESPECIALLY if there will be issues found that we will fix in 2.8.x :D On Fri, Dec 15, 2023 at 1:47 PM Elad Kalif wrote: > If There were no issues discovered so far I am +1 in making it non > experimental for 2.9 > > On Fri, Dec 15, 2023 at 2:45 PM Maciej Obuchowski > wrote: > > > Make it

Re: [PROPOSAL] NEED URGENT FEEDBACK! MySQL Badly signed repo breaks Airflow images

2023-12-15 Thread Jarek Potiuk
ices > > could give info about what kind of libraries they use for MySQL? I assume > > in AWS MWAA also use MariaDB (at least what i could see for local-runner) > > but it my assumption which required confirmation before taking it into > the > > account. > > > > On Fri,

[PROPOSAL] NEED URGENT FEEDBACK! MySQL Badly signed repo breaks Airflow images

2023-12-15 Thread Jarek Potiuk
Hello everyone, (sorry for capital letters but we need to act quickly). *TL;DR; Due to Oracle breaking MySQL repo signing, Together with Andrey we have a proposal to - immediately - switch to MariaDB clients in Airflow images for main and the upcoming Airflow 2.8.0 - giving the user an option to

Re: [VOTE] Airflow Providers prepared on December 12, 2023

2023-12-14 Thread Jarek Potiuk
:) . J On Wed, Dec 13, 2023 at 2:08 PM Wei Lee wrote: > -1 (non-binding) for the google and databricks providers. Sent > https://github.com/apache/airflow/pull/36202 for fixing the google issue > > Best, > Wei > > Jarek Potiuk 於 2023年12月13日 週三 下午6:34寫道: > > > Google an

Re: [VOTE] Airflow Providers prepared on December 12, 2023

2023-12-13 Thread Jarek Potiuk
D', 'state_message': 'Workload failed, see run > > > output > > > > for details', 'user_cancelled_or_timedout': False***, 'task': > > > > ***'spark_python_task': ***'python_file': > > > > > > > > > > > > > > 'dbfs:/mnt/pyscripts/lo

Re: [ANNOUNCE] Airflow Virtual Town Hall- Jan. 11th

2023-12-12 Thread Jarek Potiuk
; > > > The Town Hall will take place on January 11th at 8 AM PST, please > > register > > > above to join, and let me know if you have any insights on the below > > > agenda: > > > > > > AGENDA > > > - Introduction by

[VOTE] Airflow Providers prepared on December 12, 2023

2023-12-12 Thread Jarek Potiuk
Hey all, [Filling-in for Elad, who had no time to do it this time] I have just cut the new wave Airflow Providers packages. This email is calling a vote on the release, which will last for 24 hours - which means that it will end on December 13, 2023 22:00 PM UTC and until 3 binding +1 votes have

Re: [DISCUSS] Using serde for PyODBC case/other cases [was Serialization in Apache Airflow]

2023-12-11 Thread Jarek Potiuk
Tried to get others some opportunity to comment, but I see it's mostly me <> Bolke. > > > Yep. I rectified the implicitness by storing this as an explicit decision > > record > > in the ADR proposal. I think it's been - similarly as lack of > > > > Happy to help here - since there is a good

Re: [DISCUSS] Capturing Architectural decisions (ADRS?)

2023-12-11 Thread Jarek Potiuk
; really handy. It is easier to check compared to the doc. And the commit > rule doesn't need to be complicated. But yes, this kind of rule would > definitely be a barrier to new contributors, which will be something we > need to consider as well. > > Best, > Wei > > > On De

Re: [PROPOSAL] Deprecate URI Connection representation in favor of JSON

2023-12-11 Thread Jarek Potiuk
+1 to Daniels's suggestion. On Tue, Nov 28, 2023 at 5:25 AM Amogh Desai wrote: > Agree with @Daniel Standish on this. > We should rename get_uri to reduce confusion. > > I am fine with renaming to to_uri_repr and then deprecating get_uri > > > Thanks & Regards, > Amogh Desai > > On Mon, Nov

Re: [VOTE] Airflow Providers prepared on December 08, 2023

2023-12-10 Thread Jarek Potiuk
+1 (binding) for all providers except Google, an Docker where bugs were found as noted in the Github Issue. BTW. I have also found that it was easy to miss that the that removed `daskexecutor` provider should also be prepared separately (with "removal" note) - so this one will also need to be

Re: [DISCUSS] Capturing Architectural decisions (ADRS?)

2023-12-04 Thread Jarek Potiuk
e. > Ne cliquez sur aucun lien et n’ouvrez aucune pièce jointe si vous ne pouvez > pas confirmer l’identité de l’expéditeur et si vous n’êtes pas certain que > le contenu ne présente aucun risque. > > > > I love this idea. I definitely think it can improve a lot the kno

Re: Helm Chart Version 1.9.0 Unavailable

2023-12-04 Thread Jarek Potiuk
You should follow the download page - https://airflow.apache.org/docs/helm-chart/1.9.0/installing-helm-chart-from-sources.html - not downloads* link. It will forward you to the direct location. Per ASF policies - downloads.apache.org contains generally only latest maintained releases. See the

Re: [PROPOSAL] Security team rotation introduction to our process

2023-12-04 Thread Jarek Potiuk
adual introduction via a brief shadow period > where a new member would monitor the happenings but not partake in decision > making and once they are done with the shadow period they take on full > responsibility. > > -- > Regards, > Aritra Basu > > On Mon, Dec 4

[DISCUSS] Using serde for PyODBC case/other cases [was Serialization in Apache Airflow]

2023-12-04 Thread Jarek Potiuk
Hey here, Also separating it out as another topic. > > It is a choice that needs to be made at the API level - here Common.sql. In > this case the particular form of the tuple was, somewhat implicit, part of > the spec. > Yep. I rectified the implicitness by storing this as an explicit decision

[DISCUSS] Allowlist in serializatin [was: Serialization in Apache Airflow]

2023-12-04 Thread Jarek Potiuk
Let me start separate threads from the main discussion. It will be easier to follow. I'd really like to understand the actors involved and what kind of threats the allowlist is supposed to protect us. I think it would be great if we see the "attack" scenario we think allowlist will protect us

[ANNOUNCE] New committer: Utkarsh Sharma

2023-12-04 Thread Jarek Potiuk
Hello everyone, [filling-in for Kaxil who is less available nowadays and travelling without much access to computer] The Project Management Committee (PMC) for Apache Airflow has invited Utkarsh Sharma to become a committer and we are pleased to announce that they have accepted. Utkarsh had

[PROPOSAL] Security team rotation introduction to our process

2023-12-04 Thread Jarek Potiuk
Hello everyone, *TL;DR; *I have a proposal of refinements we can apply to our security team and I am looking for comments and feedback (PR is out there in [1]). In short I am proposing that we introduce rotation of the security team members, so that we can avoid burnout, give a chance to others

Re: [DISCUSS] Serialization in Apache Airflow: Clarifications and Future Directions

2023-12-03 Thread Jarek Potiuk
> > > I am reaching out to initiate a discussion on a topic that has been causing > some confusion within the community: serialization in Apache Airflow. The > purpose of this email is to shed light on the current state of > serialization and, if necessary, open the floor for discussions on the

[DISCUSS] Capturing Architectural decisions (ADRS?)

2023-12-03 Thread Jarek Potiuk
Hey everyone, I think we had a bit of clash in https://github.com/apache/airflow/pull/32319 where both "ideas" (serialization and common.sql) had not been sufficiently discussed/explained and I hope we can address it by adding (a bit) more "whys" to our (developer) documentation. I think a

[DISCUSS] Common.sql interface / API

2023-12-03 Thread Jarek Potiuk
Hey here, In parallel to the serialisation discussion, I also wanted to make sure we. raise the topic of common.io. It's been - similarly as serialization - kind of "low profile" - i.e. we've implemented it, discussed in the PRs and added a new provider, but we never - I think - in the devlist

Re: [DISCUSS] Future of Pendulum in Airflow

2023-12-01 Thread Jarek Potiuk
siveness). I hope > someone can take that away. > > Bolke > > > Sent from my iPhone > > > On 1 Dec 2023, at 13:39, Jarek Potiuk wrote: > > > > Agree. I think this is the best way forward.- we have to bite the > bullet of > > potential backwards compatib

Re: [DISCUSS] Future of Pendulum in Airflow

2023-12-01 Thread Jarek Potiuk
for datetime > types in context, which might help to get familiar types for a while. > > > On Wed, 29 Nov 2023 at 05:58, Jarek Potiuk wrote: > > > "infamous fork of " should be "fork of infamous Akka" - just to be clear > :) > > > > On Wed,

Re: [LAZY CONSENSUS] Change default docker image to point to "latest supported"

2023-12-01 Thread Jarek Potiuk
Small correction - it is "3.11" - we do not support 3.12 yet - mostly due to Pendulum holding us back. The PR switching it is here: https://github.com/apache/airflow/pull/36003 On Wed, Nov 29, 2023 at 10:05 AM Jarek Potiuk wrote: > Lazy consensus have been reached. I am going

Re: [LAZY CONSENSUS] Change default docker image to point to "latest supported"

2023-11-29 Thread Jarek Potiuk
Lazy consensus have been reached. I am going to change our tooling to mark latest available (3.12) as the "latest" image. On Wed, Nov 22, 2023 at 1:59 AM Jarek Potiuk wrote: > Hello everyone, > > Following the proposal > https://lists.apache.org/thread/xy192wmv6k1sy2wlx5v

Re: [LAZY CONSENSUS] Remove 'daskexecutor' provider

2023-11-29 Thread Jarek Potiuk
Lazy consensus has been reached. I am proceeding with the removal of daskexecutor provider. On Fri, Nov 24, 2023 at 6:51 PM Jarek Potiuk wrote: > Hello everyone, > > This is a formal call for consensus to remove the daskexecutor provider. > > All the details and propos

Re: [DISCUSS] Future of Pendulum in Airflow

2023-11-28 Thread Jarek Potiuk
"infamous fork of " should be "fork of infamous Akka" - just to be clear :) On Wed, Nov 29, 2023 at 2:57 AM Jarek Potiuk wrote: > It's a very rare occurrence that ASF accepts forks. Usually it has to be > willingly donated by those who own the IP rights, - mainly bec

Re: [DISCUSS] Future of Pendulum in Airflow

2023-11-28 Thread Jarek Potiuk
t; airflow? > > If not 'in' airflow, I wonder if ASF incubator would accept a forked > project? [ anecdotally Linux Foundation has incubated Starrocks which is a > fork of Apache Doris, so there's precedence for this to be OK in the wider > OSS ecosystem ]. > > > On Tue, Nov 28

Re: [DISCUSS] Future of Pendulum in Airflow

2023-11-28 Thread Jarek Potiuk
pport "is coming" but no-one knows when, effectively means that we should look for an alternative like NOW. J. On Wed, Nov 22, 2023 at 3:27 AM Daniel Standish wrote: > Thanks Jarek > > On Tue, Nov 21, 2023, 4:34 PM Jarek Potiuk wrote: > > > I think we miss important i

Re: [PROPOSE] Airflow Monthly Town-Hall

2023-11-28 Thread Jarek Potiuk
As Briana wrote - yeah I also think it's great to have it. We've been also discussing it at the informal "after-summit" gathering which was mostly "let's meet while we are there" - that having some opportunity for synchronous communication is cool. I think some basic rules need to be followed,

<    1   2   3   4   5   6   7   8   9   10   >