Re: [DISCUSS] Deprecation policy for apache-airflow-providers-google package

I think it's a good idea to codify the rules, indeed what Elad mentioned, we **mostly** follow a very similar approach, but it's not written anywhere. Re: points of Elad: 1. I am ok with 6 months, but I agree here we should not have "strict" limits. It's fine to have some default (i.e.

CVE-2024-25128: Apache Airlfow Vulnerability: custom, long deprecated OpenID (NOT OIDC)

CVE-2024-25128: Vulnerability in custom, long deprecated OpenID (NOT OIDC) authentication method in Flask AppBuilder Severity: moderate Affected versions: - Apache Airflow before 2.8.2 Description: When Flask-AppBuilder configuration is set to ``AUTH_TYPE`` set to ``AUTH_OID``, it allows an

Updated pandas constraints for Airflow 2.8.2 (pandas==2.1.4)

Hey here, We've just solved a teething problem with Airflow 2.8.2 pandas dependency that aa-mathias (thanks Mattthias!) reported to us (Andrey fixed it in main already by adding < 2.2 requirement for pandas). The problem is that Pandas 2.2.1 has a problem with sqlalchemy 1.4 compatibility

Re: [DISCUSS] Considering trying out uv for our CI workflows

at if you've been using breeze and were sometimes afraid > to > > > hit "y" to rebuild the image, being afraid that it will take 20 minutes > or > > so - not any more. It should be WAY faster now. > > I'm very excited about this speed up as well as our CI :) > >

Re: [DISCUSS] Considering trying out uv for our CI workflows

Summarising where we are: After ~24 hrs of operations, it looks really cool and fulfills (and actually exceeds) all my expectations. * Multiple PRs succeeded, we got quite a few constraints updated automatically after successful canary runs:

Re: Bad mixing of decorated and classic operators (users shooting themselves in their foot)

stead. > > > > > > Regarding preventing this anti-pattern, I can think of two ways: > > > > > > Add a check in BaseOperator to detect whether __init__ is called in a > > task > > > work context, and error out guiding users to use hooks instead. > &

Re: [VOTE] January 2024 PR of the Month

Thanks TP, for bringing it :) If I can split my vote between #37101 and #36797 - > that would be it On Tue, Feb 27, 2024 at 6:19 AM Tzu-ping Chung wrote: > I want to nominate #36797.[1] > > This one has been a pretty popular request as well.[2] arguably dating > even further back since dynamic

Re: Bad mixing of decorated and classic operators (users shooting themselves in their foot)

instincts of our users guide us :) Would love to hear what others think J On Tue, Feb 27, 2024 at 8:09 AM Jarek Potiuk wrote: > Hello here, > > I have seen recently at least a few times that our users started to use a > strange pattern > > @task(task_id='some_id', provide_context

Bad mixing of decorated and classic operators (users shooting themselves in their foot)

Hello here, I have seen recently at least a few times that our users started to use a strange pattern @task(task_id='some_id', provide_context=True) def some_dummy_task(**context): ti = context['ti'] cmd2 = 'echo "pushing 2"' dbt_task = BashOperator(task_id='some_dummy_task_id',

Re: [VOTE] January 2024 PR of the Month

+1 on #37101 -> I think all the changes that make the dataset feature more powerful (and especially one that our users definitely asked for) is a really cool way to make our users look at Airflow through the lens of "modern and slick". This one is definitely taking us closer to this goal. On Mon,

Re: [DISCUSS] Considering trying out uv for our CI workflows

lp here. > > But one interesting idea that airflow could look at is periodically > running test cases against either "--resolution=lowest" or > "--resolution=lowest-direct" to try and flush out bad old dependencies? > > Damian > > -Original Message- &

Re: [ANNOYUNCEMENT] Fix to Airflow 2.8.1 / 2.8.2 constraints (smtp) - downgrade smtp provider to 1.5.0

Ok. turned out that image refresh was not needed because ... smtp provider is not included. Which is a bit surprising (will add it in the future I think) but also a bit easier as constraints got updated and point to 1.5.0 of smtp provider now. On Mon, Feb 26, 2024 at 4:02 PM Jarek Potiuk wrote

[ANNOYUNCEMENT] Fix to Airflow 2.8.1 / 2.8.2 constraints (smtp) - downgrade smtp provider to 1.5.0

Hello here, One of our users (thanks @stepheatgiithub) notified us about an issue we introduced in smtp provider in december - and released in 1.6.0 version of provider (SMTP notifications were broken). Unfortunately SMTP 1.6.0 has been released as part of the 2.8.1 and 2.8.2 images (and

Re: [DISCUSS] Considering trying out uv for our CI workflows

And merged. I will keep an eye on it for the next few days. On Mon, Feb 26, 2024 at 11:47 AM Jarek Potiuk wrote: > Yes. The difference was because of caching. I forgot to mention that. This > is due to the way our CI image docker optimisation works > > The way how the image is

Re: MsSqlOperator

It's best if you open a Github Discussion for it: https://github.com/apache/airflow/discussions J. On Mon, Feb 26, 2024 at 1:40 PM Bharani Chandrika wrote: > Hi, > > > >I am trying to work with MsSqlOperator but getting import module error > but it has been installed properly as I can see

Re: [DISCUSS] Considering trying out uv for our CI workflows

r. Stefan Hartung, Dr. Christian Fischer, Dr. Markus > Forschner, > Stefan Grosch, Dr. Markus Heyn, Dr. Frank Meyer, Dr. Tanja Rückert > > -Original Message- > From: Jarek Potiuk > Sent: Montag, 26. Februar 2024 08:54 > To: Amogh Desai > Cc: dev@airflow.apache.org >

Re: [DISCUSS] Considering trying out uv for our CI workflows

waiting for reviews J. On Mon, Feb 26, 2024 at 4:50 AM Amogh Desai wrote: > Thanks for the superb investigation and effort @Jarek Potiuk > ! > > I quite like the performance improvement numbers uv brings in compared to > pip. > I see no reason not to switch to UV in prod i

Re: [DISCUSS] Considering trying out uv for our CI workflows

), but it's still significant enough. Without cache the "upgrade scenario is ~ 40s (so no 4s any more) instead of 7m with pip - so this is still a huge improvement (image size is even smaller than the one with `pip`). J, On Sun, Feb 25, 2024 at 9:17 PM Jarek Potiuk wrote: > Some more

Re: [DISCUSS] Considering trying out uv for our CI workflows

te the CI faster, but this tool will help reduce the overall time. > > Big +1 > > On Sun, Feb 25, 2024 at 12:34 AM Jarek Potiuk wrote: > > > Hello here. > > > > I have a PR https://github.com/apache/airflow/pull/37683 that > implements: > > > > * ability t

Re: [VOTE] Release Airflow 2.8.2 from 2.8.2rc3

; > > +1 non-binding > > > > On Sun, Feb 25, 2024 at 4:00 AM Hussein Awala wrote: > > > > > +1 (binding) ran the same checks and tests as the RC2; all look good. > > > > > > On Sat, Feb 24, 2024 at 8:22 PM Jarek Potiuk wrote: >

Re: [DISCUSS] Considering trying out uv for our CI workflows

Of course the PR is not yet complete - there are few things to fix there for special cases we have. On Sun, Feb 25, 2024 at 12:33 AM Jarek Potiuk wrote: > Hello here. > > I have a PR https://github.com/apache/airflow/pull/37683 that implements: > > * ability to choose either

Re: [DISCUSS] Considering trying out uv for our CI workflows

bit and maybe someone would invent the same > > solution  and you don't have to spend a personal time. > > > > So as POC I'm on it, we still need a `pip` and validate some stuff by a > pip > > because it is only one officially supported way to install Airflow but if &g

Re: [VOTE] Release Airflow 2.8.2 from 2.8.2rc3

+1 (binding): installed RC3 and ran a few DAGs. Tested reproducibility, licences, checksums, signatures. Everything looks good ! Side comment: I also found two edge-cases of edge-cases :) - where specific conditions could lead to either failure when running docker-based package preparation or

Re: [VOTE] Airflow Providers prepared on February 23, 2024

Yep. I saw they tested it in https://github.com/apache/airflow/issues/37648 . Cool :) On Fri, Feb 23, 2024 at 5:36 PM Jarek Potiuk wrote: > +1 (binding): checked reproducibility, licences, signatures, checksums - > all looks good. Have not checked if everything works, I guess Teradat

Re: [VOTE] Airflow Providers prepared on February 23, 2024

+1 (binding): checked reproducibility, licences, signatures, checksums - all looks good. Have not checked if everything works, I guess Teradata team is on it. On Fri, Feb 23, 2024 at 7:18 AM Elad Kalif wrote: > Hey all, > > I have just cut the new wave Airflow Providers packages. This email is

Re: [VOTE] Release Airflow 2.8.2 from 2.8.2rc2

+1 (binding) * Installed and ran it locally. Ran a few DAGs, inspected the wheel/sdist/ to see if all the requirements are correct. Converted the sdist to wheel with pip wheel (this was not working in the past release). Checked the release process has few small quirks removed (found out with

Re: [DISCUSS] Common.util provider?

6 > > months after we introduce some common code. > > > > So, maybe better approach would be to introduce the functionality into > > core, > > and use common.X provider as "polyfill" (to borrow JS nomenclature) > > to make sure providers could use that functionality

Re: [DISCUSS] Common.util provider?

ratsvorsitzender: Prof. Dr. Stefan Asenkerschbaumer; > > Geschäftsführung: Dr. Stefan Hartung, Dr. Christian Fischer, Dr. Markus > Forschner, > > Stefan Grosch, Dr. Markus Heyn, Dr. Frank Meyer, Dr. Tanja Rückert > > > > -Original Message- > > From: Jarek

Re: [DISCUSS] Common.util provider?

Stuttgart, Registergericht: Amtsgericht Stuttgart, HRB 14000; > Aufsichtsratsvorsitzender: Prof. Dr. Stefan Asenkerschbaumer; > Geschäftsführung: Dr. Stefan Hartung, Dr. Christian Fischer, Dr. Markus > Forschner, > Stefan Grosch, Dr. Markus Heyn, Dr. Frank Meyer, Dr. Tanja Rückert

Re: [DISCUSS] Common.util provider?

a single > place but keep individual providers fully independent… > > Jens > > Sent from Outlook for iOS<https://aka.ms/o0ukef> > > From: Jarek Potiuk > Sent: Wednesday, February 21, 2024 5:42:20 PM > To: dev@airflow.apache.org > Sub

[DISCUSS] Common.util provider?

Hello everyone, How do we feel about introducing a common.util provider? I know it's not been the original idea behind providers, but - after testing common.sql and now also having common.io, seems like more and more we would like to extract some common code that we would like providers to use,

Re: [EXTERNAL] Re: [VOTE] [RESULT] New Airflow Community Provider: Teradata

distribute the e-mail or any attachments. Instead, > please notify the sender and delete the e-mail and any attachments. Thank > you. > > Please consider the environment before printing. > > > > *From: *Jarek Potiuk > *Date: *Friday, 16 February 2024 at 4:49 PM > *To: *dev@

Re: [DISCUSS] Deprecate cli options in Airflow Configurations and airflow.api.client

>> From: Andrey Anshin > >> Sent: Monday, February 19, 2024 3:54 PM > >> To: dev@airflow.apache.org > >> Subject: Re: [DISCUSS] Deprecate cli options in Airflow Configurations > >> and airflow.api.client > >> > >> >Actually - it's no

Re: [VOTE] Airflow Providers prepared on February 19, 2024

+1 (binding). Tested signatures, licences, checksums, reproducibility. All looks good. On Mon, Feb 19, 2024 at 1:51 PM Wei Lee wrote: > +1 (non-binding) > > Best, > Wei > > > On Feb 19, 2024, at 6:59 PM, Phani Kumar > wrote: > > > > +1 non-binding. > > > > On Mon, Feb 19, 2024 at 4:15 PM

Re: [DISCUSS] Considering trying out uv for our CI workflows

ecome such stable as pip. > > -- > ,,,^..^,,, > > > On Tue, Feb 20, 2024 at 3:06 AM Jarek Potiuk wrote: > > > My opinion: > > > > I think there is a place for a number of such tools. For a long time the > > packaging team and `pip` team have been working not only on `pip` >

Re: [DISCUSS] Considering trying out uv for our CI workflows

t another > installer which stands to resolve all your problems' '. > I guess I was wrong? > > On Tue, 20 Feb 2024 at 00:53, Jarek Potiuk wrote: > > > Hey everyone, > > > > Few days ago the ruff creators have released a new tool uv - which is an > > extremely fa

[DISCUSS] Considering trying out uv for our CI workflows

Hey everyone, Few days ago the ruff creators have released a new tool uv - which is an extremely fast (written in rust) and fully featured tool generally fully compatible with `pip`. Blog post here: https://astral.sh/blog/uv It looks like It has a number of things that would make our CI cases

Re: [DISCUSS] Deprecate cli options in Airflow Configurations and airflow.api.client

Actually - it's not at all part of AIP-44 :). Airflow CLI was (consciously and deliberately) not part of it. CLI is specifically treated as a "local" tool that is executed inside the security perimeter where direct database access is needed (i.e. it falls in the same category as 'scheduler',

Re: [VOTE] Airflow Providers prepared on February 17, 2024

+1 (binding). Checked my changes, Verified reproducibility, signatures, licences, checksums, all looks good. On Sun, Feb 18, 2024 at 6:10 AM Amogh Desai wrote: > +1 non binding > > On Sun, 18 Feb 2024 at 9:37 AM, Wei Lee wrote: > > > +1 (non-binding) > > > > Tested both providers with our

Re: [LAZY CONSENSUS] Rename slack channels

The great renaming is done now. The most important part that requires an action - #development channel is now archived and if you want to continue be involved in contributing - the #contributors channel is the right one to subscribe to. On Thu, Feb 15, 2024 at 8:14 PM Jarek Potiuk wrote

Re: [VOTE] [RESULT] New Airflow Community Provider: Teradata

> > > Since our PR is merged and we’re awaiting the release, I wanted to get > some clarification on the following topics – > >1. Is this the link > > >we should be following for new

Re: [LAZY CONSENSUS] Rename slack channels

b 2024, 22:50 Briana Okyere, > > wrote: > > > > > +1 > > > > > > On Mon, Feb 12, 2024 at 11:42 AM Jarek Potiuk > wrote: > > > > > > > Hey here, > > > > > > > > Following the earlier discussion, I am calling for a

Re: Subject: Expressing Interest in Contributing to Apache Airflow

worth checking out the community page > > <https://airflow.apache.org/community/>, and specifically joining the > Airflow > > community Slack <https://apache-airflow-slack.herokuapp.com/>. > > > > On Thu, Feb 15, 2024 at 10:24 AM Jarek Potiuk wrote: > > >

Re: Subject: Expressing Interest in Contributing to Apache Airflow

Hello Rahul, There is nothing more than the Contributor' guide: https://github.com/apache/airflow/blob/main/contributing-docs/README.rst , everything is described there, including explanation on how to look for good first issues - https://github.com/apache/airflow/contribute. You do not need any

Re: [VOTE] Airflow Providers prepared on February 12, 2024

+1 (binding), checked signatures, checksums, licences, reproducibility. Checked all my changes are present in the packages. -1 (common.sql missing more-itertools dependency - already merged in main). On Mon, Feb 12, 2024 at 9:13 PM Hussein Awala wrote: > I checked the source code, the

[LAZY CONSENSUS] Rename slack channels

Hey here, Following the earlier discussion, I am calling for a lazy consensus on renaming of our slack channels: #development -> *#contributors* #first-pr-support -> *#new-contributors* #troubleshooting -> *#user-troubleshooting* #random -> *#random* (hehe) The current #development channel will

Re: [DISCUSS] Rename channels on slack

:12:05 Briana Okyere wrote: > > > Same, +1 > > > > > > On Mon, Feb 12, 2024 at 9:15 AM Oliveira, Niko > > > > > wrote: > > > > > > > I'm happy with that set of names +1 > > > > > > > > _

Re: [DISCUSS] Rename channels on slack

; wrote: > > > > > > > Good discussions going on. Personally I align with what Jed, Vincent > > and > > > > Jarek had to say. > > > > > > > > My choices would be: > > > > #dev-support -> from #development >

Re: [DISCUSS] Rename channels on slack

e from https://airflow.apache.org/community/ where > #development should be listed under "Want to contribute?" and not "Are you > a user?". I think we can modify that after we renamed the channels. > > On 2024/02/08 22:35:20 Jarek Potiuk wrote: > > Speaking of which (

Re: [DISCUSS] Rename channels on slack

#learn-contributing maybe ? On Fri, Feb 9, 2024 at 12:14 AM Jarek Potiuk wrote: > Vincent: > > > I think we can modify that after we renamed the channels. > > Yes, definitely. > > Alexander: > > > Why a special channel for the first PR? > > Yeah. I do

Re: [DISCUSS] Rename channels on slack

. I am not against the names Jarek suggested. > > On 2024/02/08 11:54:34 Jarek Potiuk wrote: > > Hey here, > > > > The number of "troubleshooting/best-practices" questions we have in > > #development channel on Slack reached the level where we have more of >

Re: [VOTE] Release Apache Airflow Helm Chart 1.12.0 based on 1.12.0rc1

+1 (binding):- checked signatures, checksums, licenses and compared content of the packages. Unfortunately it seems that the binary reproduction is still not fully working - there is some difference in "tar" layer run by Jed and myself, so I will need to do some extra checks. It's not a blocker

Re: [DISCUSS] Rename channels on slack

I see... Hmm.. Fresh start . yeah might be a good idea. On Thu, Feb 8, 2024 at 7:59 PM Jed Cunningham wrote: > That's not what I'm proposing. I should have been more verbose before, > sorry. Let me rephrase. > > Don't rename #development. Leave it where it is, restrict posting to admins > only.

Re: [DISCUSS] Rename channels on slack

>> Maybe we restrict who can post in development for a period of time with a message directing folks to the right places? > As long as we don't make it committer only. If you're contributing something and want some help/feedback, it's not welcoming to find out that you're to be restricted from

Re: [DISCUSS] Rename channels on slack

0 AM Vincent Beck > wrote: > > > > > > > I am +1 in renaming these channels because, as said, most of messages > > in > > > > @development are nothing to do there. > > > > > > > > Though, I would just rename #development to #contributing

[RESULT] [VOTE] "Require conversation resolution" setting in PRs as permanent solution

Hello everyone, Summary of the vote in https://lists.apache.org/thread/5m45dz860ovg4zyvrqscz8m1mnmmrn1g 5 binding (+1) votes: * Jarek Potiuk * Amogh Desai * Pankaj Koti * Jens Scheffer * Hussein Awala 2 binding (-1) votes: * Bolke de Bruin * Ash Berlin-Taylor 1 binding (-0.5) votes: * Jed

[DISCUSS] Rename channels on slack

Hey here, The number of "troubleshooting/best-practices" questions we have in #development channel on Slack reached the level where we have more of those than discussion about airflow development. There were few slack proposals, that we should change the names, but it's quite a significant

Re: [VOTE] "Require conversation resolution" setting in PRs as permanent solution

Since this is a controversial topic, just a kind reminder - the voting ends today in ~ 8 hrs. If you feel (even somewhat) strongly about either direction - that's the last time to cast the vote. And yes - fractional votes are possible :) - see

Re: [VOTE] Add the ability to report slack messages that don't meet code of conduct

I think the bot would be best. It would be great if the reports are made to a private channel (the bot that Andrey posted seems to be doing just that) where it can be discussed between the admin channels and action taken. Plus some way of knowing who is in the admin channel and invitation to

Re: Idea for Discussion: custom TI dependencies

I had my answer drafted before Ash sent it so I will copy it here. But I very much agree with Ash's assessment. I did not realise we could do that with task_policy but if we can, then I would go for it - having it undocumented feature for power users. Making it a feature is a big task that would

[VOTE] "Require conversation resolution" setting in PRs as permanent solution

Hello, We have been discussing and arguing and it seems that we do not have a consensus of whether "require conversation resolution" is a good idea, as promised I am casting a VOTE on it. The discussion is in this thread: https://lists.apache.org/thread/bzg00p8yolboqgwlwfnlkdfvnn996601 Voting

Re: [VOTE] AIP 61 - Hybrid Executors

+1 (binding) . I think we know the scope (more importantly we also know what's out of the scope) On Thu, Feb 1, 2024 at 6:45 AM Scheffler Jens (XC-AS/EAE-ADA-T) wrote: > +1 binding - looking forward to implementation! > > Sent from Outlook for iOS >

Re: [ANNOUNCE] Starting experimenting with "Require conversation resolution" setting

I really understand the concerns. I am also not 100% happy either with the way you have to unresolve things to see past comments, and how links to old resolved comments are difficult to solve. This is certainly a drawback I see, and yes - it's a true concern. And Ash - really my point was not to

Re: [VOTE] Airflow Providers prepared on January 30, 2024

+1 (binding) - checked reproducibility, licences, signatures, checksums - all good. On Tue, Jan 30, 2024 at 5:42 PM Elad Kalif wrote: > Hey all, > > I have just cut an ad-hoc release for the microsoft.azure provider package. > This email is calling a vote on the release, > which will last for

Re: [ANNOUNCE] Starting experimenting with "Require conversation resolution" setting

gt; > > Thanks & Regards, > > > Amogh Desai > > > > > > On Mon, Jan 29, 2024 at 7:56 PM Aritra Basu > > > wrote: > > > > > >> I personally haven't had too much friction due to the change and it > has > > >> helped me

Re: [ANNOUNCE] Starting experimenting with "Require conversation resolution" setting

assessment is. Or maybe we need more time to assess it ? I would love to hear your thoughts. J, On Sat, Dec 30, 2023 at 2:20 PM Jarek Potiuk wrote: > After an initial indentation problem in .asf.yaml it's not working as > expected. So let's see how resolving conversations will work for us.

[ANNOUNCE] Apache Airflow Providers prepared on January 26, 2024 are released

Dear Airflow community, I'm happy to announce that new versions of Airflow Providers packages prepared on January 26, 2024 were just released. Full list of PyPI packages released is added at the end of the message. The source release, as well as the binary releases, are available here:

[RESULT][VOTE] Airflow Providers - release of January 26, 2024

Hello, Apache Airflow Providers prepared on January 26, 2024 have been accepted. 3 "+1" binding votes received: - Jarek Potiuk (binding) - Husseun Awala (binding) - Elad Kalif (binding) 2 "+1" non-binding votes received: - Amogh Desai - Phani Kumar Vote thread: https://lis

Re: [PROPOSE] Add A Code of Conduct for Slack and Meetups

I think the best would be where there is a committee to discuss and decide (usually), but the people who are in the committee are known and you can choose whom to reach out to individually if you do not want to reach out to the whole committee. Google form is very impersonal and when you submit a

Re: [VOTE] Airflow Providers prepared on January 26, 2024

Just a small comment - most of this has already been checked, so if we can have PMC approvals still today, I should be able to release it tomorrow :) On Fri, Jan 26, 2024 at 1:18 PM Jarek Potiuk wrote: > Hey all, > > I have just (on behalf of Elad who is not available) cut the next se

Re: [PROPOSAL] Adding MSGraphSDK Async Operator to Airflow

Hey David - any comments on that ? On Mon, Jan 15, 2024 at 1:07 PM Elad Kalif wrote: > Hi David, > > Thanks for raising this discussion. > following the protocol established about accepting new providers - > >

[VOTE] Airflow Providers prepared on January 26, 2024

Hey all, I have just (on behalf of Elad who is not available) cut the next set of RC candidates for Airflow Providers packages. This email is calling a vote on the release, which will last for at least 24 hours - which means that it will end on January 27, 2024 12:09 PM UTC or until 3 binding +1

Re: [VOTE] Airflow Providers prepared on January 22, 2024

(Elad asked me to take over from here as he is flying tonight) I am cancelling this vote due to the problem found by Andrey - https://github.com/apache/airflow/issues/36948#issuecomment-1909664795 - the problem is that the way logger_name passing and **kwargs handling, there was a potential of

Re: [DISCUSSION] Enhanced Multi-Tenant Dataset Management in Airflow: Potential First Steps

y were > designed to be defined within a dag file, similar to a dag, and we don't > want to deal with the impediment I laid out. > > On Wed, Jan 24, 2024 at 12:45 PM Jarek Potiuk wrote: > > > On Wed, Jan 24, 2024 at 5:33 PM Constance Martineau > > wrote: > > > >

Re: [DISCUSSION] Enhanced Multi-Tenant Dataset Management in Airflow: Potential First Steps

On Wed, Jan 24, 2024 at 5:33 PM Constance Martineau wrote: > I also think it makes sense to allow people to create/update/delete > Datasets via the API and eventually UI. Even if the dataset is not > initially connected to a DAG, it's nice to be able to see in one place all > the datasets and ML

Re: [VOTE] Airflow Providers prepared on January 22, 2024

+1 (binding) - tested my changes, checked binary reproducibility, licences, signatures, checksums - all looks good. On Mon, Jan 22, 2024 at 1:11 PM Elad Kalif wrote: > Hey all, > > I have just cut the new wave Airflow Providers packages. This email is > calling a vote on the release, > which

Re: [DISCUSSION] Enhanced Multi-Tenant Dataset Management in Airflow: Potential First Steps

Clarifying: There is no (and it has never been) a problem with opening up submitting "structured" DAGs. On Tue, Jan 23, 2024 at 2:12 PM Jarek Potiuk wrote: > > I always assumed that this was the reason why it's impossible to create > dags from API, no one wanted to open

Re: [DISCUSSION] Enhanced Multi-Tenant Dataset Management in Airflow: Potential First Steps

ed config file and let respective > dag-processors create them independently of each other. > > On Tue, Jan 23, 2024 at 4:02 PM Jarek Potiuk wrote: > > > I am also pretty cool with adding/updating/datasets externally, however I > > know there are some ongoing discussions on

Re: [DISCUSSION] Enhanced Multi-Tenant Dataset Management in Airflow: Potential First Steps

I am also pretty cool with adding/updating/datasets externally, however I know there are some ongoing discussions on how to improve/change datasets and bind them together with multiple other features of Airflow - not sure what the state of those, but it would be great those effort are coordinated

Re: [VOTE] January 2024 PR of the Month

Heck, why not. I will shamelessly vote on my #36537. While it took just a few weeks to merge, It leapfrogged our legacy packaging setup to more-or-less bleeding edge from what was there since the beginning of Airflow (almost 10 years) and was already "old-ish" when I joined the project more than 4

Re: [VOTE] Accept AIP-60 (Standard URI representation for Airflow Datasets)

+1 (binding). On Fri, Jan 19, 2024 at 12:38 AM Igor Kholopov wrote: > +1 (non-binding) > > On Fri, Jan 19, 2024 at 12:03 AM Maciej Obuchowski > > wrote: > > > +1 (non-binding) > > > > On Thu, Jan 18, 2024, 22:51 Scheffler Jens (XC-AS/EAE-ADA-T) > > wrote: > > > > > +1 binding as discussed -

Re: [EXTERNAL] Re: [VOTE] New Airflow Community Provider: Teradata

Aritra Basu > wrote: > > > > > > +1 non binding > > > > > > -- > > > Regards, > > > Aritra Basu > > > > > > On Wed, Jan 17, 2024, 5:35 AM Wei Lee wrote: > > > > > >> +1 non binding > > >

Re: What goes into the next release (was [VOTE] Release Airflow 2.8.1 from 2.8.1rc1)

ote, like patch-release, and including > the link to the doc. That makes it easier for more occasional viewers. > > B. > > On Wed, 17 Jan 2024 at 13:42, Jarek Potiuk wrote: > >> I separate it out, because it seems that despite the efforts to explain >> and document how ou

Re: [DISCUSSION] Adding support for Qdrant provider

t hook > to remove redundancy. > > On Tue, 16 Jan 2024 at 19:58, Jarek Potiuk wrote: > > > > self hosted image and the cloud offering, comprehensive integration > tests > > with the > > Docker image should suffice. > > > > I am fine with that. > >

Re: What goes into the next release (was [VOTE] Release Airflow 2.8.1 from 2.8.1rc1)

ings are breaking/not breaking. And removed "how some people know/don't know" about it. I hope having that FAQ will help us to get to the point where we all know it - or at least can easily find it when we are looking for it. J. On Thu, Jan 18, 2024 at 8:42 AM Jarek Potiuk wrote: > I think i

Re: What goes into the next release (was [VOTE] Release Airflow 2.8.1 from 2.8.1rc1)

at 5:29 AM Amogh Desai wrote: > Thanks @Jarek Potiuk ! > > I vaguely knew the process but not in such detail, thanks for putting it > together in this email. > I will visit the document > https://github.com/apache/airflow?tab=readme-ov-file#what-goes-into-the-next-release > and

What goes into the next release (was [VOTE] Release Airflow 2.8.1 from 2.8.1rc1)

uded hence asking. > > B. > > Sent from my iPhone > > > On 16 Jan 2024, at 20:18, Jarek Potiuk wrote: > > > > +1 (binding): Checked all my changes, I ran airflow in a few > combinations > > (MySQL / Postgres Local/Celery executor. It looks and works w

Re: [VOTE] Release Airflow 2.8.1 from 2.8.1rc1

+1 (binding): Checked all my changes, I ran airflow in a few combinations (MySQL / Postgres Local/Celery executor. It looks and works well - run a few dags and navigated through a number of screens. Checked licences, signatures, checksums, performed a "reproducible build" check and it worked (with

Re: [VOTE] New Airflow Community Provider: Teradata

+1 binding On Tue, Jan 16, 2024 at 6:20 PM Phani Kumar wrote: > +1 non binding > > On Tue, Jan 16, 2024 at 10:43 PM K Mallam, Sunil > wrote: > > > Hello Airflow Community, > > > > Thank you very much for your comments/feedback on the Discussion Thread. > > > > I’m creating this voting thread

Re: [EXTERNAL] Re: [Discuss] New Airflow Community Provider: Teradata

thread before I start the voting > thread. > > > > So far we’ve have 4 positive responses (with 1 binding vote). Is there > > anyone else who like to share their feedback? > > > > > > Thanks, > > SM > > > > From: Jarek Potiuk > > Date

Re: [DISCUSSION] Adding support for Qdrant provider

offering, comprehensive integration tests with the > > Docker image should suffice. > > > > As you said, would appreciate any thoughts from the community. > > > > On Tue, 16 Jan, 2024, 7:11 pm Jarek Potiuk, wrote: > > > >> BTW: Dashboard links here: &g

Re: [DISCUSSION] Adding support for Qdrant provider

BTW: Dashboard links here: https://airflow.apache.org/ecosystem/#airflow-provider-system-test-dashboards On Tue, Jan 16, 2024 at 2:39 PM Jarek Potiuk wrote: > I'd love to hear from others in the community who already use Qdrant what > they think :) ? > > Few comments to Anush: >

Re: [DISCUSSION] Adding support for Qdrant provider

I'd love to hear from others in the community who already use Qdrant what they think :) ? Few comments to Anush: I did a bit of review of the links and did some usual research. 1) Re: requirements it does not introduce any big issues. Urllib3 < 2 is a bit strange (but we are anyhow limited by

Re: [LAZY CONSENSUS] Add Qdrant as a supported provider

-1 for now. I am not talking that I do not agree with having the provider, I just object to **just** start LAZY CONSENSUS without earlier discussion and explanation and justification why you think Qdrant is good for the community here and why it's not good to have Qdrant team to manage it on

[ANNOUNCE] New PMC member: Andrey Anshin (taragolis)

I have the pleasure to announce that The Project Management Committee (PMC) for Apache Airflow has invited Andrey Anshin (taragolis) to become Apache Airflow PMC Member and we are pleased to announce that he has kindly accepted it. Being a PMC member enables assistance with the management and to

Re: [LAZY CONSENSUS] Change default docker image to point to "latest supported"

ome in Airflow 2.9 and Python 3.12 being aliased from "airflow:2.9.0" and "airlfow:latest". I am reverting the changelog about it and will apply it in main with "proper" fix shortly. J. On Fri, Dec 1, 2023 at 12:41 PM Jarek Potiuk wrote: > Small correction - it is

[ANNOUNCEMENT] Modern Package management / Virtual environment management with Hatch

Hey Here, I just merged the https://github.com/apache/airflow/pull/36537 - which aims to bring Airflow into the modern packaging world of Python. It should be largely transparent, but I would recommend using the opportunity to refresh your virtual environments. Particularly I recommend you start

Re: [PROPOSAL] Standardize Airflow Packaging and build process (modern appropriate PEP-compliant tooling)

And merged :D. On Wed, Jan 10, 2024 at 6:56 PM Jarek Potiuk wrote: > I made a comparison of package files before <> after and added a few > corrections. Also I've added an extra security layer for CI building of > airflow packages - it runs inside a fully isolated Docker conta

Re: [PROPOSAL] Standardize Airflow Packaging and build process (modern appropriate PEP-compliant tooling)

d, Jan 10, 2024 at 1:36 AM Jarek Potiuk wrote: > Hey Everyone, > > I got the PR green: https://github.com/apache/airflow/pull/36537 - I got > a really comprehensive review and a number of iterations with Jens (and > approval! yay!!) and a number of comments from TP. > > I would

Re: [PROPOSAL] Standardize Airflow Packaging and build process (modern appropriate PEP-compliant tooling)

nvironment with Python 3.11. No devel installed │ └─┴─┴──┴───┘ J. On Sun, Jan 7, 2024 at 11:55 PM Jarek Potiuk wrote: > Ah .. .And comparing to the original proposal I simplified it a LOT

<    1   2   3   4   5   6   7   8   9   10   >