Re: [DISCUSS] Example dags

2025-06-01 Thread Amogh Desai
Nice! Thanks & Regards, Amogh Desai On Sat, May 31, 2025 at 4:30 PM Jarek Potiuk wrote: > And another small follow-up before we release 3.0.2 as Jed pointed out in > the previous PR, those example dags from standard provider do not really > fit the same pattern as system tests + example dags in

Re: [DISCUSS] Example dags

2025-05-31 Thread Jarek Potiuk
And another small follow-up before we release 3.0.2 as Jed pointed out in the previous PR, those example dags from standard provider do not really fit the same pattern as system tests + example dags in google, amazon and other providers - so we should likely put them in example_dags folder and for

Re: [DISCUSS] Example dags

2025-05-29 Thread Jarek Potiuk
As discussed at the last dev call - before we release 3.0.2 , I brought back example_dags back to "airflow" package - until we do the complete "Example Refurbish" - this one should bring back the same "experience" for example dags as before - including standalone airflow and airflow installed in ve

Re: [DISCUSS] Example dags

2025-05-11 Thread Amogh Desai
Sounds great Jens, thanks for creating the channel and thanks for updating us on it. Thanks & Regards, Amogh Desai On Sun, May 11, 2025 at 7:54 PM Jens Scheffler wrote: > Hi, > > following-up on the discussion we had and the verbal discussion in > Airflow 3 Dev call May 22nd (see notes in > >

Re: [DISCUSS] Example dags

2025-05-11 Thread Jens Scheffler
Hi, following-up on the discussion we had and the verbal discussion in Airflow 3 Dev call May 22nd (see notes in https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+3+Dev+call%3A+Meeting+Notes#Airflow3Devcall:MeetingNotes-22May2025) I propose the following next steps: 1) start a sepa

Re: [DISCUSS] Example dags

2025-05-07 Thread Aritra Basu
I was going to start a convo on it as well, wouldn't mind taking point on this -- Regards, Aritra Basu On Wed, 7 May 2025, 4:31 pm Jarek Potiuk, wrote: > Indeed > > On Wed, May 7, 2025 at 12:35 PM Kaxil Naik wrote: > > > Worth adding it as an agenda item for dev call -- I am sure more folks > >

Re: [DISCUSS] Example dags

2025-05-07 Thread Jarek Potiuk
Indeed On Wed, May 7, 2025 at 12:35 PM Kaxil Naik wrote: > Worth adding it as an agenda item for dev call -- I am sure more folks > might want to help out on this one. > > On Wed, 7 May 2025 at 15:44, Jarek Potiuk wrote: > > > I think it would be great to get someone to take a lead and define w

Re: [DISCUSS] Example dags

2025-05-07 Thread Kaxil Naik
Worth adding it as an agenda item for dev call -- I am sure more folks might want to help out on this one. On Wed, 7 May 2025 at 15:44, Jarek Potiuk wrote: > I think it would be great to get someone to take a lead and define what we > should do here. From what I see (and slack discussion here >

Re: [DISCUSS] Example dags

2025-05-07 Thread Jarek Potiuk
I think it would be great to get someone to take a lead and define what we should do here. From what I see (and slack discussion here https://apache-airflow.slack.com/archives/C06K9Q5G2UA/p1746587477484579), I think what we need now is to make a concrete proposal on * which examples should stay *

Re: [DISCUSS] Example dags

2025-05-01 Thread Jarek Potiuk
> Yes, probably too many example dags. Probably this is a consequence of using them as test dags. Just comment on that - I think it's more a consequence of not having a way to separate those. The idea of using test dags for examples and code snippets is an extremely sound one because it has this

Re: [DISCUSS] Example dags

2025-04-30 Thread Daniel Standish
Yeah, along Jens lines... Yes, probably too many example dags. Probably this is a consequence of using them as test dags. Probably ideally we are more selective in curating example dags and distinguishing them from test dags.

Re: [DISCUSS] Example dags

2025-04-30 Thread Aritra Basu
#x27;s a > > bigger lift to get it done, so maybe not part of the immediate migration, > > but it would be a nice overall improvement project. > > > > > > - ferruzzi > > > > > > ________ > > From: Jens Scheffler > >

Re: [DISCUSS] Example dags

2025-04-30 Thread Jarek Potiuk
Sent: Wednesday, April 30, 2025 1:56 PM > To: dev@airflow.apache.org > Subject: RE: [EXT] [DISCUSS] Example dags > > CAUTION: This email originated from outside of the organization. Do not > click links or open attachments unless you can confirm the sender and know > the content

Re: [DISCUSS] Example dags

2025-04-30 Thread Ferruzzi, Dennis
il 30, 2025 1:56 PM To: dev@airflow.apache.org Subject: RE: [EXT] [DISCUSS] Example dags CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe. AVERTISSEMENT: Ce courrier électro

Re: [DISCUSS] Example dags

2025-04-30 Thread Jens Scheffler
irflow.apache.org Subject: RE: [EXT] [DISCUSS] Example dags CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe. AVERTISSEMENT: Ce courrier électronique provient d’un expéditeur ex

Re: [DISCUSS] Example dags

2025-04-30 Thread Ferruzzi, Dennis
w the line, etc. So I think I'd go with 1. - ferruzzi From: Jarek Potiuk Sent: Wednesday, April 30, 2025 10:53 AM To: dev@airflow.apache.org Subject: RE: [EXT] [DISCUSS] Example dags CAUTION: This email originated from outside of the organization. Do not

Re: [DISCUSS] Example dags

2025-04-30 Thread Jarek Potiuk
The PR is finally green, sources updated, example dags moved to "standard provider", yet in breeze they are visible. I run all the local testing in Breeze for the moved dags and they work nicely. I think we might continue discussing what should be the "final" approach - but for now - merging that

Re: [DISCUSS] Example dags

2025-04-30 Thread Jarek Potiuk
Yep - all the dags will be visible in `breeze` in my PR https://github.com/apache/airflow/pull/49978 - just pushed a change for it: [image: image.png] On Wed, Apr 30, 2025 at 4:31 PM Jarek Potiuk wrote: > Answering Constance (somewhat it went to another thread:) > > > I know we rely on the exam

Re: [DISCUSS] Example dags

2025-04-30 Thread Jarek Potiuk
Answering Constance (somewhat it went to another thread:) > I know we rely on the example dags for dev purposes, especially for UI development. As part of moving them out of core, can we update breeze to automatically load them from their new home? Yes. Very good point. I strongly think we need

Re: [DISCUSS] Example dags

2025-04-30 Thread Constance Martineau
s/test/dev On Wed, Apr 30, 2025 at 8:49 AM Constance Martineau wrote: > I know we rely on the example dags for test purposes, especially for UI > development. As part of moving them out of core, can we update breeze to > automatically load them from their new home? > > On Wed, Apr 30, 2025 at 7:

Re: [DISCUSS] Example dags

2025-04-30 Thread Constance Martineau
I know we rely on the example dags for test purposes, especially for UI development. As part of moving them out of core, can we update breeze to automatically load them from their new home? On Wed, Apr 30, 2025 at 7:12 AM Kaxil Naik wrote: > just an idea: With "git bundles", maybe example DAGs f

Re: [DISCUSS] Example dags

2025-04-30 Thread Kaxil Naik
just an idea: With "git bundles", maybe example DAGs from standard provider can be dynamically imported? But from your options: (1) seems reasonable On Wed, 30 Apr 2025 at 15:07, Jarek Potiuk wrote: > Hello here. > > While fixing a bug in the doc generation to bring "source" link to example > d

[DISCUSS] Example dags

2025-04-30 Thread Jarek Potiuk
Hello here. While fixing a bug in the doc generation to bring "source" link to example dags https://github.com/apache/airflow/pull/49978 I realized that we have not moved a number of python/bash etc. example dags to standard provider - where they now belong. I do it as part of my PR - but as resul