Oh no.. Swords, swords, .... :) Hello Brian ! Nice to see you back in the conversations. We've been discussing things like this before a few times even over the phone, I recall.
Yes, I think it's partially marketing, I agree. And marketing that is important for the project, which we as engineers, should also at least understand. We should worry about how our work is perceived by non-engineers or at least not-so-much-deep-into-technical-correctness engineers who just want to get stuff done. This is at least my goal of "communicating" such stuff through documentation - recognise that the vast majority of our users just want to get things done and don't really care about abstract mathematical concepts, they might not even be aware of. Our role in this project is not to teach people math or algorithmics, but to help them do their job. That's our primary goal. And to that - more importantly (for the reasons i and Constance explained) we **might** soon not have "technically Directed Acyclic Graphs" - we do not know if we go that route, but IMHO, it's likely. Then from a good engineering point of view, figuring a better name that would ease the transition is not only smart, but also fair, and it will make our life easier, I am not a fan of being "100% precise and correct" if it makes my future life harder. As an experienced engineer I am happy to go down the path of making definitions more blurry, if it is still "good definition" - fitting the purpose, and making it easier to transition to the place where the old definition will no longer be true. That's what I consider being a great engineer is - to not only focus on minute details and definition correctness, but also understanding of my users, communicating with them in the way that makes it easier for them to relate to the project and "do their job", and to foresee and clear the path for future evolution of my product. I hope this explanation can provide a bit more context - at least how I understand it, and I hope you will see it's not a "because we want it" change. It has some real grounding, reasoning, expectations and is something that few of us have been thinking about for a while. So please do not see it as "marketing driven" change. For me it's "engineering driven change that takes into account marketing". J. On Tue, Feb 18, 2025 at 7:20 PM Brian Greene < br...@heisenbergwoodworking.com> wrote: > I disagree with renaming it, or removing the logical idea of directed > acyclic graph, or redefining the acronym. It’s less “math” to me, and more > “data structure”, and a lot of things work the way they do because a dag IS > a DAG. > > What’s the problem with calling something EXACTLY what it is? So rarely > does a technical team get it so right… this feels like a discussion with > marketing people, wanting to obfuscate something completely reasonable, for > no functional value. > > How much energy can be spent making this scenario more common? “what’s DAG > vs Dag vs Oh you mean DaaG :-(… or whatever”. And why would one want to? > > Brian > > On Tue, Feb 18, 2025 at 11:45 AM Jarek Potiuk <ja...@potiuk.com> wrote: > > > > -0.5 > > > Even though the new UI prefers "Dag", I don't think that means "DAG" > is > > incorrect. > > > > I think we have a bunch of related things discussed here - I am not > exactly > > sure what the -0.5 in this context means Brent. Would it be possible to > > clarify ? > > > > On Tue, Feb 18, 2025 at 6:01 PM Ambika Garg <ambikagarg1...@gmail.com> > > wrote: > > > > > +1 would vote for > > > Airflow's *D*ynamic *A*ction *G*raph (DAG) > > > > > > *Dynamic*: Reflects the capability of Airflow to handle workflows that > > > aren’t strictly acyclic, but can adapt to changes, iterations and loops > > > within broader context of workflow. > > > > > > *Action*: Emphasizes that airflow is fundamentally about orchestrating > > > actions or tasks. Whether its data processing, model training or any > > other > > > form of computation. It also conveys operational aspect of what Airflow > > > does. > > > > > > *Graph*: Maintains the core concept that workflows in Airflow are > > > visualized and managed as graphs. This term keeps the idea of > > dependencies > > > and relationships between tasks intact. > > > > > > Ambika. > > > > > > On Tue, Feb 18, 2025 at 11:46 AM Brent Bovenzi > > <br...@astronomer.io.invalid > > > > > > > wrote: > > > > > > > -0.5 > > > > > > > > Even though the new UI prefers "Dag", I don't think that means "DAG" > is > > > > incorrect. > > > > > > > > On Tue, Feb 18, 2025 at 11:36 AM Jarek Potiuk <ja...@potiuk.com> > > wrote: > > > > > > > > > I love "Dynamic Airflow Graph" +10 to that Constance. > > > > > > > > > > And yes. I fully agree with your "somewhat" acyclic statement. I > saw > > > > people > > > > > mentioning "event-driven" workflows to better describe the AI/ML, > but > > > > that > > > > > for me is a synonym of "completely unmanageable workflows" (similar > > to > > > > > micro-services) - what I was alluding to was exactly this - > something > > > > like > > > > > curren DAG with parts of it being potentially cyclic - so we are > > > > completely > > > > > aligned here :). And that's what make it "not DAG - in mathematical > > > > sense" > > > > > - even more reason to depart from it, > > > > > > > > > > J. > > > > > > > > > > On Tue, Feb 18, 2025 at 5:23 PM Constance Martineau > > > > > <consta...@astronomer.io.invalid> wrote: > > > > > > > > > > > Hey all, > > > > > > > > > > > > I appreciate the discussion about how we position the concept of > a > > > DAG > > > > in > > > > > > Airflow. While I agree with updating documentation to > de-emphasize > > > the > > > > > > strict mathematical definition, I do not think we should change > the > > > > > public > > > > > > interface to use "dag" instead of "DAG" everywhere. It adds > > > > > > unnecessary churn and breaks consistency across the ecosystem. > > > > > > > > > > > > Regarding the broader discussion on whether DAGs in Airflow > should > > > > remain > > > > > > acyclic: I acknowledge that some workflows, particularly in ML > and > > > AI, > > > > > > require cyclical behaviour (example: iterative refinement until a > > > > > condition > > > > > > is met). However, I think it's important to recognize that even > in > > > > those > > > > > > cases, the workflow itself still has an overall start and end. > It's > > > not > > > > > an > > > > > > unbounded loop but rather a controlled execution process that > > happens > > > > to > > > > > > have cycles *within* it. > > > > > > > > > > > > That being said, I do see value in reinterpreting "DAG" to better > > > align > > > > > > with how Airflow is actually used, and love shifting away from > > > > "Directed > > > > > > Acyclic Graph" in favour of something more intuitive. However > "Data > > > > > Airflow > > > > > > Graph" makes it sound like we're defining Airflow itself rather > > than > > > > > > describing the structure of workflows within it. > > > > > > > > > > > > If we want a new meaning for DAG that resonates with our base > while > > > > > keeping > > > > > > Airflow's identity clear, I am proposing three alternatives: > > > > > > > > > > > > > > > > > > 1. *Dynamic Airflow Graph*: Ties directly to Airflow while > > > > emphasizing > > > > > > flexibility > > > > > > 2. *Distributed Automation Graph*: Highlights execution across > > > > systems > > > > > > and automations without tying it strictly to data > > > > > > 3. *Directed Activity Graph*: Retains the useful meaning of > > > > "Directed" > > > > > > while shifting away from strict acyclic constraints. (My > > personal > > > > > > favourite) > > > > > > > > > > > > I think any shift in terminology should be driven by what best > > > > > communicates > > > > > > Airflow's role workflow automation, rather than just distancing > > > > ourselves > > > > > > from the mathematical definition. > > > > > > > > > > > > Curious to hear more thoughts! > > > > > > > > > > > > On Tue, Feb 18, 2025 at 10:40 AM Jarek Potiuk <ja...@potiuk.com> > > > > wrote: > > > > > > > > > > > > > Sorry for the few typos ... My slight dyslexia did not help and > > > > > > autocorrect > > > > > > > did not help either this time : dept -> debt, routed -> rooted > > and > > > a > > > > > few > > > > > > > other small typos :) . > > > > > > > > > > > > > > On Tue, Feb 18, 2025 at 4:28 PM Jarek Potiuk <ja...@potiuk.com > > > > > > wrote: > > > > > > > > > > > > > > > First of all - yes, agree with Lee and Ash, I think we do not > > > need > > > > to > > > > > > > > change the public interface, I'd also be -0 on that, but > docs, > > > > > internal > > > > > > > > strings, yes we could. > > > > > > > > > > > > > > > > But .... After thinking a bit and looking at the discussion > > here, > > > > > where > > > > > > > > people have concerns, I have an alternative take. And I know > > this > > > > > might > > > > > > > be > > > > > > > > controversial, and some people might find it confusing, but > why > > > > don't > > > > > > we > > > > > > > > come up with a new acronym for DAG, one that will be more > > airflow > > > > and > > > > > > > less > > > > > > > > "mathematics/algorithm". And I have, I think a good reason > for > > > it, > > > > > and > > > > > > I > > > > > > > > even have a proposal. Bear with me, > > > > > > > > > > > > > > > > DAG => *Data Airflow Graph* > > > > > > > > > > > > > > > > As many of us mentioned, DAG and Airflow are almost 100% 1-1 > > now > > > > > (with > > > > > > > > exception of some niche products - you know which one I speak > > > > about - > > > > > > > which > > > > > > > > are unfortunate enough to use the "Dag" in their name - I > will > > > > > explain > > > > > > > why > > > > > > > > this is unfortunate, IMHO). > > > > > > > > > > > > > > > > We are lucky to have Airflow for "A" in the acronym and we > can > > > make > > > > > it > > > > > > > > even more "synonymous" -> Airflow <> DAG with (A) being > > Airflow. > > > > > > > > > > > > > > > > Now, why I think "Directed Acyclic Graph" is unfortunate. > > > > > > > > > > > > > > > > Not only because "Directed" and "Acyclic" are mostly not in > the > > > > > > > vocabulary > > > > > > > > of most of our users - as Ryan, starting this thread > explained > > > > yes, I > > > > > > > agree > > > > > > > > with it, but also I think there is a more important reason - > I > > > > > simply > > > > > > > > believe in the future our DAGs might not (and likely will > not) > > be > > > > > > > "acyclic". > > > > > > > > > > > > > > > > I know for some it might be herezy and we have not discussed > > any > > > of > > > > > > that > > > > > > > > yet, but more and more I keep on hearing that Machine > Learning > > > > > > Workflows > > > > > > > of > > > > > > > > the future are often somewhat cyclic. In our case there is > one > > > type > > > > > of > > > > > > > > repeat-in-cycle behaviour - we have "task retry on failure" - > > but > > > > > this > > > > > > is > > > > > > > > only on failure and it basically mens "idempotent" retry - we > > > want > > > > to > > > > > > > redo > > > > > > > > something and get single result out of that and basically > > discard > > > > the > > > > > > > > previous result. But machine learning/AI workflows are > > > different. I > > > > > > had a > > > > > > > > lot of conversations recently (I even co-organized "AI > > low-level > > > > > > > > engineering and hacking" devroom to learn more about those) - > > > and I > > > > > > keep > > > > > > > on > > > > > > > > hearing that "Repeat a sequence of things several times and > > > refine" > > > > > is > > > > > > > > precisely what AI / ML workflows will need - especially with > > > > Agentic > > > > > > > > workflows - often we have cooperating agents which **could** > be > > > > part > > > > > of > > > > > > > the > > > > > > > > same DAG, but currently in Airflow where our DAGs are > > "Acyclic" > > > we > > > > > can > > > > > > > > only express "repeat several time and refine" as a single > task > > > that > > > > > > will > > > > > > > do > > > > > > > > such refining. If we would like to split such an "agent" > doing > > > > their > > > > > > own > > > > > > > > work in "cycles" and communicating with other "agents" doing > > the > > > > > same - > > > > > > > > represented as a multi-task-entity, we have no way to express > > it > > > > > using > > > > > > > our > > > > > > > > current "acyclic" semantics. > > > > > > > > > > > > > > > > But - other than our implementation makes that assumption > > > > currently - > > > > > > > > there is nothing fundamental with Airflow being able to run > > such > > > > > > > "partially > > > > > > > > cyclic workflows". We have all the pieces, we would have to > > > change > > > > > > > > dependency calculation to allow for cycles, update UI to > handle > > > > > cycles > > > > > > > > representation - and we could in theory quite seamlessly, > > support > > > > > > > "cyclic" > > > > > > > > workflows. > > > > > > > > > > > > > > > > I know, I know, that's the first time we hear it and that's > > huge > > > > > > change - > > > > > > > > of course, I do not want to start discussing any details of > it, > > > not > > > > > > > before > > > > > > > > Airflow 3.0 is out - but I think we should be open that at > some > > > > point > > > > > > of > > > > > > > > time the "Acyclic" property is going to go away, And my > > personal > > > > > > > intuition > > > > > > > > tells me that a year from now we will have acyclic workflows > > > > > (assuming > > > > > > > that > > > > > > > > all the foundational work we do on Airflow 3 will really help > > in > > > > > > > decreasing > > > > > > > > all the technical dept, and will open our minds to new > ideas). > > > > > > > > > > > > > > > > That's why I think - the sooner we get rid of "Directed > > Acyclic" > > > - > > > > > the > > > > > > > > better. that will - even for us, maintainer free us from > being > > > > routed > > > > > > in > > > > > > > > the acyclic nature of Airflow. > > > > > > > > > > > > > > > > How about we change to "Data Airflow Graph" (or any other > good > > > > > acronym > > > > > > > > which has (A)irflow instead of (A)cyclic). The name is just a > > > > > proposal. > > > > > > > > > > > > > > > > Now.. I go hide somewhere as I believe that might cause a > > > > "flamewar" > > > > > > :), > > > > > > > > unless on request of Ryan you already sheathed your swords in > > the > > > > > > > > discussion (which seems everyone did). > > > > > > > > > > > > > > > > But honestly - I would really love what you think of "let's > > just > > > > > break > > > > > > > > with Acyclic NOW" and shape our future by using a different > > > > acronym. > > > > > > > > > > > > > > > > J. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Feb 18, 2025 at 10:54 AM Ephraim Anierobi < > > > > > > > > ephraimanier...@gmail.com> wrote: > > > > > > > > > > > > > > > >> When I started contributing to Airflow, I had to read up on > > DAG, > > > > and > > > > > > > now I > > > > > > > >> know what it means. I wonder if we are about to have users > who > > > use > > > > > the > > > > > > > >> term > > > > > > > >> Dag without knowing that there's a DAG. > > > > > > > >> > > > > > > > >> Slightly concerned. > > > > > > > >> > > > > > > > >> -ephraim > > > > > > > >> > > > > > > > >> On Tue, 18 Feb 2025 at 10:26, Wei Lee <weilee...@gmail.com> > > > > wrote: > > > > > > > >> > > > > > > > >> > For user-facing things (e.g., `from airflow import DAG`), > > I’m > > > > more > > > > > > > like > > > > > > > >> > -0. But for documentation, docstring, internal things and > > > etc., > > > > we > > > > > > > >> probably > > > > > > > >> > could still change most of them? > > > > > > > >> > > > > > > > > >> > Best, > > > > > > > >> > Wei > > > > > > > >> > > > > > > > > >> > > On Feb 18, 2025, at 5:10 PM, Ash Berlin-Taylor < > > > > a...@apache.org> > > > > > > > >> wrote: > > > > > > > >> > > > > > > > > > >> > > I also don’t personally think it’s worth the pain (not > to > > > > > mention > > > > > > > >> > backcompat workaround) to rename DAG to Dag, so I’d be > -0.5 > > on > > > > > that. > > > > > > > >> > > > > > > > > > >> > > -ash > > > > > > > >> > > > > > > > > > >> > >> On 18 Feb 2025, at 04:40, Daniel Imberman < > > > > > > > daniel.imber...@gmail.com > > > > > > > >> > > > > > > > > >> > wrote: > > > > > > > >> > >> > > > > > > > >> > >> I think my biggest concern is a marketing one and not a > > > > > technical > > > > > > > >> one. > > > > > > > >> > >> > > > > > > > >> > >> As has been mentioned on the thread the terms airflow > and > > > dag > > > > > are > > > > > > > >> kind > > > > > > > >> > of > > > > > > > >> > >> synonymous and I certainly don’t want to give the > > > impression > > > > > that > > > > > > > we > > > > > > > >> are > > > > > > > >> > >> breaking more than we are breaking. > > > > > > > >> > >> > > > > > > > >> > >> I wouldn’t die on this hill, but I’m slightly > concerned. > > > > > > > >> > >> > > > > > > > >> > >> On Mon, Feb 17, 2025 at 6:00 PM Wei Lee < > > wei...@apache.org > > > > > > > > > > wrote: > > > > > > > >> > >> > > > > > > > >> > >>> I’m not sure about adding ruff rules here 🤔 I think > > ruff > > > > > rules > > > > > > > are > > > > > > > >> > best > > > > > > > >> > >>> suited for user-facing things but not the airflow code > > > base > > > > > > > itself. > > > > > > > >> If > > > > > > > >> > what > > > > > > > >> > >>> we mean is adding a rule to avoid users using "DAG" > > > *after* > > > > we > > > > > > > >> rename > > > > > > > >> > it, > > > > > > > >> > >>> it's definitely a +1000. > > > > > > > >> > >>> > > > > > > > >> > >>> I just created GitHub issues for this removing "DAG" > > > > > > > >> > >>> > > > > > > > >> > >>> https://github.com/apache/airflow/issues/46842 > > > > > > > >> > >>> https://github.com/apache/airflow/issues/46843 > > > > > > > >> > >>> > > > > > > > >> > >>> One thing I'm not sure about is whether we want to get > > rid > > > > of > > > > > > > `from > > > > > > > >> > >>> airflow import DAG` as well. 🤔 > > > > > > > >> > >>> > > > > > > > >> > >>> Best, > > > > > > > >> > >>> Wei > > > > > > > >> > >>> > > > > > > > >> > >>> On 2025/02/17 19:50:57 Jarek Potiuk wrote: > > > > > > > >> > >>>> Hard to say until it's looked at :) > > > > > > > >> > >>>> > > > > > > > >> > >>>> On Mon, Feb 17, 2025 at 8:45 PM Aritra Basu < > > > > > > > >> aritrabasu1...@gmail.com > > > > > > > >> > > > > > > > > > >> > >>>> wrote: > > > > > > > >> > >>>> > > > > > > > >> > >>>>> I can take it up, it's mostly just a doc update > right? > > > Or > > > > > are > > > > > > we > > > > > > > >> > doing > > > > > > > >> > >>> code > > > > > > > >> > >>>>> files replacement too? > > > > > > > >> > >>>>> -- > > > > > > > >> > >>>>> Regards, > > > > > > > >> > >>>>> Aritra Basu > > > > > > > >> > >>>>> > > > > > > > >> > >>>>> On Tue, 18 Feb 2025, 12:55 am Jarek Potiuk, < > > > > > ja...@potiuk.com > > > > > > > > > > > > > > >> > wrote: > > > > > > > >> > >>>>> > > > > > > > >> > >>>>>> Sounds like another +1000 files big PR is coming > :) ? > > > Any > > > > > > > >> volunteers > > > > > > > >> > >>> to > > > > > > > >> > >>>>>> make it ? It's fun. > > > > > > > >> > >>>>>> > > > > > > > >> > >>>>>> On Mon, Feb 17, 2025 at 6:52 PM Omkar P < > > > > > > > droiddev5...@gmail.com> > > > > > > > >> > >>> wrote: > > > > > > > >> > >>>>>> > > > > > > > >> > >>>>>>> +1 for ruff rules :) > > > > > > > >> > >>>>>>> > > > > > > > >> > >>>>>>> Also would be nice to introduce 'Dag' to replace > > 'DAG' > > > > in > > > > > > the > > > > > > > >> > >>> Airflow > > > > > > > >> > >>>>>>> docs, in line with the new UI changes and to make > > the > > > > > > renaming > > > > > > > >> > >>>>>>> consistent across user-facing pages. > > > > > > > >> > >>>>>>> > > > > > > > >> > >>>>>>> Regards, > > > > > > > >> > >>>>>>> Omkar > > > > > > > >> > >>>>>>> > > > > > > > >> > >>>>>>> On Mon, Feb 17, 2025 at 8:12 AM Jarek Potiuk < > > > > > > > ja...@potiuk.com> > > > > > > > >> > >>> wrote: > > > > > > > >> > >>>>>>> > > > > > > > >> > >>>>>>>> +1 :) . Maybe we could add a ruff rule for that > :) > > > > > > > >> > >>>>>>>> > > > > > > > >> > >>>>>>>> On Mon, Feb 17, 2025 at 8:27 AM Wei Lee < > > > > > wei...@apache.org > > > > > > > > > > > > > > >> > >>> wrote: > > > > > > > >> > >>>>>>>> > > > > > > > >> > >>>>>>>>> It seems that our current conclusion is to use > > "dag" > > > > or > > > > > > > "Dag" > > > > > > > >> > >>>>> instead > > > > > > > >> > >>>>>>> of > > > > > > > >> > >>>>>>>>> "DAG" whenever possible. Should we replace all > > "DAG" > > > > in > > > > > > the > > > > > > > >> > >>>>> codebase > > > > > > > >> > >>>>>>> with > > > > > > > >> > >>>>>>>>> "dag" or "Dag"? If it's too late for that (which > > it > > > > > might > > > > > > be > > > > > > > >> > >>> 🤔), > > > > > > > >> > >>>>>>> should > > > > > > > >> > >>>>>>>> we > > > > > > > >> > >>>>>>>>> at least avoid introducing new "DAG" in the > > > following > > > > > PRs? > > > > > > > >> > >>>>>>>>> > > > > > > > >> > >>>>>>>>> Best, > > > > > > > >> > >>>>>>>>> Wei > > > > > > > >> > >>>>>>>>> > > > > > > > >> > >>>>>>>>> On 2024/10/23 17:16:55 Brent Bovenzi wrote: > > > > > > > >> > >>>>>>>>>> Here's a PR to use "dag" as a word in the new > 3.0 > > > UI: > > > > > > > >> > >>>>>>>>>> https://github.com/apache/airflow/pull/43325 > > > > > > > >> > >>>>>>>>>> Let me know if that's the direction we want to > > go. > > > > > > > >> > >>>>>>>>>> > > > > > > > >> > >>>>>>>>>> On Wed, Oct 23, 2024 at 11:06 AM Bishundeo, > > > Rajeshwar > > > > > > > >> > >>>>>>>>>> <rbish...@amazon.com.invalid> wrote: > > > > > > > >> > >>>>>>>>>> > > > > > > > >> > >>>>>>>>>>> I think Brent & Daniel summarized it best, > "dag" > > > is > > > > > > > >> > >>> synonymous > > > > > > > >> > >>>>>> with > > > > > > > >> > >>>>>>>>>>> workflows in Airflow through the way we talk > and > > > > > explain > > > > > > > >> > >>> what > > > > > > > >> > >>>>>>> Airflow > > > > > > > >> > >>>>>>>>> is > > > > > > > >> > >>>>>>>>>>> all about. Although folks would ask, I don’t > > ever > > > > use > > > > > > the > > > > > > > >> > >>>>>>>> mathematical > > > > > > > >> > >>>>>>>>>>> definition of DAG. > > > > > > > >> > >>>>>>>>>>> It will be challenging and possibly confusing > > for > > > > many > > > > > > > >> > >>> users > > > > > > > >> > >>>>>> making > > > > > > > >> > >>>>>>>>> such a > > > > > > > >> > >>>>>>>>>>> change - I would rather direct that energy to > > > > > appending > > > > > > > the > > > > > > > >> > >>>>>> Oxford > > > > > > > >> > >>>>>>>>>>> definition of "dag" to include a reference to > > > > > workflows > > > > > > in > > > > > > > >> > >>>>>> Airflow. > > > > > > > >> > >>>>>>>> __ > > > > > > > >> > >>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>> -- Rajesh > > > > > > > >> > >>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>> On 2024-10-23, 3:37 AM, "Jarek Potiuk" < > > > > > > ja...@potiuk.com > > > > > > > >> > >>>>>> <mailto: > > > > > > > >> > >>>>>>>>>>> ja...@potiuk.com>> wrote: > > > > > > > >> > >>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>> CAUTION: This email originated from outside of > > the > > > > > > > >> > >>>>> organization. > > > > > > > >> > >>>>>> Do > > > > > > > >> > >>>>>>>> not > > > > > > > >> > >>>>>>>>>>> click links or open attachments unless you can > > > > confirm > > > > > > the > > > > > > > >> > >>>>> sender > > > > > > > >> > >>>>>>> and > > > > > > > >> > >>>>>>>>> know > > > > > > > >> > >>>>>>>>>>> the content is safe. > > > > > > > >> > >>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>> AVERTISSEMENT: Ce courrier électronique > provient > > > > d’un > > > > > > > >> > >>>>> expéditeur > > > > > > > >> > >>>>>>>>> externe. > > > > > > > >> > >>>>>>>>>>> Ne cliquez sur aucun lien et n’ouvrez aucune > > pièce > > > > > > jointe > > > > > > > >> > >>> si > > > > > > > >> > >>>>> vous > > > > > > > >> > >>>>>>> ne > > > > > > > >> > >>>>>>>>> pouvez > > > > > > > >> > >>>>>>>>>>> pas confirmer l’identité de l’expéditeur et si > > > vous > > > > > > n’êtes > > > > > > > >> > >>> pas > > > > > > > >> > >>>>>>>> certain > > > > > > > >> > >>>>>>>>> que > > > > > > > >> > >>>>>>>>>>> le contenu ne présente aucun risque. > > > > > > > >> > >>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>> From Guido's post: > > > > > > > >> > >>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>> "Around this time the renaming seems to have > > been > > > > > > > renamed". > > > > > > > >> > >>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>> Naming is hard. > > > > > > > >> > >>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>> J. > > > > > > > >> > >>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>> On Wed, Oct 23, 2024 at 7:11 AM Omkar P < > > > > > > > >> > >>>>> droiddev5...@gmail.com > > > > > > > >> > >>>>>>>>> <mailto: > > > > > > > >> > >>>>>>>>>>> droiddev5...@gmail.com>> wrote: > > > > > > > >> > >>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>>> Yes, completely agree with above comments, > dag > > is > > > > an > > > > > > > >> > >>> Airflow > > > > > > > >> > >>>>>> term > > > > > > > >> > >>>>>>>> now > > > > > > > >> > >>>>>>>>>>>> rather > > > > > > > >> > >>>>>>>>>>>> than just a "directed acyclic graph". Believe > > it > > > or > > > > > > not, > > > > > > > >> > >>> I've > > > > > > > >> > >>>>>>>> worked > > > > > > > >> > >>>>>>>>> with > > > > > > > >> > >>>>>>>>>>>> work > > > > > > > >> > >>>>>>>>>>>> folks who've been using dags in Airflow for > > years > > > > now > > > > > > > >> > >>> and are > > > > > > > >> > >>>>>> pro > > > > > > > >> > >>>>>>>>> devs > > > > > > > >> > >>>>>>>>>>> but > > > > > > > >> > >>>>>>>>>>>> have trouble remembering the full form of a > > DAG! > > > > For > > > > > > > >> > >>> them, > > > > > > > >> > >>>>> dag > > > > > > > >> > >>>>>> = > > > > > > > >> > >>>>>>>>> Airflow. > > > > > > > >> > >>>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>>> workflow, pipeline, flow are surely better > > terms > > > > but > > > > > > > >> > >>> will be > > > > > > > >> > >>>>> a > > > > > > > >> > >>>>>>>> major > > > > > > > >> > >>>>>>>>>>>> behavior > > > > > > > >> > >>>>>>>>>>>> change for the developer community, so we'll > > > need a > > > > > > solid > > > > > > > >> > >>>>> plan > > > > > > > >> > >>>>>> on > > > > > > > >> > >>>>>>>>> how to > > > > > > > >> > >>>>>>>>>>>> introduce it, when we do. > > > > > > > >> > >>>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>>> While we discuss this, I'd like to share > about > > > the > > > > > > Great > > > > > > > >> > >>>>>> Renaming > > > > > > > >> > >>>>>>>> in > > > > > > > >> > >>>>>>>>> core > > > > > > > >> > >>>>>>>>>>>> Python started by Guido back in 2009. We > could > > > > > probably > > > > > > > >> > >>> get > > > > > > > >> > >>>>>> some > > > > > > > >> > >>>>>>>>>>> learnings > > > > > > > >> > >>>>>>>>>>>> from there? Who knows! > > > > > > > >> > >>>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>>> Guido's blog (2009): > > > > > > > >> > >>>>>>>>>>>> > > > > > > > >> > >>>>>>>>> > > > > > > > >> > >>>>>>> > > > > > > > >> > >>>>> > > > > > > > >> > >>> > > > > > > > >> > > > > > > > > >> > > > > > > > > > > > > > > > > https://python-history.blogspot.com/2009/03/great-or-grand-renaming.html > > > > > > > >> > >>>>>>>>>>> < > > > > > > > >> > >>>>>>>>> > > > > > > > >> > >>>>>>> > > > > > > > >> > >>>>> > > > > > > > >> > >>> > > > > > > > >> > > > > > > > > >> > > > > > > > > > > > > > > > > https://python-history.blogspot.com/2009/03/great-or-grand-renaming.html > > > > > > > >> > >>>>>>>>> > > > > > > > >> > >>>>>>>>>>>> Follow-up discussion (2024): > > > > > > > >> > >>>>>>>>>>>> > > > > > > > >> > >>>>>> > > > > > > > > https://discuss.python.org/t/finishing-the-great-renaming/54082 > > > > > > > >> > >>>>>>> < > > > > > > > >> > >>>>>>>>>>> > > > > > > > >> > >>>>> > > > > > > https://discuss.python.org/t/finishing-the-great-renaming/54082 > > > > > > > > > > > > > > > >> > >>>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>>> Cheers, > > > > > > > >> > >>>>>>>>>>>> Omkar > > > > > > > >> > >>>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>>> On Wed, Oct 23, 2024 at 7:10 AM Wei Lee < > > > > > > > >> > >>> weilee...@gmail.com > > > > > > > >> > >>>>>>>>> <mailto: > > > > > > > >> > >>>>>>>>>>> weilee...@gmail.com>> wrote: > > > > > > > >> > >>>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>>>> I think we should probably just accept it as > > an > > > > > > airflow > > > > > > > >> > >>>>> term. > > > > > > > >> > >>>>>>> At > > > > > > > >> > >>>>>>>>> least, > > > > > > > >> > >>>>>>>>>>>>> that’s how I understood it when I first used > > > > > Airflow. > > > > > > I > > > > > > > >> > >>>>> feel > > > > > > > >> > >>>>>>>>> renaming > > > > > > > >> > >>>>>>>>>>> it > > > > > > > >> > >>>>>>>>>>>> at > > > > > > > >> > >>>>>>>>>>>>> this stage would require considerable effort > > > from > > > > > > > >> > >>>>> maintainers > > > > > > > >> > >>>>>>> and > > > > > > > >> > >>>>>>>>>>>> existing > > > > > > > >> > >>>>>>>>>>>>> users without providing equivalent benefits. > > > > > > > >> > >>>>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>>>> Best, > > > > > > > >> > >>>>>>>>>>>>> Wei > > > > > > > >> > >>>>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>>>>> On Oct 23, 2024, at 8:01 AM, Kaxil Naik < > > > > > > > >> > >>>>>> kaxiln...@gmail.com > > > > > > > >> > >>>>>>>>>>> <mailto:kaxiln...@gmail.com>> wrote: > > > > > > > >> > >>>>>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>>>>> Same agreed with Brent & Daniel -- maybe we > > > > > re-kindle > > > > > > > >> > >>>>> this > > > > > > > >> > >>>>>>>>> discussion > > > > > > > >> > >>>>>>>>>>>> for > > > > > > > >> > >>>>>>>>>>>>>> Airflow 4 :) -- but right now it will cause > > too > > > > > much > > > > > > > >> > >>>>>>> disruption > > > > > > > >> > >>>>>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>>>>> On Tue, 22 Oct 2024 at 21:27, Constance > > > Martineau > > > > > > > >> > >>>>>>>>>>>>>> <consta...@astronomer.io.inva <mailto: > > > > > > > >> > >>>>>>>>> consta...@astronomer.io.inva>lid> > > > > > > > >> > >>>>>>>>>>> wrote: > > > > > > > >> > >>>>>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>>>>>> In my experience, when you ask those with > > > > Airflow > > > > > > > >> > >>>>>> experience > > > > > > > >> > >>>>>>>>> what a > > > > > > > >> > >>>>>>>>>>>> dag > > > > > > > >> > >>>>>>>>>>>>> is, > > > > > > > >> > >>>>>>>>>>>>>>> they'll start talking about workflow > > > attributes > > > > - > > > > > > > >> > >>> stuff > > > > > > > >> > >>>>>> like > > > > > > > >> > >>>>>>>>> dags > > > > > > > >> > >>>>>>>>>>>> being > > > > > > > >> > >>>>>>>>>>>>> a > > > > > > > >> > >>>>>>>>>>>>>>> series of steps or tasks with owners. The > > > > > structure > > > > > > > >> > >>>>>> doesn't > > > > > > > >> > >>>>>>>>> come up. > > > > > > > >> > >>>>>>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>>>>>> Echo-ing others, at this point, my vote is > > to > > > > > > > >> > >>> embrace > > > > > > > >> > >>>>> the > > > > > > > >> > >>>>>>> name > > > > > > > >> > >>>>>>>>> and > > > > > > > >> > >>>>>>>>>>>>>>> de-emphasize the mathematical structure > > > aspect. > > > > > > > >> > >>>>>>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>>>>>> On Tue, Oct 22, 2024 at 3:47 PM Vikram > Koka > > > > > > > >> > >>>>>>>>>>>>> <vik...@astronomer.io.inva <mailto: > > > > > > > >> > >>>>> vik...@astronomer.io.inva > > > > > > > >> > >>>>>>>>> lid> > > > > > > > >> > >>>>>>>>>>>>>>> wrote: > > > > > > > >> > >>>>>>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>>>>>>> It's an interesting discussion and I > > remember > > > > > > > >> > >>>>> struggling > > > > > > > >> > >>>>>>> with > > > > > > > >> > >>>>>>>>> this > > > > > > > >> > >>>>>>>>>>>>> when I > > > > > > > >> > >>>>>>>>>>>>>>>> started working with Airflow. > > > > > > > >> > >>>>>>>>>>>>>>>> But, I also agree with the viewpoint of > it > > > > being > > > > > an > > > > > > > >> > >>>>>>>> established > > > > > > > >> > >>>>>>>>>>>> concept > > > > > > > >> > >>>>>>>>>>>>>>> now > > > > > > > >> > >>>>>>>>>>>>>>>> regardless of the origin. > > > > > > > >> > >>>>>>>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>>>>>>> I am personally leaning towards the > > > perspective > > > > > > > >> > >>> best > > > > > > > >> > >>>>>>>> expressed > > > > > > > >> > >>>>>>>>> by > > > > > > > >> > >>>>>>>>>>>>> Daniel > > > > > > > >> > >>>>>>>>>>>>>>>> Standish and Brent of using Dag as a > word, > > > > rather > > > > > > > >> > >>> than > > > > > > > >> > >>>>>> the > > > > > > > >> > >>>>>>>>> computer > > > > > > > >> > >>>>>>>>>>>>>>> science > > > > > > > >> > >>>>>>>>>>>>>>>> concept. > > > > > > > >> > >>>>>>>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>>>>>>> Best regards, > > > > > > > >> > >>>>>>>>>>>>>>>> Vikram > > > > > > > >> > >>>>>>>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>>>>>>> On Tue, Oct 22, 2024 at 9:46 AM Oliveira, > > > Niko > > > > > > > >> > >>>>>>>>>>>>>>> <oniko...@amazon.com.inva <mailto: > > > > > > > >> > >>>>>> oniko...@amazon.com.inva > > > > > > > >> > >>>>>>>>> lid > > > > > > > >> > >>>>>>>>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>>>>>>> wrote: > > > > > > > >> > >>>>>>>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>>>>>>>> I agree with the general sentiment of: > > > You're > > > > > > > >> > >>> right > > > > > > > >> > >>>>>> Ryan, > > > > > > > >> > >>>>>>>> DAG > > > > > > > >> > >>>>>>>>>>> isn't > > > > > > > >> > >>>>>>>>>>>>>>> great > > > > > > > >> > >>>>>>>>>>>>>>>>> and I'd rather workflow, but changing it > > > will > > > > > > > >> > >>> cause > > > > > > > >> > >>>>> much > > > > > > > >> > >>>>>>>> more > > > > > > > >> > >>>>>>>>>>>> wreckage > > > > > > > >> > >>>>>>>>>>>>>>>> than > > > > > > > >> > >>>>>>>>>>>>>>>>> it solves. > > > > > > > >> > >>>>>>>>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>>>>>>>> Also agree with the idea to just move > away > > > > from > > > > > > > >> > >>>>> defining > > > > > > > >> > >>>>>>>> DAG. > > > > > > > >> > >>>>>>>>> I > > > > > > > >> > >>>>>>>>>>>> think > > > > > > > >> > >>>>>>>>>>>>>>>>> we've been naturally doing that as a > > > community > > > > > > > >> > >>> for a > > > > > > > >> > >>>>>> while > > > > > > > >> > >>>>>>>> now > > > > > > > >> > >>>>>>>>>>>> anyway, > > > > > > > >> > >>>>>>>>>>>>>>> so > > > > > > > >> > >>>>>>>>>>>>>>>>> that feels like a natural step. > > > > > > > >> > >>>>>>>>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>>>>>>>> Cheers, > > > > > > > >> > >>>>>>>>>>>>>>>>> Niko > > > > > > > >> > >>>>>>>>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>>>>>>>> ________________________________ > > > > > > > >> > >>>>>>>>>>>>>>>>> From: Ash Berlin-Taylor <a...@apache.org > > > > > <mailto: > > > > > > > >> > >>>>>>>>> a...@apache.org>> > > > > > > > >> > >>>>>>>>>>>>>>>>> Sent: Tuesday, October 22, 2024 9:06:39 > AM > > > > > > > >> > >>>>>>>>>>>>>>>>> To: dev@airflow.apache.org <mailto: > > > > > > > >> > >>>>>> dev@airflow.apache.org > > > > > > > >> > >>>>>>>> > > > > > > > >> > >>>>>>>>>>>>>>>>> Subject: RE: [EXT] Airflow should > > deprecate > > > > the > > > > > > > >> > >>> term > > > > > > > >> > >>>>>> "DAG" > > > > > > > >> > >>>>>>>>> for end > > > > > > > >> > >>>>>>>>>>>>>>> users > > > > > > > >> > >>>>>>>>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>>>>>>>> CAUTION: This email originated from > > outside > > > of > > > > > the > > > > > > > >> > >>>>>>>>> organization. > > > > > > > >> > >>>>>>>>>>> Do > > > > > > > >> > >>>>>>>>>>>>> not > > > > > > > >> > >>>>>>>>>>>>>>>>> click links or open attachments unless > you > > > can > > > > > > > >> > >>> confirm > > > > > > > >> > >>>>>> the > > > > > > > >> > >>>>>>>>> sender > > > > > > > >> > >>>>>>>>>>>> and > > > > > > > >> > >>>>>>>>>>>>>>>> know > > > > > > > >> > >>>>>>>>>>>>>>>>> the content is safe. > > > > > > > >> > >>>>>>>>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>>>>>>>> AVERTISSEMENT: Ce courrier électronique > > > > provient > > > > > > > >> > >>> d’un > > > > > > > >> > >>>>>>>>> expéditeur > > > > > > > >> > >>>>>>>>>>>>>>> externe. > > > > > > > >> > >>>>>>>>>>>>>>>>> Ne cliquez sur aucun lien et n’ouvrez > > aucune > > > > > pièce > > > > > > > >> > >>>>>> jointe > > > > > > > >> > >>>>>>> si > > > > > > > >> > >>>>>>>>> vous > > > > > > > >> > >>>>>>>>>>> ne > > > > > > > >> > >>>>>>>>>>>>>>>> pouvez > > > > > > > >> > >>>>>>>>>>>>>>>>> pas confirmer l’identité de l’expéditeur > > et > > > si > > > > > > > >> > >>> vous > > > > > > > >> > >>>>>> n’êtes > > > > > > > >> > >>>>>>>> pas > > > > > > > >> > >>>>>>>>>>>> certain > > > > > > > >> > >>>>>>>>>>>>>>>> que > > > > > > > >> > >>>>>>>>>>>>>>>>> le contenu ne présente aucun risque. > > > > > > > >> > >>>>>>>>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>>>>>>>> Best argument in favour of keeping > “dags” > > > as a > > > > > > > >> > >>> term — > > > > > > > >> > >>>>>>>> getting > > > > > > > >> > >>>>>>>>> to > > > > > > > >> > >>>>>>>>>>>>> re-use > > > > > > > >> > >>>>>>>>>>>>>>>>> puns like > > https://i.imgflip.com/1xhtwh.jpg > > > < > > > > > > > >> > >>>>>>>>>>> https://i.imgflip.com/1xhtwh.jpg> > > > > > > > >> > >>>>>>>>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>>>>>>>> In all seriousness: I don’t mind either > > way, > > > > > both > > > > > > > >> > >>>>> sides > > > > > > > >> > >>>>>>> have > > > > > > > >> > >>>>>>>>> good > > > > > > > >> > >>>>>>>>>>>>>>> reasons > > > > > > > >> > >>>>>>>>>>>>>>>>> presented. > > > > > > > >> > >>>>>>>>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>>>>>>>> -a > > > > > > > >> > >>>>>>>>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>>>>>>>>> On 22 Oct 2024, at 17:03, Daniel > Standish > > > > > > > >> > >>>>>>>>>>>>>>>>> <daniel.stand...@astronomer.io.inva > > > <mailto: > > > > > > > >> > >>>>>>>>>>> daniel.stand...@astronomer.io.inva>LID> > wrote: > > > > > > > >> > >>>>>>>>>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>>>>>>>>> Yeah just say, when asked where the > name > > > > comes > > > > > > > >> > >>> from, > > > > > > > >> > >>>>>>> "well, > > > > > > > >> > >>>>>>>>> no > > > > > > > >> > >>>>>>>>>>> one > > > > > > > >> > >>>>>>>>>>>>>>>>> actually > > > > > > > >> > >>>>>>>>>>>>>>>>>> knows but..." and then make something > up. > > > > > > > >> > >>>>>>>>>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>>>>>>>>> On Tue, Oct 22, 2024 at 8:31 AM Jarek > > > Potiuk > > > > < > > > > > > > >> > >>>>>>>>> ja...@potiuk.com > > > > > > > >> > >>>>>>>>>>> <mailto:ja...@potiuk.com>> > > > > > > > >> > >>>>>>>>>>>>>>> wrote: > > > > > > > >> > >>>>>>>>>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>>>>>>>>>> Just to clarify - "directed acyclic > > graph" > > > > is > > > > > > > >> > >>> the > > > > > > > >> > >>>>>>>>>>> tongue-twister, > > > > > > > >> > >>>>>>>>>>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>>>>>>>>>> On Tue, Oct 22, 2024 at 5:29 PM Jarek > > > > Potiuk < > > > > > > > >> > >>>>>>>>> ja...@potiuk.com > > > > > > > >> > >>>>>>>>>>> <mailto:ja...@potiuk.com>> > > > > > > > >> > >>>>>>>>>>>>>>>> wrote: > > > > > > > >> > >>>>>>>>>>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>>>>>>>>>>> I like what both Daniel and Brent > > wrote. > > > I > > > > > > > >> > >>> would > > > > > > > >> > >>>>> very > > > > > > > >> > >>>>>>>> much > > > > > > > >> > >>>>>>>>> want > > > > > > > >> > >>>>>>>>>>>> to > > > > > > > >> > >>>>>>>>>>>>>>> be > > > > > > > >> > >>>>>>>>>>>>>>>>>>> able > > > > > > > >> > >>>>>>>>>>>>>>>>>>>> to say just "dag" without explaining > it > > > > > > > >> > >>> further. > > > > > > > >> > >>>>>>>>>>>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>>>>>>>>>>> For me every time I explain "DAG" at > a > > > talk > > > > > > > >> > >>> it's a > > > > > > > >> > >>>>>>>>>>>> tongue-twister, > > > > > > > >> > >>>>>>>>>>>>>>>> and > > > > > > > >> > >>>>>>>>>>>>>>>>> I > > > > > > > >> > >>>>>>>>>>>>>>>>>>>> almost stutter on trying to recall > how > > to > > > > > > > >> > >>> pronounce > > > > > > > >> > >>>>>> it > > > > > > > >> > >>>>>>>>>>> properly. > > > > > > > >> > >>>>>>>>>>>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>>>>>>>>>>> J. > > > > > > > >> > >>>>>>>>>>>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>>>>>>>>>>> On Tue, Oct 22, 2024 at 5:27 PM Brent > > > > Bovenzi > > > > > > > >> > >>>>>>>>>>>>>>>>>>> <br...@astronomer.io.inva <mailto: > > > > > > > >> > >>>>>>>> br...@astronomer.io.inva > > > > > > > >> > >>>>>>>>>> lid> > > > > > > > >> > >>>>>>>>>>>>>>>>>>>> wrote: > > > > > > > >> > >>>>>>>>>>>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>> I remember we explored renaming > "DAG" > > > when > > > > > > > >> > >>>>> starting > > > > > > > >> > >>>>>> on > > > > > > > >> > >>>>>>>>> AIP-38 > > > > > > > >> > >>>>>>>>>>> to > > > > > > > >> > >>>>>>>>>>>>>>>>>>> modernize > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>> the UI. Both "pipeline" or > "workflow" > > > are > > > > > more > > > > > > > >> > >>>>>>>>> descriptive of > > > > > > > >> > >>>>>>>>>>>> what > > > > > > > >> > >>>>>>>>>>>>>>>> one > > > > > > > >> > >>>>>>>>>>>>>>>>>>> is > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>> actually doing while Directed > Acyclic > > > > Graph > > > > > > > >> > >>> is an > > > > > > > >> > >>>>>>>>>>> implementation > > > > > > > >> > >>>>>>>>>>>>>>>>> detail. > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>> But I agree with Daniel Standish, at > > > this > > > > > > > >> > >>> point > > > > > > > >> > >>>>>> "DAG" > > > > > > > >> > >>>>>>>> has > > > > > > > >> > >>>>>>>>>>> become > > > > > > > >> > >>>>>>>>>>>>>>>> "dag" > > > > > > > >> > >>>>>>>>>>>>>>>>>>> , a > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>> word in its own right. > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>> Examples for "dag" are abound in > > > community > > > > > > > >> > >>>>>> discussion, > > > > > > > >> > >>>>>>>>> Airflow > > > > > > > >> > >>>>>>>>>>>>>>>> Summit > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>> talks, documentation and even in the > > UI. > > > > > Let's > > > > > > > >> > >>>>>> embrace > > > > > > > >> > >>>>>>>>> "dag". > > > > > > > >> > >>>>>>>>>>> A > > > > > > > >> > >>>>>>>>>>>>>>> user > > > > > > > >> > >>>>>>>>>>>>>>>>>>> just > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>> needs to learn one new word vs the > > > > technical > > > > > > > >> > >>>>> concept > > > > > > > >> > >>>>>>>>> behind > > > > > > > >> > >>>>>>>>>>> that > > > > > > > >> > >>>>>>>>>>>>>>>>> word. I > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>> think that is much less effort than > > > > > > > >> > >>> refactoring so > > > > > > > >> > >>>>>>> much > > > > > > > >> > >>>>>>>>> code, > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>> documentation, blog posts, stack > > > overflow > > > > > > > >> > >>>>> questions, > > > > > > > >> > >>>>>>>> etc. > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>> On Tue, Oct 22, 2024 at 10:51 AM > > Daniel > > > > > > > >> > >>> Standish > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>> <daniel.stand...@astronomer.io.inva > > > > > <mailto: > > > > > > > >> > >>>>>>>>>>> daniel.stand...@astronomer.io.inva>lid> > wrote: > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>>> I am skeptical. Seems like > > introducing > > > a > > > > > lot > > > > > > > >> > >>> of > > > > > > > >> > >>>>>> pain > > > > > > > >> > >>>>>>>> for > > > > > > > >> > >>>>>>>>>>>>>>>>> questionable > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>>> benefit. But, I am def sympathetic > to > > > the > > > > > > > >> > >>> idea. I > > > > > > > >> > >>>>>>> agree > > > > > > > >> > >>>>>>>>> the > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>> association > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>>> with "directed acyclic graph" is > not > > > > > helpful. > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>>> And along those lines, I offer here > > > some > > > > > less > > > > > > > >> > >>>>>>> invasive > > > > > > > >> > >>>>>>>>>>>>>>> mitigations. > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>>> One thing we can do no matter what > is > > > to > > > > > > > >> > >>>>>> de-emphasize > > > > > > > >> > >>>>>>>> the > > > > > > > >> > >>>>>>>>>>> math > > > > > > > >> > >>>>>>>>>>>>>>> nerd > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>> origins > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>>> of the name. That is to say, in > docs > > / > > > > > > > >> > >>> website / > > > > > > > >> > >>>>>> etc, > > > > > > > >> > >>>>>>>>> *never > > > > > > > >> > >>>>>>>>>>>>>>>> define* > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>>> airflow's "dag" concept as a > directed > > > > > acyclic > > > > > > > >> > >>>>>> graph. > > > > > > > >> > >>>>>>>>> Always > > > > > > > >> > >>>>>>>>>>>>>>> define > > > > > > > >> > >>>>>>>>>>>>>>>>> it > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>> as a > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>>> pipeline, collection of tasks, > > workflow > > > > > etc. > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>>> The "directed acyclic graph" part > of > > it > > > > is > > > > > > > >> > >>> like a > > > > > > > >> > >>>>>>>>> historical > > > > > > > >> > >>>>>>>>>>>>>>>>> footnote, > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>> and > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>>> we could make one mention of it > > > somewhere > > > > > > > >> > >>> hidden. > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>>> We could also start using lowercase > > in > > > > the > > > > > > > >> > >>> docs > > > > > > > >> > >>>>> in > > > > > > > >> > >>>>>>>>> general > > > > > > > >> > >>>>>>>>>>> e.g. > > > > > > > >> > >>>>>>>>>>>>>>>>>>> writing > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>>> "dag" / "dags" instead of writing > > > "DAG" / > > > > > > > >> > >>> "DAGs" > > > > > > > >> > >>>>>> etc. > > > > > > > >> > >>>>>>>> The > > > > > > > >> > >>>>>>>>>>>> upper > > > > > > > >> > >>>>>>>>>>>>>>>> case > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>> part > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>>> of it makes it look like an > acronym; > > > but > > > > > > > >> > >>> "dag" in > > > > > > > >> > >>>>>>>>> airlfow is > > > > > > > >> > >>>>>>>>>>>> just > > > > > > > >> > >>>>>>>>>>>>>>>> an > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>>> airflow concept and the association > > > with > > > > > > > >> > >>> "DAGs" > > > > > > > >> > >>>>> is > > > > > > > >> > >>>>>>> not > > > > > > > >> > >>>>>>>>> really > > > > > > > >> > >>>>>>>>>>>>>>>>>>> unhelpful. > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>>> In other words embrace that "dag" > in > > > > > airflow > > > > > > > >> > >>> is > > > > > > > >> > >>>>> its > > > > > > > >> > >>>>>>> own > > > > > > > >> > >>>>>>>>>>> thing, > > > > > > > >> > >>>>>>>>>>>> is > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>>> *not* strictly > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>>> speaking a directed acyclic graph > > > (which > > > > > > > >> > >>> nobody > > > > > > > >> > >>>>>> knows > > > > > > > >> > >>>>>>>>> about > > > > > > > >> > >>>>>>>>>>>>>>>> anyway), > > > > > > > >> > >>>>>>>>>>>>>>>>>>> and > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>>> tell them what it is in simple > terms > > > that > > > > > > > >> > >>> normal > > > > > > > >> > >>>>>>> people > > > > > > > >> > >>>>>>>>>>>>>>> understand. > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>>> On Tue, Oct 22, 2024 at 7:27 AM > Jarek > > > > > Potiuk > > > > > > > >> > >>> < > > > > > > > >> > >>>>>>>>>>> ja...@potiuk.com <mailto:ja...@potiuk.com> > > > > > > > >> > >>>>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>>>>>>>>>> wrote: > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>>>> DAG is so embedded into what we do > > > that > > > > it > > > > > > > >> > >>> will > > > > > > > >> > >>>>> be > > > > > > > >> > >>>>>>>>> extremely > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>> difficult to > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>>>> get rid of it completely. Also I > > think > > > > it > > > > > > > >> > >>> will > > > > > > > >> > >>>>>> make > > > > > > > >> > >>>>>>> a > > > > > > > >> > >>>>>>>>> lot of > > > > > > > >> > >>>>>>>>>>>>>>>>>>> "google" > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>>>> searches and "stack overflow" > > searches > > > > not > > > > > > > >> > >>>>> finding > > > > > > > >> > >>>>>>> the > > > > > > > >> > >>>>>>>>> right > > > > > > > >> > >>>>>>>>>>>>>>>>>>> answers. > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>>> This > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>>>> is one of the strengths of > Airflow - > > > > > > > >> > >>> besides the > > > > > > > >> > >>>>>>>>> community > > > > > > > >> > >>>>>>>>>>> and > > > > > > > >> > >>>>>>>>>>>>>>>> ideas > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>> that > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>>>> Bernd mentioned - is the vast > number > > > of > > > > > > > >> > >>>>> examples, > > > > > > > >> > >>>>>>>>> problems > > > > > > > >> > >>>>>>>>>>> and > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>> solutions > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>>>> you can so easily find (and we > have > > to > > > > > > > >> > >>> remember > > > > > > > >> > >>>>>> that > > > > > > > >> > >>>>>>>>> all the > > > > > > > >> > >>>>>>>>>>>> AI > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>> trained > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>>> on > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>>>> past data will be also rather > poorly > > > > > > > >> > >>> matching > > > > > > > >> > >>>>>>> queries > > > > > > > >> > >>>>>>>> of > > > > > > > >> > >>>>>>>>>>>> people. > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>>>> I am not too attached to DAG. I > > could > > > > > easily > > > > > > > >> > >>>>>> switch. > > > > > > > >> > >>>>>>>>> And if > > > > > > > >> > >>>>>>>>>>> we > > > > > > > >> > >>>>>>>>>>>>>>> do > > > > > > > >> > >>>>>>>>>>>>>>>> - > > > > > > > >> > >>>>>>>>>>>>>>>>>>> I > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>>>> would be for using workflow or > > > pipeline > > > > > > > >> > >>> instead > > > > > > > >> > >>>>> of > > > > > > > >> > >>>>>>>>> `dag` if > > > > > > > >> > >>>>>>>>>>>> not > > > > > > > >> > >>>>>>>>>>>>>>>> the > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>> above > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>>>> reason, but I think I am here with > > > Igor > > > > > > > >> > >>> that it > > > > > > > >> > >>>>>>> might > > > > > > > >> > >>>>>>>>> cause > > > > > > > >> > >>>>>>>>>>>> more > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>> problems > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>>>> than it solves. > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>>>> But I am not 100% against - if > > others > > > > will > > > > > > > >> > >>> think > > > > > > > >> > >>>>>>> it's > > > > > > > >> > >>>>>>>> a > > > > > > > >> > >>>>>>>>> good > > > > > > > >> > >>>>>>>>>>>>>>>> idea, I > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>> am > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>>> ok > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>>>> with it. > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>>>> J, > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>>>> On Tue, Oct 22, 2024 at 3:12 PM > > > Abhishek > > > > > > > >> > >>> Bhakat > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>>>> > <abhishek.bha...@astronomer.io.inva > > > > > > > >> > >>> <mailto: > > > > > > > >> > >>>>>>>>>>> abhishek.bha...@astronomer.io.inva>lid> > wrote: > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>>>> > > > > > > > >> > >>>>>>>>>>>>>>>>>>>>>>>> Agreed that the word DAG makes > very > > > > less > > > > > > > >> > >>> sense > > > > > > > >> > >>>>> to > > > > > > > >> > >>>>>>>>> someone > > > > > > > >> > >>>>>>> >
