I stayed out of this discussion for the longest time, but I really try to avoid "naming" discussions.
However, the extensions needed to Airflow for enhanced AI capabilities such as cyclic operations and "human in the loop" workflows is something I have been thinking a fair bit about. I also agree that these will need to come relatively soon, before 3.5 :) I am not advocating to reopen the discussion regarding naming, but wanted to address one of the other segways in the discussion. Vikram On Tue, Feb 18, 2025 at 4:15 PM Jarek Potiuk <ja...@potiuk.com> wrote: > Also, to hopefully at a little more humor to our "lively debate" > > > I’m not sure conflating your opinion on this with “it’s just good > engineering to make things blurry sometimes” creates a logical truth. > > I think it really depends on the definition of "engineer", for me > "engineer" is someone who builds stuff that is useful for people, not one > that is tied to naming and classifications and mathematical abstractions > (while the engineer knows and understands them perfectly well). > > This discussion reminded me of this (funny in my opinion) joke that > illustrates the difference. I hope others will find it equally funny, and > that I will not offend anyone with mathematical background (I apologise in > advance if you feel offended): > > Two engineers who are flying a test flight of a new balloon they just built > have been caught by a storm and they were lost in the middle of nowhere. > They fly pretty low and they see a man walking down. > They shout to him "Where are we now?!!" > The man answers "In a balloon!!!!" > The wind blows them far from the man and he disappears. > Baffled for a moment, those two engineers look at each other and one of > them says: "It must have been a mathematician." > "Why?" asks the second. > "Because his answer was extremely precise, and completely useless". > > Good night :) > > J. > > > > > > > On Wed, Feb 19, 2025 at 12:49 AM Jarek Potiuk <ja...@potiuk.com> wrote: > > > Yep. that will do as well for me - in the form proposed. We do not "have > > to" redefine the name, leaving it as simply "dag" and explaining the > origin > > while clearly separating from it is also a good way to segway to > "somewhat > > cyclic" workflow / graph if we decide to (which I think will happen > sooner > > than 3.5 or 4.0 Jens :)) > > > > J. > > > > > > On Wed, Feb 19, 2025 at 12:42 AM Ryan Hatter > > <ryan.hat...@astronomer.io.invalid> wrote: > > > >> Long after opening this can of worms, I also agree with Daniel S: Let's > >> define "DAG" in the context of Airflow and be done with it (at least for > >> now :) ). I've opened a docs PR attempting to do just that: > >> https://github.com/apache/airflow/pull/46875 > >> <https://github.com/apache/airflow/pull/46875> > >> > >> On Tue, Feb 18, 2025 at 6:16 PM Ferruzzi, Dennis > >> <ferru...@amazon.com.invalid> wrote: > >> > >> > My two shillings: I came to Airflow knowing what a DAG was in the > math > >> > sense, and I was a bit surprised to see it used for Airflow. Our > DAGS > >> > aren't technically DAGs and haven't been since task retries were > >> > introduced, maybe even before that. I'd support what Daniel said. > IFF > >> > we're going to change the name, I think "Workflow" works better than > >> trying > >> > to redefine an existing known term, but honestly I would advocate for > >> > switching to using "Dag" as a proper noun with some little note > >> somewhere > >> > that the name comes from DAG but we've since evolved past that strict > >> > definition. > >> > > >> > > >> > - ferruzzi > >> > > >> > > >> > ________________________________ > >> > From: Jens Scheffler <j_scheff...@gmx.de.INVALID> > >> > Sent: Tuesday, February 18, 2025 2:03 PM > >> > To: dev@airflow.apache.org > >> > Subject: RE: [EXT] Airflow should deprecate the term "DAG" for end > users > >> > > >> > CAUTION: This email originated from outside of the organization. Do > not > >> > click links or open attachments unless you can confirm the sender and > >> know > >> > the content is safe. > >> > > >> > > >> > > >> > AVERTISSEMENT: Ce courrier électronique provient d’un expéditeur > >> externe. > >> > Ne cliquez sur aucun lien et n’ouvrez aucune pièce jointe si vous ne > >> pouvez > >> > pas confirmer l’identité de l’expéditeur et si vous n’êtes pas certain > >> que > >> > le contenu ne présente aucun risque. > >> > > >> > > >> > > >> > Wow what a discussion thread. Was reading it and...: > >> > > >> > I am okay to clean up docs and agree to the others that we should NOT > >> > change code interfaces. > >> > > >> > For the marketing part I need to repeat: (Almost) Everybody touching > >> > Airflow needs an explanaition what "DAG" means. Changing the acronym > to > >> > have another meaning for DAG still needs an explanaition. For me > >> > reanming the meaning of the Acronym does not bring any benefit, also > not > >> > Marketing. > >> > > >> > I have also seen multiple times challenges for teams in our area (even > >> > outside ML) who wanted to iterate over results and we needed to > >> > implement complexer multi-DAG structures to make this possible because > >> > of DAG. If we would go in the direction as Jarek pitched (which might > be > >> > earliest 3.5 or 4.0) that a non-DAG workflow would be made possible > (I'd > >> > LOVE this!) then I would strongly opt for renaming it to "Workflow". > >> > Because everybody understands (or thinks he understands) what it is > and > >> > DAG might be one implementation of this. > >> > > >> > So my conclusion is I am not for changing the meaning of DAG. > >> > > >> > On 18.02.25 19:54, Jarek Potiuk wrote: > >> > > Ech. I would love so much if we could correct sent email same way we > >> can > >> > > correct messages in Slack :D > >> > > > >> > > On Tue, Feb 18, 2025 at 7:50 PM Daniel Standish > >> > > <daniel.stand...@astronomer.io.invalid> wrote: > >> > > > >> > >> damnit --- meant to say is *not* strictly speaking .... > >> > >> > >> > >> On Tue, Feb 18, 2025 at 10:46 AM Daniel Standish < > >> > >> daniel.stand...@astronomer.io> wrote: > >> > >> > >> > >>> Yeah I also disagree with code changes here. This thread went in > an > >> > >>> unexpected direction since I last poked my head in :) > >> > >>> > >> > >>> My thought is just in docs I would de-emphasize the mathy part of > >> this. > >> > >>> We can say a DAG is airflow's model for a collection of tasks that > >> run, > >> > >>> typically on a schedule. We could say further add, e.g. in *one > >> place* > >> > >>> somewhere, that the name DAG originated from a mathematical > concept > >> > >> called > >> > >>> directed acyclic graph. But I do not think we need to go revising > >> > >> history > >> > >>> about that and providing new words for the acronym. > >> > >>> > >> > >>> But it has always been the reality that an Airflow DAG is strictly > >> > >>> speaking a directed acyclic graph. It's something different. We > do > >> > >> forbid > >> > >>> cycles. And it does contain the info needed to construct a graph > of > >> > the > >> > >>> tasks. But it's much richer than that concept as well. > >> > >>> > >> > >>> I don't think we really need to go much further than that. But > I'd > >> > also > >> > >>> be in support of writing `dag` or `dags` in docs instead of DAG or > >> DAGs > >> > >>> because it's ugly to do so, and unnecessary, and it invokes that > >> mathy > >> > >>> concept that is both confusing and inadequate. > >> > >>> > >> > >>> > >> > >>> > >> > > >> > --------------------------------------------------------------------- > >> > To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org > >> > For additional commands, e-mail: dev-h...@airflow.apache.org > >> > > >> > > >> > > >
