Long after opening this can of worms, I also agree with Daniel S: Let's define "DAG" in the context of Airflow and be done with it (at least for now :) ). I've opened a docs PR attempting to do just that: https://github.com/apache/airflow/pull/46875 <https://github.com/apache/airflow/pull/46875>
On Tue, Feb 18, 2025 at 6:16 PM Ferruzzi, Dennis <ferru...@amazon.com.invalid> wrote: > My two shillings: I came to Airflow knowing what a DAG was in the math > sense, and I was a bit surprised to see it used for Airflow. Our DAGS > aren't technically DAGs and haven't been since task retries were > introduced, maybe even before that. I'd support what Daniel said. IFF > we're going to change the name, I think "Workflow" works better than trying > to redefine an existing known term, but honestly I would advocate for > switching to using "Dag" as a proper noun with some little note somewhere > that the name comes from DAG but we've since evolved past that strict > definition. > > > - ferruzzi > > > ________________________________ > From: Jens Scheffler <j_scheff...@gmx.de.INVALID> > Sent: Tuesday, February 18, 2025 2:03 PM > To: dev@airflow.apache.org > Subject: RE: [EXT] Airflow should deprecate the term "DAG" for end users > > CAUTION: This email originated from outside of the organization. Do not > click links or open attachments unless you can confirm the sender and know > the content is safe. > > > > AVERTISSEMENT: Ce courrier électronique provient d’un expéditeur externe. > Ne cliquez sur aucun lien et n’ouvrez aucune pièce jointe si vous ne pouvez > pas confirmer l’identité de l’expéditeur et si vous n’êtes pas certain que > le contenu ne présente aucun risque. > > > > Wow what a discussion thread. Was reading it and...: > > I am okay to clean up docs and agree to the others that we should NOT > change code interfaces. > > For the marketing part I need to repeat: (Almost) Everybody touching > Airflow needs an explanaition what "DAG" means. Changing the acronym to > have another meaning for DAG still needs an explanaition. For me > reanming the meaning of the Acronym does not bring any benefit, also not > Marketing. > > I have also seen multiple times challenges for teams in our area (even > outside ML) who wanted to iterate over results and we needed to > implement complexer multi-DAG structures to make this possible because > of DAG. If we would go in the direction as Jarek pitched (which might be > earliest 3.5 or 4.0) that a non-DAG workflow would be made possible (I'd > LOVE this!) then I would strongly opt for renaming it to "Workflow". > Because everybody understands (or thinks he understands) what it is and > DAG might be one implementation of this. > > So my conclusion is I am not for changing the meaning of DAG. > > On 18.02.25 19:54, Jarek Potiuk wrote: > > Ech. I would love so much if we could correct sent email same way we can > > correct messages in Slack :D > > > > On Tue, Feb 18, 2025 at 7:50 PM Daniel Standish > > <daniel.stand...@astronomer.io.invalid> wrote: > > > >> damnit --- meant to say is *not* strictly speaking .... > >> > >> On Tue, Feb 18, 2025 at 10:46 AM Daniel Standish < > >> daniel.stand...@astronomer.io> wrote: > >> > >>> Yeah I also disagree with code changes here. This thread went in an > >>> unexpected direction since I last poked my head in :) > >>> > >>> My thought is just in docs I would de-emphasize the mathy part of this. > >>> We can say a DAG is airflow's model for a collection of tasks that run, > >>> typically on a schedule. We could say further add, e.g. in *one place* > >>> somewhere, that the name DAG originated from a mathematical concept > >> called > >>> directed acyclic graph. But I do not think we need to go revising > >> history > >>> about that and providing new words for the acronym. > >>> > >>> But it has always been the reality that an Airflow DAG is strictly > >>> speaking a directed acyclic graph. It's something different. We do > >> forbid > >>> cycles. And it does contain the info needed to construct a graph of > the > >>> tasks. But it's much richer than that concept as well. > >>> > >>> I don't think we really need to go much further than that. But I'd > also > >>> be in support of writing `dag` or `dags` in docs instead of DAG or DAGs > >>> because it's ugly to do so, and unnecessary, and it invokes that mathy > >>> concept that is both confusing and inadequate. > >>> > >>> > >>> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org > For additional commands, e-mail: dev-h...@airflow.apache.org > >
