potiuk commented on issue #18316:
URL: https://github.com/apache/airflow/issues/18316#issuecomment-922248914
My point is not that "it's not possible" or that it's not "useful" or not
that it's not "used elsewhere". I simply think comparing it with Windows or
MacOS "Explorer/Finder" which is targeted for different users and use case is
not very good comparison. The fact that you CAN do something does not mean that
you SHOULD.
My point is that we are really not in general "filesystem" users camp where
you have non-professional users who do not understand what leading zeros are
and how they impact sorting.
The Dag ids we have are generated programmatically and company using them
builds their own dags can use and enforce their own conventions for naming
them. Rather than accepting and supporting (rather poor in terms of consistency
and organisation) naming convention where you have natural numbering I think we
should promote better organised ones,
Most of the users who create DAGs are data scientists/engineers who
understand numbers, sorting, leading 0s etc. And most of all naming your dags
with consecutive numbers: 0/1/2 etc. is actually a bad idea on it's own because
it tells exactly nothing other than this is the "next" version of the same DAG
(without giving any hint of the reasoning behind it). Having multiple versions
of the same dag with only consecutive number difference is a terrible
anti-pattern IMHO.
I believe it's good choice for Airflow to stick with lexicographic sorting
and not to promote some "bad" conventions - and since it is rather easy to make
your dags define and follow "good" conventions I see "natural sorting" as
simply bad choice for Airflow that would promote bad naming conventions.
So to explain why I marked it as "won't fix": not because I thought it as
("not new") idea. The idea is well known, but I think applying it Airflow DAG
ids is a bad idea.
But if you have arguments why natural sorting might be useful for
production, setup where data engineers have to manage multiple DAGs, I am happy
to hear the arguments.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
