potiuk commented on issue #12120: URL: https://github.com/apache/airflow/issues/12120#issuecomment-932971058
I am not sure we should keep it as "open issue" if there are no clear tasks to complete. I think the scope of the issue is very generic and it has no chance to ever end if it is defined the way it is defined. I think we should not have (or have very little) of generic issues such us "continuous update of something" without having a list of tasks this leads to. The current approach for dependency management is well estabilished when it comes to automated upgrades:: * we try put no upper boundaries of dependencies * unless we have to for some reason and we document what the reason is in setup.py * the "constraint upgrade" mechanism is specifically designed to update the constrainst as the new versions are released The limitations we have are in: * setup.cfg when they are imposed by airlfow * setup.py when they are imposed by one of the providers * Dockerfile.ci/Dockerfile if they are imposed by imperfect `pip eager-upgrade` mechanism that will prevent the combination of `airflow + providers` from working correctly when the eager-update mechanism is invoked. However we have no process currently (and this issue is not helping with it on how to get rid of the upper-bound limitations over time - this is done on an ad-hoc base. I think most of those limitations are somewhaat documented (with URLs and explanation where they came from in the setup.py/.cfg/Dockerfiles (but possibly not all of them are). I believe we should have an explanation for every single upper-bound limit that we have. Also some of the reasons for those might be alreaady gone, but we do not know it, some of them might need some work to remove (for example recent celery 5 upgrade). I think a good idea how to treat that one would be that someone ( might be a even a good first issue) reviews all the setup.py/setup.cfg/Dockerfile upper-bound limitations and verifies if the reasons are: a) documented b) still valid c) maybe even we should create a separate issue type for such limitations Then what we possibly need is some kind of periodic review of those "upper bound" - if we have list of issues for those, that might be actually possible to manage and contro and even periodically review them (I think we could even automate big parts of this when the new GitHub Issues are enabled for public projects). So I think we should close that issue and create a bunch of issues (grouped together) for every single limitation we have now and figure out what we do with those - but it requires a non-trivial amount of checking/verifying the current limits (maybe some investigations as well on where they came from), documentting and possibly organising a bit the process of regular review. Anyone :) ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
