Pascal, you can read about the reason for the action actions change on the ticket https://code.djangoproject.com/ticket/29917 ...and on this mailing list... https://groups.google.com/d/topic/django-developers/-OWoYL_zryM/discussion. I hesitate to link to things like that in the release notes because the discussions can be long and somewhat difficult to follow. Rather, as appropriate, I tried to include a short summary of the rationale for backwards incompatible changes in the release notes.
In my years working on Django, I found it reasonable to occasionally make backwards incompatible changes if there's a consensus that a deprecation isn't feasible or worthwhile. Sometimes practicality beats purity, in my experience. Luke's experience and views parallel mine. I think the current deprecation cycle works well. On Tuesday, August 6, 2019 at 4:12:23 AM UTC-4, Pkl wrote: > > Hello Luke, > > thanks for your comments, mine are below > > On Thursday, August 1, 2019 at 10:34:22 AM UTC+2, Luke Plant wrote: >> >> Hi Pascal, >> >> I know this is a very late reply, but there have been some things going >> round my head that I thought I should get down. >> >> *First,* some positive changes I think we can make from your suggestions: >> >> 1. I think in our release notes, for breaking changes that might need >> some justification, we should link to the tickets and/or django-devs >> discussions behind them. Adding full reasoning in the notes themselves >> would likely bloat them far too much, but a link could be helpful for >> anything which is not obviously necessary. >> > > Yes documenting the rationale behind breaking changes, via a ticket link > or another medium, would be indeed a nice improvement. > > >> 2. We should change the wording in our API stability docs >> <https://docs.djangoproject.com/en/stable/misc/api-stability/>, which >> does indeed initially sound like a promise never to break working code that >> follows docs: >> >> Django promises API stability and forwards-compatibility since version >> 1.0. >> >> We do have this line later on: >> >> If, for some reason, an API declared stable must be removed or replaced, >> it will be declared deprecated but will remain in the API for at least two >> feature releases. Warnings will be issued when the deprecated method is >> called. >> >> But we do make changes at a rate faster than that document would make you >> think. >> >> I suggest that as a contrast to the existing text on that page, we also >> point out that we are continually improving Django, and have a >> "(eventually) one way to do it" policy regarding how we plan APIs, which >> means we will remove old things, while always providing better >> alternatives. I'm happy to write a patch for this. >> > > As you imagine, I'd rather have had a change in actual django policy, > rather than a doc update to reflect the current status; but it'd be better > than nothing. However, the second sentence "*If, for some reason, an API > declared stable must be removed or replaced, it will be declared deprecated > but will remain in the API for at least two feature releases*" is itself > still misleading, since at least one recent major breakage violates it > <https://docs.djangoproject.com/en/2.2/releases/2.2/#admin-actions-are-no-longer-collected-from-base-modeladmin-classes> > > ; *A pronouncement of the core dev team would be needed here**: is this > behaviour change a procedure error? Or is it really the new policy, to > allow changes which create immediate silent errors, and prevent third-party > compatibility shims from getting implemented?* > > Having "only one obvious way to day it" is a useful and common commitment > in Python, but please note the "obvious". You won't have "one way to do > it", ever. If people want to use sys.stdout.write(), or do a libc-cffi call > to output text to console, instead of just calling print(), they will do > so. And compatibility shims don't change anything on this matter: the > "obvious", "standard" way of doing is just the one mentioned in official > docs and which doesn't trigger DeprecationWarnings. > > > > >> *Second,* there were things I wanted to say that might be useful to you >> (Pascal) in understanding why you are not likely to get very far with your >> current approach. >> >> Your exaggerated tone is not likely to help you. You talk about Django's >> compatibility breakages as "*ruining the day of thousands of anonymous >> plugin/website maintainers all over the world*". When we make these kind >> of changes, we always do it with very clear and detailed upgrade notes, and >> almost always with several releases emitting deprecation warnings, as per >> our policy. And no-one is forced to upgrade immediately. Those who value >> stability above other things can use LTS, which gives them a *3 year >> release cadence,* not 8 months, which is far from excessively fast >> compared to other projects. >> > > > Imho dependency hell is absolutely unrelated to detailed release notes or > LTS versions. Dependency hell is just the fact that, when each minor > version breaks compatibility, the app ecosystem inevitably explodes, > especially in its outskirts (i.e the least popular packages), into a myriad > fragments, each targeting specific django versions. > Your project remains on 1.11 LTS ? Nice, but one of your dependency (and > its latest security bugfixes) require a new feature of Django 2.0; while > another dependency is stuck on Django 1.10, either because the maintainer > vanished, or because he is himself trapped by the requirements of a > subdependency. > > That's why Semantic Versioning was invented: so that developers can > gradually update their own codebase (since each minor version brings > improvements and deprecation warnings), while still ensuring all > dependencies keep working under the same major versions (the project just > has to install the latest minor version). The concept of "lax semantic > versioning" sounds like a joke, it destroys the very purpose of this > versioning scheme; would you put a "laxly waterproof watch" under water? > > You mention Mezzanine, and indeed this CMS, being quite monolithic, should > not have big troubles with the Django compatibility policy. More generally, > as long as one sticks to a bunch of big packages (Mezzanine, > Django-rest-framework, Django-extensions...), things work fine. Dependency > hell occurs when you use modular architectures, and want to cherry pick the > most relevant modules for your needs, even those who haven not invested > into custom shims and multi-version testing, or which have not received > updates for a few years. > Such modular architectures are precious, they ensure that users don't get > trapped by feature choices of underlying frameworks; but they need this > extra bit of carefulness about softare compatibility. > > Take for example Django-CMS, a dozen external content type plugins > (ckeditor, restructuredtext, videos, musics, image galleries...), a blog > engine and its bridge (Zinnia), some utilities (easy-thumbnails, > django-filer...); and there you are, struggling to make it all work despite > large-impact breakages of the Django API. All this, not to do rocket > science, just to build what we would call a simple website. > > Let's note that Mezzanine had to fork its own dependencies, by the way, > and in big part for what reason? > > *"At the time of grappelli_safe's creation, Grappelli was incorrectly > packaged on PyPI, and had also dropped compatibility with Django 1.1 -- > grappelli_safe was therefore created to address these specific issuesAt the > time of filebrowser_safe's creation, FileBrowser was incorrectly packaged > on PyPI, and had also dropped compatibility with Django 1.1 -- > filebrowser_safe was therefore created to address these specific issues."* > Nice, more fragmentation. > > Now, I can't talk about percentages. I just *estimate *that thousands of > projects (a part of the total) are relying on modular frameworks like > django-cms, or must make lots of small django apps play together. That's > for these people that days are wasted, even if they don't realize how > unneeded all this work duplication is. For sure, dependency hell sounds > like a made-up or exaggerated concept, until you have to deal with it; to > take a harsh metaphor, people of Europe see hippos as peaceful giants, but > that's not the opinion of the 3000+ people they kill each year in Africa. > > If there is an *exageration *somewhere, I bet it's rather on all that > "*backwards > compatibility is too hard/costly/dangerous*" speech I'm being served. My > experiments at the contrary showed that backwards compatibility was easy to > setup and maintain, provided you have some tooling under the hand (tooling > that I did implement, like import aliasers). Why be worried about "*Django > codebase (becoming) a pile of backwards compatibility hacks and cruft*", > when compat-patchers precisely separate the compatibility concern from the > rest? > As for the fear of "non local reasoning" expressed in other posts, I'm > sorry, this doesn't apply here; compatibility fixers don't do evil things, > like renaming some objects and removing others. They only ADD shims, they > are activated on demand, they output logs and warnings to keep users > informed, and if a shim breaks, you have maybe 99% chances the faulty > wrapper/utility will appear in the traceback. So there is no backstabbing > here. > > By the way, some points I should have added to my initial article: > > - The most widely breaking changes are, generally, the easiest to fix. > Moving a core function/module will break every app, and can be fixed in > <10 > lines of code (tests included). Changing the corner-case behaviour of an > ORM query method is hard to maintain, but very few users should be > impacted > by this change. That's why deprecation periods should be set on a > case-by-case basis, and maybe 90% of breakages can be avoided just with > some dozens of tiny long-term shims. > - If the framework code evolves too fast for its own maintainers to > cope with compatibility shims, then it probably also evolves too fast for > plugin and project maintainers to keep track, too. In this case, it should > be called alpha-stage software, and people should wait for it to stabilize. > > > Thats why I don't buy your "*Quite quickly you get to a point where the > quantity of cruft, and the need to support all the different legacy ways, > means that adding new things becomes very difficult, and development would > have stagnated and quality dropped*". When implementing DCP, I didn't > experience this nightmare you describe. Shims can be dropped in rare cases, > when they conflict with new features or introduce security holes, and > deprecated features don't have to be supported on bugtrackers, so no hands > are tied. > > > >> *Instead, my code would be just as bad/quirky today as was did back >> then*, for two reasons: >> >> 1. Without being forced to upgrade, I probably wouldn't have done. >> > > I have trouble following you here. Why do so many developers format their > code, lint it (with pylint, xenon, doc8, mypy...), test it, package it, > tox-integrate it, document it, internationalize it? Are they compelled to > do so? Will python refuse to run if one of these best practices is not > respected? No. > Developers just keep a codebase clean because they know it's the best way > to keep it maintainable, and to attract users and contributors (+ the > happiness feeling of properly done work). > Fixing (deprecation) warnings is only one of these best practices. Why > would people skip *this *particular cleanup task, and ignore deprecation > warnings but not other alerts or test failures? Especially if, as you > mention, upgrading a single codebase to comply with new django standards is > quite straightforward. > Note that if the new way of doing things seems less intelligent to your > users than the old way (no additional clarity, no new features, no new > bugfixes), there is probably a problem with the development process, and > twisting user's arms by quickly removing old ways of doing is not a > solution. > Backwards compatibility shims are not meant as a laziness appeal for > active projects; but as a solution to dependency hell for modulare > projects, and as a last recourse for low-maintainance plugins; it avoids > the explosion of unmergeable forks while a project waits for a new > maintainer. > > By the way, your old code is not pretty, but it's understandable; and if > your project is unmaintained, I'd rather have it not pretty BUT working, > than not pretty and plain BROKEN. > > > There are many improvements, for example: >> >> - Star imports have been removed. There are lots of good reasons why >> every Python code linter complains about these. Or would you rather still >> have them? >> - In my editor I can put my cursor on a view function like `index`, >> do "jump to definition" and it does the right thing, because it is normal >> Python. Do you think dotted strings were better? >> - My list of URLs is just a list, and can be manipulated with normal >> list methods and functions (which I often need to do). Do you think >> wrapping everything in `patterns()` would be an improvement? For the sake >> of an extensibility we might just use someday (though we didn't in well >> over a decade)? >> - The addition of `path` is a great help, and has really cleaned up >> URL handling a ton, and even more in other areas of my project. I wasn't >> forced to make this change, but it's a great improvement. *However, >> this is the kind of big addition that the Django developers only >> considered >> and were able to do because the Django codebase was not a pile of >> backwards >> compatibility hacks and cruft already. *And yes, doing so meant we >> lost some *undocumented* classes which were implementation details, >> which you also seem to be complaining about. >> >> To clarify, I'm not questioning the interest of individual django > changes, just asserting that leaving the "old ways" around for a much > longer period would have harmed no one and helped a lot. (btw, string > references are still everywhere in Django, and a decent IDE like Pycharm > will easily bring you from INSTALLED_APPS, MIDDLEWARES or template_name to > the proper python/html file). > > Regarding your last points, I think we all agree that Wordpress is a > dumpster fire of security holes (with a jaw-dropping database schema > furthermore), and that AngularJS was an epic failure: an experimental toy > which was advertised as a next-gen production-ready framework, whereas it > just didn't scale - and I fell for it too (but at that time, alternatives > like ReactJS didn't feel so future-proof to me either). > Django avoided these big traps, and it's very nice. This doesn't impact > the separate topic of backwards compatibility, in which Django could shine > by just applying minor cultural changes imho. > > *To sum up,* I know that I'm not likely to change the local mindset, on > this "phobia of shims" as I dare calling it. Personally, I already solved > my recurring problems, thanks to django-compat-patchers. I'd just like *not > to get spokes in the wheels*, with non-shimmable changes, hence the > pronuncement asked above. *Can we all agree at least on this (and > correcting the documentation as you nicely proposed)?* > > It'll still pain me to see so many old but nice plugins mass-forked for > nothing, when 2 lines of project-level code would have fixed everything; > but in software world, choosing the longest work path is a recurring > problem (remember all these people who reimplement half their project in > broken > asyncio <https://veriny.tf/asyncio-a-dumpster-fire-of-bad-design/>, when > greenthreads would have solved their performance troubles for free until > further changes are needed, or Trio/Curio > <https://vorpus.org/blog/some-thoughts-on-asynchronous-api-design-in-a-post-asyncawait-world/> > > would have at least paved a robust future?) > > regards, > Pascal > > > > > > > > -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/3adb7b80-c750-4cf9-95df-128f9f6fe7cc%40googlegroups.com.