Thanks for your work on this, Aymeric - the reduced scope has really done well, I think.
Couple of comments (I agree that generally the list of work is accurate): - There's already a MIGRATION_MODULES setting which lets you set the path to a migration module by app label. We probably just need to rename it for consistency, no need to implement it! - I'm not sure that installed_apps is a good name to have to replace app_cache, but I also think that having "cache" in the name is useless. I don't mind just calling it "apps". - App labels need to be unique, and we should enforce this. It's always been the case that they had to be, and we've never checked properly. Too much stuff relies on their uniqueness. The big question is, of course, how feasible is it that this will all* land by the alpha? It's January 20th - less than a month away - and while I'd like to see this land in 1.7, if it misses that date it's out; the release is already complex enough as it is to have a behind-time complex feature drag it back further, and now your dedicated time on this is over it's going to move slower. Andrew * By "all", I mean "enough of it that it works, doesn't break anything, and is an obvious improvement" On Mon, Dec 23, 2013 at 9:50 PM, Aymeric Augustin < aymeric.augus...@polytechnique.org> wrote: > The time frame I had allocated to this project expires tomorrow. With > reference to my original list, I’ve reached goals 1, 2, 6 and 7, with the > caveat that I still have to write documentation. > > I ran out of time before looking seriously at goal 3, but it appears to be > within reach at this point. The basic ideas are: > - to populate the app cache “early”, for instance in > get_wsgi_application() and execute_from_command_line(), > - to call a conventional method on the app config, say startup(), as soon > as the app cache is populated. > Apps that need to execute code at startup would put it in > AppConfig.startup(). This solution doesn’t use signals, as there is no > earlier opportunity to register them. > > I know that I didn’t reach the standards of communication and quality > expected in the Django community while I was working on this project. I > didn’t try to have a discussion on this mailing list. I asked a few core > devs who worked on app-loading in the past to look at my code and > ruthlessly pushed it as soon as I got their feedback. Given the staggering > amount of hard problems I was trudging though, that’s all I could manage, > and it was a conscious choice. Now that master has almost reached a stable > state, I will do my best to gather feedback and fine tune the code until > the 1.7 release. > > I’ve mentioned a few times that I was maintaining a list of future tasks I > was leaving aside while I was focusing on more pressing issues. Here is it. > > > ** Must do ** > > * Fix the test suite. There are still a few failures, apparently caused by > leaking app cache state. > > * Write documentation, especially explain the app[s].py conventions and > why `for app in INSTALLED_APPS` doesn't work any more. > > * "UserWarning: No fixture named 'comment_tests' found." when running the > test suite. > > * Review previous patches. Extract relevant tests and docs, if any. > > * Review past discussions once again. > > * Close #3951 and file individual tickets for remaining tasks. This > massive ticket is a nuisance. > > > ** Cleanup / small tasks ** > > * Rename AppCache to InstalledApps? Then we should also replace app_cache > by installed_apps. Originally proposed by Arthur Koziel. The parallel with > INSTALLED_APPS is interesting but it may also cause confusion. I would also > consider simply Apps and apps, although `from django.apps import apps` > doesn't look that good. This must be done by 1.7 alpha if we want to do it. > > * Review all uses of get_app_configs() and determine if the results may be > cached. If so, add a receiver for the setting_changed signal to reset the > cache when INSTALLED_APPS changes. > > * Replace TransRealMixin by suitable receivers for the setting_changed > signal. > > * Refactor AppCommand to be able to handle apps without a models module. > > * Adapt the migrations code: > - Refactor signals to stop relying on the models module. > - Introduce MIGRATIONS_MODULE_NAME for consistency with > MODELS_MODULE_NAME. > > * Refactor get_models and get_migratable_models. They can probably be > implemented more straightforwardly. > > * Invalidate _get_models_cache appropriately or remove it entirely. This > doesn't look like a performance-sensitive area. > > * Add a comment to explain how postponing works in populate_models(), or > remove it if possible. See > https://github.com/django/django/pull/2089#discussion_r8516536. > > * populate_models() isn't exactly idempotent. A recursive invocation may > return while models aren't fully populated, and it won't set > _models_loaded. This could probably be an issue in some cases, but it would > be moot if we got rid of the postponing. > > * Consider eliminating the check for duplicate imports of models in > register_model(), which has become unnecessary with the new project layout > introduced in 1.4. It would probably be safer to do it through a > deprecation path. > > * Review the implementation of dumpdata. It looks like it could be > simplified by using AppConfigs. > > * Review how much django.apps and django.conf import — they should stay > lean to avoid import loops. > > > ** Further work / large tasks ** > > (Many of these are open questions that could be debated on this mailing > list.) > > 1) Provide an API to run code at startup, along the lines described above. > > 2) Some fields should be moved from Model._meta to Model._meta.app_config, > which should be set as part of the app cache population process. > > 3) Make it possible to change the label in AppConfig. That's half of the > original feature request in #3591 (the other half being verbose_name). > Requires 2 — and then lots of work. > > 4) Make it possible to change various prefixes (admin permissions, > database tables, dumpdata/loaddata). That was part of Arthur Koziel's GSoC > proposal. Requires 2 — and then some work. > > 5) Investigate whether app labels currently need to be unique (the docs > say so, but the code doesn’t enforce it). See > https://groups.google.com/d/msg/django-developers/gzBWU_fUdgc/zlTTkKx-s5QJ. > Consider enforcing unicity. Easier to pull off after 3. > > 6) Store application-specific settings in AppConfig? I'm not sure where to > draw the frontier between settings and app configs but this feels like > feature creep. Maybe not a wontfix but at least a someday/maybe. > > 7) Store urls and views in AppConfig? It never seemed a very practical > idea, and that’s why we ended up with lean AppConfig objects. We’re > certainly not in a position to decide this is a feature we want on Django > at this point. > > I'm not listing multiple instances of a pluggable app because I believe > the current way of formulating this feature request is incorrect and cannot > be implemented. For starters, when you import a model class, to which copy > of its app does it belong? > > -- > Aymeric. > > > > On 15 déc. 2013, at 21:08, Aymeric Augustin < > aymeric.augus...@polytechnique.org> wrote: > > > **Warning: Wall of Text** (and there’s no tl;dr). > > > > History > > > > The reference ticket for the project known as app-loading is > https://code.djangoproject.com/ticket/3591. > > > > Joseph Kocherhans attaches an initial patch when he files the ticket in > February 2007. That patch changes INSTALLED_APPS to a list of “app > directives”, which are simple data structures with a path (eg. > “django.contrib.sites”), a label (eg. “sites”), and a verbose name for the > admin (eg. _(“Sites”)). It updates various parts of Django that deal with > INSTALLED_APPS to account for this change. Vinay Sajip completes this patch > and keeps it up to date from May 2007 to May 2009. > > > > In the discussion Johannes Dollinger points out that app definitions > shouldn’t contain a verbose_name that’s only used by a contrib app, namely > the admin. I disagree for two reasons. Third-party apps could use > verbose_name to reference other apps in user interfaces. > Model._meta.verbose_name lives just fine in core and there’s no obvious > reason to treat applications differently. > > > > Significant efforts occur at the DjangoCon US 2008 sprints but they’re > ultimately unsuccessful. A post-mortem is available at: > https://code.djangoproject.com/wiki/InstalledAppsRevision. It highlights > several issues encountered when attempting to change the format of entries > in INSTALLED_APPS. > > > > At this point there’s a lot of bikeshedding about the format of > INSTALLED_APPS. It’s a red herring but it’s a recurring theme in further > discussions, especially GSoC proposals. Jannis Leidel will eventually > propose a sane API in response to Arthur Koziel’s GSoC proposal. > > > > Arthur Koziel developed an extensive patch during GSoC 2010. His > proposal is at > https://groups.google.com/forum/#!msg/django-developers/TMoght1IWHk/IUEHrOx5pFQJand > his patch at > https://github.com/django/django/tree/soc2010/app-loading. Jannis Leidel, > Travis Swicegood and Preston Holmes continued his work. Marc Tamlyn > described the goals in https://gist.github.com/mjtamlyn/2897920. The most > recent version of the branch is at > https://github.com/ptone/django/commits/app-loading. Development stalled > after DjangoCon US 2012. > > > > While a lot of work went into that patch, unfortunately, it’s hardly a > good starting point: > > - it was written at various stages of the history of Django, which means > it may contain incomplete refactorings due to later changes, > > - many parts are a “work in progress” and it’s hard to tell the intent > of the successive authors. > > However, it contains tons of tests that should still be relevant. > > > > Goals > > > > Many related issues have been closed as duplicates of #3591. As a > consequence, its scope has become frighteningly ambitious over the years. > This is certainly one of the reasons why it hasn’t been resolved yet. > > > > Since I have time and motivation to work on this project from Dec 10th > to Dec 24th, I’ve sorted its goals from the most to the least important and > selected three that will hopefully fit in this time frame: > > > > 1) Allow apps without a models module or package > > 2) Provide verbose_name for the admin > > 3) Provide a reliable initialization signal > > > > In fact, they’re quite independent from one another. > > > > Many more ideas were mentioned on the ticket; here’s a list of goals I’m > not actively pursuing, roughly sorted from most to least likely to happen: > > > > 4) Allow changing app_label in case of name clashes > > 5) Allow changing db_prefix > > 6) Clarify the app cache population sequence > > 7) Enforce consistency between the contents of INSTALLED_APPS and the > AppCache > > 8) Provide APIs to discover things at arbitrary Python or filesystems > subpaths inside apps > > 9) AppAdmin — #7497 > > 10) App-specific settings in application objects > > 11) Order of INSTALLED_APPS — #21018 > > 12) Support deploying multiple copies of an app — most likely not worth > solving, if possible at all > > > > Once again, they’re quite independent from one another. > > > > If some of these become trivial as a side effect of my work, fantastic. > Otherwise, I’ll leave them for later. My work will certainly make them > easier to achieve in the future. > > > > I will follow-up with detailed emails for each of my three goals. > > > > -- > > Aymeric. > > > > > > > > -- > You received this message because you are subscribed to the Google Groups > "Django developers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to django-developers+unsubscr...@googlegroups.com. > To post to this group, send email to django-developers@googlegroups.com. > Visit this group at http://groups.google.com/group/django-developers. > To view this discussion on the web visit > https://groups.google.com/d/msgid/django-developers/1E8B5E64-64A1-4561-A1F1-0FDD59D5336A%40polytechnique.org > . > For more options, visit https://groups.google.com/groups/opt_out. > -- You received this message because you are subscribed to the Google Groups "Django developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at http://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAFwN1uo7jkxBkcSosWV1f6tmXXERXe-rxX853%2BPCj47VV_RHZA%40mail.gmail.com. For more options, visit https://groups.google.com/groups/opt_out.
