On Fri, Apr 9, 2010 at 2:59 PM, Russell Keith-Magee
<freakboy3...@gmail.com> wrote:
> On Fri, Apr 9, 2010 at 12:33 AM, Nick Sandford <nick.sandf...@gmail.com>
> wrote:
>> An App Loading mechanism for Django
>> ====================================
>>
>> About Me
>> ----------
>> Hi everyone,
>>
>> My name is Nick Sandford, I'm an electrical engineering student at the
>> University of Western Australia.
>>
>> Background
>> -----------
>>
>> I haven't been a particularly active contributor to any open source project -
>> here is where I hope to change that. In my current work at Christ
>> Church Grammar School we use Django heavily for most of our internal
>> projects. I've
>> followed django-dev closely for a couple of years and I have poked around
>> with
>> its internals on various occasions.
>
> Ok - I clearly need to get a Perth DjUG going...
>
>> Plan
>> -----
>>
>> Implement an improved application loading mechanism into Django.
>>
>> Rationale
>> ---------
>>
>> Django current application loading code is inflexible in many cases. There
>> exists a need for a number of extra features to be added to the way
>> applications are handled such as:
>>
>> * The ability to internationalise application names.
>> * The ability to customise application names similar to ``verbose_name`` in
>> ``model._meta``
>> * Deploy the same application multiple times in a single project.
>> * Deploy two different applications with the same name.
>> * Manage settings within applications.
>>
>> Method
>> -------
>>
>> I don't intend to solve the third dot point of the previous list - it seems
>> like a difficult problem, possibly to be tackled post-GSoC. What I do intend
>> to do is to move towards 'application classes' more akin to
>> ``django.contrib.admin``.
>>
>> New syntax in ``settings.INSTALLED_APPS`` will be introduced to address the
>> previous issues. Some examples of accepted syntax:
>>
>> .. sourcecode:: python
>>
>> INSTALLED_APPS = (
>> 'django.contrib.auth',
>> app('blog.BlogApplication', 'blog_1', 'My blog', 'app1_'),
>> app('blog.BlogApplication', 'blog_2', 'Some other blog', 'app2_'),
>> app(path='tagging.Tagging', verbose_name='My tagging application'),
>> app('categories', db_prefix='cat_'),
>> app({'path': 'django.contrib.admin.AdminApplication',
>> 'label': 'admin',
>> 'verbose_name': 'Secret Admin'}),
>> )
>>
>> The ``app`` function will take four arguments, three of which are optional.
>> These are ``path``, ``label``, ``verbose_name``, and ``db_prefix``. It will
>> return an instance of an ``Application`` object, which will contain all of an
>> installed application's information. ``path`` will be the dotted path to a
>> subclass of ``Application``. The downside is that ``settings.py`` requires
>> an import, which may be against style rules.
>>
>> .. sourcecode:: python
>>
>> def app(path, label=None, verbose_name=None, db_prefix='')
>> if not path or not isinstance(path, basestring):
>> raise ImproperlyConfigured('Application path must be string.')
>> application_class = import_module(path)
>> return application_class(path, label, verbose_name, db_prefix)
>>
>> ``INSTALLED_APPS`` will then be a tuple containing strings or ``Application``
>> instances. The application loading code will iterate over ``INSTALLED_APPS``
>> and construct an internal cache of ``Application`` instances to be used with
>> ``get_models``, etc. For backwards compatibility, if an element of the tuple
>> is
>> a string, an instance of a base ``Application`` class will be created with
>> sane
>> defaults (similar to ``app_label`` at the moment).
>>
>> The ``Application`` class will be very similar to ``django.contrib.admin``'s
>> ``AdminSite`` and will hopefully simplify application settings. If you write
>> views and urls directly on the ``Application`` class itself, instead of an
>> ``from django.conf import settings`` and subsequent ``settings.API_KEY``, you
>> could just reference ``self.api_key`` for instance. This wouldn't be the
>> required, just an optional extra.
>>
>> Model classes will get a ``_meta.app`` attribute, which will be an instance
>> of the model's ``Application`` class. Models should only be associated with
>> one
>> application.
>>
>> I agree with moving ``django.db.models.loading`` to ``core.apps``, since
>> we'll
>> no longer require applications to have a ``models.py``. A reference to the
>> new functions in ``core.apps`` will still live in
>> ``django.db.models.loading``
>> for backwards compatibility.
>>
>> A subclass of ``Application`` might look like:
>>
>> .. sourcecode:: python
>>
>> from django.views.simple import direct_to_template
>> from djang.core.apps import Application
>> from blog.models import Entry, Category
>>
>> class BlogApplication(Application):
>> models = [Entry, Category]
>> api_key = 'default'
>>
>> def entry_detail(self, slug, request,
>> template_name='blog/entry_detail.html'):
>> entry = get_object_or_404(Entry, slug=slug)
>> context = {
>> 'entry': entry,
>> 'api_key': self.api_key,
>> }
>> return direct_to_template(request, template_name, context)
>>
>> Hurdles
>> --------
>>
>> There are a list of possible technical issues:
>>
>> * Introducing the new ``app`` function requires an import in settings.py
>> which might not be acceptable behaviour.
>> * Two applications with the same label.
>> * Worrying amounts of things that may affect backwards compatibility would
>> probably need addressing.
>>
>> Solutions
>> ----------
>>
>> We could alternatively introduce a new file into the project,
>> ``applications.py`` which contains purely application-related setup. This
>> might
>> be handy to manage application settings and to ensure ``settings.py`` doesn't
>> get too full of application-specific settings. This could allow us do
>> something
>> like:
>>
>> .. sourcecode:: python
>>
>> from django.core import apps
>> from blog import BlogApplication
>>
>> class MyBlogApplication(BlogApplication):
>> api_key = 'testing'
>>
>> another_blog = BlogApplication(label='blog2',
>> verbose_name=_('Another blog'),
>> api_key='anothertest')
>>
>> apps.register(MyBlogApplication, label='blog', verbose_name=_('My blog'))
>> apps.register(another_blog)
>>
>> depending on what people would like more. This could allow us not to touch
>> ``settings.INSTALLED_APPS`` at all.
>>
>> To solve the 'two applications named auth' problem, the ``AppCache`` keeps
>> track of application instances, not ``app_label``. For the case of two
>> applications with the same name, ``get_app`` should return a tuple containing
>> both application instances with that name. To ensure a single application
>> instance is returned with ``get_app``, another argument - ``path`` should be
>> added. ``get_models`` and ``get_model`` would take an application instance
>> and return models on that instance. This might affect the admin's handling of
>> applications.
>
> I'm not necessarily sold, but this is certainly an interesting idea.
> In some respects, it's a variation of a suggestion that Jannis made in
> another thread. Jannis suggested that we use dotted notation in
> INSTALLED_APPS to point at the application instances. A registration
> process is certainly another novel way of approaching the problem.
>
> My concern with using a registration process is that we could end up
> with a situation where INSTALLED_APPS no longer contains a useful
> description of what is installed. The interaction between
> INSTALLED_APPS and an app registration process also needs to be
> considered carefully:
> * Does ordering matter? If so, which takes precedence -
> INSTALLED_APPS or registrations?
> * Are registered apps added to INSTALLED_APPS as part of the
> registration process?
That was the initial idea, yes.
> * Is this something that needs to be handled as a
> "APP_LOADER='django.core.apps.loader.legacy'" which uses
> INSTALLED_APPS, and
> "APP_LOADER='django.core.apps.loader.registration'" which uses the new
> approach (i.e., that new projects won't need an INSTALLED_APPS
> setting)?
I quite like this approach, so I've incorporated it into the proposal.
Thanks! This makes it a kind-of all or nothing approach, you either
have settings.INSTALLED_APPS or you have the registration method.
Definitely simplfies trying to keep INSTALLED_APPS and the AppCache in
sync. The legacy loader can still be 'Application'-aware and you can
still iterate over INSTALLED_APPS.
>
> There are two related subproblems that are also worth investigating here:
>
> * The "Stuff that needs to happen when a Django server instance
> starts" problem - you're introducing a new task that needs to happen
> at startup. This isn't the only 'on startup' task - admin
> registration, signal registration, and the proposed logging interface
> all have similar needs. We've been handling this on a per-instance
> basis so far, but there's a greater need to address 'startup problems'
> in a common/generic way.
>
> * The "Register app/object/model with system-wide index" problem - we
> have many different implementations of this pattern in Django already;
> if we're going to add another one, it would be good to find a way to
> abstract this into a common codebase, rather than reinventing the
> wheel each time.
>
> In addition to these two, there is also a nebulous "enterprise
> settings support" issue out there. This essentially amounts to a need
> to separate application settings from deployment settings. Having a
> non-settings-based way to handle application registrations is
> certainly a start down this path.
>
> I know time is running short for you to get your proposal in, but I'd
> be interested to see any ideas you have you on how your project could
> lead to (or contribute to) solutions (or partial solutions) to these
> problems.
>
>> Timeline
>> ---------
>>
>> 1) Implement the ``Application`` class. -- 2 weeks
>> 2) Move ``django.db.models.loading`` to ``django.core.apps`` and refactor
>> ``get_app``, ``get_model``, etc. -- 1 week
>> 3) Modify the admin, management, translation, templatetags, templateloaders
>> to use ``app.path`` -- 1 week
>> 4) Testing and documentation -- 3 weeks
>> 5) Bug fixes and backwards incompatibility problems -- 1 week
>> 6) Time permitting possibly tackle the multiple instances of the same app
>> problem. (about 2 weeks).
>
> A few comments:
>
> Your estimate for point 2 seems light, especially if taken in light of
> the bigger issues of 'startup tasks' and 'registration as a pattern'
> problems I mentioned earlier.
>
> Elaboration of point 3 would be nice. What sort of modifications are required?
Some of these are described in the updated proposal, not an in-depth
look but at least acknowledging the modifications that are required.
>
> Point 4 sends up a red flag for me. Testing in particular isn't an
> afterthought - it's an integral part of development. If the work of
> step (1) doesn't include tests, it's incomplete as designed, IMHO.
I figured if i didn't explicitly SAY that there would be testing and
documentation, then the comment would have been more akin to "where's
the time set aside for testing and documentation?" But yes, I agree
testing and documentation are an integral part of the development
process.
>
> Point 6 Ain't Gonna Happen (tm). Let me tell you right now that it's
> more than 2 weeks work. The foreign key problem alone will require you
> to jump all sorts of technical hurdles. You'd be better served
> dropping this from your proposal altogether and concentrate on other
> issues.
>
Noted and dropped from my proposal :-).
> I also note that this timeline only adds up to 10 weeks. I'm going to
> guess that you're allowing for UWA's exam break -- which is fine --
> but if this is the case, you need to acknowledge it in your schedule.
>
Yeah, exams are in early June. I've added that to the timeline section
of my proposal.
>> Any feedback would be greatly appreciated, sorry for the late application and
>> good luck to all the other applicants :)
>
> On a style issue - your proposal spends a lot of time explaining the
> traditional "App() in INSTALLED_APPS" approach, which you then reject
> for various (valid) reasons, and then moves into the interesting/novel
> part. There's been plenty written on the App() approach; you would be
> better served to not mention this at all if it doesn't form part of
> what you intend to implement.
>
> 2c of advice - There are essentially two major subproblems here:
>
> 1) How to define an App. This bit isn't really controversial - it
> needs to be a class of some sort. However, describing exactly what
> this class can do and how it interacts with existing _meta structures
> is important. Your current proposal covers this, but confuses the
> issue with the old App() in INSTALLED_APPS notation.
>
> 2) How to register an App. This is the big beast that isn't
> completely solved yet. You've presented a novel approach, but it's
> important that you acknowledge in your proposal that this isn't a
> solve issue yet, and will require additional work.
>
> You would be well advised to acknowledge this split in your proposal.
Certainly. Now for some mario kart... and constant checking of gmail.
Cheers,
Nick
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
