Hi everyone,

This past week I've made some great progress in rewriting the URL 
dispatcher framework and iterating on my implementation. A big part of my 
effort to refactor my original code was to increase the performance of 
reverse() to a level similar to the legacy dispatcher, and to decouple the 
various parts of the code. I think I have now achieved both goals, so I'd 
like to get some feedback on the result. 

The current code can be found at https://github.com/django/django/pull/5578.

I will be cleaning up the code and shuffling around some of it. There's 
still a lot to be done, but the high-level design and public API are pretty 
much finished and ready for review. 

The API consists of 4 parts, most of which are extendible or replaceable: a 
Dispatcher, a set of Resolvers, Constraints and the URL configuration. 

The main entry point for users is the Dispatcher class. The dispatcher is 
responsible for resolving namespaces and reversing URLs, and handles some 
of the utility functions available to users (some more may be moved here, 
such as is_valid_path() or translate_url()). It is a thin wrapper around 
the root Resolver to allow a single entry point for both reversing and 
resolving URLs. It currently provides the following public API:

   - Dispatcher.resolve(path, request=None) -> Resolve path to a 
   ResolverMatch, or raise Resolver404. 
   - Dispatcher.resolve_namespace(viewname, current_app=None) -> Resolve 
   the namespaces in viewname, taking current_app into account. Returns 
   resolved lookup in a list.
   - Dispatcher.reverse(lookup, *args, **kwargs) -> Reverse lookup, 
   consuming *args and **kwargs. Returns a full URL path or raises 
   NoReverseMatch. 
   - Dispatcher.resolve_error_handler(view_type) -> Get error handler for 
   status code view_type from current URLConf. Fall back to default error 
   handlers. 
   - Dispatcher.ready -> (bool) Whether the dispatcher is fully 
   initialized. Used to warn about reverse() where reverse_lazy() must be 
   used. 

I'm currently looking into possible thread-safety issues with 
Dispatcher.load_namespace(). There are some parts of Django that depend on 
private API's of Dispatcher and other parts of the dispatching framework. 
To maximize extensibility, I'll look if these can use public API's where 
appropriate, or gracefully fail if a swapped-in implementation doesn't 
provide the same private API. One example is admindocs, which uses the 
Dispatcher._is_callback() function for introspection. 

If a developer wishes to completely replace the dispatcher framework, this 
would be the place to do it. This will most likely be possible by setting 
request.dispatcher to a compatible Dispatcher class. 

The BaseResolver class currently has two implementations: Resolver and 
ResolverEndpoint. The resolver's form a tree structure, where each resolver 
endpoint is a leaf node that represents a view; it's job is to resolve a path 
and request to a ResolverMatch. Users will mostly use this through 
Dispatcher.resolve(), rather than using it directly. Its public API 
consists of two functions:

   - BaseResolver.resolve(path, request=None) -> Return a ResolverMatch or 
   raise Resolver404.
   - BaseResolver.describe() -> Return a human-readable description of the 
   pattern used to match the path. This is used in error messages. 

There is a slightly more extensive API that allows a resolver to "play 
nice" with Django's resolver implementations. This allows a developer to 
replace a single layer of resolvers to implement custom logic/lookups. For 
example, you can implement a resolver that uses the first hierarchical part 
of the path as a dict lookup, rather than iterating each pattern. To make 
this possible, a resolver should accept the following arguments in its 
constructor:

   - BaseResolver.__init__(pattern, decorators=None) -> pattern is a 
   URLPattern instance (explained below). decorators is a list of 
   decorators that should be applied to each view that's a "descendant" of 
   this resolver. This list is passed down so the fully decorated view can be 
   cached. 

I'm still looking how exactly we'd allow a developer to hook in a custom 
resolver, any ideas are welcome. 

Constraints are the building blocks of the current dispatcher framework. A 
Constraint can (partially) match a path and request, and extract arguments 
from them. It can also reconstruct a partial URL from a set of arguments. 
Current implementations are a RegexPattern, LocalizedRegexPattern, LocalePrefix 
and ScriptPrefix. This is the main extension point for developers. I 
envision that over time, Django will include more constraints into core for 
common use-cases. One can for example implement a DomainConstraint or 
MethodConstraint 
to match a domain name or request method in the URL, or implement a set of 
constraints based on the parse library for better performance than the 
built-in regex-based constraints. A Constraint currently has the following 
public API:

   - Constraint.match(path, request=None) -> Match against path and request, 
   extracting arguments in the process. Returns new_path, args, kwargs or 
   raises a Resolver404.
   - Constraint.construct(url_object, *args, **kwargs) -> Reconstruct a 
   partial URL from *args and **kwargs, and add partial URL to the 
   url_object. Returns url_object, args, kwargs or raises NoReverseMatch. 
   Any arguments used by the constraint should be removed from the returned 
   arguments -- if any arguments are left when all constraints are consumed, a 
   NoReverseMatch is raised.
   - Constraint.describe() -> Return a human-readable description of the 
   constraint used to match the path. This is used in error messages. 

The biggest API problem here is in Constraint.describe(). The current 
implementation is incredibly naive when it comes to the order of 
constraints. If any constraint implements a sort of lookahead/lookbehind 
assertion, the current API doesn't provide a method to properly communicate 
that in an error message. 

The last part of the puzzle is the URL configuration. There is little 
functionality here: it is mostly an effort to standardize and normalize the 
data structure in the URL configuration files, while keeping the 
configuration decoupled from the Resolver and Dispatcher classes. The API 
for Django users will remain unchanged (except for the new decorators option). 
The public API is mostly intended for developers who wish to implement 
their own dispatcher or resolver, while maintaining compatibility with 
Django's method of configuring URLs. It consists of 4 classes: URLPattern, 
Endpoint, Include and URLConf. To illustrate:

[
    url(r'^$', views.home, name='home'),
    url(r'^accounts/', include('accounts.urls', namespace='accounts')),
    url(r'^admin/', admin.site.urls),
]


The above snippet is how you would configure a basic site with a home page, 
accounts section and admin. This URLConf would produce the following data 
structure:

[
    URLPattern([RegexPattern(r'^$')], Endpoint(views.home, 'home')),
    URLPattern([RegexPattern(r'^accounts/')], Include(URLConf(
'accounts.urls'), namespace='accounts'))
    URLPattern([RegexPattern(r'^admin/')], Include(URLConf(admin.site.
get_urls(), app_name='admin', decorators=[admin.site.admin_view]), namespace
='admin'))),
]

As you can see, the latter is a lot more verbose, while I think the compact 
URL configuration is one of the great features of Django's URLConf. That's 
why the API for configuring URLs will remain as it was, and only the 
resulting data structure will change. The only reason one would need to use 
these classes directly is to bypass the additional checks in url() and 
include() -- as happens e.g. in admin.site.urls.

----------------------------------------------------------------------------------------------------------------

Here's a small overview of what still needs to be done:

   - General clean-up.
   - Investigate thread-safety in Dispatcher.load_namespace().
   - Investigate, and where possible, replace use of private APIs by code 
   outside django.urls.
   - Provide hooks to replace the dispatcher and resolver. 
   - Expand on the describe() API to allow for more complex patterns.
   - Expand and rewrite the tests in urls_tests.
   - Document the new framework API. 

There's still plenty to do, but I feel it is finally nearing completion. 
Any help, feedback and testing is welcome to give it that final push to a 
merge. I will certainly need some help to extensively document the new API. 

Marten

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/4c27e36b-d3fb-40f2-9c15-a9b6f4975726%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Reply via email to